Qualitative insight into the industry's top vendors through 18 key performance indicators
Black Book collects ballot results on 18 performance areas of operational excellence to rank vendors by software, systems, products, equipment and outsourced service lines. Client users submit ballots per vendor per function which are juxtaposed with client size, functions outsourced, software installed, systems employed, services rendered, industry/vertical(s) affected, country of service origin and overall service and/or product line domain.
The gathered data is subjected immediately to an internal and external audit to verify completeness and accuracy and to make sure the respondent is valid while ensuring the anonymity of the client company is maintained. During the audit, each data set is reviewed by a Black Book executive and at least two other external reviewers. In this way, Black Book report clients are able to clearly see how a vendor is truly performing. The 18 criteria on operational excellence are subdivided by the client's industry, market size, geography and software category/function outsourced and reported accordingly.
Understanding the statistical confidence of Black Book data
Statistical confidence for each performance rating is based upon the number of organizations scoring the outsourcing service. Black Book identifies data confidence by one of several means:
- Top-10-ranked vendors and advisors must have a minimum of five unique clients represented. Broad categories require a minimum of 20 unique client ballots. Data that are asterisked (*) represent a sample size below required limits and are intended to be used for tracking purposes only, not ranking purposes. Performance data for an asterisked vendor's services can vary widely until a larger sample size is achieved. The margin of error can be very large and the reader is responsible for considering the possible current and future variation (margin of error) in the Black Book performance score reported.
- Vendors, associates, consultants and advisors with over 20 unique client votes are eligible for top 10 rankings and are assured to have highest confidence and lowest variation. Confidence increases as more organizations report on their outsourcing vendor. Data reported in this form are shown with a 95% confidence level (within a margin of 0.25, 0.20 or 0.15, respectively).
- Raw numbers include the quantity of completed surveys and the number of unique organizations contributing the data for the survey pool of interest.
2021 BLACK BOOKTM KEY PERFORMANCE INDICATORS VENDORS
HEALTHCARE INFORMATION SYSTEMS TECHNOLOGY COMPANIES & SERVICES FIRMS
Organizational structure meets the needs of stakeholders or customers and stakeholder satisfaction is the most important priority. Health Information
Technology software and services company ("VENDOR") is likely to recommend the vendor to similar sized health systems, hospitals, physician
groups, physicians, ancillary within the same specialties and or delivery settings. Client goals may include ONC, MACRA, Meaningful Use, HIPAA,
Strategic Growth, Transformation of technology and/or operations, et cetera. Client missions are discussed prior to implementation and considered
core to the implementation teams' objectives.
Customers are continuing to push the vendor for further enhancements to which the vendor is responsive. Vendor clients also believe that their vendors'
technology is helping them manage practices and processes more effectively, generate accurate records and reimbursement billings, track resource
utilization and cut their overhead in ways that were difficult or impossible to accomplish before the specific vendor software and services were
implemented. Vendor is responsive to make client recommendations with cutting edge improvements specific to the client, and, also broad for the entire
improvement of the population's health.
Vendor leadership provides significant and meaningful training opportunities for internal employees and client staff. Leadership strives to develop
technology staff, software client service and customer servicing consultant employees, in particular. Training modules are effective and practical so that
minimal post-implementation training is required on or off site. Regular updates are timely and require minimal additional training to implement. Areas
which require the most or additional training modules and refreshers are provided adequate resources. User group meetings, webinars, onsite training,
offsite training and educational opportunities are common practices for the vendor as offerings to their clients.
The vendor leadership honors customer relationships highly. The relationship with the vendor company elevates the customer reputation. Improving
provider practice and healthcare delivery efficiency and effectiveness is a priority of the supplier. Governance of engagement is neither complex for
buyer nor does it require vendor management attention regularly. There is no regular transparency or quality issue. There are no culture clashes or
misfits that threaten relationship's success or client's satisfaction. Global clients (non-USA) confirm their experience has included efforts to provide
local/regional expertise and employees to enhance the fit between client and vendor.
Trust in enterprise reputation is important to vendor clients as well as prospects. Client possesses an understanding that its vendor organization has
the people, executives, processes, and resources to effectively deliver the desired business and clinical results, based on its industry reputation and
past performance. There are no disconnects between promises and delivery. The client is not affected and or protected by the media coverage of their
vendor, particularly bad news. The conduct of vendor employees from the top to the bottom of the organizational chart is highly ethical as demonstrated
by superb accountability and transparency in actions and activities.
The vendor offers industry recognized horizontal functionality and vertical industry applications, and manage bundled healthcare IT services and
developing new e-Health initiatives. Vendor routinely drives operational performance improvements and results in the areas they affect. Comprehensive
offerings are constructed to meet the unique needs of the client's technology initiatives. Breadth of vendor modules offers comprehensive system
services and broad modules. Complete offerings may include EHR, Population Health, Open Health Information Exchange, Revenue Cycle Tools,
Business and Clinical Decision Support, Analytics, Care Coordination and Cybersecurity tools. Both inpatient and outpatient products are offered by
comprehensive vendors.
The client deploys at a pace acceptable to the client. Vendor solutions eliminate excessive supervision over vendor implementations. Vendor overcomes
client implementation obstacles and challenges effectively. Technical, organizational and cultural implementation obstacles are handled professionally
and punctually. Vendor implementation time meets standard expectations. Implementations are efficient and sensitive to users' specific situations which
may cause delays. There are not significant or unexpected cost overruns associated with the implementation. The vendor employs its own
implementation team or demands best practices from engaged consultants to ensure deadlines are met and budgets are adhered to honor the initial
implementation cost agreements.
Vendor products and process services are customized to meet the unique needs of specific practice client purpose, processes and physician models.
Little resistance is encountered when changing performance measurements as clients' needs vary. Extraordinary efforts are made to adapt and convert
client special needs into workable solutions with efficient cost and time considerations. Vendor software allows for modifications that are not costly or
complex. In the case of a health system with multiple delivery sites and processes, the Vendor provides experienced analysts and implementation
coordinators to develop the functionality and preferences of the client.
The vendor supports interfaces so information can be shared between necessary applications. Solutions are easily integrated to existing backend
systems as needed and connectivity feasible. Seamless interfaces to legacy applications are performed as required for optimal functioning. Human
integration and interface activities are administered precisely. Systems communicate effectively among provider groups and ancillaries. True
interoperability with other healthcare organizations is factored into implementation. Costs of integrations and interfaces is anticipated and detailed in
original contracting prior to implementations so that the client does not endure any unexpected fees or delays. Sharing of patient data is open and not
restricted by construction, platforms, pricing/fees, networking or vendor requirements.
Vendor services and solutions provides flexible pricing allowing the client to choose and pay for the precise functionality and services needed. Vendor
invests in significant infrastructure and have the ability to provide services to enterprise organizations. IT products and services meet the changing and
varied needs of the customer. Pricing is not rigid or shifting and meets needs of client. Client has the advantage and opportunity to select only the
products it needs and avoids a complete rip and replace situation of their complete IT system unless warranted to keep costs manageable.
EHR vendor team of employees is considered top in industry for professionalism and skill. Vendor attracts and retains high performing staff. Vendor is
focused on building and developing a strong employee team of producers. Employees act like owners/leaders. Company is moving towards leveraged
pay at all levels. Vendor is using effective tools to tie performance metrics to compensation policy and compensating top leaders. Human resources-
related criteria are scored from the client perspective on this indicator. Vendor employs a high percentage of employed professionals as opposed to
relying on consultants and per diem staffing to complete implementations. Vendor retains competent staff due to policies and practices that compensate,
recognize and award high performance for clients.
Vendor supplier meets agreed terms as evidenced by routine, acceptable service level reporting and industry expectations. Depth and breadth of
applications/solutions are acceptable in meeting client needs. Online reliability is maximized and outages/downtimes are minimized. Solid product and
service capacities are demonstrated consistently. Service levels are consistently met as agreed. Services and support response is maximized by vendor
team. Service outages are minimal and or anticipated and communicated. Vendor employs contingency technical staff to immediately address any
situation.
Health information technology vendor's marketing and sales statements/pitches are accurately and appropriately represented by actual product and
service deliverables. Vendor image is consistent with top analyst rankings and not inflated by advertising or presentations. Sales presentations and
proposals are delivered upon and corporate integrity/honesty in marketing and business development are highly valued. Company image and integrity
are values upheld top-down consistently. High levels of relevant client communications enhance the vendor - client/user relationship. Vendor image is
not distracted by difficult or damaging media and press coverage that leaks onto its clients. Public relations, analyst relations and media relations are
handled with skill and proficiency with the client's reputation always protected and enhanced where possible.
The vendor's cost savings are realized as generally estimated and not over-positioned or over/underestimated in ways that effect major client
satisfaction or costs. Vendor offers value-adds as a practice management partner in cost savings and avoidance initiatives and creative programs
through bundled product design. Provides true business transformation opportunities to health systems, hospitals and physicians utilizing healthcare
technology. Vendor provides opportunities to client to engage in focused training and education sessions directly corresponding to the products the
client has implemented. Vendor offers modules at little or no cost that benefit the client and user level. Other value-added options may include product
or service development directly around the respective needs of individual clients without cost or delayed in implementations.
Vendor's viability, employee turnover, financial stability and/or cultural mismatches do not threaten relationship. Senior management and the board
exemplify strong leadership principals to steward appropriate resources that impact HIT buyers. Client is confident of long term industry viability for this
vendor based on investments, client adoption, exceptional outcomes and service levels. Field management is notably competent, stable and supportive
of clients. The vendor demonstrates and provides evidence of competent, financial management, and leadership. In the case of publicly traded
companies, the stock price is consistent with a well-managed organization. For private and public HIT companies, there is no wasteful or ostentatious
use of profits as to keep pricing to clients controlled.
To provide secure and constantly dependable health information technology service offerings for physician and hospital/IDN affiliate practices and
entities, a vendor must provide the highest level of security and data back-up services. The vendor's service in these two areas is superior to the
security and back-up system of past internal systems of the healthcare organization. The vendor partners with renowned data security experts, software
and specialists. Data centers are protected and safely distributed. Vendor uses the highest levels of protection, security, authorization, privacy and
defense on behalf of the client.
Account management provides an adequate amount of onsite administration and support to clients. There exists a formal account management program
that meets client needs. External analysts, press/media and other clients reference this vendor as a services leader and top vendor correctly. Customer
services and relationship satisfaction is manifested through significant flagship clients as well as smaller and newest customers similarly. Vendor
provides appropriate number of accessible support and customer care personnel. The vendor offers support via call centers that feature client language
speakers, experienced and highly trained technicians, and hospitable and courteous relationship managers.
Product management and related technology services are considered best of breed. The vendor technology elevates customers via capabilities,
equipment, processes, deliverables, professional staff, leadership, quality assurance and innovative initiatives. Vendor services are delivered at or
above current/former in-house service levels. Technology is current and relevant to exchanging health information among providers, as well as
sufficiently offering patient access. Innovative technologies and applications, as well as managed services are made available to meet the needs of the
respective client and the entire healthcare delivery system.