Alliance Roles

Developing a Public Report:  Key Considerations to Develop and Sustain Multi-stakeholder Buy-in

Choosing the correct display for quality and cost measures is only one part of a multi-year and multi-stakeholder collaborative process that ensures full buy-in from providers, consumers, payers, and purchasers. The broader effort to develop a public report includes critical first steps before display decisions can be made. 

Exhibit 1. The Massachusetts Health Quality Partners  and Consumers Union Partnership

As part of a pilot project funded by the Robert Wood Johnson Foundation and Consumers Union, Consumer Reports published its first patient experience ratings for nearly 500 primary care physician practices in Massachusetts in June 2012. Consumer Reports used Massachusetts Health Quality Partners (MHQP) statewide patient experience survey results. Consumer Reports’ reputation as a trusted source of information for consumers—and its ability to present comparative performance information—allowed MHQP to expand its reach as a way to encourage dialogue among patients, physicians, and advocates about what quality healthcare should look like.

  1. Establish an identity within the community as a credible and unbiased, independent source. Any organization or multi-stakeholder collaborative that is attempting to provide a comparative public report must first establish itself within the community as being independent of commercial or political interests. It is imperative to communicate the organization’s mission, invite and sustain broad-based stakeholder involvement, and establish transparent decision making processes before a public report effort is introduced. Providers, consumers, payers, and purchasers must all view the organization providing the public report as a credible source for comparative data, and an organization that represents the entire healthcare system in its efforts to improve quality. See Exhibit 1 for an example of an Alliance that publically reported patient experience data in partnership with an organization that consumers view as credible and unbiased.
  1. Engage all stakeholders early in the process.  Alliances learned that the development of a community-wide public report requires the active engagement of all relevant stakeholders at its earliest inception. Community-wide public reporting efforts vary, but generally include representatives from health plans, health care providers and delivery systems, large and small businesses, state Medicaid programs, health information technology firms, researchers and statisticians, consumer representatives, consumer groups and advocates, and policy-makers. Each of these stakeholder groups must be actively involved in decisions about the processes for data collection, assessment, and display. Stakeholders should be viewed as—and feel like—partners in the process, and able to see their needs and concerns incorporated in final decisions whenever possible. While this level of engagement may appear to slow down the development process, the Alliances found that the investment actually saved time because they did not have as much opposition or resistance. Alliances also found that acting as a neutral convener and bringing together stakeholders with different perspectives allowed new relationships to develop that are necessary to improve the value of health care (e.g., employers and providers; competing health plans or providers).

Exhibit 2. Sample Language to Describe Data Sources

The Kansas City Quality Improvement Consortium provides the following information on the display page for primary care practices on “The 2011 report reflects information provided by four commercial health plans and did not include any Medicaid information.” More detailed information about the data source can be given on a separate page. This is the approach taken by the Oregon Health Quality Corp., which provides a detailed explanation of the scores for doctor’s offices and hospitals on a separate “About Quality Scores” page.

  1. Use data sources for public reports that are feasible to collect and relevant to consumers. Data sources for measuring performance, outcomes, and resource use that have been vetted and endorsed by the National Quality Forum are plentiful. But measure selection must be based on the ability to collect data accurately and consistently from area providers (e.g., clinical data through electronic medical records or other manual systems) or other sources such as claims data from health plans. Data selection discussions may need to include detailed planning on how data will be made available for public reports, how data is collected and stored, and how potential barriers to data collection can be overcome. A second important criterion in selecting data sources and measures is their relevancy to consumer decision making. The data sources used should provide information that consumers find valuable when thinking about their own care needs.
  1. Make an explicit commitment to accurate data for performance measurement. Initial resistance to public reports of quality and cost often surrounded the validity of the measures used. The multi-stakeholder collaborative producing the report must make a clear and public commitment to all stakeholders that the measures selected for display reflect an accurate assessment of the local healthcare providers. A commitment to data accuracy should be supported by a well-vetted and transparent process for data selection, collection, and interpretation that has consensus from providers and other stakeholders. Many of the Alliance websites include detailed explanations of how measures are created to help users see the data as credible and unbiased. See Exhibit 2 for sample language.
  1. Provide adequate time for data validation by providers before making reports publicly available. A strong data validation process, especially in cases where the data is converted into ratings such as “good” or “better,” can reduce opposition to public reports.  Providers should be given the opportunity to review and correct data prior to its release.[1] Providers should be involved in how measures are displayed and, in particular, setting the cut points when providers are sorted into performance categories as part of a larger effort to help consumers understand and use the data. Finally, providers should have the ability to submit new data and change a rating as improvements are accomplished. To maintain the trust and credibility of public reports, this process should be transparent to both providers and consumers. 


[1] Center for Health Care Strategies (CHCS). (2010), Opportunities for Medicaid Inclusion in Performance Measurement and Public Reporting.