This content provides information about patient-reported outcome measures (PROMs). These measures have special considerations outside the more common structure, process, and outcome measures. This information supplements the information found in the Blueprint content on the Measure Specification, Measure Testing, and Measure Use, Continuing Evaluation & Maintenance pages.

Patient-Reported Outcome Measures (PROMs)

PROMs are tools used to capture patients’ reports of their outcomes, which measure developers can use as the basis for patient-reported outcome-based performance measures (PRO-PMs). PRO-PMs are a high priority for CMS and other organizations. PROMs present some design challenges. In this document, we describe some of these challenges and approaches to address them.

Ensuring patients and families are engaged as partners in their care—a CMS goal—can also be an effective way to measure the quality of patient care. Although patient reports of their health and experience with care are not the only items that should undergo measurement, they are an important component. Historically CMS used surveys to collect patient-reported data, but the continued development of the infrastructure allows more timely collection and use of alternative collection methods (e.g., using mobile devices) of these data. Currently, academic settings develop and test PROMS for use in clinical care. Measure developers must do additional testing to use a PROM as a basis for a quality measure. The figure depicts the relationship between patient-reported outcomes (PROs), PROMs, and PRO-PMs.

Image

Figure 1. Relationship between PROs, PROMs, and PRO-PMs

Patient-Reported Outcomes (PROs)

CMS defines a PRO as any report of the status of a patient’s health condition or health behavior coming directly from the patient, without interpretation of the patient’s response by a clinician or anyone else. Self-reported patient data provide a rich data source for outcomes. This definition reflects the key areas:

• health-related quality of life (including functional status)
• symptoms and symptom burden (e.g., pain, fatigue)
• health behaviors (e.g., smoking, diet, exercise)

Patient-Reported Outcome Measures (PROMs)

PROMs are tools used to collect patient-reported outcomes. Some examples of patient self-reported data collection tools include

• Patient-Reported Outcomes Measurement Information System (PROMIS®) — Funded by the National Institutes of Health (NIH), these tools measure patient self-reported health status.
• Medicare Health Outcomes Survey (HOS) — The HOS was the first outcome measure tool used in Medicare Advantage plans. The goals of the Medicare HOS program are to gather valid and reliable health status data in Medicare managed care for use in quality improvement activities, plan accountability, public reporting, and health improvement. All managed care plans with Medicare Advantage contracts must participate.
• FOTO Patient Outcomes —This tool measures the functional status of patients who received outpatient rehabilitation through the use of self-reported health status questionnaires. The FOTO tool assesses change in functional status by comparing measurements taken at intake, during, and at discharge from rehabilitation.

However, the outcomes collected by the tools are insufficient individually for measuring performance and accountability programs cannot use them directly. Measure developers should construct quality measures that apply the outcome data collected by the tools to measure the quality of care.

Patient-Reported Outcome-Based Performance Measures (PRO-PMs)

A PRO-PM is a way to aggregate the information from patients into a reliable, valid measure of performance at the measured entity level, e.g., clinician. The same measure evaluation criteria and justification principles that apply to other outcome measures also apply to PRO-PMs.

Several PRO-PMs are available. Examples include

• Back Pain After Lumbar Discectomy/Laminotomy (CMIT Measure ID 85)
• Functional Status Assessment for Total Knee Replacement (CMIT Measure ID 279)

Approaches to Developing Patient-Reported Outcome-Based Performance Measures

Although PROs are a special type of outcome measure, the principles for development are the same. The Risk Adjustment and Risk Stratification page details the procedure for risk- adjusting outcome measures.

Choose and Define a Patient-Reported Outcome

Throughout their health care journey, patients provide many kinds of data to their clinicians. Sometimes a patient may share information with their clinician, which their clinician then interprets in the patient record. PROMs collect information directly from patients, without clinician interpretation. To choose patient-reported outcomes that will become quality measures, measure developers must first identify quality issues for a target/initial population. An appropriate outcome has clinical or policy relevance. For example, whether the patient did or did not develop a surgical site infection after cataract surgery would not be a good PRO. A patient could report redness, swelling, and drainage, but not actually whether they have an infection. A better outcome measure in this instance might be a clinically meaningful measure of improvement in vision.

Outcome quality measures must also be meaningful to the target population and usable by measured entities. Whenever possible, measure developers should consult clinical experts, including patients and patient advocates, to help them define appropriate and meaningful outcomes.

Respecify or Create a De Novo PROM

As with other types of measures, measure developers can respecify existing measures or PROMs or create a de novo PROM. There are some advantages to creating a de novo PROM, especially if its development is principally for the intended PRO-PM. Whereas when measure developers are selecting from existing PROMs, they must find the best fit among PROMs created for other purposes. Additionally, developing digital PROMs may require de novo development since some existing PROMs and their owners may not allow for or their data elements are not conducive to mapping to interoperable data standards. There are many challenges inherent in developing PROMs, including time, resource, and cost constraints, and methodological and logistical challenges, which the measure developer must acknowledge and address.

There are resources available to help measure developers respecify or create de novo PROMs. For example, the PROMIS website provides existing tools and numerous resources for PROM developers, such as standards for instrument development and validation. The journal Psychometrika routinely includes information relevant to PROM developers across fields. Psychometrika published a relevant special issue in September 2021, Advancing Methods to Assess Patient-Reported Outcomes: Lessons Learned from the Patient-Reported Outcomes Measurement Information System (PROMIS) Initiative. However, guidance tailored for health care quality measure developers to develop PROMs to support the development of PRO-PMs in health initiatives is sparse.

Other peer-reviewed journals, such as the Journal of Patient-Reported Outcomes, provide articles dedicated to these topics to help measure developers stay up to date with methods to assess PROs. CMS funded a technical expert panel (TEP), Building a Roadmap From Patient-Reported Outcome Measures to Patient-Reported Outcome Performance Measures (Building the Roadmap). The TEP produced several reports, the most recent a technical guidance report. The purpose of the TEP was to identify the attributes of high-quality PROMs and provide guidance on how to select PROMs to develop PRO-PMs. Measure developers of PROMs should consider assembling a PROM TEP or consulting experts in item set creation, psychometrics, statistics, and other specialty areas.

The Measure Lifecycle and PROMs

When developing a PROM, the Measure Lifecycle still applies. The usual starting point is information gathering with an environmental scan and literature review to identify whether there are existing tools to collect the outcome in the target population. Measure developers can use the Environmental Scan Support Tool (ESST) and the De Novo Measure Scan (DNMS) to assist with environmental scans. You must have a free CMIT account to access the DNMS. The PROMIS site has publications for PROMIS measures back to 2004. See the Environmental Scan for Quality Measurement supplemental material for more information on conducting environmental scans.

Measure developers may consider using tools with established psychometric properties (e.g., adequate data element and tool reliability and validity). While the tools are not themselves necessarily PROMs, with further testing in the health care environment, measure developers may use the information from these tools to develop and test the construct of a PROM.

PROMs use the same basic building block for specifications, e.g., title, target/initial population, description, numerator, denominator, exclusions. Questions to answer:

How to best collect the data? One method or multiple methods? Using multiple methods adds to testing complexity.

• What is the content of the PROM?
• What is the phrasing of the items? For example, consider the PROM population and reading level.
• What is the order and layout of the items? For example, should the measure developer include skip logic?

Measure developers should follow the scientific principles of questionnaire and survey development. There is a lot of published literature on these principles. PROM development is iterative with testing. Measure developers should test early and often. It is important to test these tools with the population and setting on which the PROM focuses. If respecifying an existing tool, there may be differences between the reliability and validity of a PROM in more controlled settings (e.g., clinical trials, academic research projects) compared with use in real-world practice settings, but to date, testing of most PROMs has only been in controlled settings. The test plan should assess the sources of variation that can affect validity, reliability, and usability of the PROM. If creating a PROM for electronic collection, consider the interoperability and test like other digital measures using multiple test sites and vendor products. When assessing feasibility and interoperability, the measure developer should review the concepts in the Logical Observation Identifiers Names and Codes (LOINC) terminology. LOINC has numerous health care screening, evaluation, and survey instrument items and the acceptable answers. If there are no existing pertinent concepts, The measure developer may want to request LOINC add new concepts.

Attributes of High-Quality PROMs

Measure developers must be aware that many of the desirable attributes for a PROM are subjective, making them difficult to quantify. Additionally, gold standards for what constitutes a “high-quality” PROM are lacking. When developing a new PROM, the measure developer will also face the challenge of the PROM not being in use at the outset to test the measure attributes, including gathering feedback.

This underscores the need for gathering feedback continuously from multiple interested parties throughout the PROM development process. 

Additional Considerations for PROM developers

A major consideration for measure developers developing PROMs is to ensure the ability to attribute the items to patient-reported outcomes accurately and fairly to the measured entity. Measure developers have a unique charge and perspective in survey and item development, as the design or most surveys and items is not to track provider performance. PROM developers must keep the intended use (e.g., attribution to care provision) in direct focus throughout the development and testing for eventual implementation.

Measure developers must test the PROM on a representative sample of the target population, including all sub-groups to ensure no bias, for instance, across social, geographic, and economic groups. If the PROM is multimodal (e.g., phone, mail, and online) the measure developer will have to test the effects of the various modes. If translated into additional languages, the measure developer will also need to test each translated version of the PROM. The measure developer needs to consider whether to allow proxy responses in the PROM and how well they estimate the target population’s responses.

PROM developers will need to conduct cognitive testing to ensure items are clear, relevant to the concept, and valid. PROM and PRO-PM developers must assess non-response to the instrument and specific items. Measure developers must assess the items and the whole instrument using Item Response Theory (IRT) and PROM-relevant methods, such as structural equation modeling and latent variable models. Measure developers should be aware different methods can lead to different interpretations of the tool’s performance. They will also need to evaluate any scales and determine protocols for implementing the tool.

Determine the Appropriate Performance Measure: the PRO-PM

The measure developer should report the outcomes for target/initial populations as average change or percentage improvement determined by the topic of interest. The measure developer must test all measures for reliability, usability and use, feasibility, validity, and threats to validity, including how to handle missing data and appropriate risk adjustments. To appropriately distinguish variations in performance between measured entities, the outcome must capture the results of the care given and not the influence of comorbidities or other extraneous variables. However, as in any other outcome measurement, the measure developer should not allow risk adjustment to mask gaps of care. The Risk Adjustment and Risk Stratification page contains a discussion on determining the need for risk adjustment and development, and evaluation of risk adjustment models.