During measure maintenance, it is important that measure developers analyze measure performance trends, including feedback through help desks and trainings, to determine whether the measure undergoing reevaluation is still the best or most relevant measure, and whether the measure developer needs to address any unintended consequences.

After implementation of measures, the measure developer monitors measures’ performance, addresses ongoing feedback, and continuously scans the environment regarding the measures. For example, for electronic clinical quality measures, the ONC Project Tracking System (Jira) is one method for collecting and monitoring feedback on measure implementation.

In addition, two measure maintenance activities apply to every measure: annual update and a triennial comprehensive reevaluation. A third activity, the early maintenance review, occurs only if there are significant unforeseen problems with the measure, such as a major change in the measure’s scientific evidence base.

Outcomes

CMS uses one of five outcomes following maintenance review of CMS measures. However, some programs may have different terminology, outcomes, and criteria. 

Retain or Revise

Retain - Keep the measure active with its current specifications and minor changes. Revise - Update the measure’s current specifications to reflect new information.

Criteria to Retain or Revise 

• ​​​Measure is responsive to specific program goals and statutory requirements.
• Measure addresses an important condition/topic with a performance gap and has a strong scientific evidence base to demonstrate that the measure when implemented can lead to the desired outcomes and/or more affordable care.
• Measure addresses one or more of the CMS Quality Priorities.
• Measure selection promotes alignment with CMS program attributes and across Department of Health and Human Services (HHS) programs.
• Measure developers have fully developed the measure, including the completion of testing.
• Measure reporting is feasible.
• Measure results and performance should identify opportunities for improvement. CMS will not select topped out measures.
• Use of the measure in a program does not result in negative unintended consequences (e.g., inappropriate reduced lengths of stay, overuse or inappropriate use of care or treatment, limiting access to care).
• eCQMs must be fully developed and tested, entered in the Measure Authoring Development Integrated Environment (MADiE) and created in Health Quality Measure Format using the Quality Data Model (QDM) and Clinical Quality Language (CQL).  eCQMs must pass feasibility testing.
• eCQMs must undergo reliability and validity testing, including review of the logic and value sets by the CMS partners.
• Electronically specified whenever possible.

Retire

Cease to collect or report the measure indefinitely. This applies to measures not owned or maintained by any measure steward. If it is necessary to retire a measure from a set, other replacement measures may be available to complement the remaining measures in the set.

Criteria for Retirement 

• CMS owns the measure, but will no longer use or maintain the measure.
• No longer adds value commensurate with the cost of data collection and reporting.
• Performance or improvement on a measure does not result in better outcomes.
• Collection or public reporting of a measure leads to negative unintended consequences other than patient harm.
• Does not align with current clinical guidelines or practice.
• Measure performance is so high and unvarying that CMS can no longer make meaningful distinctions and improvements in performance.
• The availability of a better measure that is more
  • Broadly applicable (across settings, populations, or conditions)
  • Proximal in time to desired outcomes for the particular topic
  • Strongly associated with desired outcomes for the particular topic
  • Aligned with other CMS/HHS programs

Suspend

• Temporarily cease to report a measure. Data collection and submission may continue. 

Remove

A specific program set no longer includes a measure for one or more reasons. This does not imply that other payors/purchasers/programs should cease using the measure. If CMS is the measure steward and another CMS program continues to use the measure, CMS may continue to maintain the measure. If another entity is the steward, the other payors/purchasers/programs that may be using the measure are responsible for determining if the steward should continue to maintain the measure.

Criteria for Removal 

• CMS no longer uses the measure in a specific CMS program. If CMS owns the measure, CMS may continue to maintain it even after removal.
• If another entity owns the measure, other payers/purchasers/programs using the measure are responsible for determining if the owner is continuing to maintain the measure.
• Collection or public reporting of a measure leads to negative unintended consequences other than patient harm.
• Measured entities cannot report the measure as currently specified.
• A measure does not align with current clinical measure performance or is so high and unvarying that CMS can no longer make meaningful distinctions and improvements in performance.
• The availability of a better measure that is more
  • Broadly applicable (across settings, populations, or conditions)
  • Proximal in time to desired outcomes for the particular topic
  • Strongly associated with desired outcomes for the particular topic
  • Aligned with other CMS/HHS programs
• Many  CMS programs have proposed, updated, and finalized factors for removal in their respective rules. For example, the Inpatient Prospective Payment System finalized eight factors for measure removal at FY 2019 IPPS/ LTCH PPS final rule (83 FR 41540 through 41544).
  • Factor 1. Measure performance among hospitals is so high and unvarying that meaningful distinctions and improvements in performance can no longer be made (that is, ‘‘topped-out’’ measures): Statistically indistinguishable performance at the 75th and 90th percentiles; and truncated coefficient of variation ≤0.10.
  • Factor 2. A measure does not align with the current clinical guidelines or practice.
    Factor 3. The availability of a more broadly applicable measure (across settings, populations, or the availability of a measure that is more proximal in time to desired patient outcomes for the particular topic).
  • Factor 4. Performance or improvement on a measure does not result in better patient outcomes. 
  • Factor 5. The availability of a measure that is more strongly associated with desired patient outcomes for the particular topic.
  • Factor 6. Collection or public reporting of a measure leads to negative unintended consequences other than patient harm. 
  • Factor 7. It is not feasible to implement the measure specifications.
  • Factor 8. The costs associated with a measure outweigh the benefit of its continued use in the program.