Testing Respecified Measures
When respecifying a measure for use in a new domain (e.g., new setting or population) or using a different data source (e.g., electronic health record [EHR] data), the measure developer should construct the measure testing to detect important changes in the functionality or properties of the measure. As applicable, review changes in
- relative frequency of critical conditions used in the existing measure specifications when applied to a new setting/population, e.g., dramatic increase in the occurrence of exclusionary conditions
- importance of the existing measure in a new setting, e.g., an existing measure addressing a highly prevalent condition may not show the same prevalence in a new setting, or evidence large disparities or suboptimal care found using the existing measure’s setting/population may not exist in the new setting/population
- location of data or the likelihood data are missing, e.g., an existing outpatient measure using an claims data source for medications in the criteria specification, when applied to Medicare patients in an inpatient setting, the measure developer may need modify to use patient record abstraction because Medicare Part A claims do not contain medication information due to bundling
- frequency of codes observed in stratified groups when applying the measure to a new setting or subpopulation
- risk adjustment model or changes that make the previous risk adjustment model inappropriate in the new setting/population
Specific to respecified electronic clinical quality measures (eCQMs), initial feasibility analysis findings are typically more qualitative, and the measure developer must confirm them by more detailed quantitative analysis and testing. Initial feasibility analysis may uncover significant industry readiness, workflow, burden, or standards constraints requiring mitigation and/or rethinking the viability of moving forward with respecification. The measure developer can complete initial feasibility assessments using the eCQM Feasibility Scorecard. Use the Feasibility Scorecard to assess
- the availability of the data element in the EHR in a structured format
- the accuracy of the data element
- if the measure developer coded the data element using recommended standards
- the impact of capturing the data element in the workflow
The components of the Scorecard assess current state data element feasibility but will not inherently provide an assessment of future data element feasibility. For example, workflow or technology changes could make data elements feasible to support respecifying registry measures to eCQMs. Therefore, the measure developer should ensure that, in cases where a respecified data element is currently not feasible, they include qualitative information about the near-term potential and level of effort to improve feasibility.
Confirming collection and storage of required data elements by the EHR reduces the risk of rework and course correction during measure respecification, which may delay the completion of the eCQM. The decision tree may be helpful in determining which data elements are critical to the eCQM. Identifying the most appropriate data elements is critical to ensure the measure’s intent does not change. If the measure developer cannot substitute critical data elements or substitution will change the intent of the measure, then developing a de novo eCQM may be the only option.