Skip to main content

Measure Testing

Develop the Testing Work Plan

Testing work plans vary depending on measure type and complexity. Measure developers can test a single measure or a set of measures. If testing targets a set of measures, the measure developer should construct a work plan that describes the full measure set. The work plan for alpha testing is usually prepared early in the measure development process; therefore, the exact number of measures for testing may not be known, and many work plan areas listed may not be appropriate. In contrast, the measure developer should prepare the beta test work plan after development of the measure specifications and should include sufficient information to help understand how sampling and planned analyses aim to meet scientific acceptability, feasibility, and usability and use criteria required for endorsement by the CMS consensus-based entity. 

To assist with recruiting testing sites, the measure developer should announce testing opportunities widely. A Call for Measure Testing Volunteers template is available. CMS measure developers may request posting of the announcement on the MMS Hub. Additionally, the Electronic Clinical Quality Improvement Resource Center Measure Collaboration Workspace has an electronic clinical quality measure (eCQM) Testing Opportunities module where measure developers can announce eCQM testing opportunities.

See the Templates and Resources for assistance with developing and documenting the testing work plan and updating measure specifications. 

    • Name(s) of measure(s)
    • Type of testing 
    • Study objective(s)
    • Timeline for testing and report completion
    • Data collection methodology
    • Description of test population, including number and distribution of test sites/data sets, when available
    • Description of data elements for collection
    • Sampling methods, if applicable
    • Provide a description of the strategy to recruit measured entities and/or obtain test data sets
    • Analysis methods planned and description of test statistics to support assessment. This will be less extensive for an alpha test. 
       
    • For a beta test, methods and analysis should address these evaluation criteria
      • Importance —including analysis of opportunities for improvement such as reducing variability in comparison groups or disparities in health care related to race, ethnicity, age, or other classifications
      • Scientific acceptability —including analysis of reliability, validity, and denominator exclusion and/or numerator exclusion appropriateness
      • Feasibility —including evaluation of reported costs or perceived burden, frequency of missing data, and description of data availability
      • Usability and use —including planned analyses to demonstrate the measure is meaningful and useful to the target audience. The technical expert panel may accomplish this by reviewing the measure results (e.g., means and detectable differences, dispersion of comparison groups)
         
    • Description and forms documenting patient confidentiality and description of Institutional Review Board compliance approval or steps to obtain data use agreements (if necessary)
       
Last Updated: Mar 2023