Measure Specification

A variety of interested parties need to be involved in the measure specification stage. In addition to the measure developer and TEP, persons, families, caregivers, and clinicians need to be involved to address the feasibility of data collection before a measure progresses too far in development. The measure developer can use focus groups, interviews, and other informal methods to engage other interested parties. The measure developer often posts a request for public comment with the measure specifications. 

Measure developers may use the Measures Management System (MMS) website, CMS MMS Hub, to announce the request for public comment. If the proposed measure is an eCQM, the measure developer may use the Office of the National Coordinator’s ONC Project Tracking System (Jira) to post the specifications and collect public comments. The eCQI Resource Center also posts requests for comments for eCQMs and points to where the user may find the specifications and provide comments.

Measure Testing

It is often appropriate for the measure developer to obtain interested party input at several points during the testing process. This process may include obtaining face validity assessments during alpha and beta testing, feasibility and burden inputs during beta testing, and other inputs based on a review of overall results. These inputs can take many forms, including formal TEPs, consultation with subject matter experts, outreach to professional associations or patient advocacy groups, and public comments. Once collection of these inputs occurs, it is important to follow up on the communications by providing additional opportunities for interested parties to comment on the results of their inputs at future stages. Follow up increases the likelihood the measure developer operationalizes the inputs consistent with the interested parties’ needs. It also improves the likelihood that interested parties will remain engaged for ongoing support on current or future measures. If a TEP reviews testing results and updated specifications, the measure developer may post those summaries for further public comment. Measure developers may post the TEP summaries.

Interested parties can volunteer to test measures under development. Additionally, through the MC Workspace eCQM Testing Opportunities module, measure developers may announce opportunities to test eCQMs.

Measure Implementation

CMS may request public comments on measures at different times in the implementation stage, for example, measures proposed for adoption in CMS programs either through the formal federal rulemaking process or through an ad hoc public comment process for measures that are not subject to rulemaking. Many CMS programs must use the rulemaking process to formally propose and finalize measures. The public may comment on proposed new measures and measures proposed to continue in a CMS program. 

Measures proposed for some programs, those subject to Section 3014 of the Patient Protection and Affordable Care Act (P.L. 111-148), must first go through a pre-rulemaking process in which measure developers submit measures for CMS to consider for adoption in CMS programs. CMS reviews the submissions and develops a Measures Under Consideration (MUC) List. Interested parties convened by the CMS CBE then provide MUC List recommendations to the United States Department of Health and Human Services (HHS) Secretary on which measures to implement in the different CMS programs. 

The measure developer convenes meetings with interested parties regarding the implementation of considered measures. Resolution of interested parties questions about the measures occurs iteratively as the measure remains under consideration. The measure implementation process is completely transparent and open to the public for comments and questions.

Measure Use, Continuing Evaluation, and Maintenance 

The MMS Hub content describes the annual update, comprehensive reevaluation, and early maintenance review as distinct and separate activities; however, in practice, these activities, like the different Measure Lifecycle stages, sometimes overlap and can occur concurrently. All steps require solicitation of inputs via public comment, usually through the rulemaking process. The measure developer should publicly report results from, and progress on, each of these review processes. Interested parties are engaged for comment and, in some cases, participate in a formal panel review. For more information on measure maintenance reviews, see Measure Maintenance Reviews and the Measure Maintenance Reviews supplemental material.

Ideally, there is alignment between the measure maintenance schedule for measures used in federal programs and the CMS consensus-based entity (CBE) endorsement maintenance cycle, which also includes requirements for public review and comment. However, in practice, these schedules may not align completely. The CMS CBE’s website lists the CMS CBE-endorsed measures and has a mechanism enabled for comment.

The ONC Project Tracking System (Jira) provides the opportunity for public comments on eCQMs in use. The measure developer may choose to include the comments as part of the Change Review Process and contribute to changes in eCQMs during the annual update. For more information on the eCQM Change Review Process and Annual Update, refer to the eCQMs Specification, Testing, Standards, Tools, and Community supplemental material.