All measures may not need formal usability testing. For example, in some cases, the technical expert panel may review measure characteristics (e.g., descriptive statistics, dispersion of comparison groups) to determine usability of the measure for performance improvement and decision-making and whether negative unintended consequences are likely. Test use after implementation of the measure. See the discussion in Measure Use, Continuing Evaluation, and Monitoring.

Formal Testing Methods

When there is a requirement for more formal testing to assess the understandability and decision-making utility of the measure with respect to intended audiences (e.g., consumers, purchasers, measured entities, policy makers), a variety of methods are available:

• Focus groups
• Structured interviews
• Surveys of potential users

These different interested party engagement methods often focus on the discriminatory ability of the measure and the meaning of the score as applied to evaluation of comparison groups or decision-making. Learn more about interested party engagement. 

Example

• The measure developer should use a survey of potential users to rate the clinical meaningfulness of the performance differences detectable by the measure or to assess the congruence of decisions based on measure summary data from a sample. Measure developers should consider developing a plan for implementation.