- Expand reliability and validity testing beyond the sample used in current submission to ensure a more diverse population and explore regional diversity within testing.
- Explore new versus current users of opioids and benzodiazepines on admission, and if feasible, stratify the data by these populations.
Proportion of inpatient hospitalizations for patients 18 years of age and older prescribed, or continued on, two or more opioids or an opioid and benzodiazepine concurrently at discharge.
Measure Specs
- General Information(active tab)
- Numerator
- Denominator
- Exclusions
- Measure Calculation
- Measure Record
- Point of Contact
General Information
Not Applicable.
No additional tools are used for data collection for eCQMs. Hospitals collect EHR data using certified electronic health record technology (CEHRT). The human readable and XML artifacts of the Health Quality Measures Format (HQMF) of the measure are contained in the eCQM specifications. No additional tools are used for data collection for eCQMs.
Numerator
Inpatient hospitalizations where the patient is prescribed or continuing to take two or more opioids or an opioid and benzodiazepine at discharge.
The numerator consists of encounters of patients prescribed two or more opioids OR an opioid and benzodiazepine at discharge.
Presence of two or more opioids at discharge Value Sets:
- Medication, Active: Schedule II and Schedule III Opioids (OID: 2.16.840.1.113762.1.4.1111.165)
- Medication, Discharge: Schedule IV Opioids (OIDs: 2.16.840.1.113883.3.3157.1004.14, 2.16.840.1.113883.3.3157.1004.16, or 2.16.840.1.113883.3.3157.1004.18)
OR
Presence of an opioid and a benzodiazepine prescription at discharge Value Sets:
- Medication, Active: Schedule II and Schedule III Opioids (OID: 2.16.840.1.113762.1.4.1111.165)
- Medication, Discharge: Schedule IV Opioids (OIDs: 2.16.840.1.113883.3.3157.1004.14, 2.16.840.1.113883.3.3157.1004.16, or 2.16.840.1.113883.3.3157.1004.18)
- Medication, Active: Benzodiazepines (OID: 2.16.840.1.113762.1.4.1125.1)
To access the value sets for the measure, please visit the Value Set Authority Center (VSAC), sponsored by the National Library of Medicine, at https://vsac.nlm.nih.gov/.
Denominator
Initial population inpatient hospitalizations (inpatient stay less than or equal to 120 days) that end during the measurement period, where the patient is 18 years of age and older at the start of the encounter and prescribed one or more new or continuing opioid or benzodiazepines at discharge.
The denominator consists of Inpatient Encounters with an Opioid or Benzodiazepine at Discharge.
Inpatient Encounters:
Encounter, Performed: Encounter Inpatient (OID: 2.16.840.1.113883.3.666.5.307).
Patients with an opioid or a benzodiazepine active on admission and continued at discharge:
- Medication, Active: Schedule II, Schedule III (OID: 2.16.840.1.113762.1.4.1111.165)
- Medication, Active: Schedule IV Opioids (OID: 2.16.840.1.113883.3.3157.1004.14, 2.16.840.1.113883.3.3157.1004.16, or 2.16.840.1.113883.3.3157.1004.18)
- Medication, Active: Schedule IV Benzodiazepines (OID: 2.16.840.1.113762.1.4.1125.1)
To access the value sets for the measure, please visit the Value Set Authority Center, sponsored by the National Library of Medicine, at https://vsac.nlm.nih.gov/.
Exclusions
The following encounters are excluded from the denominator:
- Encounters for patients with an active diagnosis of cancer that begins prior to or during the encounter
- Encounters for patients who are ordered or are receiving palliative or hospice care (including comfort measures, terminal care, and dying care) during the encounter
- Encounters for patients who are discharged to another inpatient care facility
- Encounters for patients discharged against medical advice (AMA)
- Encounters for patients who expire during the encounter
- Encounters for patients with an active diagnosis of sickle cell disease or sickle cell disease with crisis that begins prior to or during the encounter
- Encounters for patients with an active diagnosis of Opioid Use Disorder or are receiving Medications for Opioid Use Disorder that begins prior to or during the encounter
Denominator exclusions are represented using the QDM datatype and following value sets:
- Encounters for patients who are ordered or are receiving palliative or hospice care (including comfort measures, terminal care, and dying care) during the encounter
- Intervention, Performed: Palliative care (OID: 2.16.840.1.113883.3.600.1.1579)
- Intervention, Order: Palliative care (OID: 2.16.840.1.113883.3.600.1.1579).
- Encounters for patients who are discharged to another inpatient care facility
- Discharge to acute care facility (OID: 2.16.840.1.113883.3.117.1.7.1.87)
- Encounters for patients discharged against medical advice (AMA)
- Discharge against medical advice (AMA) (OID: 2.16.840.1.113883.3.117.1.7.1.308)
- Encounters for patients who expire during the encounter
- Expiration during hospital stay (OID: 2.16.840.1.113883.3.117.1.7.1.309)
- Encounters for patients with an active diagnosis of sickle cell disease or sickle cell disease with crisis that begins prior to or during the encounter
- Diagnosis: Sickle Cell Disease: (OID: 2.16.840.1.113883.3.3157.1004.22)
- Diagnosis: Sickle Cell Disease with crisis: (OID: 2.16.840.1.113762.1.4.1029.355)
- Encounters for patients with an active diagnosis of Opioid Use Disorder or are receiving Medications for Opioid Use Disorder that begins prior to or during the encounter
- Diagnosis (OIDs: 2.16.840.1.113762.1.4.1146.1109, 2.16.840.1.113762.1.4.1146.1110)
- Receiving medications for Medically assisted Treatment (MAT) (OID: 2.16.840.1.113762.1.4.1046.54)
- Encounters for patients with an active diagnosis of cancer that begins prior to or during the encounter.
- Diagnosis: Cancer (OID: 2.16.840.1.113762.1.4.1111.161),
Measure Calculation
Please see "2023 CBE Flow" diagram attached.
Not Applicable. This measure is not stratified.
Not applicable; this measure does not use a sample.
Measure Record
Point of Contact
N/A
Perry Lazar
Centers for Medicaid & Medicare Services
7500 Security Boulevard
Baltimore, MD 21244
United States
United States
Importance
Evidence
In 2022, the CDC published a guideline, The CDC Clinical Practice Guideline for Prescribing Opioids for Pain, updating the previous version of its guideline published in 2016, on the effectiveness and risks of long-term opioid treatment of chronic pain with more recent publications. Based on CDC’s GRADE criteria, the overall quality of the clinical evidence base for the effectiveness and risks of long-term opioid therapy (five systematic reviews). A “dose-dependent association” between opioid use and risk for overdose events, including death, was found consistently across two studies in the clinical evidence review and several epidemiologic studies in the contextual evidence review. Co-prescription of opioids with benzodiazepines was also found to increase risk for potentially fatal overdose in three studies included in the contextual evidence review. The studies found evidence of concurrent benzodiazepine use in 31 to 61 percent of those deceased from overdose. This guideline was published in 2022 and references the most recent systematic reviews. We are not aware of additional systematic reviews that have emerged since it was completed.
The CDC Clinical Practice Guideline for Prescribing Opioids for Pain — United States, 2022
The guideline recommends that clinicians should:
- “[Use strategies minimizing] opioid use…for both opioid-naïve and opioid-tolerant patients with acute pain when possible. If patients receiving long-term opioid therapy require additional medication for acute pain, nonopioid medications should be used when possible.”
- “Use particular caution when prescribing opioid pain medication and benzodiazepines concurrently."
- “Review increased risks for respiratory depression when opioids are taken with benzodiazepines, other sedatives, alcohol, nonprescribed or illicit drugs (e.g., heroin), or other opioids (see Recommendations 8 and 11)”
- “Closely monitor patients who are unable to taper and who continue on high-dose or otherwise high-risk opioid regimens (e.g., opioids prescribed concurrently with benzodiazepines) and should work with patients to mitigate overdose risk (e.g., by providing overdose education and naloxone) (see Recommendation 8).”
- "Discuss information from the PDMP with the patient and confirm that the patient is aware of any additional prescriptions.”
- “Discuss safety concerns, including increased risk for respiratory depression and overdose, with patients found to be receiving overlapping prescription opioids from multiple clinicians who are not coordinating the patient’s care or patients who are receiving medications that increase risk when combined with opioids (e.g., benzodiazepines) (see Recommendation 11), and offer naloxone (see Recommendation 8).
- "Discuss safety concerns with other clinicians who are prescribing controlled substances for the patient. Ideally, clinicians should first discuss concerns with the patient and inform them that they plan to coordinate care with their other clinicians to improve the patient’s safety.” "
In addition to the 2022 CDC Clinical Practice Guideline for Prescribing Opioids for Pain, opioid prescribing guidelines issued by various state agencies and professional societies for various settings agree with the recommendation to avoid concurrently prescribing opioids (American Academy of Emergency Medicine and Washington Agency Medical Directors’ Group (WAMDG)), and opioids and benzodiazepines (WAMDG, American Society of Interventional Pain Physicians, and New York City Department Of Health and Mental Hygiene) whenever possible as the combination of these medications may potentiate opioid-induced respiratory depression.
Citation:
- Deborah Dowell, MD1; Kathleen R. Ragan, MSPH; Christopher M. Jones, PharmD, DrPH; Grant T. Baldwin, PhD; Roger Chou, MD CDC Guideline for Prescribing Opioids for Chronic Pain — United States, 2022, Recommendation 11 Published November 4, 2022
- Dowell D, Ragan KR, Jones CM, Baldwin GT, Chou R. CDC Clinical Practice Guideline for Prescribing Opioids for Pain — United States, 2022. MMWR Recomm Rep 2022;71(No. RR-3):1–95. DOI: http://dx.doi.org/10.15585/mmwr.rr7103a1.
Measure Impact
We interviewed 12 providers for their feedback on the measure. Some providers indicated that it will be useful to have data on sub-group prescribing practices to provide population-based evidence-based care to look for large discrepancies.
Performance Gap
| Overall | Minimum | Decile_1 | Decile_2 | Decile_3 | Decile_4 | Decile_5 | Decile_6 | Decile_7 | Decile_8 | Decile_9 | Decile_10 | Maximum | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Mean Performance Score | 17.02 | 14.00 | 15.00 | 15.06 | 15.70 | 15.78 | 16.24 | 16.67 | 16.99 | 17.28 | 20.59 | 23.92 | 23.92 |
| N of Entities | 11 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| N of Persons / Encounters / Episodes | 18084 | 4230 | 20 | 2371 | 446 | 545 | 1699 | 330 | 2290 | 324 | 5482 | 347 | 347 |
Equity
Equity
This question was optional on the measure submission form template therefore we did not conduct this testing.
Feasibility
Feasibility
This submission was drafted prior to this field being created. Not Applicable.
Not Applicable. This measure is already fully specified therefore the Feasibility assessment during this round of testing did not influence the final measure specification. As evidence of the measure’s feasibility, there were 4,368 hospitals nation-wide that report this measure and have available measure data on the Timely and Effective Care hospital-level file on Care Compare.
Proprietary Information
Value sets are housed in the Value Set Authority Center (VSAC), which is provided by the National Library of Medicine (NLM), in coordination with the Office of the National Coordinator for Health Information Technology and the Centers for Medicare & Medicaid Services.
Viewing or downloading value sets requires a free Unified Medical Language System® (UMLS) Metathesaurus License, due to usage restrictions on some of the codes included in the value sets. Individuals interested in accessing value set content can request a UMLS license at (https://uts.nlm.nih.gov/license.html).
There are no other fees or licensing requirements to use this measure, which is in the public domain.
Scientific Acceptability
Testing Data
One large health system, representing ten hospitals total and one single hospital, in two states (MN and WI) field tested the measure from 2021-2022. All eleven hospitals are located in urban areas and are not-for-profit teaching hospitals. The hospitals varied in EHR systems (Cerner and Epic). The test sample from each health system included at least 25,504 encounters.
There are no differences in the data used for different aspects of testing.
Table 1. Testing Site Characteristics
Site Site Type State Number of Hospitals EHR System
Site 1 Hospital System MN 10 Epic
Site 2 Hospital WI 1 Cerner
Table 4 shows the characteristics of the 18,084 patients in our 11-hospital sample.
Table 4. Measure performance stratified by sex, race, ethnicity, and payer (N= 18,084)
Category Count Percent
Sex - -
Female 13425 64.18%
Male 7493 35.82%
Unknown sex 1 0%
Race - -
American Indian or Alaska Native 265 1.27%
Asian 1001 4.79%
Black or African American 1669 7.98%
Native Hawaiian or Other Pacific Islander 36 0.17%
Other Race 12 0.06%
Unknown race 838 4.01%
White 17098 81.73%
Ethnicity - -
Hispanic or Latino 408 1.95%
Not Hispanic or Latino 16345 78.13%
Unknown ethnicity 4166 19.91%
Payer - -
Medicaid 3541 17.05%
Medicare 9473 45.6%
Other 400 1.93%
Private Insurance 7349 35.38%
Self-Pay or Uninsured 10 0.05%
Reliability
Signal-to-Noise ratio tests the precision of scores across providers. To assess signal-to-noise reliability of the hospital level measure score, we utilized the beta-binomial model as described by in “The Reliability of Provider Profiling” (Adams, 2009). The distribution of reliability estimates across all facilities was examined. A reliability estimate of 0.7 has been used to define good reliability and the threshold at which meaningful differences in performance can be detected.
Reference:
Adams, John L. “The Reliability of Provider Profiling: A Tutorial.” Santa Monica, CA: RAND Corporation, 2009
The mean reliability across 11 hospitals was 0.79 (95% CI: 0.41, 0.97) and median reliability of 0.82.
Table 5. Signal to Noise calculations across hospital
year Number of hospitals Mean min p5 p10 p25 Median p75 p90 p95 IQR Maximum SD
06/2021-06/2022 11 0.79 0.14 0.41 0.68 0.72 0.82 0.95 0.97 0.97 0.23 0.98 0.24
This result indicates that the hospital-level performance rate has strong reliability, meaning that differences in hospital performance reflect true differences in quality as opposed to measurement error or noise. A reliability of 0.7 or above reflects strong precision (or reliability) in performance scores.
| | Overall | Minimum | Decile_1 | Decile_2 | Decile_3 | Decile_4 | Decile_5 | Decile_6 | Decile_7 | Decile_8 | Decile_9 | Decile_10 | Maximum |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Reliability | 0.790.14 | 0.41 | 0.41 | 0.71 | 0.72 | 0.79 | 0.82 | 0.93 | 0.95 | 0.95 | 0.97 | 0.97 | 0.97 |
| Mean Performance Score | 11 | 1 | 2 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| N of Entities | 18084 | 20 | 367 | 324 | 330 | 446 | 545 | 1699 | 2290 | 2371 | 5482 | 4230 | 4230 |
Validity
Data Element Validity
Data element validity is established by comparing the data obtained directly from EHR with the same data obtained from a different source, such as patients’ charts. To assess data element validity, we randomly selected a 25 patient encounters from the full EHR extract in each test site. For selected cases, site personnel manually abstract data elements necessary for the measure calculation from each site’s EHR. We then compared the manually abstracted and electronically extracted data to assess data element validity via agreement between the gold-standard source (manual abstraction) and the EHR extract.
We then calculated the raw agreement (percentage agreement) and the chance-corrected agreement (Gwet’s AC1) between the two data sources. The interpretation of the AC1 statistic is the same as that of Cohen’s Kappa, but AC1 is a more robust measure of interrater reliability. Kappa is sensitive to classification probabilities which in some cases lead to the low chance-corrected agreement despite the high observed agreement (the so-called Kappa paradox). This situation does not occur when using AC1. Higher values for agreement statistics demonstrate that the structured EHR data used to calculate the measure have accuracy similar to looking at the medical record overall, including clinical notes, documents, and other fields that convey information about the patient but cannot be used to calculate eCQMs. When the two measurements agree perfectly, the value of the agreement will be 1.0.
Empiric Validity
We examined validity of the measure using a hypothesis-driven validity testing approach. Under this approach, a measure is considered valid if the measure score discriminates between subgroups of patients expected to have differences in the measure rates based on findings from the literature. We evaluated differences in mean measure scores among predefined groups of patients based on the evidence. For example, consistent with the literature, Medicare patients were expected to have higher rates of concurrent opioid prescribing than non-Medicare patients (Li, 2018).
To test for the differences in the measure rates by patient subgroups, t-tests were used to compare mean group differences. We also computed Cohen's d effect size to account for small differences that are statistically significant may not always be practically or clinically meaningful. Effect size values for dichotomous variables were defined as small (0.2), medium (0.5), or large (0.8) (Cohen, 1988). For the Payer variable, analysis of variances (ANOVA) was used to test the overall differences in the measure rates between groups. We then computed Eta-squared to determine effect size for the overall difference between groups. Effect size values were categorized as small (0.01), medium (0.06), or large (0.14).
References:
Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers.
Li, J., Bell, T., Chollet, D. (2018). Patterns of Opioid Prescribing in Minnesota: 2012 and 2015. Minnesota Department of Health, Mathematica Policy Research.
Exclusions Analysis
To examine the effect of these exclusions, the number of patients with each exclusion was examined and we then computed measure rates with and without each exclusion.
Missing Data
Overall, missing data are not a threat to validity for the measure. For example, if data are missing from medication fields (for example, medication name), we interpret this to mean that the patient was not prescribed any medication at discharge, not that the patient was prescribed medication at discharge and this information is missing. Encounter type and discharge date are required for the measure calculation to assess medications that are active at discharge in the inpatient setting and ED/observation. Date of birth is also required, as it applies for patients ages 18 years and older. Rates of missing data on these items were negligible. We did not assess the frequency of missing data because we did not find any significant issues in the extracted or abstracted data.
Data Element Validity
We measured percent agreement, defined as the number of patients for whom both data sources, electronically and manually abstracted EHR data, agree on the presence or absence of a condition for a sample of 50 randomly selected patients charts. We also used Gwet's AC1 statistic to adjust percent agreement to account for chance agreement. The Kappa score can range from -1.00 to 1.00. We found high levels of percent agreement between the electronically and manually abstracted data for the denominator, denominator exclusions, and numerator.
The Kappa values calculated through data element validity testing suggest high levels of agreement between the data extract generated from the EHR systems and the manually abstracted data. The overall sample showed 88 percent agreement or higher for all data elements. In addition, agreement was almost perfect for two of the exclusionary data elements (palliative care and cancer).
Table 6. Agreement statistics for random sample data between EHR extraction and manual chart abstraction (n = 50)
Data Element Measure Part N Percent Agreement Gwet AC1
Opioid(s) at discharge Numerator, denominator 50 88 0.86
Benzodiazepine(s) at discharge Numerator, denominator 50 100 1
Discharge Disposition N/A 50 100 1
Admission date/time N/A 50 96 0.96
Discharge date/time N/A 50 96 0.96
Principal diagnosis N/A 50 100 1
Cancer Diagnosis Denominator exclusion 50 88 0.86
Palliative Hos. Denominator exclusion 50 90 0.89
Sickle Diagnosis Denominator exclusion 50 100 1
Sickle Crisis Diagnosis Denominator exclusion 50 100 1
OUD Diagnosis (ICD) Denominator exclusion 50 96 0.96
Prim. Payer N/A 50 90 0.89
Age N/A 50 98 0.98
Sex N/A 50 100 1
Race N/A 50 100 1
Ethnicity N/A 50 100 1
Empiric Validity
As noted above, we hypothesized that Medicare patients would have higher rates of concurrent opioid prescribing than non-Medicare patients. We observed meaningful differences in the measure rates across all sex, race, ethnicity, payer, and age. The Cohen’s D value for sex, race, ethnicity, and age indicated a small effect while the Cohen’s D for payer indicated a large effect. Female patients had lower performance rates than male patients, White patients had poorer performance rates compared to patients of other races, and non-Hispanic patients had worse performance rates than Hispanic or Latino patients. Medicare and Medicaid patients had poorer performance rates compared to patients with other types of insurance. We examined performed a t-test to determine differences between groups. There was no significant difference between performance rates by patients’ sex, race, or ethnicity. Differences among Medicare vs. non-Medicare beneficiaries were statistically significant.
Consistent with the literature, we observed higher measure rates (lower performance) for Medicare patients. The differences in means by were close to the definition of medium effect (0.50). We also observed that age was statistically significant which aligns without payer findings given that the Medicare population is made up primarily of beneficiaries 65+.
Table 7. hypothesis-driven validity testing results
Category Value Group 1 Mean Group 2 Mean Mean difference t-statistic P value Cohens d
Sex Female vs. male 0.16 0.19 -0.03 -1.67 0.11 0.71
Race Black vs white 0.14 0.18 -0.04 -1.19 0.26 0.51
Ethnicity Hispanic/Latino vs. Not Hispanic/Latino 0.12 0.18 -0.06 -1.95 0.08 0.89
Insurance Type/Payer Medicare vs. non-Medicare 0.22 0.13 0.09 5.56 <0.01 2.37
Age 18-64 vs. 65 and over 0.14 0.21 -0.06 -4.03 <0.01 1.72
Exclusions Analysis
Performance rates vary little regardless of applying the denominator exclusions across sites. When comparing the as specified measure with the excluded populations remove to the measure when no patients are excluded the performance rate increases from 17.12 (measure as specified) to 23.36 percent. Exclusion rates range from 19.38 to 23.32 percent. This suggests that it is unlikely that the exclusions will put any specific test site at an advantage or disadvantage.
Table 8. Patients excluded from the Initial Population and Performance Rate Without Specific Denominator Exclusions
Exclusions Count of patients in Initial Population Percent of patients in Initial Population Performance Score
As currently specified (including all exclusion criteria) - - 17.12%
No Exclusions - - 23.36%
Cancer exclusion only 5797 22.73% 19.83%
Palliative care exclusion only 1702 6.67% 20.62%
Hospice exclusion only 925 3.63% 21.31%
Discharge to another inpatient facility exclusion only 283 1.11% 23.28%
Expire during stay exclusion only 149 0.58% 23.32%
Sickle Cell Anemia Exclusion only 174 0.68% 23.23%
Left against Medical Advice Only 74 0.29% 23.31%
OUD/MOUD exclusion only 308 1.21% 22.88%
Data Element Validity
Table 6. Agreement statistics for random sample data between EHR extraction and manual chart abstraction (n = 50)
Data Element Measure Part N Percent Agreement Gwet AC1
Opioid(s) at discharge Numerator, denominator 50 88 0.86
Benzodiazepine(s) at discharge Numerator, denominator 50 100 1
Discharge Disposition N/A 50 100 1
Admission date/time N/A 50 96 0.96
Discharge date/time N/A 50 96 0.96
Principal diagnosis N/A 50 100 1
Cancer Diagnosis Denominator exclusion 50 88 0.86
Palliative Hos. Denominator exclusion 50 90 0.89
Sickle Diagnosis Denominator exclusion 50 100 1
Sickle Crisis Diagnosis Denominator exclusion 50 100 1
OUD Diagnosis (ICD) Denominator exclusion 50 96 0.96
Prim. Payer N/A 50 90 0.89
Age N/A 50 98 0.98
Sex N/A 50 100 1
Race N/A 50 100 1
Ethnicity N/A 50 100 1
Empiric Validity
Table 7. hypothesis-driven validity testing results
Category Value Group 1 Mean Group 2 Mean Mean difference t-statistic P value Cohens d
Sex Female vs. male 0.16 0.19 -0.03 -1.67 0.11 0.71
Race Black vs white 0.14 0.18 -0.04 -1.19 0.26 0.51
Ethnicity Hispanic/Latino vs. Not Hispanic/Latino 0.12 0.18 -0.06 -1.95 0.08 0.89
Insurance Type/Payer Medicare vs. non-Medicare 0.22 0.13 0.09 5.56 <0.01 2.37
Age 18-64 vs. 65 and over 0.14 0.21 -0.06 -4.03 <0.01 1.72
Exclusions Analysis
Table 8. Patients excluded from the Initial Population and Performance Rate Without Specific Denominator Exclusions
Exclusions Count of patients in Initial Population Percent of patients in Initial Population Performance Score
As currently specified (including all exclusion criteria) - - 17.12%
No Exclusions - - 23.36%
Cancer exclusion only 5797 22.73% 19.83%
Palliative care exclusion only 1702 6.67% 20.62%
Hospice exclusion only 925 3.63% 21.31%
Discharge to another inpatient facility exclusion only 283 1.11% 23.28%
Expire during stay exclusion only 149 0.58% 23.32%
Sickle Cell Anemia Exclusion only 174 0.68% 23.23%
Left against Medical Advice Only 74 0.29% 23.31%
OUD/MOUD exclusion only 308 1.21% 22.88%
Risk Adjustment
Not applicable. This is not an outcome or resource use measure.
Use & Usability
Use
Acute care hospital facility level
Usability
Based on provider interviews, data from this measure can be used to establish a baseline of prescribing practices. To improve scores, providers can monitor specific patients as well as encourage alerts for concurrent naloxone prescribing. Outreach to the community to understand these prescribing practices could take place at the hospital level.
CMS regularly receives feedback and questions from hospital abstractors about specifications and data collection through Jira, from educational webinars, testing sites, and interviews with abstractors. The measure developer and CMS take this feedback into consideration during the manual revision cycles where the team reviews the specifications to identify ways to clarify and simplify abstraction guidance and decrease data collection and clinical documentation burden.
In 2022, this measure was available for voluntary reporting and 2023 is the first year of required reporting. Since measure was only required for one year, user feedback is not available. Questions received have been focused on specification clarifications.
The measure development team monitored emerging literature on the measure concept, conducted harmonization reviews of other opioid measures to minimize reporting burden, and solicited input from stakeholders. Based on our findings, the measure now includes: (1) excluding patients who leave the hospital Against Medical Advice (AMA), (2) excluding patients with Sickle Cell Disease (SCD) from the denominator, and (3) including Schedule IV opioids within the scope of the measure.
Excluding patients who leave against medical advice (AMA). Stakeholders raised the idea that clinicians should not be held responsible for reconciling medications for patients who leave the hospital AMA.
Excluding patients with Sickle Cell Disease (SCD). The primary benefit of excluding patients with SCD is to more closely align with the Centers for Disease Control and Prevention’s recommendations in its 2022 Guideline for Prescribing Opioids for Chronic Pain.1 Excluding these patients also improves harmonization with three claims-based, health-plan-level measures stewarded by the Pharmacy Quality Alliance: Use of Opioids at High Dosage in Persons Without Cancer, Use of Opioids from Multiple Providers in Persons Without Cancer, and Use of Opioids from Multiple Providers and at High Dosage in Persons Without Cancer. During the change review process of the 2022 Annual Update, stakeholders said excluding patients with SCD would not increase the reporting burden.
Including Schedule IV opioids in the measure medications. Including Schedule IV opioids in the measure would better harmonizes the Safe Use of Opioids measure with other opioid measures, including the VA Opioid Safety Initiative and the three Pharmacy Quality Alliance opioid measures.
- Centers for Disease Control and Prevention. “CDC Advises Against Misapplication of the Guideline for Prescribing Opioids for Chronic Pain.” April 2019. Available at http://cdc.gov/media/releases/2019/s0424-advises-misapplication-guideline-prescribing-opioids.html. Accessed October 11, 2019.
This measure was only required for one year therefore data regarding trends is not available.
None were reported. We have not found evidence in the published literature that clearly demonstrates unintended consequences from implementation of the measure and will continue to monitor the published literature.
Comments
Staff Preliminary Assessment
CBE# 3316e Staff Assessment
Importance
Strengths:
- Submission relies upon a CDC guideline finding a “dose-dependent association” between opioid use and risk for overdose events, including death, was found consistently across two studies in the clinical evidence review and several epidemiologic studies in the contextual evidence review.
- Co-prescription of opioids with benzodiazepines was also found to increase risk for potentially fatal overdose in three studies included in the contextual evidence review. The studies found evidence of concurrent benzodiazepine use in 31 to 61 percent of those deceased from overdose.
- The developer examined data across 11 hospitals based on the measure specifications. Individual hospital performance rates ranged from 14% to 23.92% with mean performance of 17%, indicating room for improvement.
Limitations:
- No empirical demonstration of an association between the measure focus and a material outcome.
- Submission does not mention potential adverse effects (e.g., insufficient pain management).
- No information on whether the target population finds the measure meaningful. However, evidence provided depict the safety concerns to patients due to risk of overdose.
Rationale:
Although there is no empirical demonstration of importance, further studies are highly unlikely to have a significant impact on domain rating.
Feasibility Acceptance
Strengths:
- The measure is an eCQM, previously endorsed. The developer reports there were 4,368 hospitals nation-wide that report this measure and have available measure data on the Timely and Effective Care hospital-level file on Care Compare.
- The feasibility scorecard of data elements shows all but "Diagnosis: Opioid Use Disorder", "Discharge disposition codes", and "Encounter, Performed: Medication Assisted Treatment (MAT)" are available, accurate, and captured during care workflow.
Limitation:
- The developer reports challenges with "Diagnosis: Opioid Use Disorder", "Discharge disposition codes", and "Encounter, Performed: Medication Assisted Treatment (MAT)". To address this, the developer claims that for "Diagnosis: Opioid Use Disorder, hospitals are implementing this value set and doing education as well as automating systems (flag/alert for clinicians)." For "Discharge disposition codes, facilities can map necessary codes." However, it does not provide a feasibility plan for "Encounter, Performed: Medication Assisted Treatment (MAT)."
Rationale:
Fully specified eCQM. However, the committee may want to have the developer discuss the feasibility, accuracy, and workflow of the data elements, "Diagnosis: Opioid Use Disorder", "Discharge disposition codes", and "Encounter, Performed: Medication Assisted Treatment (MAT)."
Scientific Acceptability
Strengths:
- Measure is well defined and specified.
- Accountable entity-level reliability was assessed with signal-to-noise analysis performed on field test data collected in 2021-2022 for 18,084 persons across 11 hospitals. The median reliability is 0.82. 8 or 9 of the 11 hospitals (73-82%) have a reliability >0.6.
Limitations:
- Only 11 entities were used in the reliability calculations. Two to three of the 11 entities (18-27%) have a reliability less than the threshold of 0.6. The developer may consider expanding the number of entities for testing.
Rationale:
Two-three of the 11 entities (18-27%) have a reliability less than the threshold of 0.6. The developer may consider expanding the number of entities used for testing, and/or consider mitigation strategies for entities with low number of persons. Some possible mitigation strategies to improve these estimates could be to
- Empirical approaches outlined in the report, MAP 2019 Recommendations from the Rural Health Technical Expert Panel Final Report, https://www.qualityforum.org/WorkArea/linkit.aspx?LinkIdentifier=id&ItemID=89673
- Consider a higher minimum case volume.
- Extend the time frame.
- Focus on applying mitigation at the lower volume providers.
Strengths:
- The developer conducted data element validity testing, in which the overall sample showed 88% agreement or higher for all data elements.
- The developer also conducted accountable entity-level validity using a "known groups" approach to determine if the measure score discriminates between subgroups of patients expected to have differences in the measure rates based on findings from the literature. For this analysis, the developer hypothesized that Medicare patients would have higher rates of concurrent opioid prescribing than non-Medicare patients, which was confirmed from the analysis.
Limitations:
- For the data element validity: One large health system, representing ten hospitals total and one single hospital, in two states (MN and WI) field tested the measure from 2021-2022. All eleven hospitals are located in urban areas and are not-for-profit teaching hospitals. The hospitals varied in EHR systems (Cerner and Epic). The test sample from each health system included at least 25,504 encounters. The developer reports challenges with "Diagnosis: Opioid Use Disorder", "Discharge disposition codes", and "Encounter, Performed: Medication Assisted Treatment (MAT)". To address this, the developer claims that for "Diagnosis: Opioid Use Disorder, hospitals are implementing this value set and doing education as well as automating systems (flag/alert for clinicians)." For "Discharge disposition codes, facilities can map necessary codes." However, it does not provide a feasibility plan for "Encounter, Performed: Medication Assisted Treatment (MAT)."
- For the accountable entity-level validity: Developer does not include a rationale for the expectation of higher rates among Medicare patients.
Rationale:
- The developer reports an overall 88% agreement or higher on data elements. However, the committee may want to have the developer discuss the feasibility, accuracy, and workflow of the data elements, "Diagnosis: Opioid Use Disorder", "Discharge disposition codes", and "Encounter, Performed: Medication Assisted Treatment (MAT)."
- The developer does report accountable entity-level testing as a know groups analysis, but does not include a rationale for the expectation of higher rates among Medicare patients. The committee may consider asking the developer about this. In addition, the developer may want to consider additional empirical demonstrations of accountable entity reliability and accountable entity validity using current data for the measure, as specified. This will likely have a significant impact on this domain rating (specifically whether any association between the entity and the measure focus may be attributable to known and effective ways to decrease prescribing).
Equity
Strengths:
N/A
Limitations:
Developer did not address this optional criterion.
Rationale:
Developer did not address this optional criterion.
Use and Usability
Strengths:
- Measure is currently in use in the Hospital inpatient quality reporting (IQR) program, and pay-for-reporting program.
- Developer suggests that entities can improve performance by implementing alerts for concurrent naloxone prescribing and community outreach to better understand prescribing practices.
- Feedback is collected regularly via Jira, educational webinars, at testing sites, and in interviews with abstractors; questions received to date have sought clarification on specification.
- Developers indicate that measure specifications were changed in response to feedback from stakeholder groups and measure harmonization efforts, to exclude patients leaving AMA and patients with Sickle Cell Disease, and to include Schedule IV opioids.
- No unexpected findings were reported; developer indicates they do not expect any based on their review of the current literature.
Limitations:
- No performance trends are available yet (measure only started being required in 2023)
- Additional details on the suggested strategies would be welcomed, e.g., what these look like and where/how they have been implemented in the past.
Rationale:
- The measure is currently in use in the hospital IQR program. Developers recommend implementing alerts for concurrent prescribing and community outreach to understand prescribing practices (details not provided). Feedback on the measure is gathered through several avenues and no comments suggested respecification; however, developers have respecified the measure based on stakeholder feedback and harmonization efforts.
- No performance trends are available yet for this measure (required reporting began in 2023).
Summary
N/A
Committee Independent Review
Recommend for Endorsement
Importance
The measure is based on guidelines. Concurrent prescriptions of opioids or opioids and benzodiazepines is identified in literature to place patients at a greater risk of unintentional overdose, tying measurement to improved patient outcomes. Allow lower scores equals better performance, mean performance of 17% still indicates less than optimal performance.
Feasibility Acceptance
Data elements that are used to calculate measure performance in the eCQM measure logic are indicated from both sites as feasible. The developer could clarify how Medication Asssisted Treatment and Diagnosis of Opioid Use Disorder are used in the measure and why assessed with feasibility. As specified the measure is calculated with Inpatient Encounters where the Count of "Schedule II & III Opioid Medications" at discharge is greater than 2 or Inpatient Encounters with a Schedule II & III Opioid or "Schedule IV Benzodiazepines" at discharge. This eCQM is an episode-based measure and does not rely on a diagnosis. Discharge Disposition is used in the measure exclusion logic and includes SNOMED CT codes that the developer indicates can be used to map. Therefore, it seems as though the data elements used to calculate the eCQM are feasible. This Measure is also currently in use in the Hospital inpatient quality reporting (IQR) program and being reported.
Scientific Acceptability
Mean Reliability signal to noise >0.6. Although I agree with the staff assessment comments, that testing could be expanded the majority of entities tested met the reliability thresholds.
In the future for the optional equity category could address whether the measure is reliable for smaller health systems in rural locations versus one large health system and one independent hospital, both using large EHRs Cerner and Epic.
Data element validity sample size seems low - randomly selected a 25 patient encounters from the full EHR extract in each test site. The test sample from each health system included at least 25,504 encounters. Also curious if data elements flagged as not feasible were explored in data element testing.
Equity
Was not addressed in submitted materials but is also optional criteria that the developer may be able to speak to whether the measure is reliable and valid for smaller health systems in rural locations.
Use and Usability
Measure is currently in use in the Hospital inpatient quality reporting (IQR) program and being reported.
Summary
Overall recommend for endorsement with clarification from measure developer on feasibility assessment and scientific acceptability.
Not Recommended for Endorsement
Importance
The developer did not provide sufficient information in the importance section of the submission to meet the criteria.
- There is no evidence provided in the submission regarding why the measure is important. Everyone knows that there is an opioid epidemic in this country, but no evidence is provided regarding this issue. What is the business case for the measure/why is this measure important? Where’s the data showing that the opioid epidemic is an issue and that this measure will help to improve the outcome of overdose/death.
- The submission includes this detail “A dose-dependent association” between opioid use and risk for overdose events, including death, was found consistently across two studies in the clinical evidence review and several epidemiologic studies in the contextual evidence review. However, the measure isn’t capturing the dosage the patient is discharged on. The measure captures patients "prescribed, or continued on, two or more opioids or an opioid and benzodiazepine concurrently at discharge." There are no empirical studies with data provided that support the first portion of the numerator (i.e., prescribed, or continued on, two or more opioids at discharge).
- It is unclear if the evidence is reflective of the same measure population, as it is unclear from the measure specifications what facilities (e.g., acute care hospitals, long-term acute care hospitals, cancer hospitals, psych hospitals) this measure is captured under.
- There is a gap, but only 11 hospitals are analyzed. The specifications say that the measure is not based on a sample, so why is the gap data provided only for 11 hospitals and not all facilities captured in the measure? There is also no explanation of what types of hospitals are included in the analysis. Are they acute care, rural, cancer hospitals, psych hospitals?
- The information provided in the submission does not conclude that providers feel that this measure is meaningful. The submission states, “We interviewed 12 providers for their feedback on the measure. Some providers indicated that it will be useful to have data on sub-group prescribing practices to provide population-based evidence-based care to look for large discrepancies.” Also in light of the opioid epidemic do patients/families find the measure meaningful?
Feasibility Acceptance
- A brief description of the overall feasibility testing is needed.
- A feasibility plan for "Encounter, Performed: Medication Assisted Treatment (MAT)" is missing.
- Also the answer to the question regarding proprietary components of the measure is not answered in the submission.
Scientific Acceptability
- The average reliability across 11 hospitals was 0.79.
- There needs to be clarification in the specifications regarding what types of hospitals are included in the measure.
- The measure is not sampled, but the data provided on the measure is based on a sample of 11 hospitals. There are no details as to why only 11 hospitals were analyzed and not all the hospitals that capture the measure.
Data Element Testing
- The developer found Kappa values calculated through data element validity testing suggested high levels of agreement between the data extract generated from the EHR systems and the manually abstracted data. The overall sample showed 88 percent agreement or higher for all data elements for the denominator, denominator exclusions, and numerator.
The submission does not specify the level of analysis that was performed, only that EHRs were manually abstracted. I'm assuming the level of analysis for the data element testing is at the hospital level, but this is not stated.
Measure Score Testing
- The measure score testing provided in the submission is not applicable as the measure is not specified at the health plan level, it is specified at the hospital inpatient level.
Missing Data
- The data element 'Encounter, Performed: Medication Assisted Treatment (MAT)' was missing during feasibility testing for both testing sites. The developer does not address this in the missing data section of the submission.
Equity
NA, This is an optional criterion.
Use and Usability
- Initial endorsement was 2018, this is a maintenance review. There is no data on performance over time. However, the developer advises that the measure has only been in use for a year and trending data is not available.
Summary
Overall the measure should not be recommended for endorsement due to the submission not meeting the importance criteria.
Hospital role in deprescribing
Importance
The intent of the measure to improve the safety of this population is appropriate. There is limited data to show any trends due to the measure being reported in 2023 to show any improvements.
Feasibility Acceptance
Agree that there is limited workflow of the data elements, "Diagnosis: Opioid Use Disorder", "Discharge disposition codes", and "Encounter, Performed: Medication Assisted Treatment (MAT)."
Scientific Acceptability
I agree that there needs to be consideration for mitigation for lower volume providers.
The validity that Medicare patients have higher rates are concerning. Is this validated to ensure payors are acccurate or that this is not just a focused area?
Equity
Not addressed by measure creator.
Use and Usability
Currently being reported through the IQR program, there are no data yet regarding trends.
Summary
The measure is difficult for hospitals at time of discharge to de-prescribe. Patients who have been prescribed the opioids or benzos have often been on these medications for a long time. There is a safety concern with removing these medications at discharge. The goal would be to educate community providers (who sometimes are not associated with a hospital) to review prescribing practice. The question is, how do we hold these independent providers accountable.
Additional data and information would be helpful.
Importance
Evidence base exists to support the measure. The data from the first year of the measure indicates there is a gap with a mean performance of 17% across 11 hospitals. Additional data from a large sample of hospitals would be beneficial.
Provider feedback was not conclusive on support for the measure.
Patient population wasn't surveyed but would be helpful to consider.
Feasibility Acceptance
Workflow challenges collecting some of the data based on the feasibility plan scores. Could add more detail to the feasibility plan on how to address all of the challenges.
Scientific Acceptability
Measure reliability testing conducted at the hospital level. It meets the 0.7 threshold. The minimum reliability shown was for one of the hospitals was 0.14 and 0.41 was the median. Additional data from a broader range of hospitals might be helpful given n=11.
Validity testing conducted showing 88% agreement.
Medicare prescribing hypothesis aligns with one paper from the literature where the analysis focused on MN. Would be helpful to understand why Medicare rates would be higher, and also compare to CMS data on Medicare vs. Medicaid prescribing rates.
Equity
N/A - developer did not address this area.
Use and Usability
Currently in use in IQR.
The developer considered stakeholder feedback and made adjustments to the inclusion and exclusion criteria.
Given the mandatory reporting in 2023, no trend data is available.
Developer reported no unexpected findings.
Summary
-
I recommend endorsement of…
Importance
The submission is based on CDC guidelines finding a “dose-dependent association” between opioid use and risk for overdose events, including death combination of opioids with benzodiazepines have been found to increase the risk for potentially fatal overdose. However, it would be helpful to know if this measure was meaningful and if there was any material outcome from implementing this measure.
Feasibility Acceptance
The measure was previously endorsed and is an eCQM. 4,368 hospitals nationwide successfully reported this measure and have available measure data on Care Compare. The developer should clarify the feasibility and workflow of the data elements.
Scientific Acceptability
18-27% of entities have reliability less than the threshold of 0.6. Strategies to improve reliability could include considering a higher minimum case volume and extending the time frame.
The developer reports an overall 88% agreement or higher on data elements. However, the developer should discuss the feasibility, accuracy, and workflow of the data elements "Diagnosis: Opioid Use Disorder," "Discharge disposition codes," and "Encounter, Performed: Medication Assisted Treatment (MAT)."
Equity
The developer did not address this optional criterion.
Use and Usability
The measure is in use in the Hospital inpatient quality reporting (IQR) program and pay-for-reporting program. Developers recommend implementing alerts for concurrent prescribing and community outreach to understand prescribing practices. No performance trends are available yet for this measure.
Summary
I recommend endorsement of the measure with additional clarification from the developer, as indicated in the comments.
Endorse
Importance
Significant post inpatient morbidity/mortality associated with being on multiple opioids as well as an opioid and a benzodiazepine.
Feasibility Acceptance
EHR data is structured to capture the requisite value sets (both inclusion, numerator/denominator, and exclusion).
Scientific Acceptability
Reliability testing data provided indicates that hospital-level performance rate has strong reliability.
Validity data confirms impacts of appropriate exclusions with no findings indicating an impact on the broader validity of the rates at the hospital level.
Equity
Data were not provided on equity, but this should be addressable.
Use and Usability
Measure is in use and has been used effectively.
Summary
Continued endorsement is warranted for this measure. Recommend that the stewards consider formally evaluating equity as this domain is not met but addressable.
Endorsement Recommended
Importance
The description provides a rational on measure importance. The opioid epidemic is well documented, but it does not provide a great link to the measure’s intention. Also, there was data from only 11 hospitals (out of 4,368 that reported) and only 12 providers were interviewed. Increasing the hospitals, especially a geographic mix, would provide greater insights.
Feasibility Acceptance
The measure was designed using eCQM measure logic. The source data elements come from the Value Set Authority Center. Also, there are no proprietary components to the measure.
Scientific Acceptability
The mean reliability across 11 hospitals was 0.79 (95% CI: 0.41, 0.97) and median reliability of 0.82. Testing should be expanded to include additional hospitals.
The description states that the Kappa values suggest high levels of agreement between the EHR data extract and manually abstracted data. The overall sample showed 88%. However, the data element sample size of 25 seems low.
The findings on Medicare patients are interesting and could serve as opportunity for further investigation.
Equity
The equity was not addressed. One of the provider feedback was to look at subgroups which could be considered for future review.
Use and Usability
The data has only been in use for one year therefore there is no trend data available.
Summary
Overall, I would recommend the measure for endorsement with clarifications addressed by the measure developer on importance and scientific acceptability.
Important measure for patient safety
Importance
Safe opioid prescribing is important issue.
Feasibility Acceptance
The data provided demosntrate that this is a feasible measure
Scientific Acceptability
The data provided demonstrate that the measure may not be as reliable in hsoptials with small numbers of eligible patients Could more data be provided to show relaibility? Is there a minimum number of paitents per hospital where a hospital shoudl be excluded frm the measure?
The data provided show validity of the measure
Equity
No data is provided related to equity
Use and Usability
This is an electronic measure and has been implemented already
Summary
This is an important measure for patient safety. There some issues related to relaibility of the measure and this can be addressed with more data. It would be helpful to understand potential unintended consequences of the measure such as pain that is not well controlled.
Endorse
Importance
The importance of this measure is evident through the well documented nationwide opioid crisis. There is also extensive medical evidence reported by many reliable entities (CDC, CMS, AMA) affirms that patients who take opioids concurrently with central nervous system depressants such as benzodiazepines are at an increased risk of life threatening overdoses.
Feasibility Acceptance
Agree with staff reviewer that while this is a fully specified eCQM, developer could give additional information or data regarding feasibility, accuracy, and workflow of the data elements, "Diagnosis: Opioid Use Disorder", "Discharge disposition codes", and "Encounter, Performed: Medication Assisted Treatment (MAT)."
Scientific Acceptability
Limited entities used in reliability analysis.
Research analysis was performed at specific hospitals in specific regions of the country. This does not reliably reflect outcomes and practices on a nationwide level. There is a lack of additional reporting that would serve as supporting evidence of outcomes related to the 4368 hospitals with available performance data.
Equity
Healthcare Equity not addressed in available materials.
Use and Usability
Currently a CMS eCQM and part of the Hospital IQR Program. Trend data is not available per documentation.
Summary
This measure is important in terms of long-term patient safety and outcomes in light of the nationwide opioid epidemic. Additional data to include different areas of the country and different type of hospitals (for profit, non profit, teaching, etc) and their outcomes rather than just reporting that there were 4368 hospitals nation-wide with available measure data would strengthen the reliability and usability of this measure.
Recommend but with limitations/questions to be resolved
Importance
In review of the importance section of the evaluation it appears to be lacking the appropriate information around trending to understand if we are seeing an increase in concurrent prescriptions. Adverse impact on patients with pain that do not currently have a diagnosis that is excluded, which can lead providers to prescribe less often due to provider level analysis. In showing the research and applying it and trending the data over time will show that the impact of improvement processes can lend to positive or desirable outcome. This is now implemented, and it would be useful to understand the ideal state and what measures/tactics can be taken to improve the overall measurement outcome. Additionally, the measure title states inpatient, but the measure logic definitions, data dictionary it states ED patients and observation unit patients. It is not clear if patients from these phases in care should or should not be included. Including a hospital's ED patients and Acute Inpatient Patients may have an impact on the denominator. it would be helpful to understand why we are seeing higher rates amongst elderly 65 and older or Medicare Population.
Feasibility Acceptance
Feasibility Acceptance does appear to be missing some scorecards and results for the sites the measure was performed on. The measure stated feasibility was performed on 11 hospitals, but it appears to be combining 10 into site 1 and 1 into site 2 in feasibility score card. The type of EMR used in site 2 scorecard appears to be missing from the document. The feasibility plan is not using the correct data elements to write the plan as intended. This feasibility analysis is missing the results of sites combined. Site 2 is Cerner and has a lower volume and it is not clear if there is a need for more sites that use Cerner in this study. The emergency department, observation admissions appear to be in the data elements while this measure appears to need further specifications as it is not clear if these patient classes are included or excluded.
Scientific Acceptability
Scientific Acceptability Reliability Rating appears to need clarification around using the sites independently for testing. We see the overall results but not the Site 1 (10 hospitals on Epic) and Site 2 (1 hospital on Cerner) testing to understand if there are any anomalies amongst site groupers would be more thorough.
Scientific Acceptability Validity Rating appears to need clarification around using the sites independently for testing. We see the overall results but not the Site 1 (10 hospitals on Epic) and Site 2 (1 hospital on Cerner) testing to understand if there are any anomalies amongst site groupers would be more thorough.
Equity
This question was optional on the measure submission form template therefore we did not conduct this testing.
Use and Usability
Considerations for improve performance activities to include more of what is expected for hospitals and physicians to do to improve their performance without impacting the patient care.
Summary
It appears that we have some opportunities to understand the following: 1. Does this measure include ed and observation patients? 2. Feasibility analysis was not fully complete, including one hospital using Cerner with smallest patient volume as compared to the other 10 hospitals included into one site. 3. Actions for improvement - counter measure to improve. What exactly are hospitals and clinicians to do with this information? Alert clinicians more? 4. What is the ideal state that hospitals should aim for? 5. Why is there such a difference for patients on Medicare or >65 age? 6. Consider exclusions where education and naxlaone were given to patients for safety.
Conditional Reccomend
Importance
Unsure of how this measure will impact the intended outcome. There is no discussion of how to mitigate potential adverse effects of untreated pain.
Feasibility Acceptance
4,368 hospitals nation-wide report this measure and have available measure data on the Timely and Effective Care hospital-level file on Care Compare.
Scientific Acceptability
Only 11 hospitals were used in two states and all 11 were teaching hospitals in urban areas. Hospitals in rural and suburban areas need to be included as well as community hospitals.
Developer does not include a rationale for the expectation of higher rates among Medicare patients.
Equity
This was optional so it was not addressed
Use and Usability
Already in use and being reported
Summary
n/a
Conditional Recommend
Importance
The information underscores the significance of the CDC guideline in guiding opioid prescribing practices, and the PQM data highlights variations in performance across hospitals. The feedback from providers suggests that further granularity in the data, focusing on subgroup prescribing practices, could enhance evidence-based care and address potential discrepancies.
Feasibility Acceptance
Limitation: The developer has reported challenges with "Diagnosis: Opioid Use Disorder," "Discharge disposition codes," and lacks a feasibility plan for "Encounter, Performed: Medication Assisted Treatment (MAT)."
Recommendations:
- Diagnosis: Opioid Use Disorder:
- Implementing the suggested value set for Opioid Use Disorder is a positive step. To enhance effectiveness:
- Ensure comprehensive education for clinicians regarding the importance of accurate diagnosis and appropriate documentation.
- Continue automation efforts, but also consider refining alerts to provide actionable insights for clinicians.
- Implementing the suggested value set for Opioid Use Disorder is a positive step. To enhance effectiveness:
- Discharge Disposition Codes:
- Encourage facilities to map necessary codes for discharge disposition. This involves:
- Collaborate with coding teams to identify and integrate required codes into the discharge process.
- Provide training to relevant staff on accurate code mapping and documentation.
- Encourage facilities to map necessary codes for discharge disposition. This involves:
- Encounter, Performed: Medication Assisted Treatment (MAT):
- Develop a feasibility plan for the implementation of "Encounter, Performed: Medication Assisted Treatment (MAT)." Consider the following steps:
- Conduct a thorough assessment of current systems and workflows to identify potential integration points for MAT documentation.
- Engage with clinicians to understand their workflow and gather insights on how MAT encounters can be seamlessly documented.
- Evaluate existing technologies or develop new tools that align with clinicians' needs and facilitate efficient MAT documentation.
- Provide clear guidelines and training for clinicians on incorporating MAT encounters into their routine documentation practices.
- Pilot the implementation in a controlled environment to identify and address any challenges before widespread adoption.
- Develop a feasibility plan for the implementation of "Encounter, Performed: Medication Assisted Treatment (MAT)." Consider the following steps:
Addressing these recommendations will contribute to a more comprehensive and streamlined approach in handling "Diagnosis: Opioid Use Disorder," "Discharge disposition codes," and "Encounter, Performed: Medication Assisted Treatment (MAT)." This ensures accurate documentation, promotes adherence to guidelines, and facilitates the integration of MAT encounters into clinical workflows.
Scientific Acceptability
Strengths:
- The measure is well-defined and specified, providing clarity in its objectives.
- Accountable entity-level reliability analysis demonstrated a median reliability of 0.82, with the majority of hospitals (73-82%) exhibiting reliability values exceeding 0.6.
Limitations:
- Two to three entities (18-27%) fell below the reliability threshold of 0.6, indicating a need for improvement.
Recommendations for Acceptable Reliability:
- Expand Testing Entities or Mitigate Reliability Challenges:
- Consider expanding the number of entities for testing to address reliability concerns.
- Implement mitigation strategies as outlined in the MAP 2019 Recommendations, including exploring empirical approaches, considering a higher minimum case volume, extending the time frame, or focusing on mitigation for lower-volume providers.
Strengths:
- Data element validity testing showed 88% or higher agreement for all elements.
- Accountable entity-level validity successfully discriminated between subgroups, confirming the measure's ability to differentiate as expected.
Limitations:
- Challenges were noted with specific data elements such as "Diagnosis: Opioid Use Disorder," "Discharge disposition codes," and a lack of a feasibility plan for "Encounter, Performed: Medication Assisted Treatment (MAT)."
- The rationale for expecting higher rates among Medicare patients in accountable entity-level validity testing is not explicitly provided.
Recommendations for Acceptable Validity:
- Address Challenges with Specific Data Elements:
- Delve into the feasibility, accuracy, and workflow of challenging data elements, including "Diagnosis: Opioid Use Disorder," "Discharge disposition codes," and "Encounter, Performed: Medication Assisted Treatment (MAT)."
- Provide a detailed feasibility plan for "Encounter, Performed: Medication Assisted Treatment (MAT)."
- Clarify Rationale for Expected Higher Rates:
- Clearly articulate the rationale for expecting higher rates among Medicare patients in accountable entity-level validity testing.
- Conduct Additional Empirical Demonstrations:
- Perform additional empirical demonstrations of accountable entity reliability and validity using current data for the measure. This will enhance the measure's overall assessment and ensure robustness in validity.
Equity
Not addressed
Use and Usability
No additional comments.
Summary
The measure demonstrates notable strengths, characterized by a well-defined and specified structure, with accountable entity-level reliability showcasing a median reliability of 0.82. However, challenges are evident, particularly with two to three entities falling below the reliability threshold of 0.6, highlighting the imperative for improvement. Positive outcomes from data element validity testing are tempered by difficulties associated with specific data elements like "Diagnosis: Opioid Use Disorder," "Discharge disposition codes," and the absence of a feasibility plan for "Encounter, Performed: Medication Assisted Treatment (MAT)."
In addition, clarity is sought regarding the rationale for expecting higher rates among Medicare patients in accountable entity-level validity testing. The measure's success hinges not only on technical refinements but also on addressing feasibility concerns associated with challenging data elements.
Crucially, while equity considerations are optional, they significantly impact the measure's success. Prioritizing equity in the development process ensures the measure's applicability across diverse populations and mitigates potential disparities. Therefore, alongside technical enhancements and feasibility considerations, integrating equity considerations is imperative to create a measure aligned with best practices and promoting fairness in healthcare delivery.
Opioid Safety, Concurrent Prescribing
Importance
A variety of studies show a relationship between prescription opioid dose (MEDD or Morphine Equivalent Daily Dose or MME or Morphine Milligram Equivalents) and risk for accidental poisoning. Boehnert et al and Dunn et. al were some of the earliest papers that reported on this phenomenon. In both of these studies, hazard ratios for increased morbidity and mortality, increased exponentially above 50-90 MEDD/MME. Most deaths have been observed to be associated with doses above 60 MEDD/MME.
Park et. al and other authors have showed increased risk with the addition of benzodiazepines top opioids with risk increasing proportional to the base opioid MEDD/MME. Higher Opioid MEDD/MME when combined with a benzodiazepine drives roughly double the hazard ratio of accidental poisoning and death risk at each cut-point level. The Park et al paper showing this phenomenon associated with benzodiazepines appeared to be a class effect, however, temazepam did not appear to be associated with this phenomenon. There is a lack of follow-up on temazepam in subsequent studies to see if this phenomena can be explained.
It is less clear why two opioid prescriptions vs. one opioid prescription are thought to pose an increased risk to patients and often opioids prescriptions used in patients on chronic medications use a long acting and short acting opioid. This criteria in the metric are therefore problematic as it not explained in the measure rationale. In thinking about potential pitfalls associated with this, there is a possibility that providers, in order to meet the measure, might convert patients to a single opioid that might be less appropriate and result in a higher MEDD/MME.
The challenge, physiologically to patients that causes mortality related harm is primarily the risk of Opioid Induced Respiratory Depression (OIRD), the primary mechanism thought to be responsible for causing the harms associated with high dose MEDD/MME and opioid and benzodiazepine concurrent use. The relationship and concern with opioid and benzodiazepine co-prescribing as well as high dose opioid prescribing is more clearly defined and associated with OIRD in a dose dependent relationship and the latter portion of the metric is on more solid footing. Any dose of an opioid with a benzodiazepine is a concern for patient safety.
Recently, Yale and other groups have published guidelines that advocate for chronic opioids to not be fully stopped prior to initiation of buprenorphine or buprenorphine-naloxone using a method known as overlap therapy which involves continuing the patient's opioid and titrating up the agonist/antagonist while titrating down the pure agonist. Agonist/antagonist therapy is also being used to treat chronic pain and eliminate agonist therapy. This transition takes time and may not be able to be completely accomplished during and inpatient stay.
Benzodiazepines would be likely difficult to fully transition off of during the average hospitalization and thus an exclusion for patients transitioned to a longer-term care setting for benzodiazepine taper or who have been given a terminal script for benzodiazepine taper might be considered to incentivize this important intervention.
Smaller patient populations seemed to fare worse than larger patient populations on performance on this metric. Since most hospitals that are smaller are rural and less resourced, it might result in these hospitals not performing as well on this metric.
Despite these shortcomings, which should be addressed, the metric adds considerable value with a focus on opioid and benzodiazepine concurrent prescribing being its strongest feature. It would be advisable to consider modifying the first part of the measure to focus on MEDD/MME (as happened with the VA in the Opioid Safety Initiative) versus focusing on the number of opioid scripts written. It would also be advised to allow exceptions for benzodiazepine tapers. The addition of low dose buprenorphine or buprenorphine-naloxone concurrently prescribed at discharge might also be considered as an exclusion for this metric. Lastly, expansion of the population and the number of sites evaluated should be expanded.
Feasibility Acceptance
Feasibility assessment not systematically conducted or described. Despite 4,368 hospitals nation-wide that report this measure, that does not describe how they report or if the results reported represent something relevant. This needs to be more fully explained.
Scientific Acceptability
In studies that evaluated this metric, 11 hospitals were utilized, and all were teaching hospitals and in a single system in two states. The population used to study this metric was largely white (> 80%). This is a fairly homogenous population and lacked diversity. Sample size appears adequate 3 of 11 facilities did not meet the reliability ration of 0.7 and were .41 when measured using the beta-binomial model. This tended to be in facilities that are presumably smaller in size. Minimal populations and/or hospital size may be needed to strengthen inclusion criteria for this metric. It would be advised to study smaller hospital settings to see if this phenomena applies to all smaller hospitals or if this was isolated to the several sites studied.
AC 1 data appears to have strong agreement. Usually above 8- is considered a strong agreement. Given the relative sizes of the Medicare and Medicaid populations it is likely that the relationship seen in Medicare is relevant, especially in this analysis. This warrants additional evaluation.
Equity
This was not addressed.
Use and Usability
This measure has been in use for a year and is a current CMS eCQM. Trend data is not available.
Summary
This is a very important metric. Opioid and benzodiazepine co-prescribing is a generally dangerous practice that often takes place when several prescribers (mental health and primary care) are both jointly managing the patient for different health aspects. The metric does not address MEDD/MME which is the primary threat to OIRD, when at doses higher than 50-60 MEDD (or could use more conservative 20 MEDD) and at any dose of an opioid but especially when an opioid is co-prescribed with a benzodiazepine. Hospital physicians are in a unique place to intervene, however, there are significant challenges associated with titrating a patient off of opioids and/or benzodiazepines in a hospital setting. The study population used to evaluate this metric is large enough but not diverse enough. Medicare population risks should be further studied. It would be advised to focus this metric on opioid and benzodiazepine co-prescribing, to increase the reliability of the study and to evaluate equity impact of the metric.
Caution endorsement at this time
Importance
The opioid crisis is a significant issue for US healthcare. We need measures that can help recognize those that are at risk and perhaps should be treated differently. This fills a gap in measurement that has been lacking for a long time.
Feasibility Acceptance
This data is not routinely used in the process of care and is not available throughout all of U.S. Healthcare. Those challenges can be overcome but likely with significant time and investment in reporting and infrastructure.
Scientific Acceptability
The denominator is too broad. There are legitimate cases for prescribing multiple opioids or opioids with benzodiazepines that are not accounted for in the exclusionary criteria. Reporting of this measure would be an over inflation of the actual problem and does not tie directly back to quality of care.
As noted above this measure does not directly relate to the quality of care nor to outcomes per se. It measures a process but in a overly broad population.
Equity
Information was not provided on this section.
Use and Usability
The measure is in use in some health systems and mechanisms are in place to garner feedback.
Summary
While we need measures to address the opioid epidemic in America, this measure falls short of providing a useful purpose at this time. The denominator is too broad which could lead those who adopt this measure to potentially alter the care they provide to a population of patients for which it is appropriate. However, it is the best that we have right now so we should being using it but not stop the process of refining it into a more meaningful and purposeful measure that can drive actual clinical change.
Safe Use of Opioids - Concurrent Prescribing
Importance
As indicated, opioid overdoses are a major, and increasing, public health concern. Studies have demonstrated the impact of concurrent opioid or opioid and benzodiazepine use.
Feasibility Acceptance
Developer reported challenges with "Diagnosis: Opioid Use Disorder", "Discharge disposition codes", and "Encounter, Performed: Medication Assisted Treatment (MAT)". It would be helpful to hear the developer discuss these components.
Scientific Acceptability
Only 11 hospitals were included, of which a majority belonged to a single health system and a couple had a reliability score lower than the threshold. Consider expanding the number of entities used for testing.
The developer reported challenges with "Diagnosis: Opioid Use Disorder", "Discharge disposition codes", and "Encounter, Performed: Medication Assisted Treatment (MAT)." The developer could consider discussing these components.
Equity
Not addressed but was optional.
Use and Usability
currently in use in the Hospital inpatient quality reporting (IQR) program, and pay-for-reporting program.
Summary
Some components of the measure are not met but addressable by the developer which may lead to those components becoming met.
Recommend but requiring some issues to be resolved
Importance
Safety is an area of importance to patients.
Feasibility Acceptance
This measure is feasible to implement.
Scientific Acceptability
My main concern is with the volume threshold for public reporting. The developer wrote that "the publicly reported values (on Care Compare) are calculated for facilities nationwide in the United States that meet minimum case count requirements (> 10 cases)." The choice of this threshold is really questionable. Among 11 test sites, reliability score for the site with 20 cases was only 0.14. Can we really expect reasonable reliability scores for facilities with cases just above 10 cases?
Overall I rated "Met" for validity. However, it would be very useful for the developer to address the following two issues: 1) In the data element validity method section, the developer wrote that "to assess data element validity, we randomly selected 25 patient encounters from the full EHR extract in each test site," this would result in at least 250 cases among 11 test sites not accounting for the site with only 20 cases. However, in the data element validity results section, all reported results seem to be based on 50 cases. Were these 50 cases from all 11 sites? Were there any variations across sites in terms of testing results? 2) In the empirical validity results section, table 7, some of the differences were not significant, the developer should not over interpret the results.
Equity
No concern with equity
Use and Usability
This measure can provide useful information for providers and patients.
Summary
The key issue is the volume threshold for reporting. Ten is a really low threshold not supported by the evidence presented by the developer. Additionally, data element validity testing should be clarified and supplemented with site specific results. Empirical validity testing results should also be described taking into account statistical significance.
Need more information to assess measure
Importance
The developer cites the 2022 CDC guideline, which states there is a dose-dependent association between opioid use and risk for overdose events. In addition, they cite three studies from the CDC guideline, which reported evidence of concurrent benzodiazepine use in 31 to 61 percent of those deceased from overdose. However, they do not present graded guidelines and they do not provide evidence indicating how much concurrent use of opioids or concurrent use of opioids and benzodiazepines leads to increased risk of negative outcomes (e.g., death, overdose, hospitalization). It would be helpful to see a more detailed summary of the evidence and empirical evidence linking the concurrent prescribing of opioids/benzodiazepines with health outcomes (if available).
In addition, I did not see any information in the measure submission indicating that the developer had asked patients if the measure was meaningful to them. They could address this by asking patients for feedback on the measure (e.g., through focus groups or survey).
Feasibility Acceptance
The measure is currently in a program and will become required reporting for 2023. Therefore, there is some assumption that the measure is feasible. However, there was one data element during feasibility testing which received 0 across the board from both test sites, and for which the developer did not provide a feasibility plan. If the developer provided a feasibility plan for the Medication Assisted Treatment data element, I think they might be able to meet this requirement.
Scientific Acceptability
As the staff point out, across the small number (n=11) of test sites, 2-3 of those sites had reliability ratings <0.6. I think the developer could meet this threshold either by indicating a minimum denominator size which could improve reliability ratings above 0.6 or they could conduct testing with more sites to give a better demonstration of the reliability when more sites are evaluated.
The developer provides data element validity indicating the measure is valid. I thought the known groups validity testing was less strong as they did not provide much rationale for why Medicare beneficiaries would have higher scores, and it seems an imprecise measure could still perform worse among Medicare beneficiaries. However, since their testing demonstrates data element validity and the requirements do not seem to require both data element validity and measure score validity, I think they have met this requirement.
Equity
The develop did not submit data as this is an optional requirement.
Use and Usability
The measure is in use in the Hospital Inpatient Quality Reporting Program and will become mandatory in 2023. The developer provides two interventions that could improve scores (eg, providers monitoring specific patients and outreach to the community to understand prescribing practices). There is a feedback mechanism for this program and the developer describes several updates they have made to the measure specification in response to feedback and reviews of the literature.
Summary
The measure meets some requirements and seems as if it possibly meets other requirements, although I think we need more information in order to fully assess the measure.
Recommend for endorsement with clarifications requested
Importance
This measure is based on current guidelines for practice of limiting concurrent opioid or opioid/benzodiazepine medication prescribing. There is limited data on this measure historically given initial measurement period was 2023 so limited trend data availability
Feasibility Acceptance
Currently fully specified and in use in >4K hospitals
Scientific Acceptability
Met overall, but I would appreciate some discussion about the samples with lower reliability and lower n
reported statistics related to validity indicate reasonable strength of validity to this measure as currently written
Equity
There appears to be some potential for sampling error in the validity testing with focus on not for profit teaching locations, which may have a different patient makeup than other hospitals in the same geography
Use and Usability
Maintenance review. Currently measure is in use as part of IQR program as reported. Some states may also use this measure as part of value based payment programs
Summary
This measure is aligned with current best practice guidelines from national groups, and overall reflects adequate scientific strength to endorse. I would appreciate hearing more about the limitations noted in validity testing, exclusion diagnoses and role of equity in this measure
More detail needed before recommendation.
Importance
Importance met.
Feasibility Acceptance
Feasibility met.
Scientific Acceptability
Testing data should include hospitals from other regions of the country. It is not clear why retrospective data could not be used to increase the testing timeframe, nor why more hospitals could not be in the testing dataset. More information is needed about the validity of the measure as it relates to post-acute treatment, follow-up, and patient outcomes.
Developer does not include a rationale for the expectation of higher rates among Medicare patients.
Equity
Should be addressed.
Use and Usability
Please better describe the eCQM issues/problems.
Summary
The developers need to better explain the eCQM issues related to the measure, unintendend consquences of the measure, and increase the testing dataset and timeframe (or better explain why such a small dataset and timeframe).
Public Comments
Safe Use of Opioids – Concurrent Prescribing Measure
AOTA supports the Safe Use of Opioids – Concurrent Prescribing measure. Pain is a top reason for seeking healthcare. Managing the use of opioid prescriptions can protect many patients against long-term opioid use/misuse, which can negatively impact self-care and other life occupations.
Comment on 3316e
The American Medical Association (AMA) appreciates the opportunity to comment on Measure 3316e: Safe Use of Opioids – Concurrent Prescribing. We continue to be concerned with the measure as specified and do not support its continued endorsement until the concerns are adequately addressed.
Specifically, while we appreciate the addition of exclusions for patients with sickle cell disease or those with an active diagnosis or undergoing treatment for opioid use disorder, the measure still lacks the precision needed to ensure that only those patients for whom concurrent prescribing of two or more opioids or an opioid and benzodiazepine is appropriate are included in the denominator. As identified during the initial review of this measure in 2018, the patient population could likely include patients for whom concurrent prescribing of these medications may be appropriate, particularly those with chronic pain. Without further refinement, the AMA believes that there is a significant risk for the performance of hospitals and their physicians to be inaccurately represented. More importantly, there is a substantial risk that patients for whom these medications may be warranted will not receive appropriate therapies, leading to potential adverse outcomes, including depression, loss of function, and other negative unintended consequences.
The AMA believes that quality measurement needs to focus on how well patients’ pain is controlled, whether functional improvement goals are met, and what therapies are being used to manage pain. If pain can be well controlled and function improved without the need of these concurrent medications, then that is an indication of good patient care but the measure must precisely define the patients for which it is appropriate. We do not believe that this measure as specified is an appropriate goal as it may leave patients without access to needed therapies.
In addition, we are disappointed to see the minimum measure score reliability results of 0.41 and based on the submission information we were unable to determine if a case minimum is required to ensure a higher threshold. We believe that measures must meet minimum acceptable thresholds of 0.7 for reliability.
The AMA supports addressing the opioid crisis through quality measurement in addition to other avenues but strongly believes that any measures that are endorsed by the Consensus-Based Entity must also demonstrate that it does not compromise patient care. As a result, the AMA does not support continued endorsement of this measure until these concerns are adequately addressed.
AGS Comments on Safe Use of Opioids – Concurrent Prescribing
The American Geriatrics Society (AGS) greatly appreciates the opportunity to review and comment on the Safe Use of Opioids – Concurrent Prescribing measure (Measure #3316e).
While the AGS appreciates the addition of the cancer, palliative care, and hospice exclusions, we believe it would be important to take into consideration the population receiving post-acute rehabilitation services. Given that pain medications are a critically important part of functional status for this patient population, AGS recommends adding this group to the list of exclusions.