This electronic clinical quality measure (eCQM) assesses the proportion of inpatient hospitalizations for patients aged 18 years and older who were administered at least one medication known to cause hypoglycemia (hypoglycemic medication) during their hospitalization, and who suffered a severe hypoglycemic event (blood glucose less than 40 mg/dL) during the hospitalization.
Measure Specs
- General Information(active tab)
- Numerator
- Denominator
- Exclusions
- Measure Calculation
- Supplemental Attachment
- Point of Contact
General Information
The goal of this eCQM is to improve patient safety and prevent the adverse drug event severe hypoglycemia (blood glucose (BG) <40 mg/dL). The measure captures severe hypoglycemia in patients who are at risk. At risk patients are defined by the measure as those who receive a hypoglycemic medication during their inpatient hospitalization.
Frequency
While safety events are generally rare, hypoglycemia (BG ≤ 70 mg/dL) is one of the more frequently observed safety events. In a study published by the Office of the Inspector General (2022), adverse drug events represented 43% of all adverse events in hospitals among Medicare patients in 2018, with hypoglycemia among the top 5 most commonly occurring adverse drug events.
Additional studies indicate that severe hypoglycemia occurs in 2–5% of hospitalized patients with diabetes mellitus (Santos, 2020). Rodrick and colleagues (2024) reported that approximately 4.7% of adult patients insured by Medicare experienced a hypoglycemic event in both 2021 and 2022.
Impact
Inpatient hypoglycemia can be life-threatening and is associated with longer hospital stays and increased medical costs. Patients who experience a hypoglycemic event on average have a 4.1 day longer length of stay in the hospital (Cruz, 2020).
Performance gap and ability to improve performance
Hypoglycemic events within an inpatient setting are largely avoidable by monitoring the use of medications and nutritional status. The National Quality Forum (2011) has identified medication errors as one of its Serious Reportable Events (SREs) or “never events” and included over- or under-dosing of the hypoglycemic medication insulin as a specific example. SREs are defined as serious and harmful clinical events that are largely preventable.
Recent studies show that rates of severe hypoglycemia vary across hospitals, suggesting opportunities for improved care (Santos, 2020). Hospitals’ efforts to reduce incidents of hypoglycemia have been shown to improve patient outcomes and reduce costs (Shelton, 2021; Mattathil, 2023). For example, studies have demonstrated that surveillance of glycemic outliers and interdisciplinary approaches to glycemic management can reduce or prevent hypoglycemic events (American Diabetes Association (ADA), 2023).
References:
1. American Diabetes Association Professional Practice Committee. (2023). Diabetes care in the hospital: Standards of care in diabetes – 2024. Diabetes Care, 47(Supplement_1), S295–S306. https://doi.org/10.2337/dc24-S016
2. Cruz, P. (2020). Inpatient hypoglycemia: The challenge remains. Journal of Diabetes Science and Technology, 14(3), 560-566. https://doi.org/10.1177/1932296820918540
3. National Quality Forum. (2011). Serious reportable events in healthcare—2011 update: A consensus report. https://www.qualityforum.org/Topics/SREs/Serious_Reportable_Events.aspx
4. Mattathil R. (2023). Hypoglycemia management using a bundled care approach: A quality improvement project. Journal of Nursing Care Quality, 38(2), 141–145. https://doi.org/10.1097/NCQ.0000000000000670
5. Office of Inspector General (OIG). (2022). Adverse events in hospitals: A quarter of Medicare patients experienced harm in October 2018. https://oig.hhs.gov/oei/reports/OEI-06-18-00400.asp
6. Rodrick D., Timashenka A., & Umscheid C. Adverse events among in-hospital Medicare
patients in 2021 and 2022 (2024). AHRQ Publication No. 24-0084. Rockville, MD: Agency for Healthcare Research and Quality. https://www.ahrq.gov/sites/default/files/wysiwyg/patient-safety/quality-measures/qsrs/qsrs-2021-2022-adverse-event-data-report-nov-rev.pdf
7. Santos, C. A. Q., Conover, C., Shehab, N., Geller, A. I., Guerra, Y. S., Kramer, H., Kosacz, N. M., Zhang, H., Budnitz, D. S., & Trick, W. E. (2020). Electronic measurement of a clinical quality measure for inpatient hypoglycemic events: A multicenter validation study. Medical Care, 58(10):927-933. https://doi.org/10.1097/MLR.0000000000001398
8. Shelton, C., Demidowich, A. P., Motevalli, M., Sokolinsky, S., MacKay, P., Tucker, C., Abundo, C., Peters, E., Gooding, R., Hackett, M., Wedler, J., Alexander, L. A., Barry, L., Flynn, M., Rios, P., Fulda, C. L., Young, M. F., Kahl, B., Pummer, E., Mathioudakis, N. N., … Zilbermint, M. (2021). Retrospective quality improvement study of insulin-induced hypoglycemia and implementation of hospital-wide initiatives. Journal of Diabetes Science and Technology, 15(4), 733–740. https://doi.org/10.1177/19322968211008513
This eCQM uses electronic health record (EHR) data from hospitals to calculate the measure score. Hospitals collect EHR data using certified EHR technology (CEHRT). The measure specification is available as a package via MADiE export. The package includes a human-readable HTML file, and XML, clinical quality language (CQL), and ELM files for machine processing. No additional tools are used for data collection for eCQMs.
Numerator
Inpatient hospitalizations where a severe hypoglycemic event occurred during the encounter. A severe hypoglycemic event is:
- A glucose test with a result less than 40 mg/dL
AND
- A hypoglycemic medication was administered within 24 hours before the start of the severe hypoglycemic event (i.e., the glucose test with a result less than 40 mg/dL)
AND
- There was no subsequent repeat test for glucose with a result greater than 80 mg/dL within five minutes or less from the start of the initial glucose test with a result less than 40 mg/dL
The numerator is inpatient hospitalizations where a severe hypoglycemic event occurred. A severe hypoglycemic event is defined as:
- A glucose test with a result less than 40 mg/dL AND
- A medication known to cause hypoglycemia (hypoglycemic medication) was administered within 24 hours before the glucose test with a result less than 40 mg/dL AND
- There was no subsequent repeat test for glucose with a result greater than 80 mg/dL within five minutes or less of the initial glucose test with a result less than 40 mg/dL
Glucose levels are determined by laboratory or point-of-care (POC) tests, including capillary/glucometer blood glucose tests, and by interstitial fluid specimens from continuous glucose monitors. Glucose test results from urine specimens are not considered. The 24-hour and 5-minute timeframes are based on the time the glucose was drawn, as this reflects the time the patient was experiencing that specific glucose level.
Only one qualifying severe hypoglycemic event is counted in the numerator, and only one severe hypoglycemic event is counted per encounter.
All data elements necessary to calculate this measure’s numerator are defined within value sets available in the VSAC, and listed below:
- Glucose tests are represented in the value set Glucose Lab Test Mass Per Volume (2.16.840.1.113762.1.4.1248.34).
- Hypoglycemic medications are represented by the value set Hypoglycemics Severe Hypoglycemia (2.16.840.1.113762.1.4.1196.393).
To access the value sets for the measure, please visit the Value Set Authority Center (VSAC), sponsored by the National Library of Medicine, at https://vsac.nlm.nih.gov/.
Denominator
Inpatient hospitalizations that end during the measurement period for patients age 18 and older at the start of the encounter where at least one hypoglycemic medication administration starts during the encounter. The denominator includes inpatient admissions which began in the emergency department or in observation status when the transition between discharge from these encounters and admission to the inpatient encounter is one hour or less.
This measure includes all inpatient hospitalizations for patients aged 18 years and older at the time of admission who receive a hypoglycemic medication during their encounter. Inpatient hospitalizations include time spent in the emergency department and observation when the transition between discharge from these encounters and admission to the inpatient encounter is one hour or less.
The measurement period is one year, and the entire hospitalization must occur during the measurement period.
All data elements necessary to calculate this measure’s denominator are defined within value sets available in the VSAC, and listed below:
- Inpatient encounters are represented by the value set Encounter Inpatient (2.16.840.1.113883.3.666.5.307).
- Emergency Department visits are represented by the value set Emergency Department Visit (2.16.840.1.113883.3.117.1.7.1.292).
- Patients who had observation encounters are represented by the value set Observation Services (2.16.840.1.113762.1.4.1111.143).
- Encounters where at least one hypoglycemic medication was administered are defined by the value set Hypoglycemics Severe Hypoglycemia (2.16.840.1.113762.1.4.1196.393).
To access the value sets for the measure, please visit the Value Set Authority Center, sponsored by the National Library of Medicine, at https://vsac.nlm.nih.gov/.
Exclusions
None
None
Measure Calculation
The developer provided a measure score calculation diagram.
The measure is not stratified.
There is no minimum sample size to calculate the measure. The measure is in the Hospital Inpatient Quality Reporting (IQR) Program, which requires hospitals to have at least five patients in the initial population and one patient in the denominator for submission of quarterly data (Centers for Medicare & Medicaid Services (CMS), 2024).
Reference:
Centers for Medicare & Medicaid Services. (2024). Fiscal Year 2026 Hospital Inpatient Quality Reporting (IQR) Program Guide. https://w.qualityreportingcenter.com/globalassets/2024/05/iqr/2.-hospital-iqr-fy-2026-program-guide_vfinal508_.pdf
Supplemental Attachment
Point of Contact
Not applicable
Donta' Henson
Woodlawn, MD
United States
Michael Kerachsky
Mathematica
Princeton, NJ
United States
Importance
Evidence
Performance monitoring
Measuring glycemic events helps hospitals identify areas to address to improve patient health and cut costs. This measure provides standardized values that institutions can use to monitor performance and track the progress of quality improvement initiatives (Khan, 2022). Standardization of measurement also allows comparison of a hospital’s glycemic outcomes against a benchmark.
Recommendations for processes of care associated with measure outcome
The ADA and the Endocrine Society have published clinical practice recommendations to prevent severe hypoglycemia for individuals within the inpatient setting. Examples of relevant recommendations include:
- In people with diabetes using a personal continuous glucose monitoring (CGM) device, the use of CGM should be continued during hospitalization if clinically appropriate, with confirmatory POC glucose measurements for insulin dosing decisions and hypoglycemia assessment, if resources and training are available, and according to an institutional protocol. – Level of evidence: B (ADA, 2023a)
- Allow continuation of personal CGM in the inpatient setting with or without algorithm-driven insulin pump (ADIP) therapy rather than discontinuation – Strength of recommendation: Conditional, Certainty of evidence: Very low (McCall, 2023)
- For people with diabetes using an automated insulin delivery (AID) system along with CGM, the use of AID and CGM should be continued during hospitalization if clinically appropriate, with confirmatory POC BG measurements for insulin dosing decisions and hypoglycemia assessment, if resources and training are available, and according to an institutional protocol. – Level of evidence: C (ADA, 2023a)
- In adults with insulin-treated diabetes prior to admission who are hospitalized for noncritical illness, continue with the scheduled insulin regimen modified for nutritional status and severity of illness to maintain glucose targets in the range of 100 to 180 mg/dL (5.6 to 10.0 mmol/L). – Strength of recommendation: Strong, Certainty of evidence: Low (McCall, 2023)
- In adults with insulin-treated diabetes hospitalized for noncritical illness who are at high risk of hypoglycemia, use real-time CGM with confirmatory bedside POC BG monitoring for adjustments in insulin dosing rather than POC BG testing alone in hospital settings where resources and training are available – Strength of recommendation: Conditional, Certainty of evidence: Low (Korytkowski, 2022)
- Initiate CGM in the inpatient setting for select inpatients at high risk for hypoglycemia – Strength of recommendation: Conditional, Certainty of evidence: Very low (McCall, 2023)
- Treatment regimens should be reviewed and changed as necessary to prevent further hypoglycemia when a BG value of <70 mg/dL (3.9 mmol/L) is documented – Level of evidence: C (ADA, 2023a)
Recommendations for structures of care associated with measure outcome
Guidelines from ADA and the Endocrine Society also offer structure of care recommendations that hospitals can put in place to reduce the likelihood of severe hypoglycemia events occurring. Examples of these recommendations include:
- Institutions should implement protocols using validated written or computerized provider order entry sets for management of dysglycemia in the hospital (including emergency department, intensive care unit (ICU) and non-ICU wards, gynecology-obstetrics/delivery units, dialysis suites, and behavioral health units) that allow for a personalized approach, including glucose monitoring, insulin and/or noninsulin therapy, hypoglycemia management, diabetes self-management education, nutrition recommendations, and transitions of care – Level of evidence: B (ADA, 2023a)
- A hypoglycemia management protocol should be adopted and implemented by each hospital or hospital system. A plan for preventing and treating hypoglycemia should be established for each patient. Episodes of hypoglycemia in the hospital should be documented in the medical record and tracked for quality improvement/quality assessment. – Level of evidence: C (ADA, 2023a)
- Inpatient glycemic surveillance and management programs should leverage EHR data for inpatients at risk for hypoglycemia – Strength of recommendation: Strong, Certainty of evidence: Very low (McCall, 2023)
Measure outcome association with downstream harms
An episode of hypoglycemia increases the likelihood that a patient will experience a subsequent episode. One study found that 84% of hospitalized patients who had an episode of severe hypoglycemia had a preceding episode of hypoglycemia (ADA, 2023a). Frequent exposure to hypoglycemia might increase the risk of stroke and falls (Cruz, 2020). Hypoglycemia in the inpatient setting has also been linked to increased mortality and longer lengths of stay (Garg, 2013; Cruz, 2020).
Definitions of rating scales:
ADA level of evidence recommendations are assigned ratings of A, B, or C depending on the quality of the evidence in support of the recommendation, not the strength of the recommendation (ADA, 2023b). Recommendations with A-level evidence are based on large, well-designed randomized controlled trials or well-done meta-analyses of randomized controlled trials. Recommendations with lower levels of evidence may be equally important but are not as well supported.
The Endocrine Society uses the Grading of Recommendations Assessment, Development and Evaluation (GRADE) methodology to assess the certainty of evidence and make recommendations (Endocrine Society, 2022). Certainty of evidence is determined based on study design and quality across all evidence for an outcome. Each outcome is assigned a ranking of high, moderate, low, or very low based on consideration of all evidence, with high indicating confidence that the true effect lies close to the estimate of effect. Recommendations are either strong or conditional. A strong recommendation indicates desirable consequences clearly outweigh undesirable consequences, whereas a conditional recommendation indicates desirable consequences probably outweigh undesirable consequences.
References:
1. American Diabetes Association Professional Practice Committee. (2023a). Diabetes care in the hospital: Standards of care in diabetes – 2024. Diabetes Care, 47(Supplement_1), S295-S306. https://doi.org/10.2337/dc24-S016
2. American Diabetes Association Professional Practice Committee. (2023b). Introduction and Methodology: Standards of Care in Diabetes – 2024. Diabetes Care, 47(Supplement_1), S1-S4. https://doi.org/10.2337/dc24-SINT
3. Cruz P. (2020). Inpatient hypoglycemia: The challenge remains. Journal of Diabetes Science and Technology, 14(3), 560–566. https://doi.org/10.1177/1932296820918540
4. Endocrine Society (2022). Endocrine Society guideline methodology. https://www.endocrine.org/-/media/endocrine/files/cpg/methodology-page-refresh/endocrine_society_guideline_methodology_links.pdf
5. Garg, R., Hurwitz, S., Turchin, A., & Trivedi, A. (2013). Hypoglycemia, with or without insulin therapy, is associated with increased mortality among hospitalized patients. Diabetes Care, 36(5), 1107-1110. https://doi.org/10.2337/dc12-1296
6. Khan, S. A., & Zilbermint, M. (2022). Centers for Medicare & Medicaid Services' Hospital Harm Measures for severe hypoglycemia and hyperglycemia: Is your hospital ready?. Diabetes Spectrum : A Publication of the American Diabetes Association, 35(4), 391-397. https://doi.org/10.2337/dsi22-0008
7. Korytkowski, M.T., Muniyappa, R., Antinori-Lent, K., Donihi, A.C., Drincic, A.T, Hirsch, I.B., Luger, A., McDonnell, M.E., Murad, M.H., Nielsen, C., Pegg, C., Rushakoff, R.J., Santesso, N., & Umpierrez, G.E. (2022). Management of hyperglycemia in hospitalized adult patients in non-critical care settings: An Endocrine Society clinical practice guideline, The Journal of Clinical Endocrinology & Metabolism, 107(8), 2101-2128. https://doi.org/10.1210/clinem/dgac278
8. McCall, A. L., Lieb, D. C., Gianchandani, R., MacMaster, H., Maynard, G. A., Murad, M. H., Seaquist, E., Wolfsdorf, J. I., Wright, R. F., & Wiercioch, W. (2023). Management of individuals with diabetes at high risk for hypoglycemia: An Endocrine Society clinical practice guideline. The Journal of Clinical Endocrinology & Metabolism, 108(3), 529-562. https://doi.org/10.1210/clinem/dgac596
Measure Impact
Patient and caregiver representatives from Mathematica’s Patient Safety Technical Expert Panel (TEP) were asked if they felt that the Hospital Harm – Severe Hypoglycemia measure “is meaningful and produces information that is valuable in making care decisions.” Measure background, purpose, and definitions were reviewed prior to polling the representatives. Responses recorded from a total of six representatives were scored on a 4-point Likert scale (4 = strongly agree, 3 = agree, 2 = disagree, 1 = strongly disagree). The representatives unanimously agreed that the measure is meaningful and produces information that is valuable in making care decision, with a mean polling score of 3.7.
Performance Gap
We assessed measure performance using Hospital IQR Program data from the CMS Center for Clinical Standards and Quality (CCSQ) Centralized Data Repository (CDR) for performance period 2023 (January 1, 2023 – December 31, 2023), the first and only year data are currently available for the Hospital Harm – Severe Hypoglycemia measure. The measure was voluntarily reported by 522 hospitals with 568,405 encounters that met the Hospital IQR Program quarterly scoring thresholds of at least 5 patients in the initial population count and 1 patient in the denominator count.
Numerator and denominator counts for quarters that met the scoring threshold were combined to calculate an individual hospital’s performance period measure score.
The distribution of scores is described in Table 1 below. The overall mean performance score was 1.28% (range 0.00% - 41.80%). For this measure, a lower score indicates better quality. While a majority of hospitals met or were lower than the mean performance score, approximately 11% of hospitals performed well above the mean score, suggesting that there is room for improvement.
Multiple years of data are not yet available to observe a change in performance over time. However, the performance rates observed for the 2023 performance period, when only voluntary reporting occurred, were lower than those observed during the initial endorsement submission, 1.28% and 2.52% respectively. A major limitation to this comparison is the much smaller sample size used during the initial endorsement (13,636 patients from 6 hospitals) compared to the Hospital IQR Program dataset (568,405 patients at 522 hospitals). Any inference made from the results here and the initial endorsement should take the difference in sample size used to compute the scores into account.
| Overall | Minimum | Decile_1 | Decile_2 | Decile_3 | Decile_4 | Decile_5 | Decile_6 | Decile_7 | Decile_8 | Decile_9 | Decile_10 | Maximum | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Mean Performance Score | 1.28% | 0.00% | 0.00% | 0.00% | 0.00% | 0.12% | 0.68% | 1.09% | 1.39% | 1.73% | 2.28% | 5.55% | 41.80% |
| N of Entities | 522 | 1 | 53 | 53 | 52 | 52 | 52 | 52 | 52 | 52 | 52 | 52 | 1 |
| N of Persons / Encounters / Episodes | 568,405 | 5 | 14,219 | 5,533 | 6,210 | 39,989 | 64,377 | 98,600 | 90,221 | 141,308 | 91,549 | 16,399 | 328 |
Equity
Equity
This domain is optional for Spring 2025.
Feasibility
Feasibility
All of the data elements used within the measure are available in a structured format within EHR systems, captured as part of the typical course of patient care, and coded using nationally accepted terminology.
Value sets within the measure have been updated since the initial CBE submission, as is routine for measure maintenance. The types of codes captured to identify measure criteria are the same that were tested within the Feasibility Scorecard provided for the initial submission (attached per section 4.2), for example LOINC, ICD-10, etc. Feasibility testing for the initial submission found that it is possible to capture this information from the EHR.
This eCQM has been in the Hospital IQR Program for two years and no barriers to measure reporting have been shared with CMS through the ONC JIRA issue tracking system where implementers can share questions or comments on the measure. Administrative burden and cost associated with data collection are low because the measure uses data elements that are available in a structured format within EHR systems.
In order to maintain patient confidentiality, eCQM data are collected through the Hospital Quality Reporting (HQR) Secure Portal (https://ecqi.healthit.gov/tool/hospital-quality-reporting-hqr-system). HQR is the only CMS-approved website for secure communications and health care quality data exchange between quality improvement organizations, hospitals, physician offices, nursing homes, end-stage renal disease networks and facilities, and data vendors.
The measure scores are publicly reported in aggregate at the hospital level, therefore patient confidentiality is not a concern.
No feasibility assessment was conducted for maintenance endorsement as this is an established measure. No reports of feasibility issues have been reported by implementers, therefore no adjustments were made to the measure based on feasibility. The feasibility scorecard submitted for initial endorsement has been attached.
Proprietary Information
Scientific Acceptability
Testing Data
We acquired hospital-level and patient-level Hospital IQR Program data from the CMS Center for Clinical Standards and Quality (CCSQ) Centralized Data Repository (CDR) Hospital Quality Report (HQR) dataset. The HQR system consumes Quality Reporting Document Architecture (QRDA) files and processes the data based on measure logic to determine measure outcomes. QRDA allows a variety of EHR systems to report data in a structured, consistent format for reporting quality measure results.
Data for performance period 2023 (January 1, 2023 – December 31, 2023) were used for testing. This is the first and only year data are currently available in the CCSQ CDR for the Hospital Harm – Severe Hypoglycemia measure. Measure reporting was voluntary for 2023.
- The hospital-level dataset has 568,405 denominator eligible inpatient encounters from 522 hospitals. In this dataset, quarterly data are not included for a hospital if the facility does not meet the Hospital IQR Program threshold of at least five patients in the initial population and one patient in the denominator for submission of quarterly data.
- The patient-level dataset includes all denominator eligible patient encounters reported by hospitals, regardless of whether the encounter was excluded from the hospital-level dataset for the hospital not reaching the data threshold. As a result, there are more denominator eligible encounters included in the patient-level dataset (722,956) than denominator eligible encounters in the hospital-level dataset. Patients appear in the patient-level dataset multiple times if they had multiple eligible encounters.
Hospital-level data are used for Section 2: Importance. Both hospital-level and patient-level data are used for Section 5: Scientific Acceptability. Data from 2023 are used for all sections.
All subsection (d) hospitals within the United States are included within the Hospital IQR Program and are eligible to report the measure. Hospital IQR Program data from more than 4,000 Medicare-certified hospitals, including Veterans Administration hospitals, are made publicly available on the Care Compare and Provider Data Catalog websites. During performance period 2023, the only year data for the Hospital Harm – Severe Hypoglycemia measure are currently available, 522 hospitals voluntarily reported the measure. Of those, there are data for 475 hospitals from 43 states available for download on the Timely and Effective Care – Hospital page of the Provider Data Catalog. Denominator counts for the hospitals included in this dataset range from 25 to 14,655, with an average of 1,190 denominator-eligible patients per hospital. Hospitals are not included in this publicly available dataset if their denominator count is less than 11.
Table 1, available in attachment 5.1.4_Acceptability_3503e submitted in Section 7. Supplemental Information, uses the patient-level dataset to provide details on the characteristics of the eligible population including age, sex, race, ethnicity, and insurance payer.
Reliability
Measure Score Reliability
We assessed signal- to-noise reliability that describes how well the measure can distinguish the performance of one hospital from another (Adams and Mehrota, 2010; Yu and Mehrota, 2013). The signal is the proportion of the variability in measured performance that can be explained by real differences in performance. Scores can range from 0 to 1. A reliability of zero implies that all the variability in a measure is attributable to measurement error. A reliability of one implies that all the variability is attributable to real differences in performance.
We use the Adam’s beta-binomial method (Adams, 2009) to calculate signal-to-noise ratio reliability. Using variability between hospitals (signal: provider-to-provider variance) and variability within hospitals (noise: provider-specific-error variance), the reliability for each hospital can be defined as: reliability = signal variance / (signal variance + noise variance).
We estimate the beta-binomial signal variance using denominators and harm rates from all hospitals. The noise variance is estimated based on the denominator and harm rate of an individual hospital.
References:
1. Adams, J. (2009, June 25). The reliability of provider profiling: A tutorial. RAND Corporation. https://www.rand.org/pubs/technical_reports/TR653.html.
2. Adams J, Mehrota, A, Thoman J, McGlynn, E. (2010). Physician cost profiling – reliability and risk of misclassification. New England Journal of Medicine, 362(11): 1014-1021.
3. Yu, H, Mehrota, A, Adams J. (2013). Reliability of utilization measures for primary care physician profiling. Healthcare, 1, 22-29.
Accountable Entity–Level Reliability Testing Results
Table 2 below contains the distribution of reliability statistic estimates across hospitals for the Hospital Harm – Severe Hypoglycemia measure. Hospitals are grouped into deciles based on N, the number of denominator-eligible inpatient encounters. For example, the “reliability” row of this table contains the mean reliability estimate among hospitals for each decile of N. Similarly, the “mean performance score” row of this table contains the mean score among hospitals in each decile of N. The row “N of Persons / Encounters / Episodes” of this table contains the sum of N for hospitals in each decile group. The overall column provides the mean reliability and mean performance score for all hospitals. The minimum and maximum columns provide estimates for the hospitals with the lowest and highest N, respectively. We report the hospital with the smallest non-zero measure score in the minimum reliability column since the reliability for a hospital with a zero score is 1.0.
The overall signal-to-noise ratio of 0.829 indicates high reliability. Reliability estimates across deciles ranged from 0.674 – 0.985. While there was some variation of the reliability estimate for each decile, reliability estimates across all deciles exceeded the acceptable reliability threshold of 0.6 as determined by the CBE (Partnership for Quality Measurement (PQM), 2024).
Reference:
1. Partnership for Quality Measurement. (2024). Endorsement and Maintenance (E&M) Guidebook. https://p4qm.org/sites/default/files/2024-08/Del-3-6-Endorsement-and-Maintenance-Guidebook-Final_0.pdf
| | Overall | Minimum | Decile_1 | Decile_2 | Decile_3 | Decile_4 | Decile_5 | Decile_6 | Decile_7 | Decile_8 | Decile_9 | Decile_10 | Maximum |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Reliability | 0.829 | 0.008 | 0.842 | 0.741 | 0.674 | 0.699 | 0.790 | 0.804 | 0.866 | 0.925 | 0.966 | 0.985 | 0.992 |
| Mean Performance Score | 1.28% | 22.22% | 1.18% | 1.11% | 1.29% | 1.22% | 0.85% | 1.04% | 2.03% | 1.26% | 1.37% | 1.45% | 0.91% |
| N of Entities | 522 | 1 | 53 | 53 | 52 | 52 | 52 | 52 | 52 | 52 | 52 | 52 | 1 |
| N of Persons / Encounters / Episodes | 568,405 | 9 | 997 | 2,166 | 3,308 | 4,970 | 7,077 | 11,096 | 22,009 | 50,642 | 128,310 | 337,830 | 14,700 |
Validity
Construct Validity
We conducted correlational analyses to gather evidence of convergent validity for the Hospital Harm – Severe Hypoglycemia measure. Specifically, we calculated Spearman’s rank correlations between hospital performance scores for hospitals reporting on the Hospital Harm – Severe Hypoglycemia measure and two other hospital-level measures of patient safety or harm event: the Hospital Harm – Severe Hyperglycemia and Patient Safety Indicators (PSI) Patient Safety and Adverse Events Composite (PSI 90) measures.
The Hospital Harm – Severe Hyperglycemia measure is the most appropriate comparator measure for convergent validity testing because both measures are eCQMs, neither are risk adjusted, there is a large overlap of hospitals reporting on both measures, and both are outcome measures of similar hospital processes including inpatient care, appropriate patient monitoring, and appropriate drug administration (see logic model, Section 2.1). The PSI 90 measure, composed of 10 measures for patient safety covering a broad range of outcomes stemming from varying processes of care (see note below), also includes overlap with the Hospital Harm – Severe Hypoglycemia measure. We hypothesized that while there would be a positive correlation between both pairs of measures, there would be a stronger coefficient of correlation between the Hospital Harm – Severe Hypoglycemia and Hospital Harm – Severe Hyperglycemia measures than for the Hospital Harm – Severe Hypoglycemia and PSI 90 measures. Data for all measures were acquired from the CCSQ CDR. Data were obtained for performance year 2023 for the Hospital Harm – Severe Hyperglycemia measure and data covering performance period July 1, 2021 – June 30, 2023 were obtained for the PSI 90 measure.
Note: The PSI 90 measure is constructed from the following ten components:
1. CMS PSI 03 Pressure Ulcer Rate
2. CMS PSI 06 Iatrogenic Pneumothorax Rate
3. CMS PSI 08 In-Hospital Fall-Associated Fracture Rate
4. CMS PSI 09 Postoperative Hemorrhage or Hematoma Rate
5. CMS PSI 10 Postoperative Acute Kidney Injury Requiring Dialysis Rate
6. CMS PSI 11 Postoperative Respiratory Failure Rate
7. CMS PSI 12 Perioperative Pulmonary Embolism (PE) or Deep Vein Thrombosis (DVT) Rate
8. CMS PSI 13 Postoperative Sepsis Rate
9. CMS PSI 14 Postoperative Wound Dehiscence Rate
10. CMS PSI 15 Abdominopelvic Accidental Puncture or Laceration Rate
The Hospital Harm – Severe Hypoglycemia and Hospital Harm – Severe Hyperglycemia measures rs = 0.346 (p-value <0.001). The Hospital Harm – Severe Hypoglycemia and PSI 90 measures rs = 0.089 (p-value = 0.153).
Construct Validity
A moderate positive correlation (rs = 0.346) was observed between the Hospital Harm – Severe Hypoglycemia and Hospital Harm – Severe Hyperglycemia measures, which demonstrates that a moderately strong common construct of patient safety underlies these measures. A weak positive correlation (rs = 0.089) was observed between the Hospital Harm – Severe Hypoglycemia and PSI 90 measures. As we expected, there was a stronger correlation observed between the Hospital Harm – Severe Hypoglycemia and Hospital Harm – Severe Hyperglycemia measures than for the Hospital Harm – Severe Hypoglycemia and PSI 90 measures. These results support the assumption that facilities’ glycemic control protocols drive outcomes on the Hospital Harm - Severe Hypoglycemia and Hospital Harm - Severe Hyperglycemia measures, and the correlation between these two measures indicate each measure is accurately capturing glycemia adverse drug events resulting from the related processes of care. The weaker association between the Hospital Harm – Severe Hypoglycemia and PSI 90 measure shows that the PSI 90 measure is broadly associated as a benchmark measure of patient safety. Our interpretation is based on the following thresholds developed by Cohen (1992):
0.10 indicates a small effect size
0.30 indicates a medium effect size
0.50 indicates a large effect size
Reference:
1. Cohen J. (1992). A power primer. Psychological bulletin, 112(1), 155–159. https://doi.org/10.1037//0033-2909.112.1.155
Risk Adjustment
Risk adjustment promotes fair and accurate comparison of health care outcomes across hospitals by controlling for patient-level characteristics within the population of interest and outside of the hospital's control. Patient-level characteristics include clinical (e.g., types, number, or severity of conditions), demographic (e.g., age, gender), functional (e.g., ability to walk), and social (e.g., income, education, geography) (CMS, 2023).
The Hospital Harm – Severe Hypoglycemia measure has not been risk adjusted since its inception because severe hypoglycemia is considered to be largely preventable with proper medication and adequate medical nutrition therapy. NQF (2011) includes over- or under-dosing of the hypoglycemic medication insulin as an explicit example of SRE 4A: Patient death or serious injury associated with a medication error. SREs are clearly identifiable and measurable, universally or largely preventable, and serious, indicative of a problem in a healthcare setting’s safety systems.
To reaffirm that this measure should not be risk adjusted, we polled the Mathematica Patient Safety TEP members on whether they agreed that the Hospital Harm – Severe Hypoglycemia measure should remain unadjusted and whether hospitals should be able to effectively manage comorbidities related to severe hypoglycemic events. The 19 members present unanimously agreed that the measure should remain unadjusted.
References:
1. Centers for Medicare & Medicaid Services. (2023, August). Risk Adjustment and Risk Stratification in Quality Measurement. https://mmshub.cms.gov/sites/default/files/Risk-Adjustment-Quality-Measurement.pdf
2. National Quality Forum. (2011). Serious reportable events in healthcare—2011 update: A consensus report. https://www.qualityforum.org/Topics/SREs/Serious_Reportable_Events.aspx
Use & Usability
Use
Acute care and critical access hospitals
Acute care and critical access hospitals
Usability
Several entities have described their experiences with hospital-wide or hospital-system-wide efforts to reduce hypoglycemia among inpatients. These experiences range from quality improvement efforts to advanced informatics solutions. This broad range allows for hospitals to choose an approach that will work for them, and see results. Giroir and Calmes (2024) implemented improvement activities to lower facility incidence, including changes to insulin order sets and admission orders, and allowing deferrals for insulin administration dependent on patient nutritional status. Similarly, Kravchenko et al. (2020) were able to reduce incidents of severe hypoglycemia by updating internal protocols as part of a quality improvement initiative. Even efforts with a specific focus have shown promise. Modzelewski et al. (2021) focused on reducing incidents of hypoglycemia that resulted from treatment for hyperglycemia while Mattathil (2022) focused on reducing hypoglycemic events within a medical-surgical unit. Both saw improvements after interventions were implemented.
There are also opportunities for innovative informatics solutions to prevent hospital-acquired hypoglycemia. Mathioudakis et al. (2021) used machine learning to proactively identify hypoglycemic events within 24 hours of a BG result value. Chen et al. (2021) developed a free, open source glucometrics tool that facilities can use to monitor patients. Judson et al. (2022) describe a program that has reduced glycemic events and improved staff education though a virtual specialist consultation and written recommendations.
As documented above, a menu of options shown to be successful are available for hospitals to address patient safety in a way that works best for them.
References:
1. Chen, Y., Ning, Y., Thomas, P., Salloway, M., Tan, M. L. S., Tai, E. S., Kao, S. L., & Tan, C. S. (2021). An open source tool to compute measures of inpatient glycemic control: translating from healthcare analytics research to clinical quality improvement. JAMIA Open, 4(2), ooab033. https://doi.org/10.1093/jamiaopen/ooab033
2. Giroir, C., & Calmes, H. (2024). Hypoglycemia in the Hospitalized Patient: Interventions and Opportunities. IntechOpen. https://doi.org/10.5772/intechopen.1003906
3. Judson, T. J., Mourad, M., & Wachter, R. M. (2022). Building a Targeted Automatic E-Consult (TACo) program. Joint Commission Journal on Quality and Patient Safety, 48(2), 114–119. https://doi.org/10.1016/j.jcjq.2021.10.007
4. Kravchenko, M. I., Tate, J. M., Clerc, P. G., Forbes, W. L., Gettle, M. C., Wardian, J. L., & Colburn, J. A. (2020). Impact of structured insulin order sets on inpatient hypoglycemia and glycemic control. Endocrine Practice, 26(5), 523–528. https://doi.org/10.4158/EP-2019-0341
5. Mathioudakis, N. N., Abusamaan, M. S., Shakarchi, A. F., Sokolinsky, S., Fayzullin, S., McGready, J., Zilbermint, M., Saria, S., & Golden, S. H. (2021). Development and validation of a machine learning model to predict near-term risk of iatrogenic hypoglycemia in hospitalized patients. JAMA Network Open, 4(1), e2030913. https://doi.org/10.1001/jamanetworkopen.2020.30913
6. Mattathil R. (2023). Hypoglycemia management using a bundled care approach: A quality improvement project. Journal of Nursing Care Quality, 38(2), 141–145. https://doi.org/10.1097/NCQ.0000000000000670
7. Modzelewski, K. L., Cannavo, A., Fantasia, K. L., Korpaisarn, S., & Alexanian, S. M. (2021). A quality improvement initiative to successfully reduce the frequency of hypoglycemia during treatment of hyperglycemic crises at an academic safety-net hospital: Insights and results. Journal of Clinical & Translational Endocrinology, 26, 100269. https://doi.org/10.1016/j.jcte.2021.100269
We receive feedback on this measure through the ONC JIRA issue tracking system (https://oncprojectracking.healthit.gov/support/secure/Dashboard.jspa), which allows eCQM vendors and implementers to submit questions. From when the measure became available for review in 2023 through December 6, 2024, 31 comments were submitted through JIRA.
Most questions received through JIRA asked for clarification on how to report the measure. For these, if there was a grammatical update to the measure needed to improve clarity, we would implement that change. As one example of responding to this type of feedback, we added language to the specification clarifying that glucose results in mmol/L should be converted to mg/dL for reporting the measure.
For questions or suggestions that were technical or clinical in nature, we consulted with internal clinical or technical experts and/or the Mathematica Patient Safety TEP before making a change. As an example of this type of feedback, we asked the TEP if it would be appropriate to add (1) metformin 500 mg when used alone and/or (2) low-dose insulin to the list of qualifying hypoglycemic medications for this measure. The TEP agreed that these medications should not be included as qualifying hypoglycemic medications in the measure because it would be atypical for metformin 500 mg or low-dose insulin used alone to result in a hypoglycemic event. Based on their recommendation, we did not make an update to the measure.
2023 was the first year that this measure is available for reporting and the only year for which data are currently available. As such, there are not enough data to track performance score trends.
Implementers have not notified CMS of any unexpected findings.
Patient and caregivers from Mathematica’s Patient Safety TEP were polled on whether they believed that there were “no unintended consequences or concerns” regarding the Hospital Harm - Severe Hypoglycemia measure. Measure background, purpose, and definitions were reviewed prior to polling the representatives. Responses recorded from a total of six representatives were scored on a 4-point Likert scale (4 = strongly agree, 3 = agree, 2 = disagree, 1 = strongly disagree). The representatives unanimously agreed that there are no unintended consequences or concerns with a mean polling score of 3.3.
Comments
Staff Preliminary Assessment
CBE #3503e Staff Preliminary Assessment
Importance
Strengths
- A clear logic model is provided, depicting the relationships between inputs (e.g., electronic health records, hospital staff), activities (e.g., plans for preventing and treating hypoglycemia), and desired outcomes (e.g., Improvements in hospital practices for monitoring and detecting hypoglycemic events). This model demonstrates how the measure's implementation will lead to the anticipated outcomes.
- Hypoglycemia is a significant problem, and the developer cites evidence that hypoglycemia is among the top 5 most commonly occurring adverse drug events, severe hypoglycemia occurs in 2–5% of hospitalized patients with diabetes, and about 4.7% of adult patients insured by Medicare experienced hypoglycemic events annually.
- Data from the CMS Center for Clinical Standards and Quality (CCSQ) Centralized Data Repository (CDR) for performance period 2023 (January 1, 2023 – December 31, 2023) show a performance gap, with decile ranges from 0.00% to 5.55%, indicating variation in measure performance.
- Description of patient input supports the conclusion that the measured outcome is meaningful with at least moderate certainty. Patient input was obtained through six patient and caregiver representatives participating in Mathematica’s Patient Safety Technical Expert Panel (TEP). These patient representatives unanimously agreed that the measure is meaningful.
Limitations
- While the developer sites numerous clinical practice guidelines to support the measure, they are based on low-moderate quality evidence. The submission could be strengthened by a discussion of the quality, quantity, and consistency of the evidence.
Rationale
- The maintenance measure is rated as ‘Not Met But Addressable” due to incomplete/poorly graded evidence. Enhancements, including addressing the quality, quantity, and consistency of the evidence supporting the measure, could elevate its importance.
Closing Care Gaps
The developer did not address this optional domain.
Feasibility Assessment
Strengths
- All required data elements are routinely generated during care delivery, and required elements are available from digital or electronic sources.
- The developer completed an eCQM Scorecard showing the measure was tested in at least two EHRs (Epic and Cerner). All data required for this measure are readily available in a structured format, residing in fixed fields within the EHR systems tested. The data elements are accurate and coded using nationally accepted terminology standards, facilitating interoperability and standardization across different systems.
- The developer described how changes to measure specifications impact data structure and/or availability.
- The developer stated that no feasibility issues were found requiring adjustment of the final measure’s specifications.
The developer described the costs and burden associated with data collection and data entry, validation, and analysis. They indicated the measure has been in use in the Hospital IQR Program for two years, however, no implementers have raised concerns about measure burden. They assert administrative burden and cost for implementing the measure are low because all data elements are readily available in structured EHR fields. - The developer described how all required data elements can be collected without risk to patient confidentiality, including requiring data be submitted through the Hospital Quality Reporting (HQR) Secure Portal and only publicly reporting data in aggregate.
- There are no fees, licensing, or other requirements to use any aspect of the measure (e.g., value/code set, risk model, programming code, algorithm).
Limitations
- None identified.
Rationale
- This maintenance measure meets all criteria for 'Met' due to its well-documented feasibility assessment, clear and implementable data collection strategy, and transparent handling of patient confidentiality, burden, licensing, and fees. The eCQM Feasibility Scorecard results indicate that the measure is well-supported by the current capabilities of the tested EHR systems in terms of data availability, accuracy, adherence to standards, and workflow integration. These factors collectively ensure that the measure can be implemented effectively and sustainably in a real-world healthcare setting.
Scientific Acceptability
Strengths
- The developer performed the required reliability testing for this maintenance measure, namely, they conducted accountable entity-level (“measure score”) reliability testing at the level for which the measure is specified. Data sources used for reliability analysis are adequately described and include Hospital Inpatient Quality Reporting (IQR) Program data from the CMS CCSQ CDR Hospital Quality Report (HQR) dataset during the period of CY2023. The entities included in the analysis were characterized by 568,405 encounters across 522 hospitals.
- The developer conducted signal-to-noise reliability testing at the accountable entity-level. Based on the data shown in Table 2, it is likely that more than 70% of accountable entities likely meet the expected threshold of 0.6 at the hospital level.
Limitations
- At least one entity has a very low reliability of 0.008, a hospital with only 9 encounters. The developer may consider a minimum denominator threshold for this measure.
Rationale
- The developer performed the required reliability testing for this maintenance measure and results demonstrate sufficient reliability at the accountable entity-level.
Strengths
- Validity: The developer provides an Importance Table, Logic Model, and face validity, providing a plausible causal association between the entity response to the measure and the measure focus. Empirical support for ruling out confounders includes adequate reliability (although likely inflated by zero events), no risk-adjustment (as a serious reportable event), and a correlation with related outcome measures with construct overlap (Hospital Harm – Severe Hyperglycemia; Patient Safety and Adverse Events Composite (PSI 90)). It is notable that the developer makes an explicit quantitative claim that because the degree of construct overlap is greater for the hyperglycemia measure, the magnitude of the correlation ought to be greater (r=0.346 vs. r=0.089). Empirical support for ruling-in responsible mechanisms includes several empirical studies and reports (e.g. Treatment and Medication Protocols, Technology and Monitoring Tools, Care Structures and System Design, Education and Staff Support).
- Risk adjustment (RA): The developer did not conduct risk or case-mix adjustment. However, the developer provided the rationale that severe hypoglycemia is largely preventable with proper medication and medical nutrition therapy, and improper hypoglycemic medication administration is clearly measurable and indicative of a problem in a healthcare setting's safety systems. The rationale was supported by appropriate literature and unanimous agreement from patient safety TEP members.
Limitations
- Validity: Residual risk for confounders includes poor to fair correlation that cannot rule out confounding. The explanation is not a complete justification for the magnitude of the correlation (r=0.346) and leaves 65% of the causal factors for the not risk-adjusted performance unexplained. Residual risk for confounders not ruled out includes (chance, patient nutritional intake variability, acute illness severity and comorbidities, pre-admission insulin regimens or patient-managed devices, hospital unit type or staffing levels, timing and availability of point-of-care testing). There is no direct empirical support for whether the responsible mechanisms account for the association between the entity response to the measure and the measure focus. Residual risk for a responsible mechanism includes the potential counter-acting mechanisms (use of Electronic Health Records (EHRs) and structured surveillance systems, availability of inpatient diabetes specialty consults (including virtual consults), institutional policies allowing use of continuous glucose monitoring (CGM) and automated insulin delivery (AID) devices during hospitalization, availability of machine learning-based predictive alerts).
Rationale
- Met justification (validity): The measure developer provides some support for the causal claim that the entity response to the measure is causally related to the measure focus. The developer provides empirical support for ruling out confounders (always with some residual risk of unstated or unexamined confounders) and for ruling in responsible mechanisms (always with some residual risk that the explicit mechanisms are only partially responsible for the measure focus).
- Met justification (RA): The developer did not perform risk/case-mix adjustment or stratification, but provided appropriate reasonable rationale supported by literature and TEP agreement.
Use and Usability
Strengths
- The measure is currently used in the Centers for Medicare & Medicaid Services (CMS) Hospital IQR Program and the Centers for Medicare & Medicaid Services (CMS) Medicare Promoting Interoperability Program.
- The developer provides a summary of how accountable entities can use the measure results to improve performance. Specifically, hospitals can change insulin order sets and admission orders, update internal protocols, or use machine learning to proactively identify hypoglycemic events within 24 hours. These possible actions are reflected in the measure’s logic model.
- Stakeholders can submit questions about the measure through the Office of the National Coordinator for Health Information Technology's Jira issue tracking system. The measure has received a total of 31 comments, most of which asked for clarification about how to report the measure. Technical and clinical questions were addressed with the TEP. The developer notes the measure was not updated based on user suggestions.
- The developer argues that accountable entities can use the measure results to improve performance. However, the measure was first implemented in 2023 and there is not yet enough data to track performance.
- The developer reported no unexpected findings.
Limitations
- None identified.
Rationale
- For maintenance endorsement, the measure is actively used in at least one accountability application, with a systematic feedback approach that allows for continuous updates based on stakeholder feedback. The measure has only been in use since 2023, so sufficient time has not elapsed to observe trends in performance results. The developer reported no unexpected findings.
Public Comments
Laboratory Glucose value exclusion
While I do agree that the glucose value obtained during lab draws should be looked at, it is not feasible to have a re-draw within 5 minutes if it results <40 mg/dl considering most lab specimens do not even make it the lab to be run within 5 minutes from the "time drawn." I request glucose results from lab draws be excluded from the Numerator, or at least considered a Denominator Exception if a re-draw is done within a certain timeframe after the lab calls the PANIC value (I have only worked at a couple facilities, but both have called HI and LOW PANIC values for labs, so I would hope most/all facilities do to escalate lab concerns).
Public Comment Reply - USA Health
Thank you for your comment. This eCQM allows for results from both laboratory and point of care (POC) testing to qualify for the numerator. The measure developer understands that hospital workflows do not allow glucose tests sent to the laboratory to be retested within five minutes. Per the definition included in the measure specifications, the purpose of the repeat test within five minutes is to eliminate false positives that can occur with POC testing. There is no requirement within the measure for repeat testing following a low value. However, the measure allows for an encounter to be removed from the numerator if a repeat test within five minutes indicates that the blood glucose is above 80 mg/dL. We will consider including language in future updates that further clarifies the five minute retest.