When the measure returns for maintenance (3 years), the measure developer should have:
- Continued to explore the exclusion list to determine if changes are needed (e.g., empirical analyses with broader testing across entities) and to further clarify the conditions and justify them based on burden; and
- Conducted additional validity testing (data element in additional EHR).
The Inappropriately Broad Empiric Antibiotic Selection for Adult Hospitalized Patients with Uncomplicated Pneumonia measure is a process measure representing the annual percentage of hospitalized adults with uncomplicated community-acquired pneumonia. Here, we defined “inappropriately broad” as any antibiotic therapy targeting methicillin-resistant Staphylococcus aureus (MRSA) or Pseudomonas aeruginosa in patients without risk factors for one of those organisms. The measure will be calculated using electronic health record (EHR) data and is intended for use at the facility level for both quality improvement and pay-for-performance.
Measure Specs
- General Information(active tab)
- Numerator
- Denominator
- Exclusions
- Measure Calculation
- Supplemental Attachment
- Measure Record
- Point of Contact
General Information
The overall objective of this electronic clinical quality measure is to quantify inappropriately broad empiric antibiotic use in hospitalized adults with uncomplicated community-acquired pneumonia (CAP). Here, we defined “inappropriately broad” as any antibiotic therapy targeting methicillin-resistant Staphylococcus aureus (MRSA) or Pseudomonas aeruginosa in patients without risk factors for one of those organisms.
Antibiotic overuse is a national and international public health emergency with antibiotic resistant infections estimated to directly cause 1.27 million deaths globally and indirectly contribute to 4.95 million deaths.1 National studies by the Centers for Disease Control and Prevention (CDC) estimate that up to 50% of hospitalized patients receive antibiotic therapy, most commonly for pneumonia, and that up to 40% of antibiotic prescribing could be improved.2
Pneumonia is not only the most common reason for inpatient antibiotic use but also the most common infectious cause of mortality in the US resulting in approximately 1.4 million emergency department visits, 740 000 hospitalizations, 41 000 deaths, and $7.7 billion in inpatient costs each year in the US.3-6 Adverse consequences of unnecessarily broad empiric therapy include an increased adjusted risk of death, kidney injury, and serious secondary infections.7-9
How will this measure improve quality of care? By establishing a standardized process to assess inappropriately broad empiric antibiotic use for CAP, a larger proportion of patients will potentially receive appropriate care consistent with the 2019 American Thoracic Society/Infectious Disease Society of America (ATS/IDSA) CAP national guidelines.10 Appropriateness of antibiotic therapy for pneumonia is a priority for numerous federal and accreditation organizations–including CDC, The Joint Commission, and Centers for Medicare and Medicaid Services–and is not currently captured in typical quality improvement measures. For example, the National Healthcare Safety Network (NHSN) antimicrobial use (AU) measures focus on quantifying antibiotic use and comparing to expected values, with no assessment of appropriateness of empiric therapy. Notably, the NHSN AU initiative could be augmented with an eCQM to assess appropriateness of empiric antibiotics for CAP, the most common indication for inpatient antibiotic use.
What are the benefits or improvements in quality envisioned by this measure? Inappropriately broad empiric therapy for CAP is common. A recent report on 8,286 non-intensive care unit (ICU) hospitalized CAP patients from 67 Michigan hospitals (assessed using the chart review measure from which our eCQM was adapted) showed that 2,215 (26.7%) received inappropriately broad empiric treatment (i.e., anti-MRSA or anti-Pseudomonal coverage in patients eligible for standard CAP coverage per ATS/IDSA guidelines).11 Compared to patients who received standard CAP antibiotic treatment, patients receiving inappropriately broad antibiotics had higher 30-day readmissions, more transfers to ICU and antibiotic-associated adverse events, and longer hospitalizations.11 Similarly, a retrospective analysis of 88,605 patients with CAP across the Veterans Health Administration health care system found that adding empirical anti-MRSA therapy to standard CAP therapy) was associated with an increased adjusted risk of death, kidney injury, and secondary infections (C. difficile, vancomycin-resistant Enterococcus infection, and resistant gram-negative rod infections).8 Given that up to 90% of empiric anti-MRSA or anti-Pseudomonal antibiotic therapy is non-guideline concordant,8,11 there is substantial potential benefit to patients and the US by reducing inappropriately broad empiric antibiotic selection. National surveillance of drug-related adverse events estimated that even a small reduction in unnecessary antibiotic use could significantly decrease the direct risks of drug-related adverse event in individual patients.12 Thus, our guideline-based measure has the potential to improve patient care for a large number of patients hospitalized with CAP across the US.
The full reference list can be found in Section 2.2.
The anticipated data source for the measure is electronic health record (EHR) data from inpatient hospital admissions, including discharge diagnosis codes, pharmacy and medication administration, and imaging records. These data are all collected routinely during usual clinical care through the process of inpatient hospitalizations.
Numerator
From the denominator population, identify patients without risk factors for methicillin-resistant Staphylococcus aureus (MRSA) or Pseudomonas aeruginosa who received empiric antibiotics targeting MRSA or Pseudomonas.
From the denominator population, identify who received empiric (within first 48 hours of emergency department arrival) antibiotics targeting MRSA or Pseudomonas
- Broad Spectrum Antibiotics for MDRO Value Set, OID:2.16.840.1.113762.1.4.1264.29
- (Sheet Name: “BroadSpectrumAntibioticsMDRO”)
Denominator
Identify all adult patients with an inpatient (or observation) non-ICU hospitalization in which the discharge diagnosis includes pneumonia or sepsis/respiratory failure (RF) who received a respiratory antibiotic within 48 hours of hospitalization, received chest imaging within +/- 3 days of the hospital encounter, were not transferred from another hospital, and do not have a concurrent infection. Restrict to patients with uncomplicated pneumonia and without risk factors for MRSA or Pseudomonas.
Specific Inclusion Criteria (see Data Dictionary for detailed specifications):
- ICD10 discharge code for pneumonia or sepsis AND respiratory failure
- CAP, Sepsis, Respiratory Failure Diagnostic Value Set, OID:2.16.840.1.113762.1.4.1264.24
- (Sheet Name: “CAPSepsisRespFailureDx”)
- Received a respiratory antibiotic within the first 48 hours of hospitalization
- Antibiotic Usage for CAP Diagnosis Value Set, OID: 2.16.840.1.113762.1.4.1264.25
- (Sheet Name: “AntibioticUsageforCAP”)
- Age > 18 years on admission
- Admitted as inpatient or in observation awaiting inpatient admission
- Encounter Inpatient Value Set, OID: 2.16.840.1.113883.3.666.5.307
- (Sheet Name: “Encounterinpatient”)
- Did not die in first 48 hours of hospitalization
- code "Dead (finding)": '419099009' from "SNOMEDCT" display 'Dead (finding)'
- No time spent in an ICU in first 48 hours of hospitalization
- Intensive Care Unit Value Set, OID:2.16.840.1.113762.1.4.1029.204
- (Sheet Name: “IntensiveCareUnit”
- Not admitted from another acute care hospital or long-term acute care facility
- code "Hospital admission, transfer from other hospital or health care facility (procedure)": '4563007' from "SNOMEDCT" display 'Hospital admission, transfer from other hospital or health care facility (procedure)'
- No concurrent infection during hospitalization by ICD 10
- CAP, Concurrent Infections Value Set, OID: 2.16.840.1.113762.1.4.1264.23
- (Sheet Name: “CAPConcurentInfections”)
- Chest imaging within 3 days of admission
‘Sheet Name’ refers to specific tab in the Data Dictionary
*note–all times are calculated from arrival to the emergency department
Exclusions
To identify uncomplicated CAP patients: Exclude if:
- On mechanical ventilation in first 48 hours
- Absolute neutrophil count (ANC) <500 cells/uL
- Cystic fibrosis
- Bronchiectasis
- HIV
- Tracheostomy
- Transplant in prior year
- Hematologic malignancy
- Pulmonary complication (empyema, lung abscess, necrotizing pneumonia
To identify uncomplicated CAP patients: Exclude if:
- On mechanical ventilation in first 48 hours
- Comorbidities Indicated with CAP Value Set, OID:2.16.840.1.113762.1.4.1264.26
- (Sheet Name: “ComrobiditiesIndicatedwithCAP”)
- Absolute neutrophil count <500 cells/uL
- Complete Blood Count (with Diff), OID:1.3.6.1.4.1.6997.4.1.2.271.13.38167.1.1.999.594
- code "Neutrophils [#/volume] in Blood": '26499-4' from "LOINC" display 'Neutrophils [#/volume] in Blood'
- (Sheet Name: (“CompleteBloodCount”)
- Cystic fibrosis
- Comorbidities Indicated with CAP Value Set, OID:2.16.840.1.113762.1.4.1264.26
- (Sheet Name: “ComrobiditiesIndicatedwithCAP”)
- Bronchiectasis
- Comorbidities Indicated with CAP Value Set, OID:2.16.840.1.113762.1.4.1264.26
- (Sheet Name: “ComrobiditiesIndicatedwithCAP”)
- Human immunodeficiency virus (HIV)
- code "Human immunodeficiency virus [HIV] disease": 'B20' from "ICD10CM" display 'Human immunodeficiency virus [HIV] disease'
- code "Patient immunocompromised (finding)": '370388006' from "SNOMEDCT" display 'Patient immunocompromised (finding)' (if facility cannot report HIV status)
- Tracheostomy
- Comorbidities Indicated with CAP Value Set, OID:2.16.840.1.113762.1.4.1264.26
- (Sheet Name: “ComrobiditiesIndicatedwithCAP”)
- Transplant in prior year
- Major Transplant Value Set, OID:2.16.840.1.113883.3.464.1003.198.12.1075
- (Sheet Name: “MajorTransplant”)
- Hematologic malignancy
- Comorbidities Indicated with CAP Value Set, OID:2.16.840.1.113762.1.4.1264.26
- (Sheet Name: “ComrobiditiesIndicatedwithCAP”)
- Pulmonary complication (empyema, lung abscess, necrotizing pneumonia)
- Comorbidities Indicated with CAP Value Set, OID:2.16.840.1.113762.1.4.1264.26
- (Sheet Name: “ComrobiditiesIndicatedwithCAP”)
Exclude patients with risk factors for MRSA/Pseudomonas aeruginosa:
- Pseudomonas aeruginosa in respiratory culture in prior year
- Pseudomonas aeruginosa (Organism or Substance in Lab Results) Value Set, OID:2.16.840.1.113762.1.4.1146.1679
- (Sheet Name: “PseudomonasAeruginosa”)
- Methicillin-resistantStaphylococcus aureus in respiratory culture in prior year
- MRSA (Disorders(SNOMED), OID:2.16.840.1.113762.1.4.1146.1447
- (Sheet Name: “MRSADisorders”)
Severe pneumonia AND prior hospitalization with IV antibiotics in past 3 months
Severe pneumonia defined by ≥2 of the following on a single day in the first 48 hours of encounter:
- Respiratory rate >30 breaths/min
- Respiratory Rate Value Set, OID:2.16.840.1.113762.1.4.1045.130
- (Sheet Name: “RespiratoryRate”)
- ≥5L oxygen or SpO2<90
- code "Inhaled oxygen flow rate": '3151-8' from "LOINC" display 'Inhaled oxygen flow rate' or
- Oxygen Saturation in Blood Value Set, OID:2.16.840.1.113762.1.4.1222.1610
- (Sheet Name: “OxygenSaturation”
- White blood cell (WBC) <4000 cell/uL
- Complete Blood Count (with Diff) Value Set, OID:1.3.6.1.4.1.6997.4.1.2.271.13.38167.1.1.999.594
- (Sheet Name: “CompleteBloodCount”)
- Blood urea nitrogen (BUN) >20 mg/dL
- Blood Urea Nitrogen Value Set, OID:2.16.840.1.113762.1.4.1222.113
- (Sheet Name: “BloodUreaNitrogen”)
- Platelets <100 K/uL
- Complete Blood Count (with Diff) Value Set, OID:1.3.6.1.4.1.6997.4.1.2.271.13.38167.1.1.999.594
- (Sheet Name: “CompleteBloodCount”)
- Temperature <36 C
- Body Temperature Value Set, OID:2.16.840.1.113762.1.4.1222.113
- (Sheet Name: “BodyTemperature”)
- Systolic blood pressure (SBP) <80 mmHg
- Systolic Blood Pressure Value Set, OID:2.16.840.1.113762.1.4.1045.163
- (Sheet Name: “SystolicBloodPressure”)
AND
Prior hospitalization with IV antibiotics in past 3 months
- Prior hospitalization
- code "Hospital admission, transfer from other hospital or health care facility (procedure)": '4563007' from "SNOMEDCT" display 'Hospital admission, transfer from other hospital or health care facility (procedure)'
- IV antibiotics
- Intravenous Medication Route Value Set, OID:2.16.840.1.113762.1.4.1190.75
- (Sheet Name: “IntravenousMedicationRoute”)
‘Sheet Name’ refers to specific tab in the Data Dictionary
*note–all times are calculated from arrival to the emergency department
Measure Calculation
Step 1: Define Initial Population and Denominator
Step 2: Apply denominator exclusions
Step 3: Define the numerator
Step 4: Calculate the measure score
The measure is not stratified.
Table 3 (see Supplemental Materials attachment) indicates the minimum annual number of qualifying cases needed for the denominator to reach each target reliability level at a given facility. In order to achieve a desired reliability of 0.8, each hospital would need to include 96 cases annually. For acceptable reliability (0.7), 56 annual cases would be required, and for high reliability (0.9), 216 annual cases would be required. If a facility has fewer than the minimum number, the hospital is still encouraged to report performance on this measure.
We estimated the minimum sample size using the ICC (intraclass correlation) and the Spearman Brown prophecy formula13-15 The ICC was calculated from 109 Veterans Affairs (VA) healthcare systems. The Spearman Brown formula was used in prior inpatient CAP measures to determine minimal sample size (PN-3a: Blood Cultures Performed within 24 Hours Prior to or 24 Hours After Hospital Arrival for Patients who were Transferred or Admitted to the ICU within 24 Hours of Hospital Arrival; PN-6: Initial Antibiotic Selection for Community-Acquired Pneumonia [CAP in Immunocompetent Patients]; Inappropriate Diagnosis of CAP [CBE ID 3671]); Percent of Hospitalized Pneumonia Patients with Chest Imaging [CBE ID 4440e]).16-18 The minimum recommended sample sizes here are within the range of those previously required for inpatient CAP–for example, the minimum recommended sample size to achieve 0.80 reliability was 59 for the Inappropriate Diagnosis of Community-Acquired Pneumonia measure17 and 174 for the Percent of Hospitalized Pneumonia Patients with Chest Imaging measure, an eCQM endorsed in 2024.16
The full reference list can be found in Section 2.2.
Supplemental Attachment
Measure Record
Point of Contact
n/a
Michael Good
Salt Lake City, UT
United States
Andrea White
University of Utah, Division of General Internal Medicine
Salt Lake City, UT
United States
Importance
Evidence
Pneumonia is not only the most common reason for inpatient antibiotic use but also the most common infectious cause of mortality in the US resulting in approximately 1.4 million emergency department visits, 740,000 hospitalizations, 41,000 deaths, and $7.7 billion in inpatient costs each year in the US.3-6In a recent report on non-ICU hospitalized CAP patients from 67 Michigan hospitals, date collected using the chart review measure from which our eCQM was adapted showed that of 8,286 CAP patients, 2,215 (26.7%) received inappropriately broad empiric treatment (i.e., eligible for standard CAP coverage per ATS/IDSA guidelines). These patients had higher 30-day readmissions, more transfers to ICU and antibiotic-associated adverse events, and longer hospitalizations.11 Similarly, a retrospective analysis of 88,605 patients with CAP across the Veterans Health Administration health care system found that empirical anti-MRSA (in addition to standard CAP therapy) was associated with an increased adjusted risk of death, kidney injury, and secondary infections (C. difficile, vancomycin-resistant Enterococcus infection, and resistant gram-negative rod infections).8 Notably, associations between anti-MRSA therapy and 30-day mortality were similar among patients admitted to the ICU, those with high risk for MRSA, and those with MRSA detected on surveillance testing, suggesting that empiric anti-MRSA therapy was not associated with reduced mortality for any pneumonia patient group.8 Using data reported to the National Healthcare Safety Network (NHSN) antimicrobial use (AU) option, the effect of reducing inpatient broad-spectrum antibiotic exposure was modeled and estimated that reduction in use of broad-spectrum antibiotics by 30% could result in a 26% reduction in C. difficile infection.2 Similarly, broad spectrum antibiotic use is associated with acquisition of multi-drug resistant organisms and transmission of those organisms in nursing homes.19 Taken together, our guideline-based measure has the ability to improve patient care for a large number of patients hospitalized with CAP across the US.
The eCQM for CAP empiric antibiotic selection complements other quality efforts, including sepsis-related measures, which aim to increase early use of antibiotic therapy for patients with suspected infection. This metric targets CAP—the largest cause of sepsis—and specifically addresses correct empiric therapy and avoiding inappropriately broad empiric antibiotic selection (i.e., therapy targeting MRSA or Pseudomonas aeruginosa in patients without risk factors), not just timing of antibiotic use. In addition, we anticipate this measure can augment the NHSN AU module (which assesses overall observed vs. expected antibiotic use in hospitals but not antibiotic appropriateness) and help hospitals understand the quality of their prescribing for the most common infection treated in hospitals–CAP.
References (for full submission, not just 2.2 Evidence of Measure Importance):
1. Antimicrobial Resistance Collaborators. Global burden of bacterial antimicrobial resistance in 2019: a systematic analysis. Lancet. Feb 12 2022;399(10325):629-655. doi:10.1016/s0140-6736(21)02724-0
2. Fridkin S, Baggs J, Fagan R, et al. Vital signs: improving antibiotic use among hospitalized patients. MMWR Morbidity and mortality weekly report. Mar 7 2014;63(9):194-200.
3. Sterrantino C, Trifirò G, Lapi F, et al. Burden of community-acquired pneumonia in Italian general practice. Eur Respir J. Dec 2013;42(6):1739-42. doi:10.1183/09031936.00128713
4. Partouche H, Lepoutre A, Vaure CBD, Poisson T, Toubiana L, Gilberg S. Incidence of all-cause adult community-acquired pneumonia in primary care settings in France. Med Mal Infect. Sep 2018;48(6):389-395. doi:10.1016/j.medmal.2018.02.012
5. McDermott KW aRM. Most Frequent Principal Diagnoses for Inpatient Stays in U.S. Hospitals, 2018. 2021. https://hcup-us.ahrq.gov/reports/statbriefs/sb277-Top-Reasons-Hospital-Stays-2018.pdf#:~:text=The%20most%20frequent%20principal%20diagnoses%20for%20hospitalizations%20in,caused%20by%20tuberculosis%29%2C%20and%20diabetes%20mellitus%20with%20complication
6. Vaughn VM, Dickson RP, Horowitz JK, Flanders SA. Community-Acquired Pneumonia: A Review. Jama. Oct 15 2024;332(15):1282-1295. doi:10.1001/jama.2024.14796
7. Branch-Elliman W, O’Brien W, Strymish J, Itani K, Wyatt C, Gupta K. Association of Duration and Type of Surgical Prophylaxis With Antimicrobial-Associated Adverse Events. JAMA Surgery. 2019;154(7):590-598. doi:10.1001/jamasurg.2019.0569
8. Jones BE, Ying J, Stevens V, et al. Empirical Anti-MRSA vs Standard Antibiotic Therapy and Risk of 30-Day Mortality in Patients Hospitalized for Pneumonia. JAMA Intern Med. Apr 1 2020;180(4):552-560. doi:10.1001/jamainternmed.2019.7495
9. Attridge RT, Frei CR, Restrepo MI, et al. Guideline-concordant therapy and outcomes in healthcare-associated pneumonia. Eur Respir J. Oct 2011;38(4):878-87. doi:10.1183/09031936.00141110
10. Metlay JP, Waterer GW, Long AC, et al. Diagnosis and Treatment of Adults with Community-acquired Pneumonia. An Official Clinical Practice Guideline of the American Thoracic Society and Infectious Diseases Society of America. Am J Respir Crit Care Med. Oct 1 2019;200(7):e45-e67. doi:10.1164/rccm.201908-1581ST
11. Gandhi T, Petty L, Vaughn V, et al. Risk Factors and outcomes associated with inappropriate empiric broad-spectrum antibiotic use in hospitalized patients with community-acquired pneumonia. Antimicrobial Stewardship & Healthcare Epidemiology. 2023;3(S2):s31-s32. doi:10.1017/ash.2023.258
12. Shehab N, Patel PR, Srinivasan A, Budnitz DS. Emergency department visits for antibiotic-associated adverse events. Clin Infect Dis. Sep 15 2008;47(6):735-43. doi:10.1086/591126
13. van Ast JF, Talmon JL, Renier WO, Hasman A. An approach to knowledge base construction based on expert opinions. Methods Inf Med. 2004;43(4):427-32.
14. Zimmerman DW, Williams RH, Burkheimer GJ. Dependence of reliability of multiple-choice tests upon number of choices per item: prediction from the Spearman-Brown formula. Psychol Rep. Dec 1966;19(3):1239-43. doi:10.2466/pr0.1966.19.3f.1239
15. de Vet HCW, Mokkink LB, Mosmuller DG, Terwee CB. Spearman-Brown prophecy formula and Cronbach's alpha: different faces of reliability and opportunities for new applications. J Clin Epidemiol. May 2017;85:45-49. doi:10.1016/j.jclinepi.2017.01.013
16. Partnership for Quality Measurement. Percent of hospitalized pneumonia patients with chest imaging confirmation, CBE ID 4440e. Accessed 9/25/2024, https://p4qm.org/measures/4440e
17. Partnership for Quality Measurement. Inappropriate diagnosis of community-acquired pneumonia (CAP) in hospitalized medical patients, CBE ID 3671. Accessed 9/24/2024, https://p4qm.org/measures/3671
18. The Joint Commission. Specifications manual for Joint Commission National Quality Measures (v2015A): Pneumonia (PN). Accessed 9/25/2024, https://manual.jointcommission.org/releases/TJC2015A/Pneumonia.html#A_42Pneumonia_40PN_41_Initial_Patient_Population_42
19. Gontjes KJ, Gibson KE, Lansing BJ, et al. Association of Exposure to High-risk Antibiotics in Acute Care Hospitals With Multidrug-Resistant Organism Burden in Nursing Homes. JAMA Netw Open. Feb 1 2022;5(2):e2144959. doi:10.1001/jamanetworkopen.2021.44959
20. Graber CJ, Jones MM, Goetz MB, et al. Decreases in Antimicrobial Use Associated With Multihospital Implementation of Electronic Antimicrobial Stewardship Tools. Clin Infect Dis. Aug 22 2020;71(5):1168-1176. doi:10.1093/cid/ciz941
21. Kelly AA, Jones MM, Echevarria KL, et al. A Report of the Efforts of the Veterans Health Administration National Antimicrobial Stewardship Initiative. Infect Control Hosp Epidemiol. May 2017;38(5):513-520. doi:10.1017/ice.2016.328
22. Goetz MB, Willson T, Rubin MA, Stevens VW, Graber CJ. Antimicrobial use before and during COVID-19: data from 108 Veterans Affairs medical centers. Antimicrobial Stewardship & Healthcare Epidemiology. 2024;4(1):e109. e109. doi:10.1017/ash.2024.352
23. Spivak ES, Burk M, Zhang R, et al. Management of Bacteriuria in Veterans Affairs Hospitals. Clin Infect Dis. Sep 15 2017;65(6):910-917. doi:10.1093/cid/cix474
24. Sutton JD, Carico R, Burk M, et al. Inpatient Management of Uncomplicated Skin and Soft Tissue Infections in 34 Veterans Affairs Medical Centers: A Medication Use Evaluation. Open Forum Infect Dis. Jan 2020;7(1):ofz554. doi:10.1093/ofid/ofz554
25. Madaras-Kelly KJ, Burk M, Caplinger C, et al. Total duration of antimicrobial therapy in veterans hospitalized with uncomplicated pneumonia: Results of a national medication utilization evaluation. Journal of hospital medicine. Dec 2016;11(12):832-839. doi:10.1002/jhm.2648
26. Livorsi DJ, Branch-Elliman W, Drekonja D, et al. Research agenda for antibiotic stewardship within the Veterans' Health Administration, 2024-2028. Infect Control Hosp Epidemiol. Feb 2 2024:1-7. doi:10.1017/ice.2024.6
27. Qian ET, Casey JD, Wright A, et al. Cefepime vs Piperacillin-Tazobactam in Adults Hospitalized With Acute Infection: The ACORN Randomized Clinical Trial. Jama. Oct 24 2023;330(16):1557-1567. doi:10.1001/jama.2023.20583
28. Ciarkowski CE, Timbrook TT, Kukhareva PV, et al. A Pathway for Community-Acquired Pneumonia With Rapid Conversion to Oral Therapy Improves Health Care Value. Open Forum Infect Dis. Nov 2020;7(11):ofaa497. doi:10.1093/ofid/ofaa497
29. Shorr AF, Owens RC, Jr. Guidelines and quality for community-acquired pneumonia: measures from the Joint Commission and the Centers for Medicare and Medicaid Services. American journal of health-system pharmacy. Jun 15 2009;66(12 Suppl 4):S2-7. doi:10.2146/090087a
30. Centers for Medicare and Medicaid. Merit-Based Incentive Payment System (MIPS): Simple Pneumonia with Hospitalization Measure, 2020 performance period. Accessed 9/27/2024, https://qpp.cms.gov/docs/cost_specifications/2019-12-17-mif-ebcm-pna-hosp.pdf
31. Centers for Medicare & Medicaid Services. Quality ID #111: Pneumococcal vaccination status for older adults. . Accessed 9/27/2024, https://qpp.cms.gov/docs/QPP_quality_measure_specifications/CQM-Measures/2023_Measure_111_MedicarePartBClaims.pdf
32. Centers for Medicare & Medicaid Services. Eligible Hospital / Critical Access Hospital eCQMs: hospital quality reporting table of eCQMs. Accessed 9/27/2024, https://ecqi.healthit.gov/sites/default/files/Hybrid-EH-CAH-MeasuresTable-2023-11.pdf
33. Agency for Healthcare Research and Quality. AHRQ IQI technical documentation, version v2023. Rockville, MD. Accessed 9/27/2024, https://qualityindicators.ahrq.gov/measures/iqi_resources
34. CDC. Core Elements of Hospital Antibiotic Stewardship Programs. US Department of Health and Human Services. CDC. Accessed August 3, 2022. https://www.cdc.gov/antibiotic-use/healthcare/pdfs/hospital-core-elements-H.pdf
35. The Joint Commission. R3 Report Issue 35: New and Revised Requirements for Antibiotic Stewardship 2022. https://www.jointcommission.org/-/media/tjc/documents/standards/r3-reports/r3_antibioticstewardship_july2022_final.pdf
36. QSO-22-20-Hospitals Infection Prevention and Control and Antibiotic Stewardship Program Interpretive Guidance Update (2022).
37. van Santen KL, Edwards JR, Webb AK, et al. The Standardized Antimicrobial Administration Ratio: A New Metric for Measuring and Comparing Antibiotic Use. Clin Infect Dis. Jul 2 2018;67(2):179-185. doi:10.1093/cid/ciy075
38. Department of Health and Human Services Centers for Medicare & Medicaid Services. Hospital Inpatient Prospective Payment Systems for Acute Care Hospitals and the Long-Term Care Hospital Prospective Payment System and Policy Changes and Fiscal Year 2023 Rates; Quality Programs and Medicare Promoting Interoperability Program Requirements for Eligible Hospitals and Critical Access Hospitals; Costs Incurred for Qualified and Non-Qualified Deferred Compensation Pland; and Changes to Hospital and Critical Access Hospital Conditions of Participation. Federal Register. 2022;87 (153):487780-49499.
39. Moehring R, Vaughn VM. Development of Inpatient Stewardship Metrics: Is It Time for Public Reporting? Infect Dis Clin North Am. Dec 2023;37(4):853-871. doi:10.1016/j.idc.2023.07.006
40. Watkins RR, Lemonovich TL. Diagnosis and management of community-acquired pneumonia in adults. Am Fam Physician. Jun 1 2011;83(11):1299-306.
41. NHSN. Antimicrobial Use and Resistance (AUR) Module. Centers for Disease Control. https://www.cdc.gov/nhsn/pdfs/pscmanual/11pscaurcurrent.pdf
42. Fishman N, America SfHEo, America IDSo. Policy statement on antimicrobial stewardship by the Society for Healthcare Epidemiology of America (SHEA), the Infectious Diseases Society of America (IDSA), and the Pediatric Infectious Diseases Society (PIDS). Infection Control & Hospital Epidemiology. 2012;33(4):322-327.
43. Barlam TF, Cosgrove SE, Abbo LM, et al. Implementing an Antibiotic Stewardship Program: Guidelines by the Infectious Diseases Society of America and the Society for Healthcare Epidemiology of America. Clin Infect Dis. May 15 2016;62(10):e51-77. doi:10.1093/cid/ciw118
44. Macfarlane J, Holmes W, Macfarlane R, Britten N. Influence of patients' expectations on antibiotic management of acute lower respiratory tract illness in general practice: questionnaire study. BMJ (Clinical research ed). 1997;315(7117):1211-1214. doi:10.1136/bmj.315.7117.1211
45. Borek AJ, Campbell A, Dent E, et al. Implementing interventions to reduce antibiotic use: a qualitative study in high-prescribing practices. BMC Fam Pract. Jan 23 2021;22(1):25. doi:10.1186/s12875-021-01371-6
46. Tonkin-Crine S, Anthierens S, Francis NA, et al. Exploring patients' views of primary care consultations with contrasting interventions for acute cough: a six-country European qualitative study. NPJ Prim Care Respir Med. Jul 17 2014;24:14026. doi:10.1038/npjpcrm.2014.26
47. Centers for Disease Control and Prevention. NHSN list of common commensals. Accessed 9/30/2024, https://view.officeapps.live.com/op/view.aspx?src=https%3A%2F%2Fwww.cdc.gov%2Fnhsn%2FXLS%2Fmaster-organism-Com-Commensals-Lists.xlsx&wdOrigin=BROWSELINK)
48. Mandell LA, Wunderink RG, Anzueto A, et al. Infectious Diseases Society of America/American Thoracic Society consensus guidelines on the management of community-acquired pneumonia in adults. Clin Infect Dis. Mar 1 2007;44 Suppl 2:S27-72. doi:10.1086/511159
49. Gupta AB, Flanders SA, Petty LA, et al. Inappropriate Diagnosis of Pneumonia Among Hospitalized Adults. JAMA Intern Med. May 1 2024;184(5):548-556. doi:10.1001/jamainternmed.2024.0077
50. Vaughn VM, Gandhi TN, Hofer TP, et al. A Statewide Collaborative Quality Initiative to Improve Antibiotic Duration and Outcomes in Patients Hospitalized With Uncomplicated Community-Acquired Pneumonia. Clin Infect Dis. Aug 31 2022;75(3):460-467. doi:10.1093/cid/ciab950
51. Gupta A, Petty L, Gandhi T, et al. Overdiagnosis of urinary tract infection linked to overdiagnosis of pneumonia: a multihospital cohort study. BMJ Qual Saf. May 2022;31(5):383-386. doi:10.1136/bmjqs-2021-013565
52. Vaughn VM, Gandhi TN, Chopra V, et al. Antibiotic Overuse After Hospital Discharge: A Multi-hospital Cohort Study. Clin Infect Dis. Dec 6 2021;73(11):e4499-e4506. doi:10.1093/cid/ciaa1372
53. Gandhi TN, Vaughn VM, Petty LA, et al. 2893. The Michigan Hospital Medicine Safety Consortium: Improving Patient Care by Reducing Excessive Antibiotic Use in Patients Hospitalized with Community-Acquired Pneumonia. Open Forum Infectious Diseases. 2019;6(Supplement_2):S80-S81. doi:10.1093/ofid/ofz359.171
54. White AT, Vaughn V, Petty LA, et al. Development of patient safety measures to identify inappropriate diagnosis of common infections. Clin Infect Dis. Jan 31 2024;doi:10.1093/cid/ciae044
55. Mehta HB, Li S, An H, Goodwin JS, Alexander GC, Segal JB. Development and Validation of the Summary Elixhauser Comorbidity Score for Use With ICD-10-CM-Coded Data Among Older Adults. Annals of internal medicine. Oct 2022;175(10):1423-1430. doi:10.7326/m21-4204
56. National Quality Forum. Scientific Methods Panel Evaluation Guidelines. Accessed 10/7/2024, https://www.qualityforum.org/Measuring_Performance/Scientific_Methods_Panel.aspx
57. Centers for Medicare & Medicaid Services. MMS Blueprint Supplemental Material: Technical Expert Panels. https://www.cms.gov/files/document/blueprint-technical-expert-panels.pdf
58. Centers for Medicare & Medicaid Services. Measures Management System Risk Adjustment. Centers for Medicare & Medicaid. Accessed 8/20/24, https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/MMS/Downloads/Risk-Adjustment.pdf
59. Ramirez JA, Wiemken TL, Peyrani P, et al. Adults Hospitalized With Pneumonia in the United States: Incidence, Epidemiology, and Mortality. Clin Infect Dis. Nov 13 2017;65(11):1806-1812. doi:10.1093/cid/cix647
60. Burton DC, Flannery B, Bennett NM, et al. Socioeconomic and racial/ethnic disparities in the incidence of bacteremic pneumonia among US adults. Am J Public Health. Oct 2010;100(10):1904-11. doi:10.2105/ajph.2009.181313
61. Altawalbeh SM, Wateska AR, Nowalk MP, et al. Societal Cost of Racial Pneumococcal Disease Disparities in US Adults Aged 50 Years or Older. Appl Health Econ Health Policy. Jan 2024;22(1):61-71. doi:10.1007/s40258-023-00854-0
62. Nowalk MP, Wateska AR, Lin CJ, et al. Racial Disparities in Adult Pneumococcal Vaccination Indications and Pneumococcal Hospitalizations in the U.S. J Natl Med Assoc. Oct 2019;111(5):540-545. doi:10.1016/j.jnma.2019.04.011
63. Hausmann LR, Ibrahim SA, Mehrotra A, et al. Racial and ethnic disparities in pneumonia treatment and mortality. Med Care. Sep 2009;47(9):1009-17. doi:10.1097/MLR.0b013e3181a80fdc
64. Wiemken TL, Carrico RM, Furmanek SP, et al. Socioeconomic Position and the Incidence, Severity, and Clinical Outcomes of Hospitalized Patients With Community-Acquired Pneumonia. Public Health Rep. May/Jun 2020;135(3):364-371. doi:10.1177/0033354920912717
65. Downing NS, Wang C, Gupta A, et al. Association of Racial and Socioeconomic Disparities With Outcomes Among Patients Hospitalized With Acute Myocardial Infarction, Heart Failure, and Pneumonia: An Analysis of Within- and Between-Hospital Variation. JAMA Netw Open. Sep 7 2018;1(5):e182044. doi:10.1001/jamanetworkopen.2018.2044
Measure Impact
The 2019 ATS/IDSA CAP guidelines recommend not prescribing empiric anti-MRSA or anti-Pseudomonal therapy in uncomplicated CAP patients without risk factors for MRSA or Pseudomonas aeruginosa.10 Similarly, those guidelines recognized that prior “HCAP” (healthcare associated pneumonia) criteria had recommended anti-MRSA or anti-Pseudomonal therapy in patients for whom it was unnecessary—leading to a rapid increase in anti-MRSA or anti-Pseudomonal therapy without improved patient outcomes. Despite these updated recommendations, de-adoption of widespread use of anti-MRSA and anti-Pseudomonal therapy for CAP has been slow.
If the currently proposed measure is implemented, we expect a fairly rapid reduction in inappropriately broad empiric therapy in uncomplicated CAP patients without risk factors for MRSA or Pseudomonas aeruginosa given demonstrated improvements in empiric antibiotic use with the following large-scale measurement and antibiotic stewardship efforts:
- When the chart review version of the measure on which this eCQM was based was used as a pay-for-performance measure across 69 Michigan hospitals, inappropriate empiric broad spectrum antibiotic use saw a 42% relative decrease (from ~20% across the collaborative to ~11.6%; see Figure 1 in Supplemental Materials attachment). No unintended consequences were identified.
- The VA healthcare system (among other hospitals) has implemented interventions targeted at reducing inappropriately broad empiric therapy for uncomplicated CAP.11,20,21 Substantial decreases in antibiotic use have been demonstrated in the VA since the establishment of the national VA Antimicrobial Stewardship Task Force in 201121 that have persisted through the COVID-19 pandemic.22 Specific to the inappropriately broad empiric antibiotic use for CAP eCQM, there have been educational campaigns to improve concordance with guideline recommended therapy in the VA but no formal implementation of the measure. Regardless, with education and stewardship interventions, the proportion of patients receiving empiric anti-Pseudomonal therapy or anti-MRSA therapy has decreased from 2015 to early 2024 (see Figure 2, Supplemental Materials attachment). As a result, there have been significant reductions in methicillin resistance in S aureus, vancomycin-resistance in E faecium, and fluoroquinolone resistance across multiple pathogens.20-26
By identifying and improving the percentage of hospitalized CAP patients who receive non-guideline concordant overtreatment with anti-MRSA or anti-Pseudomonal therapy, our proposed eCQM is expected to have direct impact on patients by reducing antibiotic-associated adverse events and reducing development/acquisition of antibiotic resistance. Specifically, anti-MRSA has been associated with death, acute kidney injury, Clostridioides difficile infection, vancomycin-resistant enterococcal infection, and gram-negative rod infections and combinations of anti-MRSA and anti-Pseudomonal therapy have been associated with acute kidney injury and neurotoxicity.8,11,27 We also anticipate that reducing antibiotic overuse for the most common inpatient indication for antibiotics will have a large impact on antimicrobial resistance. Using data reported to the National Healthcare Safety Network (NHSN) antimicrobial use option, the effect of reducing inpatient broad-spectrum antibiotic exposure was modeled and estimated that reduction in use of broad-spectrum antibiotics by 30% could result in a 26% reduction in Clostridioides difficile infection.2
Potential unintended consequences include under-treatment of patients who require anti-MRSA or anti-Pseudomonal therapy. During initial measure development, we considered assessing undertreatment as part of this measure. We asked our Technical Expert Panel (TEP, 8 clinicians representing 8 different clinical specialties and national organizations [Table 4 in the Supplemental Materials attachment]) if the measure should evaluate under-treatment, and 87.5% (7/8) of them preferred eliminating under-treatment as an assessment. TEP feedback included:
- “Focus only on those who should receive non-MDRO therapy”
- “Eliminat[e] anyone who qualifies for MDRO therapy & focus on pts who do not have risk factors for MDRO therapy”
- “Exclude those with indication for MDR[O] therapy and focus on overuse”
- “There is no perfect measure here but I think simplifying the measure by taking out those who qualify for MDRO treatment would make it cleaner & more fair.”
“By concentrating on overuse- you could swing to underuse - but suspect unlikely. [looking at over and underuse] just seems too complicated to actually be used in practice- even if valid.”
Thus, we decided, based on TEP feedback, to assess only patients without a guideline-based indication for empiric anti-MRSA or anti-Pseudomonal therapy. Notably, none of the implementation studies above identified any unintended consequences during their efforts to reduce inappropriate broad spectrum antibiotic use for CAP. That said, we plan to evaluate unanticipated consequences during the measure maintenance submission.
The full reference list can be found in Section 2.2.
Pneumonia is a well-established target of clinical quality and safety monitoring. Table 6 (see Supplemental Materials attachment) lists existing and prior measures for pneumonia as well as existing measures for antibiotic use that may be influenced by measures of treatment of pneumonia.18,29-33
The CDC,34 The Joint Commission,35 and Centers for Medicare and Medicaid Services36 recommend that antibiotic stewardship programs implement interventions to promote guideline concordant empiric antibiotic selection.
Currently, there is one active quality measure related to antibiotic use that includes patients with CAP—the CDC’s NHSN AU Option. The NHSN provides a risk-adjusted benchmark comparison for participating hospitals to use to drive improvement.37 The standardized antimicrobial administration ratio (SAAR) was introduced in 2015 and is based on antibiotic days of therapy (DOT), similar in structure to risk-adjusted comparisons of health care associated infection rates. SAAR is a ratio of observed to expected DOT for a particular antimicrobial agent category and location. The 2023 CMS Inpatient Prospective Payment System rule requires NHSN AU reporting for US acute care hospitals in the Promoting Interoperability Program. Hospitals that do not meet requirements could lose funding incentives for promoting interoperability.38 The start date for enforcement of reporting will begin in late 2025. Though AU reporting will be required, hospital-specific data will be neither publicly reported nor used in pay-for-performance programs. Regardless, this mandate means that routine assessments of risk-adjusted AU comparisons to a national benchmark are now a reality for most US hospitals. As of September 2024, 4087 hospitals had reported to the NHSN AU option—a number that will continue to grow now that reporting is required.39
Notably, NHSN AU data do not evaluate for antibiotic appropriateness. Thus, we anticipate this eCQM being used in concert with the NHSN AU module to help hospitals and their antimicrobial stewardship teams understand how to improve care. We have collaborated with the CDC in designing this measure with the goal of working within their NHSN AU framework (when NHSN moves to patient-level data collection).
NHSN AU data are broad antibiotic-use metrics. No eCQMs specifically for CAP currently exist;32 though one related to pneumonia diagnosis was recently endorsed.16 A chart-review based version of this empiric antibiotic use measure has been in use in 69 Michigan hospitals as a pay-for-performance measure where it has demonstrably improved antibiotic use (see use data within this measure). By converting the empiric antibiotic use measure into an eCQM, we hope to enable broader implementation of the improvements necessary to optimize empiric antibiotic use and improve care of hospitalized patients with CAP.
The full reference list can be found in Section 2.2.
We explored the value of the pneumonia antibiotic treatment quality through two approaches: 1) literature review of guidelines, clinician and patient perspectives and 2) engagement with clinicians in technical expert panels.
As noted above, multiple national organizations consider appropriateness of antibiotic therapy for pneumonia as a priority area–including the CDC, The Joint Commission, and Centers for Medicare and Medicaid Services. Broadly, improving empiric antibiotic selection is considered part of the definition of antibiotic stewardship–antibiotic stewardship has been defined in a consensus statement from the Infectious Diseases Society of America (IDSA), the Society for Healthcare Epidemiology of America (SHEA), and the Pediatric Infectious Diseases Society (PIDS) as “coordinated interventions designed to improve and measure the appropriate use of [antibiotic] agents by promoting the selection of the optimal [antibiotic] drug regimen.”42 The IDSA in its guidelines for Implementing an Antibiotic Stewardship program provide a strong recommendation (based on moderate quality evidence) stating: “We recommend antibiotic stewardship interventions designed to reduce the use of antibiotics associated with a high risk of CDI compared with no such intervention.”43 They emphasize that reducing CDI is a priority for all ASPs and should be considered when developing stewardship interventions.43
Data specifically from patients about antibiotic selection is limited. In a study of patients presenting with lower respiratory symptoms, 83% believed antibiotics would help and 71% expected antibiotics.44 Paradoxically, qualitative studies have found patients are interested in minimizing exposure to unnecessary antibacterials particularly when objective evidence provides evidence that antibacterials are unnecessary.45,46
Technical Expert Panel (TEP)
The TEP meeting was conducted remotely via videoconference with 8 clinicians representing 8 different clinical specialties and national organizations (see Table 4, Supplemental Materials attachment). An agenda with specific sets of questions tailored to the measure development phase were provided to panelists prior to the discussion session. We also surveyed TEP members after the meeting to obtain their feedback and guidance on the proposed measure.
The importance of developing new measures addressing appropriate treatment of CAP was expressed by members of the TEP. When surveyed, 62.5% (5/8) of our TEP believed it was “important” to “develop an electronic measure of empiric antibiotic therapy for persons diagnosed with CAP” the remaining 37.5% (3/8) reported it was “very important.” All 8 “strongly agreed” that “We should try to reduce anti-MRSA/anti-Pseudomonal therapy in hospitalized patients with CAP without risk factors for MDROs.” All TEP members reported the proposed measure was important (1/8 “very important”, 6/8 “important”, 1/8 “moderately important”) for “patient outcomes (e.g., adverse events, C. difficile infection, antibiotic resistance)?”
The full reference list can be found in Section 2.2.
Performance Gap
In addition to the published studies noted above, we analyzed Veterans Affairs (VA) Computerized Patient Records System (CPRS) data from 109 VA Health Care Systems including all 47,034 eligible hospitalizations between January 1, 2022 and June 30, 2024. Performance scores by decile from those 109 hospitals are shown in Table 1 (individual hospital data in Figure 3, see Supplemental Materials attachment). In addition to VA hospitals, we analyzed University of Utah and University of Michigan hospitalizations using the eCQM applied to their Epic instances (data in Table 5, Supplemental Materials attachment).
Notably, VA and University of Utah estimates are much lower than University of Michigan. This may be because the VA health care system and University of Utah have implemented interventions to improve empiric antibiotic selection for CAP with published success.20,21,28 For example, substantial decreases in antibiotic use have been demonstrated in the VA since the establishment of the national VA Antimicrobial Stewardship Task Force in 201121 that have persisted through the COVID-19 pandemic.21 Similarly, the University of Utah created a CAP orderset that—with antibiotic stewardship efforts—improve empiric antibiotic selection.28 The range of performance across systems further demonstrates a wide gap—and opportunity for improvement—across hospitals and that data-driven antibiotic stewardship efforts can improve antibiotic use.
The full reference list can be found in Section 2.2.
Equity
Equity
Pneumonia disproportionately affects older adults, individuals from underrepresented minority communities, and those with lower socioeconomic status.59-61 Previous research has highlighted disparities in both pneumonia care processes and outcomes.62-65 The proposed eCQM aims to promote health equity by enhancing quality measurement and improving pneumonia-related care, targeting these documented inequalities.
Previously, no differences by patient race, age, gender have been found related to inappropriate broad spectrum antibiotic therapy for CAP. Rather, in the HMS study—on multivariable analysis—factors associated with inappropriate empiric BSA treatment included hospitalization or treatment with high-risk antibiotics in preceding 90 days, transfer from a post-acute care facility, hemodialysis, support with ≥3 L supplemental oxygen, severe sepsis, leukocytosis, and higher pneumonia severity index.11 In the VA study, patients receiving empirical anti-MRSA therapy demonstrated a greater comorbidity burden (renal disease, 29% vs 25%; congestive heart failure, 35% vs 30%; neoplastic disease, 34% vs 3%; and nursing home residents, 9% vs 3%), more risk factors for MRSA (7% vs 2% with history of MRSA infection, 36% vs 12% with previous hospitalization, and 42% vs 29% with previous antibiotics), and greater illness severity (median Pneumonia Severity Index, 124 [interquartile range, 95-156] vs 103 [interquartile range, 81-131]) as well as worse outcomes (16% vs 6% for 30-day all-cause mortality) compared with patients receiving standard therapy alone.8
The full reference list can be found in Section 2.2.
Feasibility
Feasibility
We implemented and tested the proposed eCQM within 3 health systems to assess: 1) whether all required data elements were routinely generated during the care of CAP hospitalizations, 2) which barriers or challenges exist when implementing and extracting the measure from current healthcare data to inform our feasibility scorecard, and 3) whether any elements were uncommon or able to be removed to simplify the measure.
First, all of the data elements collected are already part of normal healthcare system processes for care delivery and billing, so documentation of these items for the proposed eCQM do not add any additional burdens for a healthcare system including clinician workflow, diagnostic thought processes, or patient-physician interactions. During the normal course of care, inpatient clinicians must document within the EHR a discharge diagnosis of pneumonia, order antimicrobials that are recorded in the EHR, and obtain labs, vital signs, and chest imaging that generates a result in the form of structured data that is stored in the EHR. One of our main goals of the measure was to leverage the most reliable, accurate and standardized data elements and to provide a simple, transparent measure that could be consistently extracted, calculated, and interpreted without substantial data resources or expertise. To accomplish this goal, our measure only uses structured data and does not require unstructured data or natural language processing.
As with any eCQM, there is significant work to collate and clean EHR data and ensure their accuracy for measure assessment. To reduce this work, when possible, we used existing value sets—particularly those used by the CDC or NHSN where the measure may be integrated in the future—to streamline use and make the measure more feasible. Examples of harmonized value sets include:
- CDC’s list of common bacterial commensals47 (OID: 2.16.840.1.114222.24.7.281)
- CDC’s definition of Intravenous Medication Route” (OID: 2.16.840.1.113762.1.4.1190.75)
- CDC NHSN to define unit (e.g., ICU; OID: 2.16.840.1.113762.1.4.1029.204)
Data element feasibility testing:
During measure development, we assessed data element feasibility within three different health systems. This included identifying which data elements (originally present in guidelines or in the original chart-review based measure) were difficult to standardize or collect and what (if any) impact their exclusion vs. inclusion would have on measure validity. The results of this process are found in the “feasibility scorecard” and described below in the “feasibility informed measure” section.
The full reference list can be found in Section 2.2.
1) TEP-recommended Change in Measure Scope:
Originally, we anticipated our measure would assess broad spectrum antibiotic therapy for all hospitalized patients with uncomplicated CAP—including those with risk factors for MRSA or Pseudomonas aeruginosa. However, our TEP found it to be too complicated to include patients for whom empiric anti-MRSA or anti-pseudomonal therapy was appropriate. This early version of the measure also included tiers to rate the choice of empiric treatment as: preferred (guideline concordant), potentially justified, and non-guideline concordant. When the TEP was asked to comment on this version of the measure, they rated it as too complex, not feasible, and prone to “gaming.” Representative feedback includes:
- “The tiers are too complex to translate to a measure that is useful/ interpretable/ actionable for improving quality.”
- “I just worry this is overall too complicated and emphasizing overtreatment in those who have no MDRO risk factors is the cleaner thing to implement and focus on change.”
- “Overall I like this tiering but I do worry about being able to develop a fair quality metric from this. There is quite a bit of gray here (which is far too often the case with pneumonia anyway) but with a quality metric, it would be best to make it as black & white as possible.”
When the TEP was asked if the measure should assess only patients for whom anti-MRSA or anti-Pseudomonal therapy is not recommended (i.e., exclude those with risk factors for MRSA or Pseudomonas aeruginosa), 5/8 TEP members reported “strongly agree”; 2/8 reported “agree”; and 1/8 reported “neutral”. TEP comments on this simplified version of the measure included:
- “I think evaluating pts for whom MDRO therapy is not clearly indicated is cleaner and will make this metric easier and more fair. It certainly isn't perfect, but then again what quality metric is perfect!”
“This gets past aforementioned concerns for the first example [first measure version]. Per the data presented this is also 95%+ of the CAP cohort.”
2) Data Elements Removed After Feasibility Testing: We removed the following items from assessment based on the results of feasibility testing.
- CD4 count- Originally we anticipated identifying patients with AIDS based on CD4 count plus a diagnosis of HIV. CD4 count was difficult to identify in all three systems as the results were located in multiple different laboratory tests and often in unstructured fields. There were no ICD10 codes to distinguish AIDS from HIV. Because patients with HIV (regardless of CD4 count) were uncommon in all three health systems, we elected to exclude all patients with HIV rather than require assessment of CD4 count. Patients with HIV represent 0.5% of the University of Michigan population, 1.6% of the VA population, and 2.5% of the University of Utah population (prior to applying any additional exclusions). In contrast to CD4 count, HIV can be identified based on ICD-10 diagnostic codes alone, improving feasibility.
- Intravenous antibiotics at discharge- originally, as an additional way of excluding complicated pneumonia, we sought to exclude patients prescribed intravenous (IV) antibiotics at discharge. The VA CPRS database did not capture IV antibiotics after discharge in a structured field (they are documented in an “outpatient parenteral antibiotic therapy” note) and thus collecting this exclusion criteria would be an undue burden. University of Michigan showed that only 2.2% of patients had intravenous antibiotics prescribed at discharge of which 71% were excluded for an alternative reason. Given the minimal impact of this exclusion on performance—and feasibility concerns—this exclusion was dropped.
3) Data elements simplified: We simplified some data elements to improve feasibility of assessing them electronically.
- Severe pneumonia- Pneumonia severity has been defined different ways in different studies and guidance. The ATS/IDSA 2007 and 2019 CAP Guidelines10,48 define severe pneumonia as having either one major criterion or three or more minor criteria as follows:
- Major criteria
- Septic shock with need for vasopressors
- Respiratory failure requiring mechanical ventilation
- Minor criteria
- Respiratory rate ≥ 30 breaths/min
- PaO2/FiO2 ratio ≤ 250
- Multilobar infiltrates
- Confusion/disorientation
- Uremia (blood urea nitrogen level ≥ 20 mg/dL)
- Leukopenia* (white blood cell count < 4,000 cells/uL)
- Thrombocytopenia (platelet count < 100,000/uL)
- Hypothermia (core temperature < 36 C)
- Hypotension requiring aggressive fluid resuscitation
- Major criteria
*Due to infection alone (i.e., not chemotherapy induced)
For this measure, the major criteria do not apply as patients admitted to an ICU or on mechanical ventilation in the first 48 hours are excluded. Because confusion/disorientation is not captured in structured data, this element was infeasible for all three of our healthcare systems to assess and was dropped. We approximated PaO2/FiO2 ratio of ≤ 250 as ≥5L oxygen or SpO2<90. Compared to chart review (University of Michigan data), this change in definition of hypoxia had a sensitivity of 94.0% and specificity of 92.4%. Since there is no standard definition for “aggressive fluid resuscitation,” we operationalized hypotension requiring aggressive fluid resuscitation as systolic blood pressure <80. Finally, to ensure accuracy of our modified criteria, we looked across Michigan Hospital Medicine Safety Consortium (HMS) data and found that requiring ≥2 modified severity criteria (rather than ≥3 of the original criteria) to determine severity had the highest area under the receiver operating curve compared to the original definition. Thus, our modified criteria are:
- Modified Severity Criteria: ≥2 of the following on a single day in the first 48 hours of encounter:
- Respiratory rate >30 breaths/minute
- ≥5L oxygen or SpO2<90
- BUN >20 mg/dL
- WBC <4000 cells/uL
- Platelets <100,000/uL
- Temperature <36 C
- SBP <80 mmHg
In University of Michigan data, compared to chart review using the ATS/IDSA criteria, the modified severity criteria have a sensitivity of 86.9% and specificity of 77.9%.
4) Elements assessed for feasibility and NOT changed: These were elements which we assessed for feasibility but did not change.
- Prior hospitalization with IV antibiotics in prior 3 months. Per the 2017 ATS/IDSA CAP guidelines, patients should receive empiric anti-MRSA and anti-Pseudomonal antibiotics if they present with severe CAP AND have had a hospitalization during which they with IV antibiotics in the last 3 months. Thus—patients with severe CAP and a prior hospitalization with IV antibiotics in the prior 3 months should be excluded from the measure denominator. We sought to explore whether it was feasible to identify patients with a prior hospitalization with IV antibiotics. In the end, for all three of our tested EHRs this element was feasible (e.g., within University of Michigan testing cohort, sensitivity was 70% and specificity 96%). However, our TEP expressed concerns that prior hospitalization with IV antibiotics might be variably captured (particularly outside of Epic and CPRS where care is more integrated) and thus variably accurate across EHRs. Because severe CAP with prior hospitalization with IV antibiotics accounted for 10.8% of patients with uncomplicated CAP, we did not want to drop this data element completely. Thus, we conducted sensitivity analyses to assess different options.
- First, we tested measure performance when patients were not excluded from the denominator on the basis of having severe CAP with prior hospitalization with IV antibiotics. This change would result in fewer patients being excluded and classify slightly more patients (58.0% [3506/6050]) as receiving inappropriate anti-MRSA or anti-Pseudomonal empiric therapy (vs. 55.2% [2998/5431] with the exclusion applied).
- Second, we tested excluding all patients with severe CAP (rather than severe pneumonia plus prior hospitalization with IV antibiotics). Here, far more patients would be excluded but performance would slightly improve (52.9% [1906/3605]).
To be consistent with national guidelines, hospitals should maintain exclusions as we have them currently specified (severe pneumonia + prior hospitalization with IV antibiotics); however, hospitals unable to accurately identify these patients should expect results within 3% difference. We anticipate that as inter-operability improves, this data element will become more feasible for hospitals.
After presenting the measure to the TEP, we asked whether the “measure appropriately balances the need for accuracy with complexity”? 75% (6/8) responded it was the “perfect balance” with 1 not responding to the question and 1 reporting it was somewhat too simple. The TEP member worrying it was too simple thought we might call some potentially justified therapy inappropriate. Another commented, “Note- answer above - "perfect balance" - likely overstatement- I think it's the best option - but there are tradeoffs needed for the measure to remain accurate while still being feasible to use.”
The full reference list can be found in Section 2.2.
Proprietary Information
Scientific Acceptability
Testing Data
We used data from 3 healthcare systems to test the measure. Each health system had different strengths to enable different validation or reliability testing:
University of Michigan (Epic)- eCQM data were pulled from hospitalizations between 9/29/2015 and 12/11/2021
- To assess measure validity, we compared eCQM data (i.e., University of Michigan Epic data) to existing chart review data from an ongoing 69 hospital collaborative quality initiative, the Michigan Hospital Medicine Safety Consortium (HMS). HMS collects pseudo-random data from hospitalized patients with community-acquired pneumonia across all 69 hospitals. Critically, they collect all the data necessary to assess this measure (including denominator/numerator inclusions/exclusions) and have used a chart-reviewed based version of the measure to improve care for hospitalized patients with pneumonia since 2015.49-53 We specifically used University of Michigan HMS data from a matching time period (9/29/2015-12/11/2021).
Quality assurance data for HMS have been published previously.49-53 In brief, trained data abstractors collect specific data from medical records in forms with pre-specified fields (e.g., max/min allowances). Abstractors undergo random audit to ensure data quality. HMS data have been used to inform two prior NQF-endorsed quality measures (NQF 3671 and 3690).54
University of Utah (Epic)- eCQM data pulled from hospitalizations between 1/1/2021 and 5/30/2022
To assess measure validity, we compared eCQM data (i.e., University of Utah Epic data) to chart review data obtained originally for the Inappropriate Diagnosis of Pneumonia measure (NQF 3671). Cases were manually abstracted by a physician reviewer to obtain the necessary data elements to assess appropriateness of empiric therapy. Cases for abstraction were randomly selected from patients with a discharge diagnosis of pneumonia from 1/1/2021 and 5/30/2022. Electronic and chart review data were compared for key denominator/numerator variables included in both datasets.
Veterans Affairs (VA) Computerized Patient Records System (CPRS)- eCQM data were pulled from hospitalizations between January 1, 2022 and June 30, 2024 including 109 acute care hospitals.
- These data were used to assess variation across hospitals and reliability
Notably, CPRS data have been recently used to inform a Battelle-endorsed quality measure (CBE ID: 4440e).
Technical Expert Panel (TEP): The TEP was comprised of stakeholders with diverse perspectives and areas of expertise representing a variety of professional organizations, non-profit organizations, and governmental agencies (details in Supplemental Materials attachment, Table 4). Expert feedback during measure development was provided by the TEP during a 2-hour session held on April 26, 2023.
The full reference list can be found in Section 2.2.
A summary of the data sources that were used for each type of reliability and validity testing is found in Table 7 (see Supplemental Materials attachment)
The VA health system is an integrated health system that uses CPRS, with approximately 485,500 acute hospitalizations per year across 109 facilities. The University of Utah is an academic medical center that uses an Epic EHR, with approximately 14,500 total inpatient hospitalizations per year. The University of Michigan is an academic medical center that uses a different instance of the Epic EHR with approximately 49,730 total inpatient hospitalizations per year. All three sites have interoperable EHRs where data can be transferred across outpatient and inpatient settings within the health care system. Characteristics of the measured entities are in Table 8 (Supplemental Materials attachment).
Notably, the VA healthcare system and University of Utah have implemented interventions to improve empiric antibiotic selection for CAP with published success.20,21,28 For example, substantial decreases in antibiotic use have been demonstrated in the VA since the establishment of the national VA Antimicrobial Stewardship Task Force in 201121 that have persisted through the COVID-19 pandemic21. Strengths of the VA antimicrobial stewardship programs include organizational directives for antimicrobial stewardship program staffing, system-wide medication use evaluations that identify opportunities for improvements in antimicrobial use,23-25 development of regional antimicrobial stewardship collaboratives, strong implementation science engagement and utilization of informatics-based tools20 and strong, ongoing collaboration between VA research and operational partners.26 The range of performance across systems further demonstrates a wide gap—and opportunity for improvement—across hospitals and that measurement and antibiotic stewardship efforts can improve antibiotic use.
The full reference list can be found in Section 2.2.
University of Michigan, University of Utah, and VA Healthcare eCQM data include all eligible patients during their respective timeframes. Measure-level performance testing was completed without sampling, using the full denominator populations at each healthcare system (48,742 within the VA; 619 at the University of Utah; 5,553 at the University of Michigan), at the patient encounter level of analysis. Descriptive statistics of patients included in the patient encounter level testing datasets are provided for the VA cohort, the University of Michigan cohort, and the University of Utah Cohort (see Tables 9, 10, and 11 in Supplemental Materials attachment).
To test the validity of the University of Michigan population, we compared University of Michigan eCQM data to chart review data from the Michigan Hospital Medicine Safety Consortium (HMS). For HMS, abstractors screened consecutive patients via medical record review 30 days after discharge and included the first eligible patient daily, abstracting 8 eligible patients during a two-week cycle. Patients were eligible for inclusion if they were adults (≥18) admitted to general care with a billed discharge ICD-10 code of pneumonia and received antibiotics on day 1 or day 2 of hospitalization (this differs from the first 48 hours definition used in eCQM and may explain some data discrepancies). Patients who had documentation of treatment for an additional infection unrelated to pneumonia, were severely immunocompromised, were pregnant, were admitted for comfort measures, or who left against medical advice were ineligible. Quality assurance data for HMS have been published previously.49-53 In brief, trained data abstractors collect specific data from medical records in forms with pre-specified fields (e.g., max/min allowances). Abstractors undergo random audit to ensure data quality. HMS data have been used to inform two prior NQF-endorsed quality measures (NQF 3671 and 3690).54
The full reference list can be found in Section 2.2.
Reliability
Per guidance from NQF’s Scientific Methods Panel on requirements for eCQMs, data element reliability is “not required if data element validity is demonstrated.”56 Thus, please see our data element validity section rather than data element reliability.
Per guidance from NQF’s Scientific Methods Panel on requirements for eCQMs, “Reliance on data from structured data fields is expected; otherwise, unstructured data must be shown to be both reliable and valid. Reliability testing is not required if based on data from structured data fields.”56 Our measure uses only data from structured data fields, those we do not report patient/encounter or data element reliability and instead report patient/encounter and data element validity.
We conducted accountable entity-level reliability testing with two models. The first was a signal-to-noise analysis performed (within the VA dataset) using a mixed-effect logistic model run as an empty model such that the only effects in the model were the overall intercept and the hospital specific intercepts. This model enabled calculation of the hospital variance (signal), the total variance, and the residual variance (noise). The intraclass correlation was calculated from these variances. The intraclass correlation was utilized within the Spearman Brown formula in two ways: (A) to calculate the reliability for the entire hospital cohort using the median number of case abstractions for the cohort and (B) to understand minimum case abstracts necessary to achieve predetermined reliability thresholds of 0.6, 0.7, 0.8, and 0.9. All data for this analysis was from the 109 VA facilities described above. A caterpillar plot was also provided demonstrating raw metrics and shrunken estimates in this model.
We also performed an analysis of reliability using beta-binomial regression using SAS code provided in The Reliability of Provider Profiling: A Tutorial.78 VA facilities were then sorted by decile of reliability size (after exclusions)—results are shown in Table 2. The mean reliability score by beta-binomial regression was then calculated for all facilities within a decile category.
The full reference list can be found in Section 2.2.
Table 3 (see Supplemental Materials attachment) indicates the minimum annual number of qualifying cases needed for the denominator to reach each target reliability level at a given facility. In order to achieve a desired reliability of 0.8, each hospital would need to include 96 cases annually. For acceptable reliability (0.7), 56 annual cases would be required, and for high reliability (0.9), 216 annual cases would be required. The median number of eligible cases per facility within the cohort (n=135) had a reliability of 85.0%, based on data from 109 VA health care systems (2023 data). Based on our signal-to-noise analysis (using beta-binomial regression) the overall cohort reliability is 99.9%.
Based on signal-to-noise analysis, the median number of eligible cases per facility within the cohort (n=135) had a reliability of 84.7%. Based on our signal-to-noise analysis (using beta-binomial regression) the overall cohort reliability as >99.9%. Both these estimates meet the threshold for reliability for measures considered to be high stakes.
Using the current VA cohort as a representative example, the minimum number of case abstracts per hospital per year to meet pre-specified reliability thresholds of 0.7 and 0.8 are highly attainable. Within the 109 VA health care systems, 86.2% of hospitals had the minimum of 56 cases in 2023 necessary to achieve 0.7 reliability. As seen in Table 2, only the smallest decile of reliability had a mean reliability <80%. Using the VA likely underestimates reliability of the measure, as 87.2% of VA hospitals have fewer than 200 beds.
It should also be noted that this analysis was conducted among facilities all within the same healthcare system and thus may be more similar in performance to one another than a true population distribution of all hospital types and systems within the United States.
Validity
Table 12 (see Supplemental Materials attachment) summarizes validity testing of the measure during development and in its final version, including description of the testing, results, and interpretation.
A. Face Validity-National Guidelines
The inappropriately broad empiric antibiotic selection in CAP measure was based on national guidelines for pneumonia, literature review, and expert feedback and review. The 2019 ATS/IDSA CAP guidelines recommended not prescribing empiric anti-MRSA or anti-Pseudomonal therapy in uncomplicated CAP patients without risk factors for MRSA or Pseudomonas aeruginosa.
This measure is also supported by literature. For example, a recent report on 8,286 non-ICU hospitalized CAP patients from 67 Michigan hospitals (assessed using the chart review measure from which our eCQM was adapted) showed that 2,215 (26.7%) received inappropriately broad empiric treatment (i.e., anti-MRSA or anti-Pseudomonal coverage in patients eligible for standard CAP coverage per ATS/IDSA guidelines).11 Compared to patients who received standard CAP antibiotic treatment, patients receiving inappropriately broad antibiotics had higher 30-day readmissions, more transfers to ICU and antibiotic-associated adverse events, and longer hospitalizations.11 Similarly, a retrospective analysis of 88,605 patients with CAP across the Veterans Health Administration health care system found that adding empirical anti-MRSA therapy to standard CAP therapy was associated with an increased adjusted risk of death, kidney injury, and secondary infections (C. difficile, vancomycin-resistant Enterococcus infection, and resistant gram-negative rod infections).8 Given that up to 90% of empiric anti-MRSA or anti-Pseudomonal antibiotic therapy is non-guideline concordant,8,11 there is substantial potential benefit to patients and the US by reducing inappropriately broad empiric antibiotic selection. National surveillance of drug-related adverse events estimated that even a small reduction in unnecessary antibiotic use could significantly decrease the direct risks of drug-related adverse event in individual patients.12 Thus, our guideline-based measure has the potential to improve patient care for a large number of patients hospitalized with CAP across the US.
Appropriateness of antibiotic therapy for pneumonia is a priority for numerous federal and accreditation organizations–including the CDC, The Joint Commission, and Centers for Medicare and Medicaid Services. For example, the CDC developed recommendations for Antibiotic Stewardship which it published in its “Core Elements of Hospital Antibiotic Stewardship Programs.” Within the core elements, the CDC specifically states that one of the key opportunities for improving antibiotic use in CAP is “Avoid[ing] empiric use of anti-Pseudomonal beta-lactams and/or MRSA agents unless clinically indicated.”34
B. Face Validity-Expert Feedback
Throughout measure development, we obtained expert and stakeholder input via input from the Data, Design, and Publications (DDP) Committee of the Michigan Hospital Medicine Safety Consortium (HMS) early in measure development (during chart review stage)
The Data, Design, and Publications (DDP) Workgroup was an ongoing meeting of champions and experts from HMS hospitals that met to address key issues related to measure methodology, including weighing the pros and cons of measure specifications, modeling, and use (e.g., defining the measure cohort and outcome) to ensure the measure was meaningful, useful, and well-designed. The group met every 2 months during measure development and provided a forum for focused expert review and discussion of technical issues. They also provided final approval of the current submitted measure as specified.
List of DDP Workgroup Members:
- Suhasini Gudipati, MD Ascension Michigan St. Mary’s Hospital
- Tina Percha, RN, MSN Beaumont Health
- Rajiv John, MD Beaumont Health
- Lama Hsaiky, PharmD Beaumont Health
- Priscila Bercea, MPH Beaumont Health Dearborn
- Scott Kaatz, DO Henry Ford Health System
- Allison Weinmann, MD Henry Ford Health System
- Emily Nerreter, MBA Henry Ford Health System
- Danielle Osterholzer, MD Hurley Medical Center
- Lisa Dumkow PharmD Mercy Health St. Mary’s
- Anurag Malani, MD St. Joseph Mercy Ann Arbor Hospital
- Lakshmi Swaminathan, MD St. Joseph Mercy Ann Arbor Hospital
- Muhammad Nabeel, MD Sparrow Hospital
- Andrea White, PhD University of Utah Health
- Valerie Vaughn, MD, MSc University of Utah Health
- Vineet Chopra, MD, MSc University of Colorado Anschutz Medical Campus
Throughout measure development (the chart review version), we also provided opportunities from experts across the HMS collaborative to provide feedback. This included frontline clinicians, antibiotic stewards, quality improvement experts, c-suite members, and experts in quality measurement.
C/D. Assessment of Encounter-Level Validity: Comparison with Case Review
The Michigan Hospital Medicine Safety Consortium (HMS), which includes the University of Michigan, has been collecting chart-review data on patients hospitalized with CAP since 2015. We report two types of encounter-level validity using these data where we compare HMS results and our eCQM results. First, we compared the actual antibiotic empiric selection obtained via eCQM vs. via chart review. For this assessment, we report the percentage of cases where both data sets agreed in classification of whether the patient received no anti-MRSA/Pseudomonal, only anti-MRSA, only anti-Pseudomonal, and both anti-MRSA/Pseudomonal empiric antibiotics. Next, we assessed the sensitivity and specificity for the eCQM in identifying patients with inappropriately broad empiric antibiotic selection (i.e., anti-anti-MRSA or anti-Pseudomonal therapy in the absence of risk factors).
E. Assessment of Encounter-Level Validity: Critical data element validity
To assess critical data element validity, initial denominator exclusions categorized as: a) to identify hospitalized non-ICU adults with CAP, b) to narrow the population to uncomplicated CAP, and c) specific to the appropriateness of empiric antibiotics measure were compared to assess consistency across the University of Michigan, University of Utah, and VA healthcare system.
We also compared inclusion criteria, exclusion criteria, and critical data elements between HMS chart review data and eCQM data for the University of Michigan. Due to the way HMS conducts chart review, not all inclusion/exclusion criteria are collected. We report sensitivity and specificity of the eCQM assessment compared to chart review which we considered the gold standard.
F. Face Validity: National Technical Expert Panel (TEP) Feedback (N=8 experts)
Throughout measure development, we obtained expert and stakeholder input. In alignment with the CMS Measures Management System guidance on TEP,57 we held a remote TEP session to provide input and feedback from a group of recognized experts in relevant fields. To convene the TEP, we reached out to organizations whose members could potentially be impacted by the measure and asked them to nominate individuals for participation. We selected individuals to represent a range of perspectives, including Infectious Diseases physicians, pharmacists, pulmonologists, radiologists, hospitalists, emergency medicine physicians, regulatory agencies, as well as individuals with experience in quality improvement, performance measurement, diagnostic error, antibiotic stewardship, and health care quality. (TEP members are listed in Table 4 in Supplemental Materials attachment).
The TEP was convened on April 26, 2023 over a 2-hour period. In preparation for the meeting, all TEP participants were provided with an agenda and background document providing project context and data supporting the inappropriately broad empiric therapy for adult hospitalized patients with uncomplicated community-acquired pneumonia measure. The format for the TEP meeting (conducted remotely) included presentation of measure development and proposed specifications and relevant data, with open discussion among TEP members throughout. Participants were encouraged to provide expert feedback regarding the validity, feasibility, and usability of the measure during the sessions and through online questionnaires following each session.
Following the TEP, all participants completed an online survey that included questions related to validity, reliability, usability, etc. Related to measure validity, we asked TEP members:
- Please rate the following statement: “The simplified measure can be used to distinguish between better and worse performing hospitals.” 1=Strongly disagree, 2=Disagree, 3=Neutral, 4=Agree, 5=Strongly agree.
Are there any key data elements you believe are missing or not accurately captured in the inappropriate broad spectrum antibiotic therapy for CAP measure?
G. Empirical Validity: Evaluated association with other measures of CAP antibiotic treatment quality
To assess empirical validity for the inappropriately broad empiric treatment in CAP measure, we identified and assessed the measure’s correlation with other measures that target antibiotic quality for CAP. The goal was to identify if better performance on this measure was related to better performance on other relevant measures. Specifically, we assessed the association (at the hospital level) of excess antibiotic duration for CAP with overuse of broad-spectrum empiric antibiotics for CAP.
H. Empirical Validity: Evaluated association of inappropriately broad empiric antibiotic use for CAP with outcomes
We reviewed the literature for studies evaluating the impact of broad spectrum (i.e., anti-MRSA or anti-pseudomonal) empiric antibiotic use on patient outcomes in CAP.
I. Predictive Validity: Evaluated whether a reduction in inappropriate broad spectrum antibiotic use improves patient outcomes in CAP.
We reviewed published literature on how patient outcomes change with reductions in inappropriately broad empiric treatment in CAP
The full reference list can be found in Section 2.2.
C. Encounter-Level Validity: Comparison of Empiric Antibiotics with Case Review
After exclusions, 540 patients had data in both UM HMS and UM eCQM datasets. Considering HMS data the gold standard, the eCQM accurately assessed empiric antibiotics (no anti-MRSA/Pseudomonal, only anti-MRSA, only anti-Pseudomonal, and both anti-MRSA/Pseudomonal) in 96.7% (522/540) of patients (see Table 13 in Validity Results attachment).
D. Encounter-level Validity: Comparison of Measure (appropriateness of empiric antibiotics) with Case Review
Looking at the same 540 patients with both UM HMS chart review and University of Michigan eCQM data, we compared the percentage with inappropriate broad spectrum empiric antibiotic use (i.e., anti-MRSA or anti-Pseudomonal therapy) using chart review as the gold standard to obtain the sensitivity and specificity for the eCQM. The eCQM has a sensitivity of 96% and a specificity of 92% in assessing inappropriate broad spectrum empiric antibiotic use (see Table 14 in Validity Results attachment).
E. Encounter-level Validity: Critical data element validity
For each measured entity, initial denominator exclusions: a) to identify hospitalized non-ICU adults with CAP, b) to narrow the population to uncomplicated CAP, and c) specific to the appropriateness of empiric antibiotics measure (Tables 15, 16, and 17 in Validity Results attachment) were compared to assess consistency across entities.
Prior to applying exclusions, there were 592 patients hospitalized with CAP who had data in both the HMS chart review dataset and the University of Michigan eCQM data set. Below, we show the sensitivity and specificity for the empiric-specific exclusions (see Table 18, Validity Results attachment).
F. Face Validity: National Expert Panel Feedback
The eight national experts who attended our TEP agreed with the face validity and operationalization of the measure. They believed that patients we identified as receiving inappropriate broad spectrum empiric antibiotic therapy did, in fact, receive inappropriate therapy. The main feedback from the TEP was to exclude patients with risk factors for broad spectrum therapy from the denominator (which we did). (see TEP survey results in the Validity Testing Results attachment).
G. Empirical Validity: Association with Other Measures of CAP Antibiotic Treatment Quality
To assess empirical validity for the inappropriately broad empiric therapy in CAP measure, we assessed the association (at the hospital level) of excess antibiotic duration for CAP with overuse of broad-spectrum empiric antibiotics for CAP with the 109 VA hospitals. We found that the two measures were weakly correlated at the hospital level (R=0.3, p=0.0014, see Figure 4 in Validity Results attachment).
H. Empirical Validity: Evaluated association of inappropriately broad empiric antibiotic use for CAP with outcomes
Across 67 HMS hospitals, 26.7% (2,215/8,286) of included patients were empirically treated with inappropriate broad-spectrum antibiotics. The median broad-spectrum antibiotic treatment duration was 3 days (IQR, 2.5). Factors associated with inappropriate empiric broad-spectrum antibiotic treatment included hospitalization or treatment with high-risk antibiotics in preceding 90 days, transfer from a post-acute care facility, hemodialysis, support with ≥3 L supplemental oxygen, severe sepsis, leukocytosis, and higher pneumonia severity index. After adjustment, inappropriately broad empiric treatment was associated with higher 30-day readmissions, more transfers to ICU, longer hospitalizations, and more antibiotic-associated adverse events (see Table 19 in Validity Results attachment).11
Among 88,605 CAP patients across the Veterans Health Administration health care system, empirical anti-MRSA therapy in addition to standard CAP therapy was associated with an increased adjusted risk of death, kidney injury, and secondary infections (C. difficile, vancomycin-resistant Enterococcus infection, and gram-negative rod infections) compared to standard CAP treatment alone.8
I. Predictive Validity: Evaluated whether a reduction in inappropriate broad spectrum antibiotic use improves patient outcomes in CAP.
The effect of reducing inpatient broad-spectrum antibiotic exposure was modeled and estimated that reduction in use of broad-spectrum antibiotics by 30% could result in a 26% reduction in C. difficile infection.2
The full reference list can be found in Section 2.2.
We assessed multiple types and levels of validity during and after measure development. First, the measure has high face validity as it is based on national guidelines and recommendations from national organizations (e.g., CDC) and was approved by national experts representing 8 national organizations. Compared to 540 case reviews (using HMS data at the University of Michigan), the measure had high sensitivity and specificity at the patient level for identifying inappropriate broad spectrum antibiotic use. Empiric-specific exclusions also had high sensitivity and specificity when compared to HMS chart review cases. The exception is the exclusion of severe CAP with prior hospitalization with IV antibiotics-here, we used a slightly less sensitive definition (sensitivity of 86.9% and specificity of 77.9%) to improve feasibility.
Across hospitals, we found performance on the empiric broad spectrum measure to be weakly correlated with performance on the excess duration measure, supporting empirical validity. Finally, the chart-review version of the measure on which this eCQM was based has been shown to be associated with patient outcomes and, critically, models suggest that improvement in the measure may reduce Clostridioides difficile infection.
Together, these results suggest that the inappropriate broad spectrum empiric antibiotic measure is highly valid and, most importantly, that hospitals can improve patient outcomes by improving their performance on the measure.
Risk Adjustment
Use & Usability
Use
Acute care hospitals
Usability
Acute care hospitals are now required to have antimicrobial stewardship teams to optimize antibiotic use in hospitalized patients. There are multiple strategies to improve empiric antibiotic selection for CAP including: a) ordersets to standardize care, b) audit and feedback by pharmacists, c) nudging or behavioral interventions to reduce anti-MRSA or anti-Pseudomonal therapy, and d) creation of guidelines and education.
Each strategy has its own barriers, and feasibility varies by institution. For example, smaller hospitals have an easier time implementing audit and feedback as their patient population is small and pharmacists may have fewer patients to cover. In contrast, larger institutions may have more information technology infrastructure to help construct ordersets or nudges using the EHR.
What we do know is that diverse hospitals are able to improve empiric antibiotic selection in accordance with national guidelines. A chart-review based form of this measure has been implemented within the Michigan Hospital Medicine Safety (HMS) consortium since 2022. After years of reporting measure data to the hospitals (since 2015), HMS announced in 2022 that it would become a pay-for-performance measure with an initial goal of ≤15% of patients receiving inappropriately broad-spectrum empiric antibiotics. As hospitals improved this goal was changed to ≤10% in 2024. A 42% relative decrease in inappropriate empiric anti-MRSA/anti-Pseudomonal therapy was observed across 69 HMS hospitals from quarter 2 of 2020 (22%) to quarter 2 of 2024 (13%, P<0.0001; unpublished data with permission from HMS DDP committee (see Figure 1. Supplemental Materials attachment) By quarter 4 of 2023, the majority of participating HMS hospitals met the performance goal (overall collaborative average was 13%); thus, the metric was converted into a sustainability measure for the collaborative (goal ≤10% for collaborative).
Tri-annual Collaborative Wide Meetings
Individuals from participating hospitals meet in person three times a year. We encourage hospitals to send their Clinical Data Abstractors, physician champions, and quality leads, as well as other individuals from their hospital that might be interested in participation. These meetings take place three times per year – in March, July, and November. Traditionally, meetings took place in-person at venues across Michigan. In 2020 and 2021, these meetings were hosted via an on-line format due to COVID-19.
The tri-annual meetings provide individuals from member hospitals with the opportunity to engage with each other in a variety of formats. Each meeting includes a formal discussion of the data from each of the HMS initiatives—including data on empiric antibiotic therapy for CAP—for the previous quarter, presentations from member hospitals and expert guests, breakout/work group sessions, and networking opportunities. These meetings allow individuals from member hospitals to network with individuals from other hospitals who have excelled in those areas to seek ideas on how to improve their performance. It also allows for an opportunity for feedback and to answer questions related to their performance.
Site-specific Reports on Measure Performance
Tri-annually, each participating hospital receives a printed and email version of a site-specific data report. These reports are also available daily within the database/registry (see below). These reports provide an in-depth look into the performance of each site. For example, we provide hospital data on the number of patients with CAP treated with inappropriate broad-spectrum empiric antibiotic therapy, details on antibiotic use and outcomes (e.g., adverse events), longitudinal performance, and data on how individual hospitals compare to other hospitals in the state. Hospitals also receive a list of all patients who were considered treated with an “inappropriate broad spectrum empiric therapy” to enable them to return to their hospital and conduct case reviews of those patients. Each hospital is encouraged to review these cases with their local team to perform audit and feedback, identify trends, and assist with overall quality improvement. This also provides an opportunity for measure feedback—for example, hospitals might find an error in case classification. Early during measure development this case-specific feedback was critical for improving measure validity.
Live Database Reports
Each of the HMS databases are equipped with the ability to view live reports utilizing Business Objects software. These reports provide updated data every 24 hours regarding measures (site performance and collaborative performance), fallout case information, demographics, critical/non-critical data errors, completeness of abstracted cases, and case classification information.
Individuals who participate in the collaborative either as a Clinical Data Abstractor or a quality administrator have the ability to log into the HMS databases and view these reports at their leisure. The software that HMS utilizes also allows for these reports to be exported as Excel files or PDFs for hospital-specific customization. This information is often utilized by participating hospitals at committee meetings or for presentations to track progress and inform quality improvement efforts. They also assist the Clinical Data Abstractor to identify errors in their abstraction and resolve them in real time. These reports also allow hospitals to review individual fallout cases and their clinical scenarios to inform individual clinicians or groups of clinicians of their performance and provide targeted education.
Throughout measure development, we received feedback on the measure performance/validity through the above mechanisms as well as: 1) Expert Feedback from Data Design and Publications Committee and Michigan Hospital Medicine Safety (HMS) Consortium Hospital Experts/Representatives and 2) “Fall-out” Feedback. Briefly, measure performance feedback allowed us to refine the measures to the current version. The Data, Design, and Publications Committee approved the measures for use across HMS. There have been no unanticipated outcomes/ill effects.
Using the chart-review based form of this measure as a pay-for-performance measure within the Michigan Hospital Medicine Safety consortium, hospitals were able to reduce empiric broad spectrum antibiotic use.
A 42% relative decrease in inappropriate empiric anti-MRSA/anti-Pseudomonal therapy was observed across 69 HMS hospitals from quarter 2 of 2020 (22%) to quarter 2 of 2024 (13%, P<0.0001; unpublished data with permission from HMS DDP committee, see Figure 1. By quarter 4 of 2023, the majority of participating HMS hospitals met the performance goal of ≤15% of uncomplicated CAP patients who received inappropriate broad empiric coverage (overall collaborative average was 13%); thus, the metric was converted into a sustainability measure for the collaborative (goal ≤10% across all hospitals).
There were no unexpected findings during the 3 years that this measure has been a pay-for-performance measure.
Comments
Staff Preliminary Assessment
4545e Staff Assessment
Importance
Strengths:
- Logic Model: A clear logic model is provided, depicting the relationships between inputs (e.g., ATS/IDSA Guidelines, EHR Systems), activities (e.g., increased attention to anti-MDRO therapy per guidelines, patient care, antibiotic stewardship teams working to improve prescribing), and desired outcomes (e.g., improved adherence to ATS/IDSA Guidelines, reduced inappropriate antibiotic exposure, improved patient care and safety). This model demonstrates how the measure's implementation will lead to the anticipated outcomes.
- Performance Gap: Data from VA Healthcare Systems (January 1, 2022-June 30, 2024) shows a performance gap, with decile ranges from 13.5% to 42.4%, indicating variation in measure performance.
- Existing Measures: The proposed measure addresses a healthcare need not sufficiently covered by existing measures. No eCQMs specifically for CAP currently exist. The developer collaborated with the CDC to design this measure with the goal of working within their NHSN AU framework.
- Literature Review: The measure is supported by a comprehensive literature review, including citing the Infection Disease Society of America (IDSA) guidelines for Implementing an Antibiotic Stewardship program which provide a strong recommendation (based on moderate quality evidence) that recommend antibiotic stewardship interventions designed to reduce the use of antibiotics associated with a high risk of Clostridium difficile infection (CDI) compared with no such intervention.
Limitations:
- Patient Input: Meaningfulness to patients was partially assessed. The information available in the literature was limited and the TEP only included clinicians.
Rationale:
- This new measure is supported by a clear logic model linking inputs like ATS/IDSA Guidelines and EHR Systems to activities and outcomes such as improved guideline adherence and patient safety, addresses a gap in existing measures for CAP, as evidenced by variable performance data and a comprehensive literature review. However, the rating is "Not Met but Addressable" as the information about meaningfulness to patients is limited to evidence from guidelines and a TEP that included only clinicians.
Feasibility Acceptance
Strengths:
- Feasibility Assessment: A comprehensive feasibility assessment has been conducted, demonstrating that all necessary people, tools, tasks, and technologies have been considered for the implementation of this measure. This includes a detailed analysis of the resources required and the operational steps needed to integrate the measure into current healthcare practices.
- A completed eCQM Feasibility Scorecard is attached, showing the measure was tested in more than three EHRs (Epic- University of Utah, CPRS- Veterans Affairs, Epic- University of Michigan). Most data required for this measure are readily available in a structured format, residing in fixed fields within the EHR systems tested. Lastly, the data are captured during the course of care, ensuring that the measure does not require additional steps or disrupt normal clinical workflows.
- Adjustments Based on Feasibility: The feasibility assessment informed the final measure design, indicating decisions made to adjust the measure in response to data availability and other practical considerations.
Limitations:
- The feasibility plan in the feasibility scorecard notes that "transfer from other hospital" is not currently tracked in a structured field in some EHRs. However, they plan to continue to assess the feasibility of search methods where the element is not encoded and evaluate improvements.
Rationale:
- This new measure meets all criteria for 'Met' due to its well-documented feasibility assessment, clear and implementable data collection strategy, ensuring practical implementation within the healthcare system.
- The eCQM Feasibility Scorecard results indicate that the measure is well-supported by the current capabilities of the tested EHR systems in terms of data availability, accuracy, adherence to standards, and workflow integration. These factors collectively ensure that the measure can be implemented effectively and sustainably in a real-world healthcare setting.
Scientific Acceptability
Strengths:
- Data Sources and Dates: Data used for testing were sourced from EHRs during the period January 1, 2022 to June 30, 2024. These data are all collected routinely during usual clinical care through the process of inpatient hospitalizations. The 109 hospitals included in the analysis were characterized by being in the VA healthcare system which spans urban and rural areas.
- Accountable Entity Level Reliability: The developer conducted signal-to-noise reliability testing at the accountable entity-level. More than 90% of accountable entities meet the expected threshold of 0.6 with a minimum reliability of 0.291.
Limitations:
- None.
Rationale:
- The results demonstrate sufficient reliability at the accountable entity-level.
Strengths:
- The developer assessed face validity, encounter-level validity, and accountable entity-level validity. For face validity, TEP (N=8) demonstrated consensus with the statement "The measure as specified can be used to distinguish between better and worse quality hospitals." For encounter-level validity, based on 540 case reviews (using HMS data at the University of Michigan), the measure had high sensitivity and specificity for identifying empiric antibiotic assessment and for assessing inappropriate anti-MRSA or anti-Pseudomonal. For denominator exclusion data elements (N=592), sensitivity ranged from 60% (prior hospitalization with IV antibiotic in prior 3 months) to 100%, and specificity from 77.9% to 99.3%. For accountable-entity validity, the developer presented an association study (a weak correlation with overuse of broad-spectrum empiric antibiotics for CAP) and a 'modeled" association study (change in broad-spectrum antibiotics associated with a change in C. difficile infection).
Limitations:
- For face validity, the main limitation may be the size and composition of the TEP (N=8) which lessens the persuasive impact of the consensus statement. For patient- or encounter-level validity, the main limitation is the use of only one EHR vendor and the use of of data limited to HMS data at the University of Michigan, which may have limited external validity for less resource intensive environments. For accountable-entity validity, the main limitation is the lack of mechanism studies on the features of the mechanism complex (i.e. what worse performing entities do not do that better performing entities do). The "modeled" association study (change in outcome) does support the rule out non-causal connections and confounding. The patient level analysis demonstrating an association between the measure focus and material outcomes (i.e., higher 30-day readmissions, more transfers to ICU, longer hospitalizations, and more antibiotic-associated adverse events ) speaks to importance rather than validity.
Rationale:
- The validity testing results support a relatively strong inference of validity for the measure, confirming that the measure accurately reflects performance on quality or resource use and can distinguish good from poor performance. However, additional data element testing within at least two EHR vendors is needed.
Equity
Strengths:
- The developer referred to literature on health inequities, finding no differences by patient race, age, gender have been found related to inappropriate broad spectrum antibiotic therapy for community-acquired pneumonia.
Limitations:
- However, this content alone does not address the equity criterion. The developer should consider empirically testing for differences in performance scores across multiple sociocontextual variables (e.g., race, ethnicity, urbanicity/rurality, socioeconomic status (SES), gender, gender identity, sexual orientation, age).
Rationale:
- Though the developer provided some information on inequities, no empirical testing was completed.
Use and Usability
Strengths:
- Current Use: The measure is currently used in the Michigan Hospital Medicine Safety Consortium.
- Performance Trends: The developer reports changes in performance from 2020 – 2024 using the chart-review based form of this measure, in which there was a 42% relative decrease in inappropriate empiric anti-MRSA/anti-pseudomonal therapy observed across 69 HMS hospitals.
- Feedback Mechanism: Feedback is collected during tri-annual collaborative wide meetings, and site-specific data reports allow hospitals to repot any errors in classifications or other abstraction errors. They also get expert feedback from Data Design and Publications Committee and Michigan Hospital Medicine Safety Consortium Hospital Experts/ representatives.
Limitations:
- Measure Updates: The developer could provide more detail on whether any feedback received led to changes in the measure specifications.
Rationale:
- This new measures is currently used in at least one accountability application, and the measure provides actionable information for improvement. The developer describes approaches to collecting feedback on this measure, but more detail is needed on how specific feedback received led to changes in the measure specifications. Although performance results for the chart-based version of the measure showed positive results, more data on performance trends for the eCQM is needed. The developer reports no unexpected findings.
Committee Independent Review
Overall, the intent of the…
Importance
The measure is supported by a comprehensive literature review showing the focus on decreasing patients receiving inappropriately broad-spectrum empiric antibiotics
Feasibility Acceptance
Comprehensive feasibility assessment and scorecard showing current EHR systems have the appropriate capabilities to capture the data.
Scientific Acceptability
Data was reviewed for EPIC EHR and data reviewed from VA’s CPRS.
The size of the TEP (N=8) is a limitation
Equity
The developer did not address equity criterion.
Use and Usability
Data suggests usability
Summary
Overall, the intent of the measure is supported and has shown facilities focused on this decreases patients receiving inappropriately broad-spectrum empiric antibiotics
Question:
- Are pregnant women included in denominator?
Comments noted.
Importance
Agree with staff assessment that direct patient input on measure's meaningfulness is needed.
Feasibility Acceptance
Agree with staff assessment.
Scientific Acceptability
Agree with staff assessment.
The staff assessment noted that the data element validity testing needed to be conducted with 2+ EHRs, but I could not find that in the guidebook.
The staff assessment also noted that the TEP only included 8 members, but there is nothing in the guidebook about a minimum number.
Equity
Optional
Use and Usability
Agree with staff assessment.
Summary
Comments noted.
Promising, but look for future diminishing returns
Importance
Although evidence from the literature is limited, the two studies provided (45 and 46 in the reference list) make the case that patients are concerned about antibiotic overutilization, as well as the demonstrable relationship to clinical outcomes of importance to patients, particularly older adults. Two small-scale studies I found suggests an opportunity for patient engagement on the subject (https://pubmed.ncbi.nlm.nih.gov/26809477/, https://academic.oup.com/jacamr/article/2/3/dlaa046/5879268). As well, I note that PFPS US has commented in strong support of the measure.
Feasibility Acceptance
Concur with staff assessment; feasibility assessment has been completed.
Scientific Acceptability
Concur with staff assessment.
In this case, given the limited number of data elements in use and the strong confidence in feasibility, I don’t foresee any issues with validity when applied in a different EHR.
I would emphasize that it is regrettable that patients were not included in the TEP, but it is not sufficient to disqualify the measure’s validity.
Equity
Concur with staff assessment; it seems reasonable to test for the influence of socioeconomic factors.
Use and Usability
Concur with staff assessment. The measure is in use in an accountability application.
Summary
Implementation of this eCQM has shown a substantial and rapid performance improvement (inappropriate prescribing was nearly cut in half) – extrapolating outwards to a national implementation is an exciting prospect for public health.
Although the balance of risks strongly favors measure implementation, there is a risk of unintended consequences that I agree should continue to be monitored. Something else to watch for over the long term: from the performance data we can infer that as rates of overprescribing tend towards zero (already down to 11% post-implementation of the measure), there may be diminishing returns in continued use of the measure.
reasonable measure
Importance
The patient perspective is important. Srictly not met but addressable, but hard to imagine much opposition in that regard.
Feasibility Acceptance
satisfied with staff assessment
Scientific Acceptability
satisfied with staff assessment
limited testing, but adequate
Equity
agree with staff
Use and Usability
agree with preliminary assessment
Summary
There are a few areas for improvement but the body of information we have points to a measure that could be endorseable
similar to other pneumonia measure
Importance
Technically not met because they did not provide evidence that the target population values the measured process, but all other evidence is strong.
Feasibility Acceptance
no concerns
Scientific Acceptability
56 beds for acceptable reliability is sufficiently small in my mind, but curious if other Committee members think this will capture enough hospitals.
results are encouraging.
Equity
Unclear if the testing they presented was only showing significant differences or if they did not test for sociocontextual differences at all based on prior literature. It is important to report findings even if no significant differences were found so encourage this in subsequent submissions if that is the case.
Use and Usability
no concerns
Summary
similar concerns as to the other pneumonia measure by this developer. More evidence of patient value for the measure and more evidence of how this addresses inequities in healthcare, if at all. However, sufficient for initial endorsement.
CBE #4545e
Importance
Pneumonia is most common reason for IP Abx, and most common infectious mortality.
Feasibility Acceptance
Agree with Staff Preliminary Assessment.
Scientific Acceptability
Agree with Staff Preliminary Assessment.
Agree with Staff Preliminary Assessment.
Equity
Agree with Staff Preliminary Assessment.
Use and Usability
Agree with Staff Preliminary Assessment.
Summary
Support this measure. A manageable eCQM and a reasonable definition of "inappropriately broad" abx selection, makes this a strong measure and good first step in using eCQMs to better facilitate Antimicrobial Stewardship.
Endorseable
Importance
Evidence indicates the importance of this rating.
Feasibility Acceptance
The feasibility assessment and scorecard demonstrate this measure's feasibility.
Scientific Acceptability
The developers used robust and recent data. Results from the analysis also suggest meets this criteria.
The workgroup consists of clinicians and researchers without patient representation and involvement.
The TEP has a small sample size.
Equity
In addition to my comments in the validity rating, the developers did not address the inequities noted in their literature review.
Use and Usability
The measure is in use, suggesting its usability. Moreover, the data provided supports the importance of the measure.
Summary
The developers should consider incorporating the committee's feedback and the staff assessment.
Public Comments
PQM CBE 4545e: Inappropriately Broad Empiric Antibiotic
Pneumonia is not only the most common reason for inpatient antibiotic use but also the most common infectious cause of mortality in the US resulting in approximately 1.4 million emergency department visits, 740,000 hospitalizations, 41,000 deaths, and $7.7 billion in inpatient costs each year in the US. The CDC, The Joint Commission, and Centers for Medicare and Medicaid Services recommend that antibiotic stewardship programs implement interventions to promote guideline concordant empiric antibiotic selection.
Antibiotic overuse is a national and international public health emergency with antibiotic resistant infections estimated to directly cause 1.27 million deaths globally and indirectly contribute to 4.95 million deaths. By identifying and improving the percentage of hospitalized CAP patients who receive non-guideline concordant overtreatment with anti-MRSA or anti-Pseudomonal therapy, our proposed eCQM is expected to have direct impact on patients by reducing antibiotic-associated adverse events and reducing development/acquisition of antibiotic resistance.
Certainly I see antibiotic overuse in CAP patient in our rural facility even with a robust stewardship program. I believe a simplified guideline and quality project on a national scale will help with this antibiotic overtreatment problem with see with CAP. We know in most cases 5 days of antibiotic therapy is enough to treat. In my health system we had good treatment days of therapy on the inpatient side but patients were sent home on extended days of treatment from an additional 5 to 10 days. Antibiotic resistance is a real and happening threat in healthcare today. We have make lots of progress in the last ten years, there is still more work to do . Please pass this guideline initiative .
Simple metric for better care
I'm glad to see a simple and useful metric is being proposed to lessen inappropriate (and potentially worse) antibiotics for community acquired pneumonia (CAP). CAP is one of the most common diagnoses and patients often get suboptimal treatment targeting MRSA or Pseudomonas, although standard options like ceftriaxone/azithromycin better covers organisms causing CAP.
I think this metric will make sense to clinicians and will actually be found to be wrong in many patients in some hospitals, where the sepsis oriented vancopime predominates.
CDC supports measure 4545e
The Centers for Disease Control and Prevention (CDC) is strongly supportive of measures PDQM CBE 4540e and 4545e addressing the agents selection and duration of therapy for uncomplicated community acquired pneumonia. Over the past decade, the United States has made major advances in efforts to improve antibiotic use in hospitals as part of efforts to combat the public health crisis of antimicrobial resistance. Nearly all hospitals now have dedicated programs to improve antibiotic use (antibiotic stewardship programs), as required by the Centers for Medicare and Medicaid Services (CMS). The vast majority also now have access to risk adjusted benchmarks of the quantity of their antibiotic use through the CDC’s National Healthcare Safety Network Antibiotic Use Option. This reporting is also now required by CMS.
However, efforts to improve hospital antibiotic use remain challenged by an inability to quickly and easily assess the quality of antibiotic prescribing. Improving the quality of antibiotic use is the ultimate goal of antibiotic stewardship and we need measures that will help us target this goal directly. Studies have consistently shown that assessing the guideline concordance of antibiotic prescribing for common infections consistently leads to better use.
These measures represent a critical 1st step into a new era for improving hospital antibiotic use. They are targeting the single most common reason that antibiotics are prescribed in hospitals. In many hospitals, 50% or more of all antibiotic prescribing is for pneumonia. And because they are electronic, they will allow antibiotic stewardship programs to assess the quality of prescribing without having to conduct labor intensive chart reviews. As you can see from the data in the application, use of the manual versions of these measures led to significant improvements in prescribing.
CDC is currently working towards a pathway for hospitals to be able to report patient level antibiotic use data into the National Healthcare Safety Network through Fast Healthcare Interoperability Resources (FHIR). Patient level data will allow CDC to begin analyzing and reporting on antibiotic use quality in addition to quantity. To do that, we will, of course, need validated measures, like PDQM CBE 4540e and 4545e. CDC urges the panel to endorse these measures.
Comment on this measure
PQM CBE 4545e: Inappropriately Broad Empiric Antibiotic Selection for Adult Hospitalized Patients with Uncomplicated Community-Acquired Pneumonia
Impact to Patient Outcomes
Alignment with other measures
Supporting this quality measure is a step towards enhancing the overall quality of care for patients with CAP, promoting responsible antibiotic use, improving patient outcomes and reducing healthcare costs.
Comments on CBE# 4545e
The Society of Hospital Medicine (SHM) encourages the E&M Committee to consider the impact of the measure and potential unintended consequences or pressures created by the measure. It is uncommon for hospitals to have antibiotic stewardship programs that extend into post-discharge prescribing. This may be challenging to implement particularly for smaller, rural or less resourced hospitals. While SHM broadly supports the clinical concept of this measure, we also question whether it is best positioned as a hospital-level measure (as currently structured) or whether it may be more actionable as a provider-level measure.
SIDP Support For CBE 4545e
SIDP supports PQM endorsement of CBE 4545e to address overly broad empiric antibiotic selection in the treatment of hospitalized patients with community acquired pneumonia. In the absence of risk factors, broad spectrum antibiotics targeted towards methicillin resistant Staphylococcus aureus and Pseudomonas aeruginosa are unnecessary and may contribute to significant patient harm, including increased mortality, extended duration of hospitalization, and increased C. difficile. CBE4540e will ensure that more patients receive up-front treatment with narrower-spectrum, guideline-concordant therapy. Antimicrobial stewardship programs will be able to use the electronic measure to track and report hospital performance and identify opportunities for intervention. SIDP notes patients can meet sepsis criteria and not have risk factors for broad spectrum therapy; administering timely, narrow spectrum antibiotics for community-acquired pneumonia should meet both the sepsis and this proposed CAP measure.
SIDP Support For CBE 4545e
SIDP supports PQM endorsement of CBE 4545e to address overly broad empiric antibiotic selection in the treatment of hospitalized patients with community acquired pneumonia. In the absence of risk factors, broad spectrum antibiotics targeted towards methicillin resistant Staphylococcus aureus and Pseudomonas aeruginosa are unnecessary and may contribute to significant patient harm, including increased mortality, extended duration of hospitalization, and increased C. difficile. CBE4540e will ensure that more patients receive up-front treatment with narrower-spectrum, guideline-concordant therapy. Antimicrobial stewardship programs will be able to use the electronic measure to track and report hospital performance and identify opportunities for intervention. SIDP notes patients can meet sepsis criteria and not have risk factors for broad spectrum therapy; administering timely, narrow spectrum antibiotics for community-acquired pneumonia should meet both the sepsis and this proposed CAP measure.
DCASIP Supports 4545e
Electronic measures to better assess antimicrobial stewardship interventions are sorely needed. The proposed measures CBE 4540e and 4545e, which aim to electronically assess empiric antibiotic selection and treatment duration for adult patients hospitalized with uncomplicated community-acquired pneumonia (CAP), show significant progress in development of electronic clinical quality measures (eCQMs). These metrics are built on evidence-based practices to improve antibiotic durations for a leading indication for antibiotic use in US hospitals.
Successful programs to optimize treatment durations for uncomplicated CAP, such as those described by the Michigan Hospital Medicine Safety Consortium have been hard to replicate in other quality measurement programs because they relied upon labor-intensive manual data collection processes. Laborious manual data capture can detract from the important multifaceted efforts of antimicrobial stewardship programs (ASPs). The eCQMs developed and tested by the groups in Michigan and Utah have shown it is possible to capture similar data electronically in 3 systems: two health systems with Epic and the VA system. Endorsing these measures will allow more widespread development and testing of the feasibility of electronic capture of these data in more varied electronic health record (EHR) data systems. In our experience with electronic data reporting from >70 hospitals in the SE USA, we know how burdensome data collection requirements can be, especially for hospitals with minimal resources or IT support. Thus, we emphasize that it will be essential to engage EHR vendors as key partners in further deployment and implementation of these eCQMs as an important next phase of this work which will greatly benefit stewards to help improve quality of patient care.
The Duke Center for Antimicrobial Stewardship and Infection Prevention (DCASIP) supports the proposed measures CBE 4540e and 4545e. We are excited to see important next steps toward having electronic measures to support our stewardship work.
PQM CBE 4545e: Inappropriately Broad Empiric Antibiotic
PFPS US is strongly supportive of measures PDQM CBE 4540e and 4545e, which address the selection of agents and duration of therapy for uncomplicated community-acquired pneumonia. Over the past decade, significant strides have been made across the United States to improve antibiotic use in hospitals as part of broader efforts to combat the public health crisis of antimicrobial resistance. Most hospitals now have dedicated antibiotic stewardship programs (ASPs) to promote responsible antibiotic use, as required by the Centers for Medicare and Medicaid Services (CMS). Additionally, many hospitals have access to risk-adjusted benchmarks of their antibiotic use through the National Healthcare Safety Network Antibiotic Use Option, a reporting system also required by CMS.
Despite these advances, improving the quality of hospital antibiotic use remains a challenge. Current efforts are hindered by the difficulty of quickly and easily assessing the appropriateness of antibiotic prescribing. The ultimate goal of antibiotic stewardship is to ensure high-quality antibiotic use, and targeted measures are essential for achieving this goal. Research consistently shows that evaluating the guideline adherence of antibiotic prescribing for common infections leads to more effective and appropriate use.
Measures like PDQM CBE 4540e and 4545e are critical first steps in a new era of hospital antibiotic stewardship. These measures target pneumonia — the single most common reason for hospital antibiotic prescriptions. In some hospitals, pneumonia accounts for over 50% of all antibiotic prescribing. Importantly, since these measures are electronic, they will enable stewardship programs to assess prescribing quality without the need for labor-intensive chart reviews. Data from previous applications of manual versions of these measures demonstrate that they significantly improve prescribing practices.
PFPS US is committed to supporting pathways for hospitals to report patient-level antibiotic use data through Fast Healthcare Interoperability Resources (FHIR). Access to patient-level data will enable analysis and reporting on the quality, rather than just the quantity, of antibiotic use. However, achieving this requires validated measures like PDQM CBE 4540e and 4545e. PFPS US urges the panel to endorse these critical measures, which will play a pivotal role in advancing the quality of hospital antibiotic use nationwide.
Organization: PFPS US