Inappropriately Broad Empiric Antibiotic Selection for Adult Hospitalized Patients with Uncomplicated Community-Acquired Pneumonia

CBE ID

4545e

1.4 Project

Initial Recognition and Management

1.1 New or Maintenance

New

Previous Endorsement Cycle

Fall 2024

Is Under Review

Yes

1.3 Measure Description

The Inappropriately Broad Empiric Antibiotic Selection for Adult Hospitalized Patients with Uncomplicated Pneumonia measure is a process measure representing the annual percentage of hospitalized adults with uncomplicated community-acquired pneumonia. Here, we defined “inappropriately broad” as any antibiotic therapy targeting methicillin-resistant Staphylococcus aureus (MRSA) or Pseudomonas aeruginosa in patients without risk factors for one of those organisms. The measure will be calculated using electronic health record (EHR) data and is intended for use at the facility level for both quality improvement and pay-for-performance.

1.5 Measure Type

Process

1.6 Composite Measure

No

1.7 Electronic Clinical Quality Measure (eCQM)

Yes

1.8 Level Of Analysis

Facility

1.9 Care Setting

Hospital: Acute Care Facility

Hospital: Critical Access

Hospital: Inpatient

1.10 Measure Rationale

The overall objective of this electronic clinical quality measure is to quantify inappropriately broad empiric antibiotic use in hospitalized adults with uncomplicated community-acquired pneumonia (CAP). Here, we defined “inappropriately broad” as any antibiotic therapy targeting methicillin-resistant Staphylococcus aureus (MRSA) or Pseudomonas aeruginosa in patients without risk factors for one of those organisms.

Antibiotic overuse is a national and international public health emergency with antibiotic resistant infections estimated to directly cause 1.27 million deaths globally and indirectly contribute to 4.95 million deaths.¹ National studies by the Centers for Disease Control and Prevention (CDC) estimate that up to 50% of hospitalized patients receive antibiotic therapy, most commonly for pneumonia, and that up to 40% of antibiotic prescribing could be improved.²

Pneumonia is not only the most common reason for inpatient antibiotic use but also the most common infectious cause of mortality in the US resulting in approximately 1.4 million emergency department visits, 740 000 hospitalizations, 41 000 deaths, and $7.7 billion in inpatient costs each year in the US.^3-6 Adverse consequences of unnecessarily broad empiric therapy include an increased adjusted risk of death, kidney injury, and serious secondary infections.^7-9

How will this measure improve quality of care? By establishing a standardized process to assess inappropriately broad empiric antibiotic use for CAP, a larger proportion of patients will potentially receive appropriate care consistent with the 2019 American Thoracic Society/Infectious Disease Society of America (ATS/IDSA) CAP national guidelines.¹⁰ Appropriateness of antibiotic therapy for pneumonia is a priority for numerous federal and accreditation organizations–including CDC, The Joint Commission, and Centers for Medicare and Medicaid Services–and is not currently captured in typical quality improvement measures. For example, the National Healthcare Safety Network (NHSN) antimicrobial use (AU) measures focus on quantifying antibiotic use and comparing to expected values, with no assessment of appropriateness of empiric therapy. Notably, the NHSN AU initiative could be augmented with an eCQM to assess appropriateness of empiric antibiotics for CAP, the most common indication for inpatient antibiotic use.

What are the benefits or improvements in quality envisioned by this measure? Inappropriately broad empiric therapy for CAP is common. A recent report on 8,286 non-intensive care unit (ICU) hospitalized CAP patients from 67 Michigan hospitals (assessed using the chart review measure from which our eCQM was adapted) showed that 2,215 (26.7%) received inappropriately broad empiric treatment (i.e., anti-MRSA or anti-Pseudomonal coverage in patients eligible for standard CAP coverage per ATS/IDSA guidelines).¹¹ Compared to patients who received standard CAP antibiotic treatment, patients receiving inappropriately broad antibiotics had higher 30-day readmissions, more transfers to ICU and antibiotic-associated adverse events, and longer hospitalizations.¹¹ Similarly, a retrospective analysis of 88,605 patients with CAP across the Veterans Health Administration health care system found that adding empirical anti-MRSA therapy to standard CAP therapy) was associated with an increased adjusted risk of death, kidney injury, and secondary infections (C. difficile, vancomycin-resistant Enterococcus infection, and resistant gram-negative rod infections).⁸ Given that up to 90% of empiric anti-MRSA or anti-Pseudomonal antibiotic therapy is non-guideline concordant,^8,11 there is substantial potential benefit to patients and the US by reducing inappropriately broad empiric antibiotic selection. National surveillance of drug-related adverse events estimated that even a small reduction in unnecessary antibiotic use could significantly decrease the direct risks of drug-related adverse event in individual patients.¹² Thus, our guideline-based measure has the potential to improve patient care for a large number of patients hospitalized with CAP across the US.

The full reference list can be found in Section 2.2.

1.11 Measure Webpage

https://example.com

1.20 Testing Data Sources

Electronic Health Records

1.25 Data Sources

The anticipated data source for the measure is electronic health record (EHR) data from inpatient hospital admissions, including discharge diagnosis codes, pharmacy and medication administration, and imaging records. These data are all collected routinely during usual clinical care through the process of inpatient hospitalizations.

1.14 Numerator

From the denominator population, identify patients without risk factors for methicillin-resistant Staphylococcus aureus (MRSA) or Pseudomonas aeruginosa who received empiric antibiotics targeting MRSA or Pseudomonas.

1.14a Numerator Details

From the denominator population, identify who received empiric (within first 48 hours of emergency department arrival) antibiotics targeting MRSA or Pseudomonas

Broad Spectrum Antibiotics for MDRO Value Set, OID:2.16.840.1.113762.1.4.1264.29
(Sheet Name: “BroadSpectrumAntibioticsMDRO”)

1.15 Denominator

Identify all adult patients with an inpatient (or observation) non-ICU hospitalization in which the discharge diagnosis includes pneumonia or sepsis/respiratory failure (RF) who received a respiratory antibiotic within 48 hours of hospitalization, received chest imaging within +/- 3 days of the hospital encounter, were not transferred from another hospital, and do not have a concurrent infection. Restrict to patients with uncomplicated pneumonia and without risk factors for MRSA or Pseudomonas.

1.15a Denominator Details

Specific Inclusion Criteria (see Data Dictionary for detailed specifications):

ICD10 discharge code for pneumonia or sepsis AND respiratory failure
CAP, Sepsis, Respiratory Failure Diagnostic Value Set, OID:2.16.840.1.113762.1.4.1264.24
(Sheet Name: “CAPSepsisRespFailureDx”)
Received a respiratory antibiotic within the first 48 hours of hospitalization
Antibiotic Usage for CAP Diagnosis Value Set, OID: 2.16.840.1.113762.1.4.1264.25
(Sheet Name: “AntibioticUsageforCAP”)
Age > 18 years on admission
Admitted as inpatient or in observation awaiting inpatient admission
Encounter Inpatient Value Set, OID: 2.16.840.1.113883.3.666.5.307
(Sheet Name: “Encounterinpatient”)
Did not die in first 48 hours of hospitalization
code "Dead (finding)": '419099009' from "SNOMEDCT" display 'Dead (finding)'
No time spent in an ICU in first 48 hours of hospitalization
Intensive Care Unit Value Set, OID:2.16.840.1.113762.1.4.1029.204
(Sheet Name: “IntensiveCareUnit”
Not admitted from another acute care hospital or long-term acute care facility
code "Hospital admission, transfer from other hospital or health care facility (procedure)": '4563007' from "SNOMEDCT" display 'Hospital admission, transfer from other hospital or health care facility (procedure)'
No concurrent infection during hospitalization by ICD 10
CAP, Concurrent Infections Value Set, OID: 2.16.840.1.113762.1.4.1264.23
(Sheet Name: “CAPConcurentInfections”)
Chest imaging within 3 days of admission

‘Sheet Name’ refers to specific tab in the Data Dictionary
*note–all times are calculated from arrival to the emergency department

1.15d Age Group

Adults (18-64 years)

Older Adults (65 years and older)

1.15b Denominator Exclusions

To identify uncomplicated CAP patients: Exclude if:
On mechanical ventilation in first 48 hours
Absolute neutrophil count (ANC) <500 cells/uL
Cystic fibrosis
Bronchiectasis
HIV
Tracheostomy
Transplant in prior year
Hematologic malignancy
Pulmonary complication (empyema, lung abscess, necrotizing pneumonia

1.15c Denominator Exclusions Details

To identify uncomplicated CAP patients: Exclude if:
On mechanical ventilation in first 48 hours
Comorbidities Indicated with CAP Value Set, OID:2.16.840.1.113762.1.4.1264.26
(Sheet Name: “ComrobiditiesIndicatedwithCAP”)
Absolute neutrophil count <500 cells/uL
Complete Blood Count (with Diff), OID:1.3.6.1.4.1.6997.4.1.2.271.13.38167.1.1.999.594
code "Neutrophils [#/volume] in Blood": '26499-4' from "LOINC" display 'Neutrophils [#/volume] in Blood'
(Sheet Name: (“CompleteBloodCount”)
Cystic fibrosis
Comorbidities Indicated with CAP Value Set, OID:2.16.840.1.113762.1.4.1264.26
(Sheet Name: “ComrobiditiesIndicatedwithCAP”)
Bronchiectasis
Comorbidities Indicated with CAP Value Set, OID:2.16.840.1.113762.1.4.1264.26
(Sheet Name: “ComrobiditiesIndicatedwithCAP”)
Human immunodeficiency virus (HIV)
code "Human immunodeficiency virus [HIV] disease": 'B20' from "ICD10CM" display 'Human immunodeficiency virus [HIV] disease'
code "Patient immunocompromised (finding)": '370388006' from "SNOMEDCT" display 'Patient immunocompromised (finding)' (if facility cannot report HIV status)
Tracheostomy
Comorbidities Indicated with CAP Value Set, OID:2.16.840.1.113762.1.4.1264.26
(Sheet Name: “ComrobiditiesIndicatedwithCAP”)
Transplant in prior year
Major Transplant Value Set, OID:2.16.840.1.113883.3.464.1003.198.12.1075
(Sheet Name: “MajorTransplant”)
Hematologic malignancy
Comorbidities Indicated with CAP Value Set, OID:2.16.840.1.113762.1.4.1264.26
(Sheet Name: “ComrobiditiesIndicatedwithCAP”)
Pulmonary complication (empyema, lung abscess, necrotizing pneumonia)
Comorbidities Indicated with CAP Value Set, OID:2.16.840.1.113762.1.4.1264.26
(Sheet Name: “ComrobiditiesIndicatedwithCAP”)

Exclude patients with risk factors for MRSA/Pseudomonas aeruginosa:
Pseudomonas aeruginosa in respiratory culture in prior year
Pseudomonas aeruginosa (Organism or Substance in Lab Results) Value Set, OID:2.16.840.1.113762.1.4.1146.1679
(Sheet Name: “PseudomonasAeruginosa”)
Methicillin-resistantStaphylococcus aureus in respiratory culture in prior year
MRSA (Disorders(SNOMED), OID:2.16.840.1.113762.1.4.1146.1447
(Sheet Name: “MRSADisorders”)

Severe pneumonia AND prior hospitalization with IV antibiotics in past 3 months

Severe pneumonia defined by ≥2 of the following on a single day in the first 48 hours of encounter:
Respiratory rate >30 breaths/min
Respiratory Rate Value Set, OID:2.16.840.1.113762.1.4.1045.130
(Sheet Name: “RespiratoryRate”)
≥5L oxygen or SpO2<90
code "Inhaled oxygen flow rate": '3151-8' from "LOINC" display 'Inhaled oxygen flow rate' or
Oxygen Saturation in Blood Value Set, OID:2.16.840.1.113762.1.4.1222.1610
(Sheet Name: “OxygenSaturation”
White blood cell (WBC) <4000 cell/uL
Complete Blood Count (with Diff) Value Set, OID:1.3.6.1.4.1.6997.4.1.2.271.13.38167.1.1.999.594
(Sheet Name: “CompleteBloodCount”)
Blood urea nitrogen (BUN) >20 mg/dL
Blood Urea Nitrogen Value Set, OID:2.16.840.1.113762.1.4.1222.113
(Sheet Name: “BloodUreaNitrogen”)
Platelets <100 K/uL
Complete Blood Count (with Diff) Value Set, OID:1.3.6.1.4.1.6997.4.1.2.271.13.38167.1.1.999.594
(Sheet Name: “CompleteBloodCount”)
Temperature <36 C
Body Temperature Value Set, OID:2.16.840.1.113762.1.4.1222.113
(Sheet Name: “BodyTemperature”)
Systolic blood pressure (SBP) <80 mmHg
Systolic Blood Pressure Value Set, OID:2.16.840.1.113762.1.4.1045.163
(Sheet Name: “SystolicBloodPressure”)
AND
Prior hospitalization with IV antibiotics in past 3 months
Prior hospitalization
code "Hospital admission, transfer from other hospital or health care facility (procedure)": '4563007' from "SNOMEDCT" display 'Hospital admission, transfer from other hospital or health care facility (procedure)'
IV antibiotics
Intravenous Medication Route Value Set, OID:2.16.840.1.113762.1.4.1190.75
(Sheet Name: “IntravenousMedicationRoute”)

‘Sheet Name’ refers to specific tab in the Data Dictionary
*note–all times are calculated from arrival to the emergency department

1.12 Attach MAT Output

IBEAntibioticsCAP-v0.0.000-QDM.zip

1.13 Attach Data Dictionary

DataDictionary_4545e_empiric.xlsx

1.13a Data dictionary not attached

No

1.16 Type of Score

Rate/proportion

1.17 Measure Score Interpretation

Better quality = Lower score

1.18 Calculation of Measure Score

Step 1: Define Initial Population and Denominator
Step 2: Apply denominator exclusions
Step 3: Define the numerator
Step 4: Calculate the measure score

1.18a Attach measure score calculation diagram, if applicable

PQM CBE 4545e Score Calculation Diagram.pdf

1.19 Measure Stratification Details

The measure is not stratified.

1.26 Minimum Sample Size

Table 3 (see Supplemental Materials attachment) indicates the minimum annual number of qualifying cases needed for the denominator to reach each target reliability level at a given facility. In order to achieve a desired reliability of 0.8, each hospital would need to include 96 cases annually. For acceptable reliability (0.7), 56 annual cases would be required, and for high reliability (0.9), 216 annual cases would be required. If a facility has fewer than the minimum number, the hospital is still encouraged to report performance on this measure.

We estimated the minimum sample size using the ICC (intraclass correlation) and the Spearman Brown prophecy formula^13-15 The ICC was calculated from 109 Veterans Affairs (VA) healthcare systems. The Spearman Brown formula was used in prior inpatient CAP measures to determine minimal sample size (PN-3a: Blood Cultures Performed within 24 Hours Prior to or 24 Hours After Hospital Arrival for Patients who were Transferred or Admitted to the ICU within 24 Hours of Hospital Arrival; PN-6: Initial Antibiotic Selection for Community-Acquired Pneumonia [CAP in Immunocompetent Patients]; Inappropriate Diagnosis of CAP [CBE ID 3671]); Percent of Hospitalized Pneumonia Patients with Chest Imaging [CBE ID 4440e]).^16-18 The minimum recommended sample sizes here are within the range of those previously required for inpatient CAP–for example, the minimum recommended sample size to achieve 0.80 reliability was 59 for the Inappropriate Diagnosis of Community-Acquired Pneumonia measure¹⁷ and 174 for the Percent of Hospitalized Pneumonia Patients with Chest Imaging measure, an eCQM endorsed in 2024.¹⁶

The full reference list can be found in Section 2.2.

7.1 Supplemental Attachment

Supplemental Materials for PQM CBE 4545e.docx

Steward

University of Utah

Steward Organization POC Email

[email protected]

Steward Organization URL

https://www.utah.edu

Steward Organization Copyright

n/a

Measure Developer Secondary Point Of Contact

Andrea White
University of Utah, Division of General Internal Medicine
30 North Mario Capecchi Drive, 3rd Floor South
Salt Lake City, UT 84112
United States

Measure Developer Secondary Point Of Contact Email

[email protected]

2.1 Attach Logic Model

PQM CBE 4545e Measure Logic Diagram.pdf

2.2 Evidence of Measure Importance

Pneumonia is not only the most common reason for inpatient antibiotic use but also the most common infectious cause of mortality in the US resulting in approximately 1.4 million emergency department visits, 740,000 hospitalizations, 41,000 deaths, and $7.7 billion in inpatient costs each year in the US.^3-6In a recent report on non-ICU hospitalized CAP patients from 67 Michigan hospitals, date collected using the chart review measure from which our eCQM was adapted showed that of 8,286 CAP patients, 2,215 (26.7%) received inappropriately broad empiric treatment (i.e., eligible for standard CAP coverage per ATS/IDSA guidelines). These patients had higher 30-day readmissions, more transfers to ICU and antibiotic-associated adverse events, and longer hospitalizations.¹¹ Similarly, a retrospective analysis of 88,605 patients with CAP across the Veterans Health Administration health care system found that empirical anti-MRSA (in addition to standard CAP therapy) was associated with an increased adjusted risk of death, kidney injury, and secondary infections (C. difficile, vancomycin-resistant Enterococcus infection, and resistant gram-negative rod infections).⁸ Notably, associations between anti-MRSA therapy and 30-day mortality were similar among patients admitted to the ICU, those with high risk for MRSA, and those with MRSA detected on surveillance testing, suggesting that empiric anti-MRSA therapy was not associated with reduced mortality for any pneumonia patient group.⁸ Using data reported to the National Healthcare Safety Network (NHSN) antimicrobial use (AU) option, the effect of reducing inpatient broad-spectrum antibiotic exposure was modeled and estimated that reduction in use of broad-spectrum antibiotics by 30% could result in a 26% reduction in C. difficile infection.² Similarly, broad spectrum antibiotic use is associated with acquisition of multi-drug resistant organisms and transmission of those organisms in nursing homes.¹⁹ Taken together, our guideline-based measure has the ability to improve patient care for a large number of patients hospitalized with CAP across the US.

The eCQM for CAP empiric antibiotic selection complements other quality efforts, including sepsis-related measures, which aim to increase early use of antibiotic therapy for patients with suspected infection. This metric targets CAP—the largest cause of sepsis—and specifically addresses correct empiric therapy and avoiding inappropriately broad empiric antibiotic selection (i.e., therapy targeting MRSA or Pseudomonas aeruginosa in patients without risk factors), not just timing of antibiotic use. In addition, we anticipate this measure can augment the NHSN AU module (which assesses overall observed vs. expected antibiotic use in hospitals but not antibiotic appropriateness) and help hospitals understand the quality of their prescribing for the most common infection treated in hospitals–CAP.

References (for full submission, not just 2.2 Evidence of Measure Importance):

1.            Antimicrobial Resistance Collaborators. Global burden of bacterial antimicrobial resistance in 2019: a systematic analysis. Lancet. Feb 12 2022;399(10325):629-655. doi:10.1016/s0140-6736(21)02724-0
2.            Fridkin S, Baggs J, Fagan R, et al. Vital signs: improving antibiotic use among hospitalized patients. MMWR Morbidity and mortality weekly report. Mar 7 2014;63(9):194-200.
3.            Sterrantino C, Trifirò G, Lapi F, et al. Burden of community-acquired pneumonia in Italian general practice. Eur Respir J. Dec 2013;42(6):1739-42. doi:10.1183/09031936.00128713
4.            Partouche H, Lepoutre A, Vaure CBD, Poisson T, Toubiana L, Gilberg S. Incidence of all-cause adult community-acquired pneumonia in primary care settings in France. Med Mal Infect. Sep 2018;48(6):389-395. doi:10.1016/j.medmal.2018.02.012
5.            McDermott KW aRM. Most Frequent Principal Diagnoses for Inpatient Stays in U.S. Hospitals, 2018. 2021. https://hcup-us.ahrq.gov/reports/statbriefs/sb277-Top-Reasons-Hospital-Stays-2018.pdf#:~:text=The%20most%20frequent%20principal%20diagnoses%20for%20hospitalizations%20in,caused%20by%20tuberculosis%29%2C%20and%20diabetes%20mellitus%20with%20complication
6.            Vaughn VM, Dickson RP, Horowitz JK, Flanders SA. Community-Acquired Pneumonia: A Review. Jama. Oct 15 2024;332(15):1282-1295. doi:10.1001/jama.2024.14796
7.            Branch-Elliman W, O’Brien W, Strymish J, Itani K, Wyatt C, Gupta K. Association of Duration and Type of Surgical Prophylaxis With Antimicrobial-Associated Adverse Events. JAMA Surgery. 2019;154(7):590-598. doi:10.1001/jamasurg.2019.0569
8.            Jones BE, Ying J, Stevens V, et al. Empirical Anti-MRSA vs Standard Antibiotic Therapy and Risk of 30-Day Mortality in Patients Hospitalized for Pneumonia. JAMA Intern Med. Apr 1 2020;180(4):552-560. doi:10.1001/jamainternmed.2019.7495
9.            Attridge RT, Frei CR, Restrepo MI, et al. Guideline-concordant therapy and outcomes in healthcare-associated pneumonia. Eur Respir J. Oct 2011;38(4):878-87. doi:10.1183/09031936.00141110
10.          Metlay JP, Waterer GW, Long AC, et al. Diagnosis and Treatment of Adults with Community-acquired Pneumonia. An Official Clinical Practice Guideline of the American Thoracic Society and Infectious Diseases Society of America. Am J Respir Crit Care Med. Oct 1 2019;200(7):e45-e67. doi:10.1164/rccm.201908-1581ST
11.          Gandhi T, Petty L, Vaughn V, et al. Risk Factors and outcomes associated with inappropriate empiric broad-spectrum antibiotic use in hospitalized patients with community-acquired pneumonia. Antimicrobial Stewardship & Healthcare Epidemiology. 2023;3(S2):s31-s32. doi:10.1017/ash.2023.258
12.          Shehab N, Patel PR, Srinivasan A, Budnitz DS. Emergency department visits for antibiotic-associated adverse events. Clin Infect Dis. Sep 15 2008;47(6):735-43. doi:10.1086/591126
13.          van Ast JF, Talmon JL, Renier WO, Hasman A. An approach to knowledge base construction based on expert opinions. Methods Inf Med. 2004;43(4):427-32.
14.          Zimmerman DW, Williams RH, Burkheimer GJ. Dependence of reliability of multiple-choice tests upon number of choices per item: prediction from the Spearman-Brown formula. Psychol Rep. Dec 1966;19(3):1239-43. doi:10.2466/pr0.1966.19.3f.1239
15.          de Vet HCW, Mokkink LB, Mosmuller DG, Terwee CB. Spearman-Brown prophecy formula and Cronbach's alpha: different faces of reliability and opportunities for new applications. J Clin Epidemiol. May 2017;85:45-49. doi:10.1016/j.jclinepi.2017.01.013
16.          Partnership for Quality Measurement. Percent of hospitalized pneumonia patients with chest imaging confirmation, CBE ID 4440e. Accessed 9/25/2024, https://p4qm.org/measures/4440e
17.          Partnership for Quality Measurement. Inappropriate diagnosis of community-acquired pneumonia (CAP) in hospitalized medical patients, CBE ID 3671. Accessed 9/24/2024, https://p4qm.org/measures/3671
18.          The Joint Commission. Specifications manual for Joint Commission National Quality Measures (v2015A): Pneumonia (PN). Accessed 9/25/2024, https://manual.jointcommission.org/releases/TJC2015A/Pneumonia.html#A_42Pneumonia_40PN_41_Initial_Patient_Population_42
19.          Gontjes KJ, Gibson KE, Lansing BJ, et al. Association of Exposure to High-risk Antibiotics in Acute Care Hospitals With Multidrug-Resistant Organism Burden in Nursing Homes. JAMA Netw Open. Feb 1 2022;5(2):e2144959. doi:10.1001/jamanetworkopen.2021.44959
20.          Graber CJ, Jones MM, Goetz MB, et al. Decreases in Antimicrobial Use Associated With Multihospital Implementation of Electronic Antimicrobial Stewardship Tools. Clin Infect Dis. Aug 22 2020;71(5):1168-1176. doi:10.1093/cid/ciz941
21.          Kelly AA, Jones MM, Echevarria KL, et al. A Report of the Efforts of the Veterans Health Administration National Antimicrobial Stewardship Initiative. Infect Control Hosp Epidemiol. May 2017;38(5):513-520. doi:10.1017/ice.2016.328
22.          Goetz MB, Willson T, Rubin MA, Stevens VW, Graber CJ. Antimicrobial use before and during COVID-19: data from 108 Veterans Affairs medical centers. Antimicrobial Stewardship & Healthcare Epidemiology. 2024;4(1):e109. e109. doi:10.1017/ash.2024.352
23.          Spivak ES, Burk M, Zhang R, et al. Management of Bacteriuria in Veterans Affairs Hospitals. Clin Infect Dis. Sep 15 2017;65(6):910-917. doi:10.1093/cid/cix474
24.          Sutton JD, Carico R, Burk M, et al. Inpatient Management of Uncomplicated Skin and Soft Tissue Infections in 34 Veterans Affairs Medical Centers: A Medication Use Evaluation. Open Forum Infect Dis. Jan 2020;7(1):ofz554. doi:10.1093/ofid/ofz554
25.          Madaras-Kelly KJ, Burk M, Caplinger C, et al. Total duration of antimicrobial therapy in veterans hospitalized with uncomplicated pneumonia: Results of a national medication utilization evaluation. Journal of hospital medicine. Dec 2016;11(12):832-839. doi:10.1002/jhm.2648
26.          Livorsi DJ, Branch-Elliman W, Drekonja D, et al. Research agenda for antibiotic stewardship within the Veterans' Health Administration, 2024-2028. Infect Control Hosp Epidemiol. Feb 2 2024:1-7. doi:10.1017/ice.2024.6
27.          Qian ET, Casey JD, Wright A, et al. Cefepime vs Piperacillin-Tazobactam in Adults Hospitalized With Acute Infection: The ACORN Randomized Clinical Trial. Jama. Oct 24 2023;330(16):1557-1567. doi:10.1001/jama.2023.20583
28.          Ciarkowski CE, Timbrook TT, Kukhareva PV, et al. A Pathway for Community-Acquired Pneumonia With Rapid Conversion to Oral Therapy Improves Health Care Value. Open Forum Infect Dis. Nov 2020;7(11):ofaa497. doi:10.1093/ofid/ofaa497
29.          Shorr AF, Owens RC, Jr. Guidelines and quality for community-acquired pneumonia: measures from the Joint Commission and the Centers for Medicare and Medicaid Services. American journal of health-system pharmacy. Jun 15 2009;66(12 Suppl 4):S2-7. doi:10.2146/090087a
30.          Centers for Medicare and Medicaid. Merit-Based Incentive Payment System (MIPS): Simple Pneumonia with Hospitalization Measure, 2020 performance period. Accessed 9/27/2024, https://qpp.cms.gov/docs/cost_specifications/2019-12-17-mif-ebcm-pna-hosp.pdf
31.          Centers for Medicare & Medicaid Services. Quality ID #111: Pneumococcal vaccination status for older adults. . Accessed 9/27/2024, https://qpp.cms.gov/docs/QPP_quality_measure_specifications/CQM-Measures/2023_Measure_111_MedicarePartBClaims.pdf
32.          Centers for Medicare & Medicaid Services. Eligible Hospital / Critical Access Hospital eCQMs: hospital quality reporting table of eCQMs. Accessed 9/27/2024, https://ecqi.healthit.gov/sites/default/files/Hybrid-EH-CAH-MeasuresTable-2023-11.pdf
33.          Agency for Healthcare Research and Quality. AHRQ IQI technical documentation, version v2023. Rockville, MD. Accessed 9/27/2024, https://qualityindicators.ahrq.gov/measures/iqi_resources
34.          CDC. Core Elements of Hospital Antibiotic Stewardship Programs. US Department of Health and Human Services. CDC. Accessed August 3, 2022. https://www.cdc.gov/antibiotic-use/healthcare/pdfs/hospital-core-elements-H.pdf
35.          The Joint Commission. R3 Report Issue 35: New and Revised Requirements for Antibiotic Stewardship 2022. https://www.jointcommission.org/-/media/tjc/documents/standards/r3-reports/r3_antibioticstewardship_july2022_final.pdf
36.          QSO-22-20-Hospitals Infection Prevention and Control and Antibiotic Stewardship Program Interpretive Guidance Update (2022).
37.          van Santen KL, Edwards JR, Webb AK, et al. The Standardized Antimicrobial Administration Ratio: A New Metric for Measuring and Comparing Antibiotic Use. Clin Infect Dis. Jul 2 2018;67(2):179-185. doi:10.1093/cid/ciy075
38.          Department of Health and Human Services Centers for Medicare & Medicaid Services. Hospital Inpatient Prospective Payment Systems for Acute Care Hospitals and the Long-Term Care Hospital Prospective Payment System and Policy Changes and Fiscal Year 2023 Rates; Quality Programs and Medicare Promoting Interoperability Program Requirements for Eligible Hospitals and Critical Access Hospitals; Costs Incurred for Qualified and Non-Qualified Deferred Compensation Pland; and Changes to Hospital and Critical Access Hospital Conditions of Participation. Federal Register. 2022;87 (153):487780-49499.
39.          Moehring R, Vaughn VM. Development of Inpatient Stewardship Metrics: Is It Time for Public Reporting? Infect Dis Clin North Am. Dec 2023;37(4):853-871. doi:10.1016/j.idc.2023.07.006
40.          Watkins RR, Lemonovich TL. Diagnosis and management of community-acquired pneumonia in adults. Am Fam Physician. Jun 1 2011;83(11):1299-306.
41.          NHSN. Antimicrobial Use and Resistance (AUR) Module. Centers for Disease Control. https://www.cdc.gov/nhsn/pdfs/pscmanual/11pscaurcurrent.pdf
42.          Fishman N, America SfHEo, America IDSo. Policy statement on antimicrobial stewardship by the Society for Healthcare Epidemiology of America (SHEA), the Infectious Diseases Society of America (IDSA), and the Pediatric Infectious Diseases Society (PIDS). Infection Control & Hospital Epidemiology. 2012;33(4):322-327.
43.          Barlam TF, Cosgrove SE, Abbo LM, et al. Implementing an Antibiotic Stewardship Program: Guidelines by the Infectious Diseases Society of America and the Society for Healthcare Epidemiology of America. Clin Infect Dis. May 15 2016;62(10):e51-77. doi:10.1093/cid/ciw118
44.          Macfarlane J, Holmes W, Macfarlane R, Britten N. Influence of patients' expectations on antibiotic management of acute lower respiratory tract illness in general practice: questionnaire study. BMJ (Clinical research ed). 1997;315(7117):1211-1214. doi:10.1136/bmj.315.7117.1211
45.          Borek AJ, Campbell A, Dent E, et al. Implementing interventions to reduce antibiotic use: a qualitative study in high-prescribing practices. BMC Fam Pract. Jan 23 2021;22(1):25. doi:10.1186/s12875-021-01371-6
46.          Tonkin-Crine S, Anthierens S, Francis NA, et al. Exploring patients' views of primary care consultations with contrasting interventions for acute cough: a six-country European qualitative study. NPJ Prim Care Respir Med. Jul 17 2014;24:14026. doi:10.1038/npjpcrm.2014.26
47.          Centers for Disease Control and Prevention. NHSN list of common commensals. Accessed 9/30/2024, https://view.officeapps.live.com/op/view.aspx?src=https%3A%2F%2Fwww.cdc.gov%2Fnhsn%2FXLS%2Fmaster-organism-Com-Commensals-Lists.xlsx&wdOrigin=BROWSELINK)
48.          Mandell LA, Wunderink RG, Anzueto A, et al. Infectious Diseases Society of America/American Thoracic Society consensus guidelines on the management of community-acquired pneumonia in adults. Clin Infect Dis. Mar 1 2007;44 Suppl 2:S27-72. doi:10.1086/511159
49.          Gupta AB, Flanders SA, Petty LA, et al. Inappropriate Diagnosis of Pneumonia Among Hospitalized Adults. JAMA Intern Med. May 1 2024;184(5):548-556. doi:10.1001/jamainternmed.2024.0077
50.          Vaughn VM, Gandhi TN, Hofer TP, et al. A Statewide Collaborative Quality Initiative to Improve Antibiotic Duration and Outcomes in Patients Hospitalized With Uncomplicated Community-Acquired Pneumonia. Clin Infect Dis. Aug 31 2022;75(3):460-467. doi:10.1093/cid/ciab950
51.          Gupta A, Petty L, Gandhi T, et al. Overdiagnosis of urinary tract infection linked to overdiagnosis of pneumonia: a multihospital cohort study. BMJ Qual Saf. May 2022;31(5):383-386. doi:10.1136/bmjqs-2021-013565
52.          Vaughn VM, Gandhi TN, Chopra V, et al. Antibiotic Overuse After Hospital Discharge: A Multi-hospital Cohort Study. Clin Infect Dis. Dec 6 2021;73(11):e4499-e4506. doi:10.1093/cid/ciaa1372
53.          Gandhi TN, Vaughn VM, Petty LA, et al. 2893. The Michigan Hospital Medicine Safety Consortium: Improving Patient Care by Reducing Excessive Antibiotic Use in Patients Hospitalized with Community-Acquired Pneumonia. Open Forum Infectious Diseases. 2019;6(Supplement_2):S80-S81. doi:10.1093/ofid/ofz359.171
54.          White AT, Vaughn V, Petty LA, et al. Development of patient safety measures to identify inappropriate diagnosis of common infections. Clin Infect Dis. Jan 31 2024;doi:10.1093/cid/ciae044
55.          Mehta HB, Li S, An H, Goodwin JS, Alexander GC, Segal JB. Development and Validation of the Summary Elixhauser Comorbidity Score for Use With ICD-10-CM-Coded Data Among Older Adults. Annals of internal medicine. Oct 2022;175(10):1423-1430. doi:10.7326/m21-4204
56.          National Quality Forum. Scientific Methods Panel Evaluation Guidelines. Accessed 10/7/2024, https://www.qualityforum.org/Measuring_Performance/Scientific_Methods_Panel.aspx
57.          Centers for Medicare & Medicaid Services. MMS Blueprint Supplemental Material: Technical Expert Panels. https://www.cms.gov/files/document/blueprint-technical-expert-panels.pdf
58.          Centers for Medicare & Medicaid Services. Measures Management System Risk Adjustment. Centers for Medicare & Medicaid. Accessed 8/20/24, https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/MMS/Downloads/Risk-Adjustment.pdf
59.          Ramirez JA, Wiemken TL, Peyrani P, et al. Adults Hospitalized With Pneumonia in the United States: Incidence, Epidemiology, and Mortality. Clin Infect Dis. Nov 13 2017;65(11):1806-1812. doi:10.1093/cid/cix647
60.          Burton DC, Flannery B, Bennett NM, et al. Socioeconomic and racial/ethnic disparities in the incidence of bacteremic pneumonia among US adults. Am J Public Health. Oct 2010;100(10):1904-11. doi:10.2105/ajph.2009.181313
61.          Altawalbeh SM, Wateska AR, Nowalk MP, et al. Societal Cost of Racial Pneumococcal Disease Disparities in US Adults Aged 50 Years or Older. Appl Health Econ Health Policy. Jan 2024;22(1):61-71. doi:10.1007/s40258-023-00854-0
62.          Nowalk MP, Wateska AR, Lin CJ, et al. Racial Disparities in Adult Pneumococcal Vaccination Indications and Pneumococcal Hospitalizations in the U.S. J Natl Med Assoc. Oct 2019;111(5):540-545. doi:10.1016/j.jnma.2019.04.011
63.          Hausmann LR, Ibrahim SA, Mehrotra A, et al. Racial and ethnic disparities in pneumonia treatment and mortality. Med Care. Sep 2009;47(9):1009-17. doi:10.1097/MLR.0b013e3181a80fdc
64.          Wiemken TL, Carrico RM, Furmanek SP, et al. Socioeconomic Position and the Incidence, Severity, and Clinical Outcomes of Hospitalized Patients With Community-Acquired Pneumonia. Public Health Rep. May/Jun 2020;135(3):364-371. doi:10.1177/0033354920912717
65.          Downing NS, Wang C, Gupta A, et al. Association of Racial and Socioeconomic Disparities With Outcomes Among Patients Hospitalized With Acute Myocardial Infarction, Heart Failure, and Pneumonia: An Analysis of Within- and Between-Hospital Variation. JAMA Netw Open. Sep 7 2018;1(5):e182044. doi:10.1001/jamanetworkopen.2018.2044

2.3 Anticipated Impact

The 2019 ATS/IDSA CAP guidelines recommend not prescribing empiric anti-MRSA or anti-Pseudomonal therapy in uncomplicated CAP patients without risk factors for MRSA or Pseudomonas aeruginosa.¹⁰ Similarly, those guidelines recognized that prior “HCAP” (healthcare associated pneumonia) criteria had recommended anti-MRSA or anti-Pseudomonal therapy in patients for whom it was unnecessary—leading to a rapid increase in anti-MRSA or anti-Pseudomonal therapy without improved patient outcomes. Despite these updated recommendations, de-adoption of widespread use of anti-MRSA and anti-Pseudomonal therapy for CAP has been slow.

If the currently proposed measure is implemented, we expect a fairly rapid reduction in inappropriately broad empiric therapy in uncomplicated CAP patients without risk factors for MRSA or Pseudomonas aeruginosa given demonstrated improvements in empiric antibiotic use with the following large-scale measurement and antibiotic stewardship efforts:
When the chart review version of the measure on which this eCQM was based was used as a pay-for-performance measure across 69 Michigan hospitals, inappropriate empiric broad spectrum antibiotic use saw a 42% relative decrease (from ~20% across the collaborative to ~11.6%; see Figure 1 in Supplemental Materials attachment). No unintended consequences were identified.
The VA healthcare system (among other hospitals) has implemented interventions targeted at reducing inappropriately broad empiric therapy for uncomplicated CAP.^11,20,21 Substantial decreases in antibiotic use have been demonstrated in the VA since the establishment of the national VA Antimicrobial Stewardship Task Force in 2011²¹ that have persisted through the COVID-19 pandemic.²² Specific to the inappropriately broad empiric antibiotic use for CAP eCQM, there have been educational campaigns to improve concordance with guideline recommended therapy in the VA but no formal implementation of the measure. Regardless, with education and stewardship interventions, the proportion of patients receiving empiric anti-Pseudomonal therapy or anti-MRSA therapy has decreased from 2015 to early 2024 (see Figure 2, Supplemental Materials attachment). As a result, there have been significant reductions in methicillin resistance in S aureus, vancomycin-resistance in E faecium, and fluoroquinolone resistance across multiple pathogens.^20-26

By identifying and improving the percentage of hospitalized CAP patients who receive non-guideline concordant overtreatment with anti-MRSA or anti-Pseudomonal therapy, our proposed eCQM is expected to have direct impact on patients by reducing antibiotic-associated adverse events and reducing development/acquisition of antibiotic resistance. Specifically, anti-MRSA has been associated with death, acute kidney injury, Clostridioides difficile infection, vancomycin-resistant enterococcal infection, and gram-negative rod infections and combinations of anti-MRSA and anti-Pseudomonal therapy have been associated with acute kidney injury and neurotoxicity.^8,11,27 We also anticipate that reducing antibiotic overuse for the most common inpatient indication for antibiotics will have a large impact on antimicrobial resistance. Using data reported to the National Healthcare Safety Network (NHSN) antimicrobial use option, the effect of reducing inpatient broad-spectrum antibiotic exposure was modeled and estimated that reduction in use of broad-spectrum antibiotics by 30% could result in a 26% reduction in Clostridioides difficile infection.²

Potential unintended consequences include under-treatment of patients who require anti-MRSA or anti-Pseudomonal therapy. During initial measure development, we considered assessing undertreatment as part of this measure. We asked our Technical Expert Panel (TEP, 8 clinicians representing 8 different clinical specialties and national organizations [Table 4 in the Supplemental Materials attachment]) if the measure should evaluate under-treatment, and 87.5% (7/8) of them preferred eliminating under-treatment as an assessment. TEP feedback included:
“Focus only on those who should receive non-MDRO therapy”
“Eliminat[e] anyone who qualifies for MDRO therapy & focus on pts who do not have risk factors for MDRO therapy”
“Exclude those with indication for MDR[O] therapy and focus on overuse”
“There is no perfect measure here but I think simplifying the measure by taking out those who qualify for MDRO treatment would make it cleaner & more fair.”
“By concentrating on overuse- you could swing to underuse - but suspect unlikely. [looking at over and underuse] just seems too complicated to actually be used in practice- even if valid.”

Thus, we decided, based on TEP feedback, to assess only patients without a guideline-based indication for empiric anti-MRSA or anti-Pseudomonal therapy. Notably, none of the implementation studies above identified any unintended consequences during their efforts to reduce inappropriate broad spectrum antibiotic use for CAP. That said, we plan to evaluate unanticipated consequences during the measure maintenance submission.

The full reference list can be found in Section 2.2.

2.5 Health Care Quality Landscape

Pneumonia is a well-established target of clinical quality and safety monitoring. Table 6 (see Supplemental Materials attachment) lists existing and prior measures for pneumonia as well as existing measures for antibiotic use that may be influenced by measures of treatment of pneumonia.^18,29-33

The CDC,³⁴ The Joint Commission,³⁵ and Centers for Medicare and Medicaid Services³⁶ recommend that antibiotic stewardship programs implement interventions to promote guideline concordant empiric antibiotic selection.

Currently, there is one active quality measure related to antibiotic use that includes patients with CAP—the CDC’s NHSN AU Option. The NHSN provides a risk-adjusted benchmark comparison for participating hospitals to use to drive improvement.³⁷ The standardized antimicrobial administration ratio (SAAR) was introduced in 2015 and is based on antibiotic days of therapy (DOT), similar in structure to risk-adjusted comparisons of health care associated infection rates. SAAR is a ratio of observed to expected DOT for a particular antimicrobial agent category and location. The 2023 CMS Inpatient Prospective Payment System rule requires NHSN AU reporting for US acute care hospitals in the Promoting Interoperability Program. Hospitals that do not meet requirements could lose funding incentives for promoting interoperability.³⁸ The start date for enforcement of reporting will begin in late 2025. Though AU reporting will be required, hospital-specific data will be neither publicly reported nor used in pay-for-performance programs. Regardless, this mandate means that routine assessments of risk-adjusted AU comparisons to a national benchmark are now a reality for most US hospitals. As of September 2024, 4087 hospitals had reported to the NHSN AU option—a number that will continue to grow now that reporting is required.³⁹

Notably, NHSN AU data do not evaluate for antibiotic appropriateness. Thus, we anticipate this eCQM being used in concert with the NHSN AU module to help hospitals and their antimicrobial stewardship teams understand how to improve care. We have collaborated with the CDC in designing this measure with the goal of working within their NHSN AU framework (when NHSN moves to patient-level data collection).

NHSN AU data are broad antibiotic-use metrics. No eCQMs specifically for CAP currently exist;³² though one related to pneumonia diagnosis was recently endorsed.¹⁶ A chart-review based version of this empiric antibiotic use measure has been in use in 69 Michigan hospitals as a pay-for-performance measure where it has demonstrably improved antibiotic use (see use data within this measure). By converting the empiric antibiotic use measure into an eCQM, we hope to enable broader implementation of the improvements necessary to optimize empiric antibiotic use and improve care of hospitalized patients with CAP.

The full reference list can be found in Section 2.2.

2.6 Meaningfulness to Target Population

We explored the value of the pneumonia antibiotic treatment quality through two approaches: 1) literature review of guidelines, clinician and patient perspectives and 2) engagement with clinicians in technical expert panels.

As noted above, multiple national organizations consider appropriateness of antibiotic therapy for pneumonia as a priority area–including the CDC, The Joint Commission, and Centers for Medicare and Medicaid Services. Broadly, improving empiric antibiotic selection is considered part of the definition of antibiotic stewardship–antibiotic stewardship has been defined in a consensus statement from the Infectious Diseases Society of America (IDSA), the Society for Healthcare Epidemiology of America (SHEA), and the Pediatric Infectious Diseases Society (PIDS) as “coordinated interventions designed to improve and measure the appropriate use of [antibiotic] agents by promoting the selection of the optimal [antibiotic] drug regimen.”⁴² The IDSA in its guidelines for Implementing an Antibiotic Stewardship program provide a strong recommendation (based on moderate quality evidence) stating: “We recommend antibiotic stewardship interventions designed to reduce the use of antibiotics associated with a high risk of CDI compared with no such intervention.”⁴³ They emphasize that reducing CDI is a priority for all ASPs and should be considered when developing stewardship interventions.⁴³

Data specifically from patients about antibiotic selection is limited. In a study of patients presenting with lower respiratory symptoms, 83% believed antibiotics would help and 71% expected antibiotics.⁴⁴ Paradoxically, qualitative studies have found patients are interested in minimizing exposure to unnecessary antibacterials particularly when objective evidence provides evidence that antibacterials are unnecessary.^45,46

Technical Expert Panel (TEP)
The TEP meeting was conducted remotely via videoconference with 8 clinicians representing 8 different clinical specialties and national organizations (see Table 4, Supplemental Materials attachment). An agenda with specific sets of questions tailored to the measure development phase were provided to panelists prior to the discussion session. We also surveyed TEP members after the meeting to obtain their feedback and guidance on the proposed measure.

The importance of developing new measures addressing appropriate treatment of CAP was expressed by members of the TEP. When surveyed, 62.5% (5/8) of our TEP believed it was “important” to “develop an electronic measure of empiric antibiotic therapy for persons diagnosed with CAP” the remaining 37.5% (3/8) reported it was “very important.” All 8 “strongly agreed” that “We should try to reduce anti-MRSA/anti-Pseudomonal therapy in hospitalized patients with CAP without risk factors for MDROs.” All TEP members reported the proposed measure was important (1/8 “very important”, 6/8 “important”, 1/8 “moderately important”) for “patient outcomes (e.g., adverse events, C. difficile infection, antibiotic resistance)?”

The full reference list can be found in Section 2.2.

2.4 Performance Gap

In addition to the published studies noted above, we analyzed Veterans Affairs (VA) Computerized Patient Records System (CPRS) data from 109 VA Health Care Systems including all 47,034 eligible hospitalizations between January 1, 2022 and June 30, 2024. Performance scores by decile from those 109 hospitals are shown in Table 1 (individual hospital data in Figure 3, see Supplemental Materials attachment). In addition to VA hospitals, we analyzed University of Utah and University of Michigan hospitalizations using the eCQM applied to their Epic instances (data in Table 5, Supplemental Materials attachment).

Notably, VA and University of Utah estimates are much lower than University of Michigan. This may be because the VA health care system and University of Utah have implemented interventions to improve empiric antibiotic selection for CAP with published success.^20,21,28 For example, substantial decreases in antibiotic use have been demonstrated in the VA since the establishment of the national VA Antimicrobial Stewardship Task Force in 2011²¹ that have persisted through the COVID-19 pandemic.²¹ Similarly, the University of Utah created a CAP orderset that—with antibiotic stewardship efforts—improve empiric antibiotic selection.²⁸ The range of performance across systems further demonstrates a wide gap—and opportunity for improvement—across hospitals and that data-driven antibiotic stewardship efforts can improve antibiotic use.

The full reference list can be found in Section 2.2.

2.4a Attach Performance Gap Results

Table 1 Performance Scores by Decile.pdf

3.1 Feasibility Assessment

We implemented and tested the proposed eCQM within 3 health systems to assess: 1) whether all required data elements were routinely generated during the care of CAP hospitalizations, 2) which barriers or challenges exist when implementing and extracting the measure from current healthcare data to inform our feasibility scorecard, and 3) whether any elements were uncommon or able to be removed to simplify the measure.

First, all of the data elements collected are already part of normal healthcare system processes for care delivery and billing, so documentation of these items for the proposed eCQM do not add any additional burdens for a healthcare system including clinician workflow, diagnostic thought processes, or patient-physician interactions. During the normal course of care, inpatient clinicians must document within the EHR a discharge diagnosis of pneumonia, order antimicrobials that are recorded in the EHR, and obtain labs, vital signs, and chest imaging that generates a result in the form of structured data that is stored in the EHR. One of our main goals of the measure was to leverage the most reliable, accurate and standardized data elements and to provide a simple, transparent measure that could be consistently extracted, calculated, and interpreted without substantial data resources or expertise. To accomplish this goal, our measure only uses structured data and does not require unstructured data or natural language processing.

As with any eCQM, there is significant work to collate and clean EHR data and ensure their accuracy for measure assessment. To reduce this work, when possible, we used existing value sets—particularly those used by the CDC or NHSN where the measure may be integrated in the future—to streamline use and make the measure more feasible. Examples of harmonized value sets include:
CDC’s list of common bacterial commensals⁴⁷ (OID: 2.16.840.1.114222.24.7.281)
CDC’s definition of Intravenous Medication Route” (OID: 2.16.840.1.113762.1.4.1190.75)
CDC NHSN to define unit (e.g., ICU; OID: 2.16.840.1.113762.1.4.1029.204)

Data element feasibility testing:
During measure development, we assessed data element feasibility within three different health systems. This included identifying which data elements (originally present in guidelines or in the original chart-review based measure) were difficult to standardize or collect and what (if any) impact their exclusion vs. inclusion would have on measure validity. The results of this process are found in the “feasibility scorecard” and described below in the “feasibility informed measure” section.

The full reference list can be found in Section 2.2.

3.2 Attach Feasibility Scorecard

Feasibility Scorecard PQM CBE 4545e.xlsx

3.3 Feasibility Informed Final Measure

1) TEP-recommended Change in Measure Scope:
Originally, we anticipated our measure would assess broad spectrum antibiotic therapy for all hospitalized patients with uncomplicated CAP—including those with risk factors for MRSA or Pseudomonas aeruginosa. However, our TEP found it to be too complicated to include patients for whom empiric anti-MRSA or anti-pseudomonal therapy was appropriate. This early version of the measure also included tiers to rate the choice of empiric treatment as: preferred (guideline concordant), potentially justified, and non-guideline concordant. When the TEP was asked to comment on this version of the measure, they rated it as too complex, not feasible, and prone to “gaming.” Representative feedback includes:
“The tiers are too complex to translate to a measure that is useful/ interpretable/ actionable for improving quality.”
“I just worry this is overall too complicated and emphasizing overtreatment in those who have no MDRO risk factors is the cleaner thing to implement and focus on change.”
“Overall I like this tiering but I do worry about being able to develop a fair quality metric from this. There is quite a bit of gray here (which is far too often the case with pneumonia anyway) but with a quality metric, it would be best to make it as black & white as possible.”
When the TEP was asked if the measure should assess only patients for whom anti-MRSA or anti-Pseudomonal therapy is not recommended (i.e., exclude those with risk factors for MRSA or Pseudomonas aeruginosa), 5/8 TEP members reported “strongly agree”; 2/8 reported “agree”; and 1/8 reported “neutral”. TEP comments on this simplified version of the measure included:
“I think evaluating pts for whom MDRO therapy is not clearly indicated is cleaner and will make this metric easier and more fair. It certainly isn't perfect, but then again what quality metric is perfect!”
“This gets past aforementioned concerns for the first example [first measure version]. Per the data presented this is also 95%+ of the CAP cohort.”

2) Data Elements Removed After Feasibility Testing: We removed the following items from assessment based on the results of feasibility testing.
CD4 count- Originally we anticipated identifying patients with AIDS based on CD4 count plus a diagnosis of HIV. CD4 count was difficult to identify in all three systems as the results were located in multiple different laboratory tests and often in unstructured fields. There were no ICD10 codes to distinguish AIDS from HIV. Because patients with HIV (regardless of CD4 count) were uncommon in all three health systems, we elected to exclude all patients with HIV rather than require assessment of CD4 count. Patients with HIV represent 0.5% of the University of Michigan population, 1.6% of the VA population, and 2.5% of the University of Utah population (prior to applying any additional exclusions). In contrast to CD4 count, HIV can be identified based on ICD-10 diagnostic codes alone, improving feasibility.
Intravenous antibiotics at discharge- originally, as an additional way of excluding complicated pneumonia, we sought to exclude patients prescribed intravenous (IV) antibiotics at discharge. The VA CPRS database did not capture IV antibiotics after discharge in a structured field (they are documented in an “outpatient parenteral antibiotic therapy” note) and thus collecting this exclusion criteria would be an undue burden. University of Michigan showed that only 2.2% of patients had intravenous antibiotics prescribed at discharge of which 71% were excluded for an alternative reason. Given the minimal impact of this exclusion on performance—and feasibility concerns—this exclusion was dropped.

3) Data elements simplified: We simplified some data elements to improve feasibility of assessing them electronically.
Severe pneumonia- Pneumonia severity has been defined different ways in different studies and guidance. The ATS/IDSA 2007 and 2019 CAP Guidelines^10,48 define severe pneumonia as having either one major criterion or three or more minor criteria as follows:
Major criteria
Septic shock with need for vasopressors
Respiratory failure requiring mechanical ventilation
Minor criteria
Respiratory rate ≥ 30 breaths/min
PaO2/FiO2 ratio ≤ 250
Multilobar infiltrates
Confusion/disorientation
Uremia (blood urea nitrogen level ≥ 20 mg/dL)
Leukopenia* (white blood cell count < 4,000 cells/uL)
Thrombocytopenia (platelet count < 100,000/uL)
Hypothermia (core temperature < 36 C)
Hypotension requiring aggressive fluid resuscitation
*Due to infection alone (i.e., not chemotherapy induced)

For this measure, the major criteria do not apply as patients admitted to an ICU or on mechanical ventilation in the first 48 hours are excluded. Because confusion/disorientation is not captured in structured data, this element was infeasible for all three of our healthcare systems to assess and was dropped. We approximated PaO2/FiO2 ratio of ≤ 250 as ≥5L oxygen or SpO2<90. Compared to chart review (University of Michigan data), this change in definition of hypoxia had a sensitivity of 94.0% and specificity of 92.4%. Since there is no standard definition for “aggressive fluid resuscitation,” we operationalized hypotension requiring aggressive fluid resuscitation as systolic blood pressure <80. Finally, to ensure accuracy of our modified criteria, we looked across Michigan Hospital Medicine Safety Consortium (HMS) data and found that requiring ≥2 modified severity criteria (rather than ≥3 of the original criteria) to determine severity had the highest area under the receiver operating curve compared to the original definition. Thus, our modified criteria are:

Modified Severity Criteria: ≥2 of the following on a single day in the first 48 hours of encounter:
Respiratory rate >30 breaths/minute
≥5L oxygen or SpO2<90
BUN >20 mg/dL
WBC <4000 cells/uL
Platelets <100,000/uL
Temperature <36 C
SBP <80 mmHg
In University of Michigan data, compared to chart review using the ATS/IDSA criteria, the modified severity criteria have a sensitivity of 86.9% and specificity of 77.9%.

4) Elements assessed for feasibility and NOT changed: These were elements which we assessed for feasibility but did not change.
Prior hospitalization with IV antibiotics in prior 3 months. Per the 2017 ATS/IDSA CAP guidelines, patients should receive empiric anti-MRSA and anti-Pseudomonal antibiotics if they present with severe CAP AND have had a hospitalization during which they with IV antibiotics in the last 3 months. Thus—patients with severe CAP and a prior hospitalization with IV antibiotics in the prior 3 months should be excluded from the measure denominator. We sought to explore whether it was feasible to identify patients with a prior hospitalization with IV antibiotics. In the end, for all three of our tested EHRs this element was feasible (e.g., within University of Michigan testing cohort, sensitivity was 70% and specificity 96%). However, our TEP expressed concerns that prior hospitalization with IV antibiotics might be variably captured (particularly outside of Epic and CPRS where care is more integrated) and thus variably accurate across EHRs. Because severe CAP with prior hospitalization with IV antibiotics accounted for 10.8% of patients with uncomplicated CAP, we did not want to drop this data element completely. Thus, we conducted sensitivity analyses to assess different options.
First, we tested measure performance when patients were not excluded from the denominator on the basis of having severe CAP with prior hospitalization with IV antibiotics. This change would result in fewer patients being excluded and classify slightly more patients (58.0% [3506/6050]) as receiving inappropriate anti-MRSA or anti-Pseudomonal empiric therapy (vs. 55.2% [2998/5431] with the exclusion applied).
Second, we tested excluding all patients with severe CAP (rather than severe pneumonia plus prior hospitalization with IV antibiotics). Here, far more patients would be excluded but performance would slightly improve (52.9% [1906/3605]).
To be consistent with national guidelines, hospitals should maintain exclusions as we have them currently specified (severe pneumonia + prior hospitalization with IV antibiotics); however, hospitals unable to accurately identify these patients should expect results within 3% difference. We anticipate that as inter-operability improves, this data element will become more feasible for hospitals.

After presenting the measure to the TEP, we asked whether the “measure appropriately balances the need for accuracy with complexity”? 75% (6/8) responded it was the “perfect balance” with 1 not responding to the question and 1 reporting it was somewhat too simple. The TEP member worrying it was too simple thought we might call some potentially justified therapy inappropriate. Another commented, “Note- answer above - "perfect balance" - likely overstatement- I think it's the best option - but there are tradeoffs needed for the measure to remain accurate while still being feasible to use.”

The full reference list can be found in Section 2.2.

3.4 Proprietary Information

Not a proprietary measure and no proprietary components

4.1.3 Characteristics of Measured Entities

The VA health system is an integrated health system that uses CPRS, with approximately 485,500 acute hospitalizations per year across 109 facilities. The University of Utah is an academic medical center that uses an Epic EHR, with approximately 14,500 total inpatient hospitalizations per year. The University of Michigan is an academic medical center that uses a different instance of the Epic EHR with approximately 49,730 total inpatient hospitalizations per year. All three sites have interoperable EHRs where data can be transferred across outpatient and inpatient settings within the health care system. Characteristics of the measured entities are in Table 8 (Supplemental Materials attachment).

Notably, the VA healthcare system and University of Utah have implemented interventions to improve empiric antibiotic selection for CAP with published success.^20,21,28 For example, substantial decreases in antibiotic use have been demonstrated in the VA since the establishment of the national VA Antimicrobial Stewardship Task Force in 2011²¹ that have persisted through the COVID-19 pandemic²¹. Strengths of the VA antimicrobial stewardship programs include organizational directives for antimicrobial stewardship program staffing, system-wide medication use evaluations that identify opportunities for improvements in antimicrobial use,^23-25 development of regional antimicrobial stewardship collaboratives, strong implementation science engagement and utilization of informatics-based tools²⁰ and strong, ongoing collaboration between VA research and operational partners.²⁶ The range of performance across systems further demonstrates a wide gap—and opportunity for improvement—across hospitals and that measurement and antibiotic stewardship efforts can improve antibiotic use.

The full reference list can be found in Section 2.2.

4.1.1 Data Used for Testing

We used data from 3 healthcare systems to test the measure. Each health system had different strengths to enable different validation or reliability testing:

University of Michigan (Epic)- eCQM data were pulled from hospitalizations between 9/29/2015 and 12/11/2021
To assess measure validity, we compared eCQM data (i.e., University of Michigan Epic data) to existing chart review data from an ongoing 69 hospital collaborative quality initiative, the Michigan Hospital Medicine Safety Consortium (HMS). HMS collects pseudo-random data from hospitalized patients with community-acquired pneumonia across all 69 hospitals. Critically, they collect all the data necessary to assess this measure (including denominator/numerator inclusions/exclusions) and have used a chart-reviewed based version of the measure to improve care for hospitalized patients with pneumonia since 2015.^49-53 We specifically used University of Michigan HMS data from a matching time period (9/29/2015-12/11/2021).
Quality assurance data for HMS have been published previously.^49-53 In brief, trained data abstractors collect specific data from medical records in forms with pre-specified fields (e.g., max/min allowances). Abstractors undergo random audit to ensure data quality. HMS data have been used to inform two prior NQF-endorsed quality measures (NQF 3671 and 3690).⁵⁴

University of Utah (Epic)- eCQM data pulled from hospitalizations between 1/1/2021 and 5/30/2022
To assess measure validity, we compared eCQM data (i.e., University of Utah Epic data) to chart review data obtained originally for the Inappropriate Diagnosis of Pneumonia measure (NQF 3671). Cases were manually abstracted by a physician reviewer to obtain the necessary data elements to assess appropriateness of empiric therapy. Cases for abstraction were randomly selected from patients with a discharge diagnosis of pneumonia from 1/1/2021 and 5/30/2022. Electronic and chart review data were compared for key denominator/numerator variables included in both datasets.

Veterans Affairs (VA) Computerized Patient Records System (CPRS)- eCQM data were pulled from hospitalizations between January 1, 2022 and June 30, 2024 including 109 acute care hospitals.
These data were used to assess variation across hospitals and reliability
Notably, CPRS data have been recently used to inform a Battelle-endorsed quality measure (CBE ID: 4440e).

Technical Expert Panel (TEP): The TEP was comprised of stakeholders with diverse perspectives and areas of expertise representing a variety of professional organizations, non-profit organizations, and governmental agencies (details in Supplemental Materials attachment, Table 4). Expert feedback during measure development was provided by the TEP during a 2-hour session held on April 26, 2023.

The full reference list can be found in Section 2.2.

4.1.4 Characteristics of Units of the Eligible Population

University of Michigan, University of Utah, and VA Healthcare eCQM data include all eligible patients during their respective timeframes. Measure-level performance testing was completed without sampling, using the full denominator populations at each healthcare system (48,742 within the VA; 619 at the University of Utah; 5,553 at the University of Michigan), at the patient encounter level of analysis. Descriptive statistics of patients included in the patient encounter level testing datasets are provided for the VA cohort, the University of Michigan cohort, and the University of Utah Cohort (see Tables 9, 10, and 11 in Supplemental Materials attachment).

To test the validity of the University of Michigan population, we compared University of Michigan eCQM data to chart review data from the Michigan Hospital Medicine Safety Consortium (HMS). For HMS, abstractors screened consecutive patients via medical record review 30 days after discharge and included the first eligible patient daily, abstracting 8 eligible patients during a two-week cycle. Patients were eligible for inclusion if they were adults (≥18) admitted to general care with a billed discharge ICD-10 code of pneumonia and received antibiotics on day 1 or day 2 of hospitalization (this differs from the first 48 hours definition used in eCQM and may explain some data discrepancies). Patients who had documentation of treatment for an additional infection unrelated to pneumonia, were severely immunocompromised, were pregnant, were admitted for comfort measures, or who left against medical advice were ineligible. Quality assurance data for HMS have been published previously.^49-53 In brief, trained data abstractors collect specific data from medical records in forms with pre-specified fields (e.g., max/min allowances). Abstractors undergo random audit to ensure data quality. HMS data have been used to inform two prior NQF-endorsed quality measures (NQF 3671 and 3690).⁵⁴

The full reference list can be found in Section 2.2.

4.1.2 Differences in Data

A summary of the data sources that were used for each type of reliability and validity testing is found in Table 7 (see Supplemental Materials attachment)

4.2.1 Level(s) of Reliability Testing Conducted

Person– or encounter–level (i.e., data element) (e.g., inter–abstractor reliability)

Accountable Entity-Level (e.g., signal-to-noise analysis)

4.2.2 Method(s) of Reliability Testing

Per guidance from NQF’s Scientific Methods Panel on requirements for eCQMs, data element reliability is “not required if data element validity is demonstrated.”⁵⁶ Thus, please see our data element validity section rather than data element reliability.

Per guidance from NQF’s Scientific Methods Panel on requirements for eCQMs, “Reliance on data from structured data fields is expected; otherwise, unstructured data must be shown to be both reliable and valid. Reliability testing is not required if based on data from structured data fields.”⁵⁶ Our measure uses only data from structured data fields, those we do not report patient/encounter or data element reliability and instead report patient/encounter and data element validity.

We conducted accountable entity-level reliability testing with two models. The first was a signal-to-noise analysis performed (within the VA dataset) using a mixed-effect logistic model run as an empty model such that the only effects in the model were the overall intercept and the hospital specific intercepts. This model enabled calculation of the hospital variance (signal), the total variance, and the residual variance (noise). The intraclass correlation was calculated from these variances. The intraclass correlation was utilized within the Spearman Brown formula in two ways: (A) to calculate the reliability for the entire hospital cohort using the median number of case abstractions for the cohort and (B) to understand minimum case abstracts necessary to achieve predetermined reliability thresholds of 0.6, 0.7, 0.8, and 0.9. All data for this analysis was from the 109 VA facilities described above. A caterpillar plot was also provided demonstrating raw metrics and shrunken estimates in this model.

We also performed an analysis of reliability using beta-binomial regression using SAS code provided in The Reliability of Provider Profiling: A Tutorial.⁷⁸ VA facilities were then sorted by decile of reliability size (after exclusions)—results are shown in Table 2. The mean reliability score by beta-binomial regression was then calculated for all facilities within a decile category.

The full reference list can be found in Section 2.2.

4.2.3 Reliability Testing Results

Table 3 (see Supplemental Materials attachment) indicates the minimum annual number of qualifying cases needed for the denominator to reach each target reliability level at a given facility. In order to achieve a desired reliability of 0.8, each hospital would need to include 96 cases annually. For acceptable reliability (0.7), 56 annual cases would be required, and for high reliability (0.9), 216 annual cases would be required. The median number of eligible cases per facility within the cohort (n=135) had a reliability of 85.0%, based on data from 109 VA health care systems (2023 data). Based on our signal-to-noise analysis (using beta-binomial regression) the overall cohort reliability is 99.9%.

4.2.3a Attach Additional Reliability Testing Results

Table 2 Accountable Entity Level Reliability 4545e.pdf

4.2.4 Interpretation of Reliability Results

Based on signal-to-noise analysis, the median number of eligible cases per facility within the cohort (n=135) had a reliability of 84.7%. Based on our signal-to-noise analysis (using beta-binomial regression) the overall cohort reliability as >99.9%. Both these estimates meet the threshold for reliability for measures considered to be high stakes.

Using the current VA cohort as a representative example, the minimum number of case abstracts per hospital per year to meet pre-specified reliability thresholds of 0.7 and 0.8 are highly attainable. Within the 109 VA health care systems, 86.2% of hospitals had the minimum of 56 cases in 2023 necessary to achieve 0.7 reliability. As seen in Table 2, only the smallest decile of reliability had a mean reliability <80%. Using the VA likely underestimates reliability of the measure, as 87.2% of VA hospitals have fewer than 200 beds.

It should also be noted that this analysis was conducted among facilities all within the same healthcare system and thus may be more similar in performance to one another than a true population distribution of all hospital types and systems within the United States.

4.3.1 Level(s) of Validity Testing Conducted

Person– or encounter–level (i.e., data element) (e.g., sensitivity and specificity)

Accountable entity–level (e.g., criterion validity)

4.3.2 Type of accountable entity-level validity testing conducted

Empirical validity testing at the accountable entity-level (e.g., criterion validity, construct validity, known groups analysis)

Systematic assessment of face validity of performance measure score as an indicator of quality or resource use (i.e., the score is an accurate reflection of performance on quality or resource use and can distinguish good from poor performance).

4.3.3 Method(s) of Validity Testing

Table 12 (see Supplemental Materials attachment) summarizes validity testing of the measure during development and in its final version, including description of the testing, results, and interpretation.

A. Face Validity-National Guidelines
The inappropriately broad empiric antibiotic selection in CAP measure was based on national guidelines for pneumonia, literature review, and expert feedback and review. The 2019 ATS/IDSA CAP guidelines recommended not prescribing empiric anti-MRSA or anti-Pseudomonal therapy in uncomplicated CAP patients without risk factors for MRSA or Pseudomonas aeruginosa.

This measure is also supported by literature. For example, a recent report on 8,286 non-ICU hospitalized CAP patients from 67 Michigan hospitals (assessed using the chart review measure from which our eCQM was adapted) showed that 2,215 (26.7%) received inappropriately broad empiric treatment (i.e., anti-MRSA or anti-Pseudomonal coverage in patients eligible for standard CAP coverage per ATS/IDSA guidelines).¹¹ Compared to patients who received standard CAP antibiotic treatment, patients receiving inappropriately broad antibiotics had higher 30-day readmissions, more transfers to ICU and antibiotic-associated adverse events, and longer hospitalizations.¹¹ Similarly, a retrospective analysis of 88,605 patients with CAP across the Veterans Health Administration health care system found that adding empirical anti-MRSA therapy to standard CAP therapy was associated with an increased adjusted risk of death, kidney injury, and secondary infections (C. difficile, vancomycin-resistant Enterococcus infection, and resistant gram-negative rod infections).⁸ Given that up to 90% of empiric anti-MRSA or anti-Pseudomonal antibiotic therapy is non-guideline concordant,^8,11 there is substantial potential benefit to patients and the US by reducing inappropriately broad empiric antibiotic selection. National surveillance of drug-related adverse events estimated that even a small reduction in unnecessary antibiotic use could significantly decrease the direct risks of drug-related adverse event in individual patients.¹² Thus, our guideline-based measure has the potential to improve patient care for a large number of patients hospitalized with CAP across the US.

Appropriateness of antibiotic therapy for pneumonia is a priority for numerous federal and accreditation organizations–including the CDC, The Joint Commission, and Centers for Medicare and Medicaid Services. For example, the CDC developed recommendations for Antibiotic Stewardship which it published in its “Core Elements of Hospital Antibiotic Stewardship Programs.” Within the core elements, the CDC specifically states that one of the key opportunities for improving antibiotic use in CAP is “Avoid[ing] empiric use of anti-Pseudomonal beta-lactams and/or MRSA agents unless clinically indicated.”³⁴

B. Face Validity-Expert Feedback
Throughout measure development, we obtained expert and stakeholder input via input from the Data, Design, and Publications (DDP) Committee of the Michigan Hospital Medicine Safety Consortium (HMS) early in measure development (during chart review stage)

The Data, Design, and Publications (DDP) Workgroup was an ongoing meeting of champions and experts from HMS hospitals that met to address key issues related to measure methodology, including weighing the pros and cons of measure specifications, modeling, and use (e.g., defining the measure cohort and outcome) to ensure the measure was meaningful, useful, and well-designed. The group met every 2 months during measure development and provided a forum for focused expert review and discussion of technical issues. They also provided final approval of the current submitted measure as specified.
List of DDP Workgroup Members:
Suhasini Gudipati, MD Ascension Michigan St. Mary’s Hospital
Tina Percha, RN, MSN Beaumont Health
Rajiv John, MD Beaumont Health
Lama Hsaiky, PharmD Beaumont Health
Priscila Bercea, MPH Beaumont Health Dearborn
Scott Kaatz, DO Henry Ford Health System
Allison Weinmann, MD Henry Ford Health System
Emily Nerreter, MBA Henry Ford Health System
Danielle Osterholzer, MD Hurley Medical Center
Lisa Dumkow PharmD Mercy Health St. Mary’s
Anurag Malani, MD St. Joseph Mercy Ann Arbor Hospital
Lakshmi Swaminathan, MD St. Joseph Mercy Ann Arbor Hospital
Muhammad Nabeel, MD Sparrow Hospital
Andrea White, PhD University of Utah Health
Valerie Vaughn, MD, MSc University of Utah Health
Vineet Chopra, MD, MSc University of Colorado Anschutz Medical Campus
Throughout measure development (the chart review version), we also provided opportunities from experts across the HMS collaborative to provide feedback. This included frontline clinicians, antibiotic stewards, quality improvement experts, c-suite members, and experts in quality measurement.

C/D. Assessment of Encounter-Level Validity: Comparison with Case Review
The Michigan Hospital Medicine Safety Consortium (HMS), which includes the University of Michigan, has been collecting chart-review data on patients hospitalized with CAP since 2015. We report two types of encounter-level validity using these data where we compare HMS results and our eCQM results. First, we compared the actual antibiotic empiric selection obtained via eCQM vs. via chart review. For this assessment, we report the percentage of cases where both data sets agreed in classification of whether the patient received no anti-MRSA/Pseudomonal, only anti-MRSA, only anti-Pseudomonal, and both anti-MRSA/Pseudomonal empiric antibiotics. Next, we assessed the sensitivity and specificity for the eCQM in identifying patients with inappropriately broad empiric antibiotic selection (i.e., anti-anti-MRSA or anti-Pseudomonal therapy in the absence of risk factors).

E. Assessment of Encounter-Level Validity: Critical data element validity
To assess critical data element validity, initial denominator exclusions categorized as: a) to identify hospitalized non-ICU adults with CAP, b) to narrow the population to uncomplicated CAP, and c) specific to the appropriateness of empiric antibiotics measure were compared to assess consistency across the University of Michigan, University of Utah, and VA healthcare system.

We also compared inclusion criteria, exclusion criteria, and critical data elements between HMS chart review data and eCQM data for the University of Michigan. Due to the way HMS conducts chart review, not all inclusion/exclusion criteria are collected. We report sensitivity and specificity of the eCQM assessment compared to chart review which we considered the gold standard.

F. Face Validity: National Technical Expert Panel (TEP) Feedback (N=8 experts)
Throughout measure development, we obtained expert and stakeholder input. In alignment with the CMS Measures Management System guidance on TEP,⁵⁷ we held a remote TEP session to provide input and feedback from a group of recognized experts in relevant fields. To convene the TEP, we reached out to organizations whose members could potentially be impacted by the measure and asked them to nominate individuals for participation. We selected individuals to represent a range of perspectives, including Infectious Diseases physicians, pharmacists, pulmonologists, radiologists, hospitalists, emergency medicine physicians, regulatory agencies, as well as individuals with experience in quality improvement, performance measurement, diagnostic error, antibiotic stewardship, and health care quality. (TEP members are listed in Table 4 in Supplemental Materials attachment).

The TEP was convened on April 26, 2023 over a 2-hour period. In preparation for the meeting, all TEP participants were provided with an agenda and background document providing project context and data supporting the inappropriately broad empiric therapy for adult hospitalized patients with uncomplicated community-acquired pneumonia measure. The format for the TEP meeting (conducted remotely) included presentation of measure development and proposed specifications and relevant data, with open discussion among TEP members throughout. Participants were encouraged to provide expert feedback regarding the validity, feasibility, and usability of the measure during the sessions and through online questionnaires following each session.

Following the TEP, all participants completed an online survey that included questions related to validity, reliability, usability, etc. Related to measure validity, we asked TEP members:
Please rate the following statement: “The simplified measure can be used to distinguish between better and worse performing hospitals.” 1=Strongly disagree, 2=Disagree, 3=Neutral, 4=Agree, 5=Strongly agree.
Are there any key data elements you believe are missing or not accurately captured in the inappropriate broad spectrum antibiotic therapy for CAP measure?

G. Empirical Validity: Evaluated association with other measures of CAP antibiotic treatment quality
To assess empirical validity for the inappropriately broad empiric treatment in CAP measure, we identified and assessed the measure’s correlation with other measures that target antibiotic quality for CAP. The goal was to identify if better performance on this measure was related to better performance on other relevant measures. Specifically, we assessed the association (at the hospital level) of excess antibiotic duration for CAP with overuse of broad-spectrum empiric antibiotics for CAP.

H. Empirical Validity: Evaluated association of inappropriately broad empiric antibiotic use for CAP with outcomes
We reviewed the literature for studies evaluating the impact of broad spectrum (i.e., anti-MRSA or anti-pseudomonal) empiric antibiotic use on patient outcomes in CAP.

I. Predictive Validity: Evaluated whether a reduction in inappropriate broad spectrum antibiotic use improves patient outcomes in CAP.
We reviewed published literature on how patient outcomes change with reductions in inappropriately broad empiric treatment in CAP

The full reference list can be found in Section 2.2.

4.3.4 Validity Testing Results

C. Encounter-Level Validity: Comparison of Empiric Antibiotics with Case Review
After exclusions, 540 patients had data in both UM HMS and UM eCQM datasets. Considering HMS data the gold standard, the eCQM accurately assessed empiric antibiotics (no anti-MRSA/Pseudomonal, only anti-MRSA, only anti-Pseudomonal, and both anti-MRSA/Pseudomonal) in 96.7% (522/540) of patients (see Table 13 in Validity Results attachment).
D. Encounter-level Validity: Comparison of Measure (appropriateness of empiric antibiotics) with Case Review
Looking at the same 540 patients with both UM HMS chart review and University of Michigan eCQM data, we compared the percentage with inappropriate broad spectrum empiric antibiotic use (i.e., anti-MRSA or anti-Pseudomonal therapy) using chart review as the gold standard to obtain the sensitivity and specificity for the eCQM. The eCQM has a sensitivity of 96% and a specificity of 92% in assessing inappropriate broad spectrum empiric antibiotic use (see Table 14 in Validity Results attachment).
E. Encounter-level Validity: Critical data element validity
For each measured entity, initial denominator exclusions: a) to identify hospitalized non-ICU adults with CAP, b) to narrow the population to uncomplicated CAP, and c) specific to the appropriateness of empiric antibiotics measure (Tables 15, 16, and 17 in Validity Results attachment) were compared to assess consistency across entities.

Prior to applying exclusions, there were 592 patients hospitalized with CAP who had data in both the HMS chart review dataset and the University of Michigan eCQM data set. Below, we show the sensitivity and specificity for the empiric-specific exclusions (see Table 18, Validity Results attachment).
F. Face Validity: National Expert Panel Feedback
The eight national experts who attended our TEP agreed with the face validity and operationalization of the measure. They believed that patients we identified as receiving inappropriate broad spectrum empiric antibiotic therapy did, in fact, receive inappropriate therapy. The main feedback from the TEP was to exclude patients with risk factors for broad spectrum therapy from the denominator (which we did). (see TEP survey results in the Validity Testing Results attachment).
G. Empirical Validity: Association with Other Measures of CAP Antibiotic Treatment Quality
To assess empirical validity for the inappropriately broad empiric therapy in CAP measure, we assessed the association (at the hospital level) of excess antibiotic duration for CAP with overuse of broad-spectrum empiric antibiotics for CAP with the 109 VA hospitals. We found that the two measures were weakly correlated at the hospital level (R=0.3, p=0.0014, see Figure 4 in Validity Results attachment).
H. Empirical Validity: Evaluated association of inappropriately broad empiric antibiotic use for CAP with outcomes
Across 67 HMS hospitals, 26.7% (2,215/8,286) of included patients were empirically treated with inappropriate broad-spectrum antibiotics. The median broad-spectrum antibiotic treatment duration was 3 days (IQR, 2.5). Factors associated with inappropriate empiric broad-spectrum antibiotic treatment included hospitalization or treatment with high-risk antibiotics in preceding 90 days, transfer from a post-acute care facility, hemodialysis, support with ≥3 L supplemental oxygen, severe sepsis, leukocytosis, and higher pneumonia severity index. After adjustment, inappropriately broad empiric treatment was associated with higher 30-day readmissions, more transfers to ICU, longer hospitalizations, and more antibiotic-associated adverse events (see Table 19 in Validity Results attachment).¹¹

Among 88,605 CAP patients across the Veterans Health Administration health care system, empirical anti-MRSA therapy in addition to standard CAP therapy was associated with an increased adjusted risk of death, kidney injury, and secondary infections (C. difficile, vancomycin-resistant Enterococcus infection, and gram-negative rod infections) compared to standard CAP treatment alone.⁸
I. Predictive Validity: Evaluated whether a reduction in inappropriate broad spectrum antibiotic use improves patient outcomes in CAP.
The effect of reducing inpatient broad-spectrum antibiotic exposure was modeled and estimated that reduction in use of broad-spectrum antibiotics by 30% could result in a 26% reduction in C. difficile infection.²

The full reference list can be found in Section 2.2.

4.3.4a Attach Additional Validity Testing Results

Validity Results Attachment PQM CBE 4545e.docx

4.3.5 Interpretation of Validity Results

We assessed multiple types and levels of validity during and after measure development. First, the measure has high face validity as it is based on national guidelines and recommendations from national organizations (e.g., CDC) and was approved by national experts representing 8 national organizations. Compared to 540 case reviews (using HMS data at the University of Michigan), the measure had high sensitivity and specificity at the patient level for identifying inappropriate broad spectrum antibiotic use. Empiric-specific exclusions also had high sensitivity and specificity when compared to HMS chart review cases. The exception is the exclusion of severe CAP with prior hospitalization with IV antibiotics-here, we used a slightly less sensitive definition (sensitivity of 86.9% and specificity of 77.9%) to improve feasibility.

Across hospitals, we found performance on the empiric broad spectrum measure to be weakly correlated with performance on the excess duration measure, supporting empirical validity. Finally, the chart-review version of the measure on which this eCQM was based has been shown to be associated with patient outcomes and, critically, models suggest that improvement in the measure may reduce Clostridioides difficile infection.
Together, these results suggest that the inappropriate broad spectrum empiric antibiotic measure is highly valid and, most importantly, that hospitals can improve patient outcomes by improving their performance on the measure.

4.4.1 Methods used to address risk factors

No risk adjustment or stratification

Risk adjustment approach

Off

Risk adjustment approach

Off

Conceptual model for risk adjustment

Off

Conceptual model for risk adjustment

Off

5.1 Contributions Towards Advancing Health Equity

Pneumonia disproportionately affects older adults, individuals from underrepresented minority communities, and those with lower socioeconomic status.^59-61 Previous research has highlighted disparities in both pneumonia care processes and outcomes.^62-65 The proposed eCQM aims to promote health equity by enhancing quality measurement and improving pneumonia-related care, targeting these documented inequalities.

Previously, no differences by patient race, age, gender have been found related to inappropriate broad spectrum antibiotic therapy for CAP. Rather, in the HMS study—on multivariable analysis—factors associated with inappropriate empiric BSA treatment included hospitalization or treatment with high-risk antibiotics in preceding 90 days, transfer from a post-acute care facility, hemodialysis, support with ≥3 L supplemental oxygen, severe sepsis, leukocytosis, and higher pneumonia severity index.¹¹ In the VA study, patients receiving empirical anti-MRSA therapy demonstrated a greater comorbidity burden (renal disease, 29% vs 25%; congestive heart failure, 35% vs 30%; neoplastic disease, 34% vs 3%; and nursing home residents, 9% vs 3%), more risk factors for MRSA (7% vs 2% with history of MRSA infection, 36% vs 12% with previous hospitalization, and 42% vs 29% with previous antibiotics), and greater illness severity (median Pneumonia Severity Index, 124 [interquartile range, 95-156] vs 103 [interquartile range, 81-131]) as well as worse outcomes (16% vs 6% for 30-day all-cause mortality) compared with patients receiving standard therapy alone.⁸

The full reference list can be found in Section 2.2.

6.1.1 Current Status

Yes

6.1.2 Current or Planned Use(s)

Public Reporting

Payment Program

Quality Improvement with Benchmarking (external benchmarking to multiple organizations)

Quality Improvement (Internal to the specific organization)

6.1.4 Program Details

The Michigan Hospital Medicine Safety Consortium; Supported supported by the Blue Cross Blue Shield of Michigan and Blue Care Network as part of the B, https://www.mi-hms.org/quality-initiatives/antimicrobial-use-initiative, The aim of the antimicrobial use initiative is to formally measure and improve the appropriate use of antibiotics including selection of the right ant, 69 acute care hospitals in Michigan, Acute care hospitals

6.2.1 Actions of Measured Entities to Improve Performance

Acute care hospitals are now required to have antimicrobial stewardship teams to optimize antibiotic use in hospitalized patients. There are multiple strategies to improve empiric antibiotic selection for CAP including: a) ordersets to standardize care, b) audit and feedback by pharmacists, c) nudging or behavioral interventions to reduce anti-MRSA or anti-Pseudomonal therapy, and d) creation of guidelines and education.

Each strategy has its own barriers, and feasibility varies by institution. For example, smaller hospitals have an easier time implementing audit and feedback as their patient population is small and pharmacists may have fewer patients to cover. In contrast, larger institutions may have more information technology infrastructure to help construct ordersets or nudges using the EHR.

What we do know is that diverse hospitals are able to improve empiric antibiotic selection in accordance with national guidelines. A chart-review based form of this measure has been implemented within the Michigan Hospital Medicine Safety (HMS) consortium since 2022. After years of reporting measure data to the hospitals (since 2015), HMS announced in 2022 that it would become a pay-for-performance measure with an initial goal of ≤15% of patients receiving inappropriately broad-spectrum empiric antibiotics. As hospitals improved this goal was changed to ≤10% in 2024. A 42% relative decrease in inappropriate empiric anti-MRSA/anti-Pseudomonal therapy was observed across 69 HMS hospitals from quarter 2 of 2020 (22%) to quarter 2 of 2024 (13%, P<0.0001; unpublished data with permission from HMS DDP committee (see Figure 1. Supplemental Materials attachment) By quarter 4 of 2023, the majority of participating HMS hospitals met the performance goal (overall collaborative average was 13%); thus, the metric was converted into a sustainability measure for the collaborative (goal ≤10% for collaborative).

6.2.2 Feedback on Measure Performance

Tri-annual Collaborative Wide Meetings
Individuals from participating hospitals meet in person three times a year. We encourage hospitals to send their Clinical Data Abstractors, physician champions, and quality leads, as well as other individuals from their hospital that might be interested in participation. These meetings take place three times per year – in March, July, and November. Traditionally, meetings took place in-person at venues across Michigan. In 2020 and 2021, these meetings were hosted via an on-line format due to COVID-19.
The tri-annual meetings provide individuals from member hospitals with the opportunity to engage with each other in a variety of formats. Each meeting includes a formal discussion of the data from each of the HMS initiatives—including data on empiric antibiotic therapy for CAP—for the previous quarter, presentations from member hospitals and expert guests, breakout/work group sessions, and networking opportunities. These meetings allow individuals from member hospitals to network with individuals from other hospitals who have excelled in those areas to seek ideas on how to improve their performance. It also allows for an opportunity for feedback and to answer questions related to their performance.
Site-specific Reports on Measure Performance
Tri-annually, each participating hospital receives a printed and email version of a site-specific data report. These reports are also available daily within the database/registry (see below). These reports provide an in-depth look into the performance of each site. For example, we provide hospital data on the number of patients with CAP treated with inappropriate broad-spectrum empiric antibiotic therapy, details on antibiotic use and outcomes (e.g., adverse events), longitudinal performance, and data on how individual hospitals compare to other hospitals in the state. Hospitals also receive a list of all patients who were considered treated with an “inappropriate broad spectrum empiric therapy” to enable them to return to their hospital and conduct case reviews of those patients. Each hospital is encouraged to review these cases with their local team to perform audit and feedback, identify trends, and assist with overall quality improvement. This also provides an opportunity for measure feedback—for example, hospitals might find an error in case classification. Early during measure development this case-specific feedback was critical for improving measure validity.
Live Database Reports
Each of the HMS databases are equipped with the ability to view live reports utilizing Business Objects software. These reports provide updated data every 24 hours regarding measures (site performance and collaborative performance), fallout case information, demographics, critical/non-critical data errors, completeness of abstracted cases, and case classification information.
Individuals who participate in the collaborative either as a Clinical Data Abstractor or a quality administrator have the ability to log into the HMS databases and view these reports at their leisure. The software that HMS utilizes also allows for these reports to be exported as Excel files or PDFs for hospital-specific customization. This information is often utilized by participating hospitals at committee meetings or for presentations to track progress and inform quality improvement efforts. They also assist the Clinical Data Abstractor to identify errors in their abstraction and resolve them in real time. These reports also allow hospitals to review individual fallout cases and their clinical scenarios to inform individual clinicians or groups of clinicians of their performance and provide targeted education.

6.2.3 Consideration of Measure Feedback

Throughout measure development, we received feedback on the measure performance/validity through the above mechanisms as well as: 1) Expert Feedback from Data Design and Publications Committee and Michigan Hospital Medicine Safety (HMS) Consortium Hospital Experts/Representatives and 2) “Fall-out” Feedback. Briefly, measure performance feedback allowed us to refine the measures to the current version. The Data, Design, and Publications Committee approved the measures for use across HMS. There have been no unanticipated outcomes/ill effects.

6.2.4 Progress on Improvement

Using the chart-review based form of this measure as a pay-for-performance measure within the Michigan Hospital Medicine Safety consortium, hospitals were able to reduce empiric broad spectrum antibiotic use.

A 42% relative decrease in inappropriate empiric anti-MRSA/anti-Pseudomonal therapy was observed across 69 HMS hospitals from quarter 2 of 2020 (22%) to quarter 2 of 2024 (13%, P<0.0001; unpublished data with permission from HMS DDP committee, see Figure 1. By quarter 4 of 2023, the majority of participating HMS hospitals met the performance goal of ≤15% of uncomplicated CAP patients who received inappropriate broad empiric coverage (overall collaborative average was 13%); thus, the metric was converted into a sustainability measure for the collaborative (goal ≤10% across all hospitals).

6.2.5 Unexpected Findings

There were no unexpected findings during the 3 years that this measure has been a pay-for-performance measure.

Enter a comment below

Your name (optional)

Organization (optional)

Email

Subject (optional)

Comment

About text formats

Math question 1 + 4 =
Solve this simple math problem and enter the result. E.g. for 1+3, enter 4.

This question tests whether or not you are a human visitor to prevent automated spam submissions.

Submitted by Marc J Meyer B… (not verified) on Tue, 11/19/2024 - 11:37
Permalink

PQM CBE 4545e: Inappropriately Broad Empiric Antibiotic

Pneumonia is not only the most common reason for inpatient antibiotic use but also the most common infectious cause of mortality in the US resulting in approximately 1.4 million emergency department visits, 740,000 hospitalizations, 41,000 deaths, and $7.7 billion in inpatient costs each year in the US. The CDC, The Joint Commission, and Centers for Medicare and Medicaid Services recommend that antibiotic stewardship programs implement interventions to promote guideline concordant empiric antibiotic selection.
Antibiotic overuse is a national and international public health emergency with antibiotic resistant infections estimated to directly cause 1.27 million deaths globally and indirectly contribute to 4.95 million deaths. By identifying and improving the percentage of hospitalized CAP patients who receive non-guideline concordant overtreatment with anti-MRSA or anti-Pseudomonal therapy, our proposed eCQM is expected to have direct impact on patients by reducing antibiotic-associated adverse events and reducing development/acquisition of antibiotic resistance.
Certainly I see antibiotic overuse in CAP patient in our rural facility even with a robust stewardship program. I believe a simplified guideline and quality project on a national scale will help with this antibiotic overtreatment problem with see with CAP. We know in most cases 5 days of antibiotic therapy is enough to treat. In my health system we had good treatment days of therapy on the inpatient side but patients were sent home on extended days of treatment from an additional 5 to 10 days. Antibiotic resistance is a real and happening threat in healthcare today. We have make lots of progress in the last ten years, there is still more work to do . Please pass this guideline initiative .

Organization

Southwest Health System

Breadcrumb

Inappropriately Broad Empiric Antibiotic Selection for Adult Hospitalized Patients with Uncomplicated Community-Acquired Pneumonia

Enter a comment below

PQM CBE 4545e: Inappropriately Broad Empiric Antibiotic