When the measure returns for maintenance (3 years), the measure developer should have:
- Continued to explore the exclusion list to determine if changes are needed (e.g., empirical analyses with broader testing across entities) and to further clarify the conditions and justify them based on burden; and
- Conducted additional validity testing (data element in additional EHR).
The Excess Antibiotic Duration for Adult Hospitalized Patients with Uncomplicated Community-Acquired Pneumonia measure is a process measure representing the annual percentage of hospitalized adults with uncomplicated community-acquired pneumonia who receive an excess antibiotic duration. The measure will be calculated using electronic health record (EHR) data and is intended for use at the facility level for both quality improvement and pay-for-performance.
Measure Specs
- General Information(active tab)
- Numerator
- Denominator
- Exclusions
- Measure Calculation
- Supplemental Attachment
- Measure Record
- Point of Contact
General Information
The overall objective of this electronic clinical quality measure is to quantify excess antibiotic duration in hospitalized adults with uncomplicated community-acquired pneumonia (CAP).
Antibiotic overuse is a national and international public health emergency with antibiotic resistant infections estimated to directly cause 1.27 million deaths globally and indirectly contribute to 4.95 million deaths.1 National studies by the Centers for Disease Control and Prevention (CDC) estimate that up to 50% of hospitalized patients receive antibiotic therapy, most commonly for pneumonia, and that up to 40% of antibiotic prescribing could be improved.2 Because of these harms, the CDC developed recommendations for Antibiotic Stewardship which it published in its “Core Elements of Hospital Antibiotic Stewardship Programs.”3 These recommendations include “Assessing how often patients are discharged on the correct antibiotics for the recommended duration.” Specifically, the CDC recommends, “most cases of uncomplicated pneumonia can be treated for 5 days when a patient has a timely clinical response.”3 The 5-day treatment duration is based on national 5-day guideline recommendations for uncomplicated CAP, multiple randomized clinical trials showing the safety of short vs. long durations,4-14 and retrospective observational studies showing higher antibiotic-associated adverse events in patients who receive excess antibiotic durations.15
Pneumonia is not only the most common reason for inpatient antibiotic use but also the most common infectious cause of mortality in the US resulting in approximately 1.4 million emergency department visits, 740 000 hospitalizations, 41 000 deaths, and $7.7 billion in inpatient costs each year in the US.16-19 Given studies suggesting that up to ⅔ of patients hospitalized and treated for CAP have excess duration,15,20 there is substantial potential benefit to patients and the US by reducing excess treatment durations. For patients, shorter durations are associated with fewer antibiotic-associated adverse events and less risk of developing antibiotic resistant infections.15,21-23 When used as a pay-for-performance measure across 41 Michigan hospitals, the chart review measure from which our eCQM was adapted increased appropriate use of short duration therapy for CAP by 22% and decreased adverse events (driven by antibiotic-associated harm; adjusted Odds Ratio (aOR) per quarter, 0.98 [95% confidence interval: .96-.99]).24 Taken together, our guideline-based measure has the demonstrated ability to improve patient care for a large number of patients hospitalized with CAP across the US.
How will this measure improve quality of care? By establishing a standardized process for assessing treatment duration for CAP, a larger proportion of patients will receive appropriate care that is consistent with the 2019 ATS/IDSA CAP national guidelines.25 Appropriateness of antibiotic therapy for pneumonia is a priority for numerous federal and accreditation organizations–including the CDC, The Joint Commission, and Centers for Medicare and Medicaid Services–and is not currently captured in typical quality improvement measures. For example, National Healthcare Safety Network (NHSN) antimicrobial use (AU) module focuses on quantifying antibiotic use and comparing it to expected values, with no assessment of appropriateness or duration of therapy. Notably, the NHSN AU initiative could be augmented with an eCQM to assess appropriateness of antibiotic duration for CAP, the most common indication for inpatient antibiotic use.
What are the benefits or improvements in quality envisioned by this measure? Duration of antibiotic therapy for CAP often exceeds national guidelines. Michigan Hospital Medicine Safety Consortium and CDC data analyses report excessive duration of antibiotics for >70% of CAP cases in the US.15,20 The consequences of excess duration of therapy include adverse events in 20% of patients who receive unnecessary antibiotics, with increasing risk for each excess day of therapy for pneumonia.15 Conversely, a quality initiative (based on a chart review version of this measure) to adhere to 5-day antibiotic duration in CAP patients implemented across 41 hospitals was associated with a higher percentage of appropriate duration and fewer adverse events.24
The full reference list can be found in Section 2.2
The anticipated data source for the measure is electronic health record (EHR) data from inpatient hospital admissions, including discharge diagnosis codes, pharmacy and medication administration, and imaging records. These data are all collected routinely during usual clinical care through the process of inpatient hospitalizations.
Numerator
From the denominator population, identify patients who received excess antibiotic duration (defined as >/= 7 days of total antibiotic therapy including inpatient and discharge antibiotics).
- Compute "Total Antibiotic Duration" = Days administered, inpatient + Days prescribed, discharge
- Antibiotic Usage for CAP Diagnosis Value Set, OID: 2.16.840.1.113762.1.4.1264.25
- (Sheet Name: “AntibioticUsageforCAP” in Data Dictionary)
- If Total Antibiotic Duration >= 7, then case is in the numerator
Denominator
Identify all adult patients with an inpatient (or observation) non-intensive care unite (ICU) hospitalization in which the discharge diagnosis includes pneumonia or sepsis AND respiratory failure who received a respiratory antibiotic within 48 hours of hospitalization, received chest imaging within +/- 3 days of the hospital encounter, were not transferred from another hospital, and do not have a concurrent infection. Restrict to patients with uncomplicated pneumonia that qualify for a 5-day duration according to national guidelines.
Specific Inclusion Criteria (see Data Dictionary for detailed specifications):
- ICD10 discharge code for pneumonia or sepsis AND respiratory failure
- CAP, Sepsis, Respiratory Failure Diagnostic Value Set, OID:2.16.840.1.113762.1.4.1264.24
- (Sheet Name: “CAPSepsisRespFailureDx”)
- Received a respiratory antibiotic within the first 48 hours of hospitalization
- Antibiotic Usage for CAP Diagnosis Value Set, OID: 2.16.840.1.113762.1.4.1264.25
- (Sheet Name: “AntibioticUsageforCAP”)
- Age > 18 years on admission
- Admitted as inpatient or in observation awaiting inpatient admission
- Encounter Inpatient Value Set, OID: 2.16.840.1.113883.3.666.5.307
- (Sheet Name: “Encounterinpatient”)
- Did not die in first 48 hours of hospitalization
- code "Dead (finding)": '419099009' from "SNOMEDCT" display 'Dead (finding)'
- No time spent in an ICU in first 48 hours of hospitalization
- Intensive Care Unit Value Set, OID:2.16.840.1.113762.1.4.1029.204
- (Sheet Name: “IntensiveCareUnit”
- Not admitted from another acute care hospital or long-term acute care facility
- code "Hospital admission, transfer from other hospital or health care facility (procedure)": '4563007' from "SNOMEDCT" display 'Hospital admission, transfer from other hospital or health care facility (procedure)'
- No concurrent infection during hospitalization by ICD 10
- CAP, Concurrent Infections Value Set, OID: 2.16.840.1.113762.1.4.1264.23
- (Sheet Name: “CAPConcurentInfections”)
- Chest imaging within 3 days of admission
- Chest Imaging for Pneumonia Grouping Definition Value Set, OID:2.16.840.1.113762.1.4.1264.12
(Sheet Name: “ChestImagingforPneumonia”)
‘Sheet Name’ refers to specific tab in the Data Dictionary
*note–all times are calculated from arrival to the emergency department
Exclusions
To identify uncomplicated CAP patients: Exclude if:
- On mechanical ventilation in first 48 hours
- Absolute neutrophil count (ANC) <500 cells/uL
- Cystic fibrosis
- Bronchiectasis
- HIV
- Tracheostomy
- Transplant in prior year,
- Hematologic malignancy
- Pulmonary complication (empyema, lung abscess, necrotizing pneumonia)
Exclude cases in which duration is unknown:
- Died during hospitalization
- Discharged to another hospital
- Discharged to inpatient/home hospice
- Antibiotic prescribed at discharge, but duration is missing from record
Exclude conditions requiring longer duration:
- Transferred to ICU during hospitalization
- Bacteremic with non-skin commensal
- Staphylococcus aureus in respiratory culture
- Pseudomonas in respiratory culture
- Legionella pneumonia
- Time to clinical stability > 5 days
Exclude conditions where treatment likely not for uncomplicated CAP:
- <3 days of antibiotics
- >14-day total antibiotic duration
To identify uncomplicated CAP patients: Exclude if:
- On mechanical ventilation in first 48 hours
- Comorbidities Indicated with CAP Value Set, OID:2.16.840.1.113762.1.4.1264.26
- (Sheet Name: “ComrobiditiesIndicatedwithCAP”)
- Absolute neutrophil count <500 cells/uL
- Complete Blood Count (with Diff), OID:1.3.6.1.4.1.6997.4.1.2.271.13.38167.1.1.999.594
- code "Neutrophils [#/volume] in Blood": '26499-4' from "LOINC" display 'Neutrophils [#/volume] in Blood'
- (Sheet Name: (“CompleteBloodCount”)
- Cystic fibrosis
- Comorbidities Indicated with CAP Value Set, OID:2.16.840.1.113762.1.4.1264.26
- (Sheet Name: “ComrobiditiesIndicatedwithCAP”)
- Bronchiectasis
- Comorbidities Indicated with CAP Value Set, OID:2.16.840.1.113762.1.4.1264.26
- (Sheet Name: “ComrobiditiesIndicatedwithCAP”)
- Human immunodeficiency virus (HIV)
- code "Human immunodeficiency virus [HIV] disease": 'B20' from "ICD10CM" display 'Human immunodeficiency virus [HIV] disease'
- code "Patient immunocompromised (finding)": '370388006' from "SNOMEDCT" display 'Patient immunocompromised (finding)' (if facility cannot report HIV status)
- Tracheostomy
- Comorbidities Indicated with CAP Value Set, OID:2.16.840.1.113762.1.4.1264.26
- (Sheet Name: “ComrobiditiesIndicatedwithCAP”)
- Transplant in prior year
- Major Transplant Value Set, OID:2.16.840.1.113883.3.464.1003.198.12.1075
- (Sheet Name: “MajorTransplant”)
- Hematologic malignancy
- Comorbidities Indicated with CAP Value Set, OID:2.16.840.1.113762.1.4.1264.26
- (Sheet Name: “ComrobiditiesIndicatedwithCAP”)
- Pulmonary complication (empyema, lung abscess, necrotizing pneumonia)
- Comorbidities Indicated with CAP Value Set, OID:2.16.840.1.113762.1.4.1264.26
- (Sheet Name: “ComrobiditiesIndicatedwithCAP”)
Exclude cases in which duration is unknown:
- Died during hospitalization
- Patient Expired Value Set, OID: 2.16.840.1.113883.3.117.1.7.1.309 – discharge finding of dead
- (Sheet Name: “PatientExpired”)
- Discharged to another hospital
- Discharge Services Hospital Inpatient
- (Sheet Name: “DischargeServicesHospital”)
- Discharge To Acute Care Facility Value Set, OID:2.16.840.1.113883.3.117.1.7.1.87
- (Sheet Name: “(DischargeAcuteCare”)
- Discharged to inpatient/home hospice
- Discharged to Health Care Facility for Hospice Care Value Set, OID:2.16.840.1.113883.3.117.1.7.1.207
- (Sheet Name: “DischargetoFacilityforHospice”)
- Discharged to Home for Hospice Care Value Set, OID:2.16.840.1.113883.3.117.1.7.1.209
- (Sheet Name: “DischargeHomeforHospice”)
Exclude conditions requiring longer duration:
- Transferred to ICU during hospitalization
- Intensive Care Unit Value Set, OID:2.16.840.1.113762.1.4.1029.204
- (Sheet Name: “IntensiveCareUnit”)
- Bacteremic with non-skin commensal
- Bacteria (Tests in Blood by Culture and Identification Method) Value Set, OID:2.16.840.1.113762.1.4.1146.1460
- (Sheet Name: “Bacteremic”)
- code "Bacteria identified in Blood by Culture": '600-7' from "LOINC" display 'Bacteria identified in Blood by Culture'
- Bacterial Skin Commensals Value Set, OID: 2.16.840.1.114222.24.7.281
- (Sheet Name: “BacterialCommensals”
- Staphylococcus aureus in respiratory culture
- Staphylococcus aureus (Organism or Substance in Lab Results) Value Set, OID:2.16.840.1.113762.1.4.1146.1156
- (Sheet Name: “StaphylococcusAureus”)
- Pseudomonas aeruginosa in respiratory culture
- Pseudomonas aeruginosa (Organism or Substance in Lab Results) Value Set, OID:2.16.840.1.113762.1.4.1146.1679
- (Sheet Name: “PseudomonasAeruginosa”)
- Legionella pneumonia
- Legionella Presence by Urine Antigen or DNA Value Set, OID:2.16.840.1.113762.1.4.1264.27
- (Sheet Name: “LegionellaPresence”)
- Time to clinical stability > 5 days, uses:
- Body Temperature Value Set, OID:2.16.840.1.113762.1.4.1045.152
- (Sheet Name: “BodyTemperature”)
- Systolic Blood Pressure Value Set, OID:2.16.840.1.113762.1.4.1045.163
- (Sheet Name: “SystolicBloodPressure”)
Exclude conditions where treatment likely not for uncomplicated CAP:
- <3 days of antibiotics
- See CQL code
- >14-day total antibiotic duration
- See CQL code
‘Sheet Name’ refers to specific tab in the Data Dictionary
*note–all times are calculated from arrival to the emergency department
Measure Calculation
Step 1: Define Initial Population and Denominator
Step 2: Apply denominator exclusions
Step 3: Define the numerator
Step 4: Calculate the measure score: ((numerator/denominator)*100)
The measure is not stratified.
Table 3 (see Supplemental Materials attachment) indicates the minimum annual number of qualifying cases needed for the denominator to reach each target reliability level at a given facility. In order to achieve a desired reliability of 0.8, each hospital would need to include 49 cases annually. For acceptable reliability (0.7), 19 annual cases would be required, and for high reliability (0.9), 110 annual cases would be required. If a facility has fewer than the minimum number, the hospital is still encouraged to report performance on this measure.
We estimated the minimum sample size using the ICC (intraclass correlation) and the Spearman Brown prophecy formula26-28 The ICC was calculated based on data from 109 Veterans Affairs (VA) facilities between Jan 2022-June 2024. The Spearman Brown formula was used in prior inpatient CAP measures to determine minimal sample size (PN-3a: Blood Cultures Performed within 24 Hours Prior to or 24 Hours After Hospital Arrival for Patients who were Transferred or Admitted to the ICU within 24 Hours of Hospital Arrival; PN-6: Initial Antibiotic Selection for Community-Acquired Pneumonia [CAP in Immunocompetent Patients]; Inappropriate Diagnosis of CAP [CBE ID 3671]); Percent of Hospitalized Pneumonia Patients with Chest Imaging [CBE ID 4440e]).29-31 The minimum recommended sample sizes here are within the range of those previously required for inpatient CAP–for example, the minimum recommended sample size to achieve 0.80 reliability was 59 for the Inappropriate Diagnosis of Community-Acquired Pneumonia measure30 and 174 for the Percent of Hospitalized Pneumonia Patients with Chest Imaging measure, an eCQM endorsed in 2024.29
The full reference list can be found in Section 2.2.
Supplemental Attachment
Measure Record
Point of Contact
n/a
Michael Good
Salt Lake City, UT
United States
Andrea White
University of Utah, Division of General Internal Medicine
Salt Lake City, UT
United States
Importance
Evidence
Pneumonia is not only the most common reason for inpatient antibiotic use but also the most common infectious cause of mortality in the US resulting in approximately 1.4 million emergency department visits, 740,000 hospitalizations, 41,000 deaths, and $7.7 billion in inpatient costs each year in the US.16-19 Studies suggest up to ⅔ of patients hospitalized and treated for CAP have excess duration and that shorter durations are associated with fewer antibiotic-associated adverse events and less risk of developing antibiotic resistant infections.15,21-23 When used as a pay-for-performance measure across 41 Michigan hospitals, the chart review measure from which our eCQM was adapted increased appropriate use of short duration therapy for CAP by 22% and decreased adverse events (driven by lower antibiotic-associated harm; aOR per quarter, 0.98 [95% confidence interval: .96-.99]).24 Taken together, our guideline-based measure has the demonstrated ability to improve patient care for a large number of patients hospitalized with CAP across the US.
The eCQM for CAP duration also balances with other quality efforts, including sepsis-related measures, which aim to increase early use of antibiotic therapy for patients with suspected infection. This metric targets those same patients later during hospitalization when a diagnosis of CAP has been made and clinicians are determining the final appropriate therapy. In addition, we anticipate this measure can augment the NHSN AU module (which assesses overall observed vs. expected antibiotic use in hospitals) and help hospitals understand the quality of their prescribing for the most common infection treated in hospitals–CAP.
References (for full submission, not just 2.2 Evidence of Measure Importance):
1. Antimicrobial Resistance Collaborators. Global burden of bacterial antimicrobial resistance in 2019: a systematic analysis. Lancet. Feb 12 2022;399(10325):629-655. doi:10.1016/s0140-6736(21)02724-0
2. Fridkin S, Baggs J, Fagan R, et al. Vital signs: improving antibiotic use among hospitalized patients. MMWR Morbidity and mortality weekly report. Mar 7 2014;63(9):194-200.
3. CDC. Core Elements of Hospital Antibiotic Stewardship Programs. US Department of Health and Human Services. CDC. Accessed August 3, 2022. https://www.cdc.gov/antibiotic-use/healthcare/pdfs/hospital-core-elements-H.pdf
4. Dunbar LM, Khashab MM, Kahn JB, Zadeikis N, Xiang JX, Tennenberg AM. Efficacy of 750-mg, 5-day levofloxacin in the treatment of community-acquired pneumonia caused by atypical pathogens. Curr Med Res Opin. Apr 2004;20(4):555-63. doi:10.1185/030079904125003304
5. Zhao X, Wu JF, Xiu QY, et al. A randomized controlled clinical trial of levofloxacin 750 mg versus 500 mg intravenous infusion in the treatment of community-acquired pneumonia. Diagn Microbiol Infect Dis. Oct 2014;80(2):141-7. doi:10.1016/j.diagmicrobio.2013.11.008
6. Pakistan Multicentre Amoxycillin Short Course Therapy pneumonia study. Clinical efficacy of 3 days versus 5 days of oral amoxicillin for treatment of childhood pneumonia: a multicentre double-blind trial. Lancet. Sep 14 2002;360(9336):835-41. doi:10.1016/s0140-6736(02)09994-4
7. Greenberg D, Givon-Lavi N, Sadaka Y, Ben-Shimol S, Bar-Ziv J, Dagan R. Short-course antibiotic treatment for community-acquired alveolar pneumonia in ambulatory children: a double-blind, randomized, placebo-controlled trial. Pediatr Infect Dis J. Feb 2014;33(2):136-42. doi:10.1097/inf.0000000000000023
8. el Moussaoui R, de Borgie CA, van den Broek P, et al. Effectiveness of discontinuing antibiotic treatment after three days versus eight days in mild to moderate-severe community acquired pneumonia: randomised, double blind study. BMJ (Clinical research ed). Jun 10 2006;332(7554):1355. doi:10.1136/bmj.332.7554.1355
9. Uranga A, Espana PP, Bilbao A, et al. Duration of Antibiotic Treatment in Community-Acquired Pneumonia: A Multicenter Randomized Clinical Trial. JAMA Intern Med. Sep 1 2016;176(9):1257-65. doi:10.1001/jamainternmed.2016.3633
10. Harris JA, Kolokathis A, Campbell M, Cassell GH, Hammerschlag MR. Safety and efficacy of azithromycin in the treatment of community-acquired pneumonia in children. Pediatr Infect Dis J. Oct 1998;17(10):865-71. doi:10.1097/00006454-199810000-00004
11. Dinh A, Ropers J, Duran C, et al. Discontinuing β-lactam treatment after 3 days for patients with community-acquired pneumonia in non-critical care wards (PTC): a double-blind, randomised, placebo-controlled, non-inferiority trial. Lancet. Mar 27 2021;397(10280):1195-1203. doi:10.1016/s0140-6736(21)00313-5
12. Ginsburg AS, Mvalo T, Nkwopara E, et al. Amoxicillin for 3 or 5 Days for Chest-Indrawing Pneumonia in Malawian Children. N Engl J Med. Jul 2 2020;383(1):13-23. doi:10.1056/NEJMoa1912400
13. Pernica JM, Harman S, Kam AJ, et al. Short-Course Antimicrobial Therapy for Pediatric Community-Acquired Pneumonia: The SAFER Randomized Clinical Trial. JAMA Pediatr. May 1 2021;175(5):475-482. doi:10.1001/jamapediatrics.2020.6735
14. Dinh A, Davido B, Bouchand F, Duran C, Ropers J, Crémieux AC. Honey, I Shrunk the Antibiotic Therapy. Clin Infect Dis. Jun 1 2018;66(12):1981-1982. doi:10.1093/cid/ciy047
15. Vaughn VM, Flanders SA, Snyder A, et al. Excess Antibiotic Treatment Duration and Adverse Events in Patients Hospitalized with Pneumonia: A Multihospital Cohort Study. Annals of internal medicine. Aug 6 2019;171(3):153-163. doi:10.7326/M18-3640
16. Sterrantino C, Trifirò G, Lapi F, et al. Burden of community-acquired pneumonia in Italian general practice. Eur Respir J. Dec 2013;42(6):1739-42. doi:10.1183/09031936.00128713
17. Partouche H, Lepoutre A, Vaure CBD, Poisson T, Toubiana L, Gilberg S. Incidence of all-cause adult community-acquired pneumonia in primary care settings in France. Med Mal Infect. Sep 2018;48(6):389-395. doi:10.1016/j.medmal.2018.02.012
18. McDermott KW aRM. Most Frequent Principal Diagnoses for Inpatient Stays in U.S. Hospitals, 2018. 2021. https://hcup-us.ahrq.gov/reports/statbriefs/sb277-Top-Reasons-Hospital-Stays-2018.pdf#:~:text=The%20most%20frequent%20principal%20diagnoses%20for%20hospitalizations%20in,caused%20by%20tuberculosis%29%2C%20and%20diabetes%20mellitus%20with%20complication
19. Vaughn VM, Dickson RP, Horowitz JK, Flanders SA. Community-Acquired Pneumonia: A Review. Jama. Oct 15 2024;332(15):1282-1295. doi:10.1001/jama.2024.14796
20. Yi SH, Hatfield KM, Baggs J, et al. Duration of Antibiotic Use Among Adults With Uncomplicated Community-Acquired Pneumonia Requiring Hospitalization in the United States. Clin Infect Dis. Apr 17 2018;66(9):1333-1341. doi:10.1093/cid/cix986
21. Schrag SJ, Pena C, Fernandez J, et al. Effect of short-course, high-dose amoxicillin therapy on resistant pneumococcal carriage: a randomized trial. JAMA. Jul 4 2001;286(1):49-56. doi:10.1001/jama.286.1.49
22. Wistrom J, Norrby SR, Myhre EB, et al. Frequency of antibiotic-associated diarrhoea in 2462 antibiotic-treated hospitalized patients: a prospective study. The Journal of antimicrobial chemotherapy. Jan 2001;47(1):43-50. doi:10.1093/jac/47.1.43
23. Tamma PD, Avdic E, Li DX, Dzintars K, Cosgrove SE. Association of Adverse Events With Antibiotic Use in Hospitalized Patients. JAMA Internal Medicine. 2017;
24. Vaughn VM, Gandhi TN, Hofer TP, et al. A Statewide Collaborative Quality Initiative to Improve Antibiotic Duration and Outcomes in Patients Hospitalized With Uncomplicated Community-Acquired Pneumonia. Clin Infect Dis. Aug 31 2022;75(3):460-467. doi:10.1093/cid/ciab950
25. Metlay JP, Waterer GW, Long AC, et al. Diagnosis and Treatment of Adults with Community-acquired Pneumonia. An Official Clinical Practice Guideline of the American Thoracic Society and Infectious Diseases Society of America. Am J Respir Crit Care Med. Oct 1 2019;200(7):e45-e67. doi:10.1164/rccm.201908-1581ST
26. van Ast JF, Talmon JL, Renier WO, Hasman A. An approach to knowledge base construction based on expert opinions. Methods Inf Med. 2004;43(4):427-32.
27. Zimmerman DW, Williams RH, Burkheimer GJ. Dependence of reliability of multiple-choice tests upon number of choices per item: prediction from the Spearman-Brown formula. Psychol Rep. Dec 1966;19(3):1239-43. doi:10.2466/pr0.1966.19.3f.1239
28. de Vet HCW, Mokkink LB, Mosmuller DG, Terwee CB. Spearman-Brown prophecy formula and Cronbach's alpha: different faces of reliability and opportunities for new applications. J Clin Epidemiol. May 2017;85:45-49. doi:10.1016/j.jclinepi.2017.01.013
29. Partnership for Quality Measurement. Percent of hospitalized pneumonia patients with chest imaging confirmation, CBE ID 4440e. Accessed 9/25/2024, https://p4qm.org/measures/4440e
30. Partnership for Quality Measurement. Inappropriate diagnosis of community-acquired pneumonia (CAP) in hospitalized medical patients, CBE ID 3671. Accessed 9/24/2024, https://p4qm.org/measures/3671
31. The Joint Commission. Specifications manual for Joint Commission National Quality Measures (v2015A): Pneumonia (PN). Accessed 9/25/2024, https://manual.jointcommission.org/releases/TJC2015A/Pneumonia.html#A_42Pneumonia_40PN_41_Initial_Patient_Population_42
32. Kazakova SV, Baggs J, McDonald LC, et al. Association Between Antibiotic Use and Hospital-onset Clostridioides difficile Infection in US Acute Care Hospitals, 2006-2012: An Ecologic Analysis. Clin Infect Dis. Jan 1 2020;70(1):11-18. doi:10.1093/cid/ciz169
33. Branch-Elliman W, O'Brien W, Strymish J, Itani K, Wyatt C, Gupta K. Association of Duration and Type of Surgical Prophylaxis With Antimicrobial-Associated Adverse Events. JAMA Surg. Jul 1 2019;154(7):590-598. doi:10.1001/jamasurg.2019.0569
34. Brown KA, Fisman DN, Moineddin R, Daneman N. The magnitude and duration of Clostridium difficile infection risk associated with antibiotic therapy: a hospital cohort study. PLoS One. 2014;9(8):e105454. doi:10.1371/journal.pone.0105454
35. Russo TA, Spellberg B, Johnson JR. Important Complexities of the Antivirulence Target Paradigm: A Novel Ostensibly Resistance-Avoiding Approach for Treating Infections. J Infect Dis. Mar 15 2016;213(6):901-3. doi:10.1093/infdis/jiv533
36. Curran J, Lo J, Leung V, et al. Estimating daily antibiotic harms: an umbrella review with individual study meta-analysis. Clin Microbiol Infect. Apr 2022;28(4):479-490. doi:10.1016/j.cmi.2021.10.022
37. Poku E, Cooper K, Cantrell A, et al. Systematic review of time lag between antibiotic use and rise of resistant pathogens among hospitalized adults in Europe. JAC Antimicrob Resist. Feb 2023;5(1):dlad001. doi:10.1093/jacamr/dlad001
38. Spellberg B. The new antibiotic mantra—“shorter is better”. JAMA internal medicine. 2016;176(9):1254-1255.
39. Madaras-Kelly KJ, Burk M, Caplinger C, et al. Total duration of antimicrobial therapy in veterans hospitalized with uncomplicated pneumonia: Results of a national medication utilization evaluation. Journal of hospital medicine. Dec 2016;11(12):832-839. doi:10.1002/jhm.2648
40. Weber BR, Noble BN, Bearden DT, et al. Antibiotic prescribing upon discharge from the hospital to long-term care facilities: Implications for antimicrobial stewardship requirements in post-acute settings. Infect Control Hosp Epidemiol. Jan 2019;40(1):18-23. doi:10.1017/ice.2018.288
41. Low M, Neuberger A, Hooton TM, et al. Association between urinary community-acquired fluoroquinolone-resistant Escherichia coli and neighbourhood antibiotic consumption: a population-based case-control study. Lancet Infect Dis. Apr 2019;19(4):419-428. doi:10.1016/S1473-3099(18)30676-5
42. Gandhi T, Petty L, Vaughn V, et al. Risk Factors and outcomes associated with inappropriate empiric broad-spectrum antibiotic use in hospitalized patients with community-acquired pneumonia. Antimicrobial Stewardship & Healthcare Epidemiology. 2023;3(S2):s31-s32. doi:10.1017/ash.2023.258
43. Graber CJ, Jones MM, Goetz MB, et al. Decreases in Antimicrobial Use Associated With Multihospital Implementation of Electronic Antimicrobial Stewardship Tools. Clin Infect Dis. Aug 22 2020;71(5):1168-1176. doi:10.1093/cid/ciz941
44. Kelly AA, Jones MM, Echevarria KL, et al. A Report of the Efforts of the Veterans Health Administration National Antimicrobial Stewardship Initiative. Infect Control Hosp Epidemiol. May 2017;38(5):513-520. doi:10.1017/ice.2016.328
45. Goetz MB, Willson T, Rubin MA, Stevens VW, Graber CJ. Antimicrobial use before and during COVID-19: data from 108 Veterans Affairs medical centers. Antimicrobial Stewardship & Healthcare Epidemiology. 2024;4(1):e109. e109. doi:10.1017/ash.2024.352
46. Ciarkowski CE, Timbrook TT, Kukhareva PV, et al. A Pathway for Community-Acquired Pneumonia With Rapid Conversion to Oral Therapy Improves Health Care Value. Open Forum Infect Dis. Nov 2020;7(11):ofaa497. doi:10.1093/ofid/ofaa497
47. Shorr AF, Owens RC, Jr. Guidelines and quality for community-acquired pneumonia: measures from the Joint Commission and the Centers for Medicare and Medicaid Services. American journal of health-system pharmacy. Jun 15 2009;66(12 Suppl 4):S2-7. doi:10.2146/090087a
48. Centers for Medicare and Medicaid. Merit-Based Incentive Payment System (MIPS): Simple Pneumonia with Hospitalization Measure, 2020 performance period. Accessed 9/27/2024, https://qpp.cms.gov/docs/cost_specifications/2019-12-17-mif-ebcm-pna-hosp.pdf
49. Centers for Medicare & Medicaid Services. Quality ID #111: Pneumococcal vaccination status for older adults. . Accessed 9/27/2024, https://qpp.cms.gov/docs/QPP_quality_measure_specifications/CQM-Measures/2023_Measure_111_MedicarePartBClaims.pdf
50. Centers for Medicare & Medicaid Services. Eligible Hospital / Critical Access Hospital eCQMs: hospital quality reporting table of eCQMs. Accessed 9/27/2024, https://ecqi.healthit.gov/sites/default/files/Hybrid-EH-CAH-MeasuresTable-2023-11.pdf
51. Agency for Healthcare Research and Quality. AHRQ IQI technical documentation, version v2023. Rockville, MD. Accessed 9/27/2024, https://qualityindicators.ahrq.gov/measures/iqi_resources
52. CDC. Core Elements of Hospital Antibiotic Stewardship Programs. (US Department of Health and Human Services, CDC) (2019. Available at https://www.cdc.gov/antibiotic-use/core-elements/hospital.html.).
53. The Joint Commission. R3 Report Issue 35: New and Revised Requirements for Antibiotic Stewardship 2022. https://www.jointcommission.org/-/media/tjc/documents/standards/r3-reports/r3_antibioticstewardship_july2022_final.pdf
54. QSO-22-20-Hospitals Infection Prevention and Control and Antibiotic Stewardship Program Interpretive Guidance Update (2022).
55. van Santen KL, Edwards JR, Webb AK, et al. The Standardized Antimicrobial Administration Ratio: A New Metric for Measuring and Comparing Antibiotic Use. Clin Infect Dis. Jul 2 2018;67(2):179-185. doi:10.1093/cid/ciy075
56. Department of Health and Human Services Centers for Medicare & Medicaid Services. Hospital Inpatient Prospective Payment Systems for Acute Care Hospitals and the Long-Term Care Hospital Prospective Payment System and Policy Changes and Fiscal Year 2023 Rates; Quality Programs and Medicare Promoting Interoperability Program Requirements for Eligible Hospitals and Critical Access Hospitals; Costs Incurred for Qualified and Non-Qualified Deferred Compensation Pland; and Changes to Hospital and Critical Access Hospital Conditions of Participation. Federal Register. 2022;87 (153):487780-49499.
57. Moehring R, Vaughn VM. Development of Inpatient Stewardship Metrics: Is It Time for Public Reporting? Infect Dis Clin North Am. Dec 2023;37(4):853-871. doi:10.1016/j.idc.2023.07.006
58. Watkins RR, Lemonovich TL. Diagnosis and management of community-acquired pneumonia in adults. Am Fam Physician. Jun 1 2011;83(11):1299-306.
59. NHSN. Antimicrobial Use and Resistance (AUR) Module. Centers for Disease Control. https://www.cdc.gov/nhsn/pdfs/pscmanual/11pscaurcurrent.pdf
60. Fishman N, America SfHEo, America IDSo. Policy statement on antimicrobial stewardship by the Society for Healthcare Epidemiology of America (SHEA), the Infectious Diseases Society of America (IDSA), and the Pediatric Infectious Diseases Society (PIDS). Infection Control & Hospital Epidemiology. 2012;33(4):322-327.
61. Barlam TF, Cosgrove SE, Abbo LM, et al. Implementing an Antibiotic Stewardship Program: Guidelines by the Infectious Diseases Society of America and the Society for Healthcare Epidemiology of America. Clin Infect Dis. May 15 2016;62(10):e51-77. doi:10.1093/cid/ciw118
62. Borek AJ, Campbell A, Dent E, et al. Implementing interventions to reduce antibiotic use: a qualitative study in high-prescribing practices. BMC Fam Pract. Jan 23 2021;22(1):25. doi:10.1186/s12875-021-01371-6
63. Tonkin-Crine S, Anthierens S, Francis NA, et al. Exploring patients' views of primary care consultations with contrasting interventions for acute cough: a six-country European qualitative study. NPJ Prim Care Respir Med. Jul 17 2014;24:14026. doi:10.1038/npjpcrm.2014.26
64. Centers for Disease Control and Prevention. NHSN list of common commensals. Accessed 9/30/2024, https://view.officeapps.live.com/op/view.aspx?src=https%3A%2F%2Fwww.cdc.gov%2Fnhsn%2FXLS%2Fmaster-organism-Com-Commensals-Lists.xlsx&wdOrigin=BROWSELINK)
65. Mandell LA, Wunderink RG, Anzueto A, et al. Infectious Diseases Society of America/American Thoracic Society consensus guidelines on the management of community-acquired pneumonia in adults. Clin Infect Dis. Mar 1 2007;44 Suppl 2:S27-72. doi:10.1086/511159
66. Aliberti S, Zanaboni AM, Wiemken T, et al. Criteria for clinical stability in hospitalised patients with community-acquired pneumonia. Eur Respir J. Sep 2013;42(3):742-9. doi:10.1183/09031936.00100812
67. Halm EA, Fine MJ, Marrie TJ, et al. Time to clinical stability in patients hospitalized with community-acquired pneumonia: implications for practice guidelines. Jama. May 13 1998;279(18):1452-7. doi:10.1001/jama.279.18.1452
68. Gupta AB, Flanders SA, Petty LA, et al. Inappropriate Diagnosis of Pneumonia Among Hospitalized Adults. JAMA Intern Med. May 1 2024;184(5):548-556. doi:10.1001/jamainternmed.2024.0077
69. Gupta A, Petty L, Gandhi T, et al. Overdiagnosis of urinary tract infection linked to overdiagnosis of pneumonia: a multihospital cohort study. BMJ Qual Saf. May 2022;31(5):383-386. doi:10.1136/bmjqs-2021-013565
70. Vaughn VM, Gandhi TN, Chopra V, et al. Antibiotic Overuse After Hospital Discharge: A Multi-hospital Cohort Study. Clin Infect Dis. Dec 6 2021;73(11):e4499-e4506. doi:10.1093/cid/ciaa1372
71. Gandhi TN, Vaughn VM, Petty LA, et al. 2893. The Michigan Hospital Medicine Safety Consortium: Improving Patient Care by Reducing Excessive Antibiotic Use in Patients Hospitalized with Community-Acquired Pneumonia. Open Forum Infectious Diseases. 2019;6(Supplement_2):S80-S81. doi:10.1093/ofid/ofz359.171
72. White AT, Vaughn V, Petty LA, et al. Development of patient safety measures to identify inappropriate diagnosis of common infections. Clin Infect Dis. Jan 31 2024;doi:10.1093/cid/ciae044
73. Spivak ES, Burk M, Zhang R, et al. Management of Bacteriuria in Veterans Affairs Hospitals. Clin Infect Dis. Sep 15 2017;65(6):910-917. doi:10.1093/cid/cix474
74. Sutton JD, Carico R, Burk M, et al. Inpatient Management of Uncomplicated Skin and Soft Tissue Infections in 34 Veterans Affairs Medical Centers: A Medication Use Evaluation. Open Forum Infect Dis. Jan 2020;7(1):ofz554. doi:10.1093/ofid/ofz554
75. Livorsi DJ, Branch-Elliman W, Drekonja D, et al. Research agenda for antibiotic stewardship within the Veterans' Health Administration, 2024-2028. Infect Control Hosp Epidemiol. Feb 2 2024:1-7. doi:10.1017/ice.2024.6
76. Mehta HB, Li S, An H, Goodwin JS, Alexander GC, Segal JB. Development and Validation of the Summary Elixhauser Comorbidity Score for Use With ICD-10-CM-Coded Data Among Older Adults. Annals of internal medicine. Oct 2022;175(10):1423-1430. doi:10.7326/m21-4204
77. National Quality Forum. Scientific Methods Panel Evaluation Guidelines. Accessed 10/7/2024, https://www.qualityforum.org/Measuring_Performance/Scientific_Methods_Panel.aspx
78. Adams JL. The Reliability of Provider Profiling: A Tutorial. RAND Corporation; 2009.
79. Centers for Medicare & Medicaid Services. MMS Blueprint Supplemental Material: Technical Expert Panels. https://www.cms.gov/files/document/blueprint-technical-expert-panels.pdf
80. Centers for Medicare & Medicaid Services. Measures Management System Risk Adjustment. Centers for Medicare & Medicaid. Accessed 8/20/24, https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/MMS/Downloads/Risk-Adjustment.pdf
81. Ramirez JA, Wiemken TL, Peyrani P, et al. Adults Hospitalized With Pneumonia in the United States: Incidence, Epidemiology, and Mortality. Clin Infect Dis. Nov 13 2017;65(11):1806-1812. doi:10.1093/cid/cix647
82. Burton DC, Flannery B, Bennett NM, et al. Socioeconomic and racial/ethnic disparities in the incidence of bacteremic pneumonia among US adults. Am J Public Health. Oct 2010;100(10):1904-11. doi:10.2105/ajph.2009.181313
83. Altawalbeh SM, Wateska AR, Nowalk MP, et al. Societal Cost of Racial Pneumococcal Disease Disparities in US Adults Aged 50 Years or Older. Appl Health Econ Health Policy. Jan 2024;22(1):61-71. doi:10.1007/s40258-023-00854-0
84. Nowalk MP, Wateska AR, Lin CJ, et al. Racial Disparities in Adult Pneumococcal Vaccination Indications and Pneumococcal Hospitalizations in the U.S. J Natl Med Assoc. Oct 2019;111(5):540-545. doi:10.1016/j.jnma.2019.04.011
85. Hausmann LR, Ibrahim SA, Mehrotra A, et al. Racial and ethnic disparities in pneumonia treatment and mortality. Med Care. Sep 2009;47(9):1009-17. doi:10.1097/MLR.0b013e3181a80fdc
86. Wiemken TL, Carrico RM, Furmanek SP, et al. Socioeconomic Position and the Incidence, Severity, and Clinical Outcomes of Hospitalized Patients With Community-Acquired Pneumonia. Public Health Rep. May/Jun 2020;135(3):364-371. doi:10.1177/0033354920912717
87. Downing NS, Wang C, Gupta A, et al. Association of Racial and Socioeconomic Disparities With Outcomes Among Patients Hospitalized With Acute Myocardial Infarction, Heart Failure, and Pneumonia: An Analysis of Within- and Between-Hospital Variation. JAMA Netw Open. Sep 7 2018;1(5):e182044. doi:10.1001/jamanetworkopen.2018.2044
88. Vaughn VM, Ratz D, Greene MT, et al. Antibiotic Stewardship Strategies and Their Association With Antibiotic Overuse After Hospital Discharge: An Analysis of the Reducing Overuse of Antibiotics at Discharge (Road) Home Framework. Clin Infect Dis. Sep 29 2022;75(6):1063-1072. doi:10.1093/cid/ciac104
89. Daniels LM, Weber DJ. Interventions to improve antibiotic prescribing at hospital discharge: A systematic review. Infect Control Hosp Epidemiol. Jan 2021;42(1):96-99. doi:10.1017/ice.2020.367
Measure Impact
Harms from longer antibiotic durations are increasingly observed, including side effects,15,23 Clostridioides difficile infection,32-34 acute kidney injury,33 disruption of normal flora35 and emergence of antibacterial resistance.36,37 Over the past 2 decades, several studies have demonstrated non-inferior clinical outcomes with shorter durations of antibiotic therapy compared to longer durations.8,9,11,38,39 ATS/IDSA CAP guidelines in 201925 recommended a duration of therapy no more than 5 days of antibiotics if the patient reaches clinical stability. Since these recommendations, additional clinical trials suggested durations even shorter than 5 days could be adequate for selected patients reaching clinical stability sooner.11 Excess antibiotic duration has also been shown to increase antibiotic resistance and harm in nursing homes and other post-hospitalization patient communities.40,41
Taken together, our guideline-based measure has the demonstrated ability to safely reduce antibiotic use while improving patient care for a large number of patients hospitalized with CAP across the US.
If the currently proposed measure is implemented, we expect a fairly rapid reduction in excessive antibiotic duration for patients hospitalized with uncomplicated CAP. Such a reduction is expected given improvements already seen in antibiotic duration when such measures are implemented.24
When used as a pay-for-performance measure across 41 Michigan hospitals, the chart review measure from which our eCQM was adapted increased appropriate use of short duration therapy for CAP by 22% (from 22.1% to 45.9%) and decreased adverse events (driven by decreased antibiotic-associated harm; aOR per quarter, 0.98 [95% confidence interval: .96-.99]).24 There was no change in 30-day mortality, hospital readmission, or urgent visit.
- The VA healthcare system (among other hospitals) has implemented interventions targeted at reducing excess antibiotic duration for uncomplicated CAP.42-44 Substantial decreases in antibiotic use have been demonstrated in the VA since the establishment of the national VA Antimicrobial Stewardship Task Force in 201144 that have persisted through the COVID-19 pandemic.45 Specific to the excess antibiotic duration for uncomplicated CAP eCQM, there have been educational campaigns to improve concordance with guideline recommended therapy in the VA but no formal implementation of the measure. Regardless, with education and stewardship interventions, mean antibiotic duration has decreased steadily from 2015 to 2020, and these improvements were maintained through the first part of 2024 (Figure 1, see Supplemental Materials attachment).
These improvements are expected to have direct impact on patients by reducing antibiotic-associated adverse events24 (including Clostridioides difficile infection) and reducing development/acquisition of antibiotic resistance. Furthermore, reducing antibiotic duration for the number one most common inpatient indication for antibiotics is expected to have nationwide public health improvements by reducing antibiotic resistance.
Potential unintended consequences include under-treatment of patients who require longer antibiotic durations. We anticipate this impact to be minimal. We asked the Technical Expert Panel (TEP, 8 clinicians representing 8 different clinical specialties and national organizations [Table 4 in the Supplemental Materials attachment]) whether we should measure “under-treatment”, and no (0/8) TEP members responded that we should. Half (4/8) responded “there’s no such thing as undertreatment,” the other 4 reported:
- “operationally defining undertreatment in an eCQM may be prone to validity issues due to missing data"
- “if you exclude < 3 days of Rx AND those who do not stabilize til after 5 days, then there is honestly no undertreatment. I think there is a concern in those who take >5 days to stabilize, but per your data thats <1% of the cohort and i would not waste effort for that”
- “No [undertreatment] is very rare”
- “I think it depends if we can determine if these patients were thought to have pneumonia or not. Maybe the admitting diagnosis is pneumonia but discharge diagnosis is not, and abx were stopped early - that would be ok [to treat for a shorter duration] and not under-treatment. But if thought to have pneumonia and treated <3-5 days, and esp if not clinically stable, that would be under-treatment.”
Notably, newer clinical trial evidence suggests even shorter durations (3 days) may be safe for hospitalized patients with CAP;8,11 however, we are targeting a higher range (5 days with one day for errors in calculation) for initial measure implementation to minimize potential harms from treating patients with short durations who may necessitate longer durations. This strategy was safe and improved patient outcomes when implemented across 41 Michigan hospitals.24 That said, we anticipate evaluating for unanticipated consequences during the measure maintenance submission.
When asked about other unintended consequences, the main concerns from our TEP included feasibility issues around assessing clinical stability, particularly for critical access (CAH) and small hospitals with less information technology (IT) infrastructure. These concerns informed our approach to simplifying the measure (see feasibility section).
The full reference list can be found in Section 2.2.
Pneumonia is a well-established target of clinical quality and safety monitoring. Table 6 (see Supplemental Materials attachment) lists existing and prior measures for pneumonia as well as existing measures for antibiotic use that may be influenced by measures of treatment of pneumonia.31,47-51
Most prior and existing pneumonia performance measures have focused on initial treatment of pneumonia during the first 48 hours of treatment. Antibiotic duration has not been a focus of previous measures despite the CDC,52 The Joint Commission,53 and Centers for Medicare and Medicaid Services54 recommending that antibiotic stewardship programs implement interventions to improve antibiotic duration.
Currently, the CDC’s NHSN AU Option provides a risk-adjusted benchmark comparison for participating hospitals to use to drive improvement.55 The standardized antimicrobial administration ratio (SAAR) was introduced in 2015 and is based on antibiotic days of therapy (DOT), similar in structure to risk-adjusted comparisons of health care associated infection rates. SAAR is a ratio of observed to expected DOT for a particular antimicrobial agent category and location. The 2023 CMS Inpatient Prospective Payment System rule requires NHSN AU reporting for US acute care hospitals in the Promoting Interoperability Program. Hospitals that do not meet requirements could lose funding incentives for promoting interoperability.56 The start date for enforcement of reporting will begin in late 2025. Though AU reporting will be required, hospital-specific data will be neither publicly reported nor used in pay-for-performance programs. Regardless, this mandate means that routine assessments of risk-adjusted AU comparisons to a national benchmark are now a reality for most US hospitals. As of September 2024, 4087 hospitals had reported to the NHSN AU option—a number that will continue to grow now that reporting is required.57
Notably, NHSN AU data do not include discharge data (which accounts for 90% of excess antibiotic duration)15 and therefore does not allow hospitals to understand the total duration of therapy for CAP. Thus, we anticipate this eCQM being used in concert with the NHSN AU module to help hospitals and their antimicrobial stewardship teams understand how to improve care. We have collaborated with the CDC in designing this measure with the goal of working within their NHSN AU framework (when NHSN moves to patient-level data collection).
NHSN AU data are broad antibiotic-use metrics. No eCQMs specifically for CAP currently exist;50 though one related to pneumonia diagnosis was recently endorsed.29 A chart-review based version of this duration measure has been in use in 69 Michigan hospitals as a pay-for-performance measure where it has demonstrably improved patient care.24 By converting the chart review duration measure into an eCQM, we hope to enable broader implementation of the improvements necessary to reduce antibiotic duration to improve care of hospitalized patients with CAP.
The full reference list can be found in Section 2.2.
We explored the value of the pneumonia antibiotic treatment quality through two approaches: 1) literature review of guidelines, clinician and patient perspectives and 2) engagement with clinicians in a technical expert panel (TEP).
As noted above, multiple national organizations consider appropriateness of antibiotic therapy for pneumonia as a priority area–including the CDC, The Joint Commission, and Centers for Medicare and Medicaid Services. Broadly, improving duration of therapy is considered part of the definition of antibiotic stewardship–Antibiotic stewardship has been defined in a consensus statement from the Infectious Diseases Society of America (IDSA), the Society for Healthcare Epidemiology of America (SHEA), and the Pediatric Infectious Diseases Society (PIDS) as “coordinated interventions designed to improve and measure the appropriate use of [antibiotic] agents by promoting the selection of the optimal [antibiotic] drug regimen including dosing, duration of therapy, and route of administration.”60 In addition, IDSA in its guidelines for Implementing an Antibiotic Stewardship program provide a strong recommendation (based on moderate quality evidence that “Antimicrobial stewardship programs (ASPs) implement guidelines and strategies to reduce antibiotic therapy to the shortest effective duration.”61 IDSA guidelines also specifically call out CAP and CAP duration of therapy as an important target, “ASP interventions for CAP have increased the proportion of patients receiving appropriate therapy…”61
Data specifically from patients about antibiotic duration is limited. However, qualitative studies have found patients are interested in minimizing exposure to unnecessary antibacterials particularly when objective evidence provides evidence that antibacterials are unnecessary.62,63
Technical Expert Panel (TEP)
The TEP meeting was conducted remotely via videoconference with 8 clinicians representing 8 different clinical specialties and national organizations (see Table 4, Supplemental Materials attachment). An agenda with specific sets of questions tailored to the measure development phase were provided to panelists prior to the discussion session. We also surveyed TEP members after the meeting to obtain their feedback and guidance on the proposed measure.
When surveyed, 75% (6/8) of our TEP believed it was “very important” to “develop an electronic measure of the duration of antibiotic therapy for persons diagnosed with CAP” the remaining 25% (2/8) reported it was “important.” When surveyed, 75% (6/8) members of our TEP believed duration of therapy for CAP was “important” or “very important” for “patient outcomes (e.g., adverse events, C. difficile infection, antibiotic resistance)”.
Comments include:
- “Excessive treatment is very common and this could reduce antibiotic burden and improve patient safety without worse outcomes.”
- “I think this will educate and drive change of our provider if it provides good feedback for facilities. It much needed I certainly see this here. I just complete a discharge [antibiotic days of therapy] and was not surprised by what I found.”
- “This is one of the easiest, most evidence based interventions that can be made - one that clearly is safe to do and there is clear harm if we do not optimize.”
- “The measure would support a standard, operational definition and practice for antibiotic treatment and prescribing.”
- “There is a large gap in adherence to recommended length of therapy”
The full reference list can be found in Section 2.2.
Performance Gap
Published studies suggest wide variation in antibiotic duration for hospitalized patients with CAP with rates of excess duration for CAP varying from 38-95% across hospitals.2,15,20,39
In addition to the published studies noted above, we analyzed Veterans Affairs (VA) Computerized Patient Records System (CPRS) data from 109 VA health care systems including all 28,238 eligible hospitalizations between January 1, 2022 and June 30, 2024. Performance scores by decile from those 109 hospitals are shown in Table 1 (individual hospital data in Figure 2, see Supplemental Materials attachment). In addition to VA health care systems, we analyzed University of Utah and University of Michigan hospitalizations using the eCQM applied to their Epic instances (see Table 5, Supplemental Materials attachment). These eCQM results may underestimate the performance gap, as University of Utah, University of Michigan, and the VA healthcare systems have all implemented interventions to improve antibiotic duration for CAP with published success.24,43,44,46 For example, substantial decreases in antibiotic use have been demonstrated in the VA since the establishment of the national VA Antimicrobial Stewardship Task Force in 201144 that have persisted through the COVID-19 pandemic.45 The improvements made at University of Utah, University of Michigan, and the VA healthcare system all demonstrate that these measures can be improved with data-driven antibiotic stewardship efforts.
The full reference list can be found in Section 2.2.
Equity
Equity
Pneumonia disproportionately affects older adults, individuals from underrepresented minority communities, and those with lower socioeconomic status.81-83 Previous research has highlighted disparities in both pneumonia care processes and outcomes.84-87 The proposed eCQM aims to promote health equity by enhancing quality measurement and improving pneumonia-related care, targeting these documented inequalities.
Previously, no differences by patient race, age, gender have been found related to excess duration for CAP.15,70 Rather, on multivariable analysis, having a respiratory culture (positive or negative result) or a nonculture diagnostic test, a longer hospital stay, high-risk antibiotic use in the prior 90 days, and not having total treatment duration documented in the discharge summary were associated with higher rates of excess duration.15,70
Notably, hospital characteristics including being self-reported as for-profit or being a smaller (e.g., more rural) hospital were associated with more excess antibiotic duration.15,70 We anticipate that the measure would therefore be most helpful at reducing inequities in these populations.
The full reference list can be found in Section 2.2.
Feasibility
Feasibility
We implemented and tested the proposed eCQM within 3 health systems to assess: 1) whether all required data elements were routinely generated during the care of CAP hospitalizations, 2) which barriers or challenges exist when implementing and extracting the measure from current healthcare data to inform our feasibility scorecard, and 3) whether any elements were uncommon or able to be removed to simplify the measure.
First, all of the data elements collected are already part of normal healthcare system processes for care delivery and billing, so documentation of these items for the proposed eCQM do not add any additional burdens for a healthcare system including clinician workflow, diagnostic thought processes, or patient-physician interactions. During the normal course of care, inpatient clinicians must document within the EHR a discharge diagnosis of pneumonia, order antimicrobials that are recorded in the EHR, and obtain labs, vital signs, and chest imaging that generates a result in the form of structured data that is stored in the EHR. One of our main goals of the measure was to leverage the most reliable, accurate and standardized data elements and to provide a simple, transparent measure that could be consistently extracted, calculated, and interpreted without substantial data resources or expertise. To accomplish this goal, our measure only uses structured data and does not require unstructured data or natural language processing.
As with any eCQM, there is significant work to collate and clean EHR data and ensure their accuracy for measure assessment. To reduce this work, when possible, we used existing value sets—particularly those used by the CDC or NHSN where the measure may be integrated in the future—to streamline use and make the measure more feasible. Examples of harmonized value sets include:
- CDC’s list of common bacterial commensals64 (OID: 2.16.840.1.114222.24.7.281)
- CDC’s definition of Intravenous Medication Route” (OID: 2.16.840.1.113762.1.4.1190.75)
- CDC NHSN to define unit (e.g., ICU; OID: 2.16.840.1.113762.1.4.1029.204)
Data element feasibility testing:
During measure development, we assessed within three different health systems which data elements (originally present in guidelines or in the original chart-review based measures) were difficult to standardize or collect and what (if any) impact their exclusion vs. inclusion would have on measure validity. The results of this process are found in the “feasibility scorecard” and described below in the “feasibility informed measure” section.
The full reference list can be found in Section 2.2.
Data Elements Removed After Feasibility Testing:
- CD4 count- originally we anticipated excluding patients with acquired immunodeficiency syndrome based on CD4 count plus a diagnosis of HIV. CD4 count was difficult to identify in all three systems as the results were located in multiple different laboratory tests and often in unstructured fields. There were no ICD10 codes to distinguish acquired immunodeficiency syndrome from HIV. Because patients with HIV (regardless of CD4 count) were uncommon in all three health systems, we elected to exclude all patients with HIV rather than require assessment of CD4 count. Patients with HIV represent 0.5% of the University of Michigan population, 1.6% of the VA population, and 2.1% of the University of Utah population (prior to applying any additional exclusions). In contrast to CD4 count, HIV can be identified based on ICD-10 diagnostic codes alone, improving feasibility.
- Monday/Wednesday/Friday antibiotics- originally, as a way of excluding chronic suppressive antibiotics, we sought to exclude patients prescribed Monday/Wednesday/Friday dosing at discharge. The VA CPRS database did not capture this type of dosing in a structured field and thus assessing this would be an undue burden. Testing in the other two systems showed that only 0.5% of patients had this type of dosing and 59% of those patients were excluded for an alternative reason. Thus, this exclusion was dropped.
- Intravenous antibiotics at discharge- originally, as an additional way of excluding complicated pneumonia, we sought to exclude patients prescribed intravenous (IV) antibiotics at discharge. The VA CPRS database did not capture IV antibiotics after discharge in a structured field (they are documented in an “outpatient parenteral antibiotic therapy” note) and thus collecting this exclusion criteria would be an undue burden. University of Michigan showed that only 2.2% of patients had intravenous antibiotics prescribed at discharge of which 71% were excluded for an alternative reason. Given the minimal impact of this exclusion on performance—and feasibility concerns—this exclusion was dropped.
Data elements simplified:
- Time to clinical stability- 62.5% (5/8) of our TEP strongly agreed and 37.5% (3/8) agreed that patients who did not achieve clinical stability by hospital day 5 should be excluded from our measure. National guidelines and clinical trials11,25 define clinical stability as being afebrile (≤37.8 °C) with no more than one of the following: heart rate >100 beats per minute, respiratory rate >24 breaths per minute, hypoxic (i.e., SpO2 <90% or PaO2 <60 mmHg), and systolic blood pressure <90 mmHg. Our TEP was concerned about the feasibility for assessing this definition, particularly for hospitals with fewer information technology resources (e.g., critical access hospitals)—e.g., "Computationally, calculating clinical stability could be resource intensive for EHR vendors/data intermediaries, due to the large volume of vitals data and complex logic to connect values”.
As this exclusion criterion applied to >10% of patients at the University of Michigan, we could not drop the criterion altogether. Literature review demonstrated that the guideline-based stability criteria were designed originally not to determine duration but to determine suitability for discharge and that “altered mental status” was only originally meant to be applied for suitability for discharge, not clinical stability.65-67 Thus, we dropped altered mental status as a stability criterion. This did not impact measure exclusions as <1% of patients had altered mental status with no other signs of clinical stability beyond day 5 of hospitalization. Our TEP panel agreed with this change with one member reporting “that seems to simply the complexity and still produce the end result.”
Next, we tested a simpler definition that defined clinical stability as afebrile (max temperature for the calendar day ≤37.8 °C) and normotensive (lowest systolic blood pressure for the calendar day ≥90 mmHg). We also considered a patient to be clinically stable on the day of discharge (this was derived from the original chart review version of the measure used in Michigan hospitals and suggested by a TEP member—"Include discharge <5 days as an exclusion/proxy for clinical stability”).24 This modified definition was feasible for all three sites where we tested the measure and was highly specific (98% specificity).
The modified version of clinical stability (i.e., afebrile and normotensive) did not identify all patients who (based on chart review) were unstable beyond day 5 (sensitivity 29%). To assess whether the low sensitivity would impact the measures’ validity, we assessed how the two definitions would impact the University of Michigan’s measure performance. Using the original, expanded vital sign criteria would result in more patients being excluded and 53.7% (1734/3227) of patients being quantified as having excess duration. Using the modified (and easier) definition would result in 54.5% (1861/3417) of patients being classified as having an excess duration. Given that the two definitions produced comparable results and did not change University of Michigan’s decile of performance, we suggest (and report here) the more feasible, modified stability criteria.
After presenting the measure to the TEP, we asked whether the “measure appropriately balances the need for accuracy with complexity”? 75% (6/8) responded it was the “perfect balance” with 1 reporting it was too complex and 1 reporting it was somewhat too simple. The TEP member reporting it was too complex suggested, “mov[ing] the "clinical stability" definition to denominator exceptions.” In the final measure, time to clinical stability (>5 days) is a denominator exclusion. The TEP member stating that the measure was too simple was concerned about potential overtreatment of viral pneumonia, stating, “I think the measure should be as expressed, but wonder if a clarifying statement that a course of antibiotics that is 5 days may be excessive in viral pneumonia and a shorter than 5 days may be appropriate, particularly in cases of viral pneumonia (positive viral test and/or PCT <0.25). e.g. something to avoid the message that treatment duration MUST be 5 days.”
The full reference list can be found in Section 2.2.
Proprietary Information
Scientific Acceptability
Testing Data
We used data from 3 healthcare systems to test the measure. Each health system had different strengths to enable different validation or reliability testing:
University of Michigan (Epic)- eCQM data were pulled from hospitalizations between 9/29/2015 and 12/11/2021
- To assess measure validity, we compared eCQM data (i.e., University of Michigan Epic data) to existing chart review data from an ongoing 69 hospital collaborative quality initiative, the Michigan Hospital Medicine Safety Consortium (HMS). HMS collects pseudo-random data from hospitalized patients with community-acquired pneumonia across all 69 hospitals. Critically, they collect all the data necessary to assess this measure (including denominator/numerator inclusions/exclusions) and have used a chart-reviewed based version of the measure to improve care for hospitalized patients with pneumonia since 2015.24,68-71 We specifically used University of Michigan HMS data from a matching time period (9/29/2015-12/11/2021).
- Quality assurance data for HMS have been published previously.24,68-71 In brief, trained data abstractors collect specific data from medical records in forms with pre-specified fields (e.g., max/min allowances). Abstractors undergo random audit to ensure data quality. HMS data have been used to inform two prior NQF-endorsed quality measures (NQF 3671 and 3690).72
University of Utah (Epic)- eCQM data pulled from hospitalizations between 1/1/2021 and 5/30/2022
- To assess measure validity, we compared eCQM data (i.e., University of Utah Epic data) to chart review data obtained originally for the Inappropriate Diagnosis of Pneumonia measure (NQF 3671). Cases were manually abstracted by a physician reviewer to obtain the necessary data elements to assess antibiotic duration. Cases for abstraction were randomly selected from patients with a discharge diagnosis of pneumonia from 1/1/2021 and 5/30/2022. Electronic and chart review data were compared for key denominator/numerator variables included in both datasets.
Veterans Affairs (VA) Computerized Patient Records System (CPRS)- eCQM data were pulled from hospitalizations between January 1, 2022 and June 30, 2024 including 109 health care systems.
- These data were used to assess variation across hospitals and reliability
Notably, CPRS data have been recently used to inform a Battelle-endorsed quality measure (CBE ID: 4440e).
Technical Expert Panel (TEP): The TEP was comprised of stakeholders with diverse perspectives and areas of expertise representing a variety of professional organizations, non-profit organizations, and governmental agencies (details in Table 4, Supplemental Materials attachment). Expert feedback during measure development was provided by the TEP during a 2-hour session held on April 24, 2023.
The full reference list can be found in Section 2.2.
A summary of the data sources that were used for each type of reliability and validity testing is found in Table 7 (see Supplemental Materials attachment)
The VA health system is an integrated health system that uses CPRS, with approximately 485,500 acute hospitalizations per year across 109 facilities. The University of Utah is an academic medical center that uses an Epic EHR, with approximately 14,500 total inpatient hospitalizations per year. The University of Michigan is an academic medical center that uses a different instance of the Epic EHR with approximately 49,730 total inpatient hospitalizations per year. All three sites have interoperable EHRs where data can be transferred across outpatient and inpatient settings within the healthcare system. Detailed characteristics of measured entities are in Table 8 (see Supplemental Materials attachment)
The University of Utah, University of Michigan, and the VA healthcare system have all implemented interventions to improve antibiotic duration for CAP with published success.24,43,44,46 For example, substantial decreases in antibiotic use have been demonstrated in the VA since the establishment of the national VA Antimicrobial Stewardship Task Force in 201144 that have persisted through the COVID-19 pandemic45. Strengths of the VA antimicrobial stewardship programs include organizational directives for antimicrobial stewardship program staffing, system-wide medication use evaluations that identify opportunities for improvements in antimicrobial use,39,73,74 development of regional antimicrobial stewardship collaboratives, strong implementation science engagement and utilization of informatics-based tools43 and strong, ongoing collaboration between VA research and operational partners75. The improvements made at University of Utah, University of Michigan, and the VA healthcare system all demonstrate that these measures can be improved with antibiotic stewardship efforts.
The full reference list can be found in Section 2.2.
University of Michigan, University of Utah, and VA Healthcare eCQM data include all eligible patients during their respective timeframes. Measure-level performance testing was completed without sampling, using the full denominator populations at each healthcare system 28,238 within the VA; 313 at the University of Utah; 3,417 at the University of Michigan), at the patient encounter level of analysis. Descriptive statistics of patients included in the patient encounter level testing datasets are provided for the VA cohort, the University of Michigan cohort, and the University of Utah cohort (see Tables 9, 10, and 11 in Supplemental Materials attachment).
To test the validity of the University of Michigan population, we compared University of Michigan eCQM data to chart review data from the Michigan Hospital Medicine Safety Consortium (HMS). For HMS, abstractors screened consecutive patients via medical record review 30 days after discharge and included the first eligible patient daily, abstracting 8 eligible patients during a two-week cycle. Patients were eligible for inclusion if they were adults (≥18) admitted to general care with a billed discharge ICD-10 code of pneumonia and received antibiotics on day 1 or day 2 of hospitalization (this differs from the first 48 hours definition used in eCQM and may explain some data discrepancies). Patients who had documentation of treatment for an additional infection unrelated to pneumonia, were severely immunocompromised, were pregnant, were admitted for comfort measures, or who left against medical advice were ineligible. Quality assurance data for HMS have been published previously.24,68-71 In brief, trained data abstractors collect specific data from medical records in forms with pre-specified fields (e.g., max/min allowances). Abstractors undergo random audit to ensure data quality. HMS data have been used to inform two prior NQF-endorsed quality measures (NQF 3671 and 3690).72
The full reference list can be found in Section 2.2.
Reliability
Per guidance from NQF’s Scientific Methods Panel on requirements for eCQMs, data element reliability is “not required if data element validity is demonstrated.”77 Thus, please see our data element validity section rather than data element reliability.
Per guidance from NQF’s Scientific Methods Panel on requirements for eCQMs, “Reliance on data from structured data fields is expected; otherwise, unstructured data must be shown to be both reliable and valid. Reliability testing is not required if based on data from structured data fields.”77 Our measure uses only data from structured data fields, thus we do not report patient/encounter or data element reliability and instead report patient/encounter and data element validity.
We did conduct accountable entity-level reliability testing with two models. The first was a signal-to-noise analysis performed (within the VA dataset) using a mixed-effect logistic model run as an empty model such that the only effects in the model were the overall intercept and the hospital specific intercepts. This model enabled calculation of the hospital variance (signal), the total variance, and the residual variance (noise). The intraclass correlation was calculated from these variances. The intraclass correlation was utilized within the Spearman Brown formula in two ways: (A) to calculate the reliability for the entire hospital cohort using the median number of case abstractions for the cohort and (B) to understand minimum case abstracts necessary to achieve predetermined reliability thresholds of 0.6, 0.7, 0.8, and 0.9. All data for this analysis was from the 109 VA facilities described above. A caterpillar plot was also provided demonstrating raw metrics and shrunken estimates in this model.
We also performed an analysis of reliability using beta-binomial regression using SAS code provided in The Reliability of Provider Profiling: A Tutorial.78 VA facilities were then sorted by decile of reliability size (after exclusions)—results are shown in Table 2. The mean reliability score by beta-binomial regression was then calculated for all facilities within a decile category.
The full reference list can be found in Section 2.2.
Table 3 (see Supplemental Materials attachment) indicates the minimum annual number of qualifying cases needed for the denominator to reach each target reliability level at a given facility. In order to achieve a desired reliability of 0.8, each hospital would need to include 49 cases annually. For acceptable reliability (0.7), 19 annual cases would be required, and for high reliability (0.9), 110 annual cases would be required. The median number of eligible cases per facility within the cohort (n=49) had a reliability of 80.1%, based on data from 109 VA health care systems (2023 data). Based on our signal-to-noise analysis (using beta-binomial regression) the overall cohort reliability is >99.9%.
Based on signal-to-noise analysis, the median number of eligible cases per facility within the cohort (n=49) had a reliability of 80.1%. Based on our signal-to-noise analysis (using beta-binomial regression) the overall cohort reliability is >99.9%. Both these estimates meet the threshold for reliability for measures considered to be high stakes.
Using the current VA cohort as a representative example, the minimum number of case abstracts per hospital per year to meet pre-specified reliability thresholds of 0.7 and 0.8 are attainable for half of facilities in one year. Within the 109 VA hospitals, 84.4% of hospitals had the minimum of 19 cases in 2023 necessary to achieve 0.7 reliability. As seen in Table 2 above, only the smallest decile of reliability had a mean reliability <80%. Using the VA likely underestimates reliability of the measure, as 87.2% of VA hospitals have fewer than 200 beds.
It should also be noted that this analysis was conducted among facilities all within the same healthcare system and thus may be more similar in performance to one another than a true population distribution of all hospital types and systems within the United States.
Validity
We conducted multiple types of validity testing as summarized in Table 12 (see Supplemental Materials attachment) and described in detail below.
A. Face Validity-National Guidelines
The excess duration of antibiotic therapy in CAP measure was based on national guidelines for pneumonia, literature review including randomized clinical trials, and with addition expert feedback and review.
The CDC developed recommendations for Antibiotic Stewardship which it published in its “Core Elements of Hospital Antibiotic Stewardship Programs.” These recommendations include “Assessing how often patients are discharged on the correct antibiotics for the recommended duration.” Specifically, the CDC recommends, “most cases of uncomplicated pneumonia can be treated for 5 days when a patient has a timely clinical response.”3 The 5-day treatment duration is based on national 5-day guideline recommendations for uncomplicated CAP, multiple randomized clinical trials showing the safety of short vs. long durations,4-13 and retrospective observational studies showing higher antibiotic-associated adverse events in patients who receive excess (i.e., >5 day) antibiotic durations.
B. Face Validity-Expert Feedback
Throughout measure development, we obtained expert and stakeholder input via input from the Data, Design, and Publications Committee of the Michigan Hospital Medicine Safety Consortium (HMS) early in measure development (during chart review stage)
The Data, Design, and Publications Workgroup was an ongoing meeting of champions and experts from HMS hospitals that met to address key issues related to measure methodology, including weighing the pros and cons of measure specifications, modeling, and use (e.g., defining the measure cohort and outcome) to ensure the measure was meaningful, useful, and well-designed. The group met every 2 months during measure development and provided a forum for focused expert review and discussion of technical issues. They also provided final approval of the current submitted measure as specified.
List of DDP Workgroup Members:
- Suhasini Gudipati, MD Ascension Michigan St. Mary’s Hospital
- Tina Percha, RN, MSN Beaumont Health
- Rajiv John, MD Beaumont Health
- Lama Hsaiky, PharmD Beaumont Health
- Priscila Bercea, MPH Beaumont Health Dearborn
- Scott Kaatz, DO Henry Ford Health System
- Allison Weinmann, MD Henry Ford Health System
- Emily Nerreter, MBA Henry Ford Health System
- Danielle Osterholzer, MD Hurley Medical Center
- Lisa Dumkow PharmD Mercy Health St. Mary’s
- Anurag Malani, MD St. Joseph Mercy Ann Arbor Hospital
- Lakshmi Swaminathan, MD St. Joseph Mercy Ann Arbor Hospital
- Muhammad Nabeel, MD Sparrow Hospital
- Andrea White, PhD University of Utah Health
- Valerie Vaughn, MD, MSc University of Utah Health
- Vineet Chopra, MD, MSc University of Colorado Anschutz Medical Campus
Throughout measure development (the chart review version), we also provided opportunities from experts across the HMS collaborative to provide feedback. This included frontline clinicians, antibiotic stewards, quality improvement experts, c-suite members, and experts in quality measurement.
C/D. Assessment of Encounter-Level Validity: Comparison with Case Review
The Michigan Hospital Medicine Safety Consortium (HMS), which includes the University of Michigan, has been collecting chart-review data on patients hospitalized with CAP since 2015. We report two types of encounter-level validity using these data where we compare HMS results and our eCQM results. First, we compared the actual antibiotic duration (inpatient + outpatient) obtained via eCQM vs. via chart review. For this assessment, we report the percentage of cases that fell within ½ day agreement and the percentage that fell within 1.5 day agreement. Next, we assessed the sensitivity and specificity for the eCQM in identifying patients with an excess antibiotic duration (i.e., antibiotic duration of ≥ 7 days).
E. Assessment of Encounter-Level Validity: Critical data element validity
To assess critical data element validity, initial denominator exclusions categorized as: a) to identify hospitalized non-ICU adults with CAP, b) to narrow the population to uncomplicated CAP, and c) specific to the excess antibiotic duration measure were compared to assess consistency across the University of Michigan, University of Utah, and VA healthcare system.
We also compared inclusion criteria, exclusion criteria, and critical data elements between HMS chart review data and eCQM data for the University of Michigan. Due to the way HMS conducts chart review, not all inclusion/exclusion criteria are collected. We report sensitivity and specificity of the eCQM assessment compared to chart review which we considered the gold standard.
F. Face Validity: National Technical Expert Panel (TEP) Feedback (N=8 experts)
Throughout measure development, we obtained expert and stakeholder input. In alignment with the CMS Measures Management System guidance on TEP,79 we held a remote TEP session to provide input and feedback from a group of recognized experts in relevant fields. To convene the TEP, we reached out to organizations whose members could potentially be impacted by the measure and asked them to nominate individuals for participation. We selected individuals to represent a range of perspectives, including Infectious Diseases physicians, pharmacists, pulmonologists, radiologists, hospitalists, emergency medicine physicians, regulatory agencies, as well as individuals with experience in quality improvement, performance measurement, diagnostic error, antibiotic stewardship, and health care quality (TEP members are listed in Table 4 in Supplemental Materials attachment).
The TEP was convened on April 24, 2023 over a 2-hour period. In preparation for the meeting, each TEP participant received an agenda and background document providing project context and data supporting the excess antibiotic duration for adult hospitalized patients with uncomplicated community-acquired pneumonia measure. The format for the TEP meeting (conducted remotely) included presentation of measure development and proposed specifications and relevant data, with open discussion among TEP members throughout. Participants were encouraged to provide expert feedback regarding the validity, feasibility, and usability of the measure during the sessions and through online questionnaires following each session.
Following the meeting, all participants completed an online survey that included questions related to validity, reliability, usability, etc. Related to measure validity, we asked TEP members:
- Please rate the following statement: “The measure as specified can be used to distinguish between better and worse quality hospitals.” 1=Strongly disagree, 2=Disagree, 3=Neutral, 4=Agree, 5=Strongly agree.
- Are there any key data elements you believe are missing or not accurately captured in excess antibiotic duration for CAP measure?
G. Empirical Validity: Evaluated association with other measures of CAP antibiotic treatment quality
To assess empirical validity for the excess duration of antibiotic therapy in CAP measure, we identified and assessed the measure’s correlation with other measures that target antibiotic quality for CAP. The goal was to identify if better performance on this measure was related to better performance on other relevant measures. Specifically, we assessed the association (at the hospital level) of excess antibiotic duration for CAP with overuse of broad-spectrum empiric antibiotics for CAP.
H. Empirical Validity: Evaluated association of excess antibiotic duration for CAP with outcomes
We also assessed the association of excess antibiotic duration for CAP with 30-day patient outcomes and antibiotic-associated adverse events. This work and the associated methods have been previously published.15 In brief, we evaluated the association of each day of excess antibiotic therapy with patient outcomes at 30-days. Specifically, we were interested in the effect of each day of excess antibiotic use on patient-reported antibiotic-associated adverse events (obtained through 30-day phone calls). We used logit generalized estimating equation models adjusted for patient characteristics and probability of treatment to assess patient outcomes associated with each day of excess antibiotic use.
I. Predictive Validity: Evaluated whether improvement in excess duration results in improved patient outcomes.
The most critical validity test for any measure is whether measure improvement leads to improved outcome. Previously, we reported results from HMS improvement activities between February 23, 2017 and February 5, 2020 (HMS improvement activities continue through present but have not been published). Over that time period, HMS targeted appropriate 5-day antibiotic treatment through benchmarking, sharing best practices, and pay-for-performance incentives based on the chart-review version of the eCQM described here. Changes in outcomes, including appropriate receipt of 5 ± 1–day antibiotic treatment and 30-day post discharge composite adverse events (i.e., deaths, readmissions, urgent visits, and antibiotic-associated adverse events), were assessed over time (per 3-month quarter), using logistic regression and controlling for hospital clustering.24
The full reference list can be found in Section 2.2.
C. Encounter-level Validity: Comparison with Case Review
After exclusions, 450 patients were included in both University of Michigan (UM) Michigan Hospital Medicine Safety Consortium (HMS) and UM eCQM data. It is important to note that HMS does not collect ½ day data—so the eCQM data are more accurate in terms of antibiotic duration. To account for the differential feasibility of counting ½ days in antibiotic prescriptions, we elected to define excess duration as patients receiving ≥7 days rather than >5 days—providing a grace period of 1.5 days for an error in counting. Considering UM HMS data the gold standard, 91.3% (411/450) of patients had a total antibiotic duration (including inpatient and discharge data) that fell within a half day of the eCQM total duration, and 96.4% (434/450) were accurate within 1.5 days (Table 13 in Validity Results attachment). For inpatient duration, 439/454 (96.7%) of patients had an inpatient antibiotic duration that fell within a half day of the eCQM inpatient duration, and 447/454 (98.5%) were accurate within 1.5 days (Table 14 in Validity Results attachment). Finally, for discharge duration, 427/458 (93.2%) of patients had a discharge antibiotic duration that fell within a half day of the eCQM discharge duration, and 435/458 (95.0%) were accurate within 1.5 days (Table 15 in Validity Results attachment). Notably, inpatient duration was slightly more accurate than discharge duration where occasionally discharge durations are written as free text or not otherwise accurate as discrete data.
D. Encounter-level Validity: Comparison with Case Review
Looking at the same 450 patients with both UM HMS chart review and University of Michigan eCQM data, we next compared the percentage with excess antibiotic duration (i.e., ≥7-day total antibiotic duration) of those considered eligible for a 5-day duration using chart review as the “gold standard” to obtain the sensitivity and specificity for the eCQM. The eCQM has a sensitivity of 96% and a specificity of 93% in assessing excess antibiotic duration (see Table 16, Validity Results attachment).
E. Encounter-level Validity: Critical data element validity
For each measured entity, initial denominator exclusions: a) to identify hospitalized non-ICU adults with CAP, b) to narrow the population to uncomplicated CAP, and c) specific to the appropriateness of empiric antibiotics measure were compared to assess consistency across entities (Tables 17, 18, and 19, respectively, see Validity Results attachment).
Prior to applying exclusions, there were 592 patients hospitalized with CAP who had data in both the UM HMS chart review dataset and the UM eCQM data set. For duration-specific exclusions (e.g., died during hospitalization, transferred to ICU, etc), we determined the sensitivity and specificity of UM eCQM relative to UM HMS chart review. For 11/12 exclusions, sensitivity ranged from 88%-100%, with 7/12 exclusions with sensitivity of 100%. One exclusion, “time to clinical stability >5 days” had a sensitivity of 29%. We discussed this exclusion in the feasibility section. For all duration-specific exclusions, specificity was either 99% or 100% (for details, see Table 20 in Validity Results attachment).
F. Face Validity: National Expert Panel Feedback
The eight national experts who attended our TEP agreed with the face validity and operationalization of the measure. They believed that patients we identified as having excess duration did, in fact, have excess duration.
TEP Survey results:
In response to the question: “How much do you agree with the following statement? The measure, as specified can be used to distinguish between better and worse performing hospitals,” the TEP were unanimously in agreement, with 2/8 (25%) reporting “Strongly agree”, and 6/8 (75%) reporting “Agree” (on a scale from 1 “Strongly disagree” to 5 “Strongly agree).”
We also asked the TEP: “Are there any key data elements you believe are missed or not appropriately used in the determination of the appropriate duration of therapy for CAP measure?” Of the 3 responses to this free text question, two responded “no”, with one adding “I think we tried to shorten them in the meeting,” supporting face validity of the measure.
The third response was:
“The clinical stability definition is computationally and operationally complex and I am concerned that the initial pilot results will not play out in a larger population with disparate EHRs- particularly due to outlier vital signs values (elevated HR on ambulation) and data completeness/timeliness of supplemental oxygen documentation, particularly when oxygen orders are PRN.” In response, we simplified the definition of clinical stability for the measure (see discussion in feasibility section).
Two TEP members responded to the free text question: “Please use the space below to provide any additional comments or suggestions related to the proposed measure of the duration of therapy for CAP.” The first response supports face validity of the measure:
“This is a great attempt to accomplish decrease [days of therapy] for pneumonia. Compressing to make it extractable and provide good data is difficult. Certainly, HMS has been able to show it works.”
The second comment was: “Computationally, calculating clinical stability could be resource intensive for EHR vendors/data intermediaries, due to the large volume of vitals data and complex logic to connect values. Suggest seeking vendor feedback as part of the feasibility evaluation for this data element,” In response, we simplified the definition of clinical stability for the measure (see discussion in feasibility section).
G. Empirical Validity: association with other measures of CAP antibiotic treatment quality
To assess empirical validity for the excess antibiotic duration in CAP measure, we assessed the association (at the hospital level) of excess antibiotic duration for CAP with overuse of broad-spectrum empiric antibiotics for CAP with the 109 VA hospitals. We found that the two measures were weakly correlated at the hospital level (R=0.3, p=0.0014, see Figure 3 in Validity Results attachment).
H. Empirical Validity: Evaluated association of excess antibiotic duration for CAP with outcomes
The association of excess antibiotic duration for CAP with 30-day patient outcomes and antibiotic-associated adverse events have been previously published.15 Results show that for each day of excess antibiotic use, 30-daypatient-reported antibiotic-associated adverse events were more likely (aOR for each excess day 1.03 (1.00-1.06). Details are in Table 21 in Validity Results attachment.
I. Predictive Validity: Evaluated whether improvement in excess duration results in improved patient outcomes.
Between February 23, 2017 and February 5, 2020 a total of 41 HMS hospitals and 6553 patients were included. When the chart-review form of our proposed eCQM was used as a pay-for-performance measure, the percentage of patients treated with an appropriate 5 ± 1–day duration increased from 22.1% (predicted probability, 20.9% [95% confidence interval: 17.2%–25.0%]) to 45.9% (predicted probability, 43.9% [36.8%–51.2%]; adjusted odds ratio [aOR] per quarter, 1.10 [1.07–1.14]). Thirty-day composite adverse events occurred in 18.5% of patients (1166 of 6319) and decreased over time (aOR per quarter, 0.98 [95% confidence interval: .96–.99]) owing to a decrease in antibiotic-associated adverse events (aOR per quarter, 0.91 [.87–.95]).24 No outcomes worsened during this time period.
We assessed multiple types and levels of validity during and after measure development. First, the measure has high face validity as it is based on national guidelines and recommendations from national organizations (e.g., CDC) and was approved by national experts representing 8 national organizations. Compared to 450 case reviews (using HMS data at the University of Michigan), the measure had high sensitivity and specificity at the patient level for identifying excess antibiotic duration. We noted during assessment that antibiotic durations could be off by a ½ day due to variation in how inpatient + outpatient was added (e.g., a patient given antibiotics inpatient on the day of discharge and prescribed antibiotics to start that evening could be miscounted). Due to this variation, we added a grace period of 1.5 days to the measure (i.e., each patient in the denominator should only receive 5 days total duration—however, we only consider it an excess duration if they receive ≥ 7 total days of antibiotic therapy). Duration-specific exclusions also had high sensitivity and specificity when compared to 450 HMS chart review cases. The exception is the exclusion “time to clinical stability >5 days”—here, at the suggestion of our TEP, we used a less sensitive definition to improve feasibility. This change improved feasibility without sacrificing overall measure accuracy as hospital level performance did not change with the different definitions (see feasibility section for details).
Across hospitals, we found performance on the excess duration measure to be weakly correlated with performance on the inappropriate broad spectrum antibiotic measure, supporting empirical validity. Finally, the chart-review version of the measure on which this eCQM was based has been shown to be associated with patient outcomes and, critically, improvement in the measure is associated with an improvement in antibiotic-associated adverse events.
Together, these results suggest that the excess antibiotic duration measure is highly valid and, most importantly, that hospitals can improve patient outcomes by improving their performance on the measure.
Risk Adjustment
Use & Usability
Use
Acute care hospitals
Usability
Acute care hospitals are now required to have antimicrobial stewardship teams to optimize antibiotic use in hospitalized patients. There are multiple published methods to improve antibiotic duration for CAP including: a) ordersets to standardize care, b) audit and feedback by pharmacists, c) nudging or behavioral interventions to reduce duration, d) creation of guidelines and education, and e) discharge-focused interventions.88,89
Each strategy has its own barriers and feasibility varies by institution. For example, smaller hospitals have an easier time implementing audit and feedback as their patient population is small and pharmacists may have fewer patients to cover. In contrast, larger institutions may have more information technology infrastructure to help construct ordersets or nudges using the EHR.
What we do know is that diverse hospitals are able to improve on a chart-review based form of this measure. This work has been previously published but is summarized here.24 Notably the publication is for the first 41 hospitals and includes data from 2017-2020. The consortium has continued to use antibiotic duration as a pay-for-performance measure (through at least end of 2025) and now includes 69 hospitals.
The aim of the antimicrobial use initiative in the Michigan Hospital Medicine Safety Consortium is to formally measure and improve the appropriate use of antibiotics including selection of the right antibiotic for the right clinical condition for the right duration. HMS seeks to decrease antibiotic-related complications and decrease antimicrobial resistance on a population health basis by:
- ensuring that patients with uncomplicated community-acquired pneumonia (CAP) do not receive more than five days of antibiotics.
- reducing the use of inappropriate empiric broad-spectrum antibiotics for patients with uncomplicated CAP.
- reducing the testing and treatment of asymptomatic bacteriuria (ASB).
- reducing antibiotic treatment of questionable pneumonia.
In short, when implemented as a pay-for-performance measure within the Michigan Hospital Medicine Safety consortium, hospitals were able to reduce antibiotic duration for CAP.24 Over the study period, the proportion of patients hospitalized with CAP who were treated with a 5-day antibiotic duration increased from 22.1% (predicted probability adjusted for hospital clustering, 20.9% [95% CI, 17.2%–25.0%]) to 45.9% (43.9% [36.8%–51.2%]; P < .001). Each quarter was associated with higher odds of receiving an appropriate 5-day antibiotic duration (aOR per quarter, 1.10 [95% CI, 1.07–1.14]), for an annual aOR of 1.49 (1.33–1.66). Improvement in appropriate antibiotic duration over time was driven by a decrease in antibiotic duration at discharge (median [interquartile range] discharge duration, 5 [3–7] days in 2017 and 3 [2–5] days in 2020; P < .001).
Nearly all hospitals saw improvement including small hospitals, rural hospitals, and large academic hospitals.24 There was evidence of variation in adoption; hospitals with low baseline rates had, on average, larger increases in rates of appropriate treatment. The estimate of the variance of the random slopes across hospitals was 0.005 on the log odds scale (95% CI: .001–.008), which implies a wide variation in rates of change with 95% of hospitals falling into a range of 0.96–1.27 for OR per quarter. However, the hospital characteristics we evaluated (e.g., bed size, rurality) did not explain the variation in slope of improvement across hospitals
As they reduced duration, hospitals also improved 30-day patient outcomes (driven by a reduction in antibiotic-associated adverse events).24 After adjustments, composite adverse events decreased over time (aOR 0.98 [95% CI: .96–.99]). This decrease appeared to be driven by a decrease in antibiotic-associated adverse events over time (physician-reported aOR per quarter, 0.93 [95% CI: .87–.99]; patient-reported aOR per quarter 0.89 [.84–.95]). The occurrence of deaths, readmissions, urgent visits, and CDI did not change over time.
The full reference list can be found in Section 2.2.
Tri-annual Collaborative Wide Meetings
Individuals from participating hospitals meet in person three times a year. We encourage hospitals to send their Clinical Data Abstractors, physician champions, and quality leads, as well as other individuals from their hospital that might be interested in participation. These meetings take place three times per year – in March, July, and November. Traditionally, meetings took place in-person at venues across Michigan. In 2020 and 2021, these meetings were hosted via an on-line format due to COVID-19.
The tri-annual meetings provide individuals from member hospitals with the opportunity to engage with each other in a variety of formats. Each meeting includes a formal discussion of the data from each of the HMS initiatives—including data on antibiotic duration for CAP—for the previous quarter, presentations from member hospitals and expert guests, breakout/work group sessions, and networking opportunities. These meetings allow individuals from member hospitals to network with individuals from other hospitals who have excelled in those areas to seek ideas on how to improve their performance. It also allows for an opportunity for feedback and to answer questions related to their performance.
Site-specific Reports on Measure Performance
Tri-annually, each participating hospital receives a printed and email version of a site-specific data report. These reports are also available daily within the database/registry (see below). These reports provide an in-depth look into the performance of each site. For example, we provide hospital data on the number of patients with CAP treated with an excess duration, details on antibiotic use and outcomes (e.g., adverse events), longitudinal performance, and data on how individual hospitals compare to other hospitals in the state. Hospitals also receive a list of all patients who were considered treated with an “excess antibiotic duration” to enable them to return to their hospital and conduct case reviews of those patients. Each hospital is encouraged to review these cases with their local team to perform audit and feedback, identify trends, and assist with overall quality improvement. This also provides an opportunity for measure feedback—for example, hospitals might find an error in case classification. Early during measure development this case-specific feedback was critical for improving measure validity.
Live Database Reports
Each of the HMS databases are equipped with the ability to view live reports utilizing Business Objects software. These reports provide updated data every 24 hours regarding measures (site performance and collaborative performance), fallout case information, demographics, critical/non-critical data errors, completeness of abstracted cases, and case classification information.
Individuals who participate in the collaborative either as a Clinical Data Abstractor or a quality administrator have the ability to log into the HMS databases and view these reports at their leisure. The software that HMS utilizes also allows for these reports to be exported as Excel files or PDFs for hospital-specific customization. This information is often utilized by participating hospitals at committee meetings or for presentations to track progress and inform quality improvement efforts. They also assist the Clinical Data Abstractor to identify errors in their abstraction and resolve them in real time. These reports also allow hospitals to review individual fallout cases and their clinical scenarios to inform individual clinicians or groups of clinicians of their performance and provide targeted education.
Throughout measure development, we received feedback on the measure performance/validity through the above mechanisms as well as: 1) Expert Feedback from Data Design and Publications Committee and Michigan Hospital Medicine Safety (HMS) Consortium Hospital Experts/Representatives and 2) “Fall-out” Feedback. Briefly, measure performance feedback allowed us to refine the measures to the current version. The Data, Design, and Publications Committee approved the measures for use across HMS. There have been no unanticipated outcomes/ill effects.
Using the chart-review based form of this measure as a pay-for-performance measure within the Michigan Hospital Medicine Safety consortium, hospitals were able to reduce duration.24 Over the study period, the proportion of patients treated with a 5-day antibiotic duration increased from 22.1% (predicted probability adjusted for hospital clustering, 20.9% [95% CI, 17.2%–25.0%]) to 45.9% (43.9% [36.8%–51.2%]; P < .001). Each quarter was associated with higher odds of receiving an appropriate 5-day antibiotic duration (aOR per quarter, 1.10 [95% CI, 1.07–1.14]), for an annual aOR of 1.49 (1.33–1.66). As they reduced duration, hospitals also improved 30-day patient outcomes (driven by a reduction in antibiotic-associated adverse events).24 After adjustments, composite adverse events decreased over time (aOR 0.98 [95% CI: .96–.99]). This decrease appeared to be driven by a decrease in antibiotic-associated adverse events over time (physician-reported aOR per quarter, 0.93 [95% CI: .87–.99]; patient-reported aOR per quarter 0.89 [.84–.95]). The occurrence of deaths, readmissions, urgent visits, and CDI did not change over time.
The full reference list can be found in Section 2.2.
There were no unexpected findings during the 7 years that this measure has been a pay-for-performance measure.
Comments
Staff Preliminary Assessment
4540e Staff Assessment
Importance
Strengths:
- Logic Model: A clear logic model is provided, depicting the relationships between inputs (e.g., ATS/IDSA Guidelines, EHR Systems), activities (e.g., increased attention to duration of therapy per guidelines, patient care, antibiotic stewardship teams working to improve prescribing), and desired outcomes (e.g., improved adherence to ATS/IDSA Guidelines, reduced inappropriate antibiotic exposure, improved patient care and safety). This model demonstrates how the measure's implementation will lead to the anticipated outcomes.
Performance Gap: Data from VA Healthcare Systems (January 1, 2022-June 30, 2024) shows a performance gap, with decile ranges from 20.5% to 65.6%, indicating variation in measure performance. - Existing Measures: The proposed measure addresses a healthcare need not sufficiently covered by existing measures. No eCQMs specifically for CAP currently exist. The developer collaborated with the CDC to design this measure with the goal of working within their NHSN AU framework.
- Literature Review: The measure is supported by a comprehensive literature review, including citing the Infection Disease Society of America (IDSA) guidelines for Implementing an Antibiotic Stewardship program which provide a strong recommendation (based on moderate quality evidence that “Antimicrobial stewardship programs (ASPs) implement guidelines and strategies to reduce antibiotic therapy to the shortest effective duration" and specifically call out CAP and CAP duration of therapy as an important target.
Limitations:
- Patient Input: Meaningfulness to patients was partially assessed. The information available in the literature was limited and the TEP only included clinicians.
Rationale:
- This new measure is supported by a clear logic model linking inputs like ATS/IDSA Guidelines and EHR Systems to activities and outcomes such as improved guideline adherence and patient safety, addresses a gap in existing measures for CAP, as evidenced by variable performance data and a comprehensive literature review. However, the rating is "Not Met but Addressable" as the information about meaningfulness to patients is limited to evidence from guidelines and a TEP that included only clinicians.
Feasibility Acceptance
Strengths:
- Feasibility Assessment: A comprehensive feasibility assessment has been conducted, demonstrating that all necessary people, tools, tasks, and technologies have been considered for the implementation of this measure. This includes a detailed analysis of the resources required and the operational steps needed to integrate the measure into current healthcare practices.
- A completed eCQM Feasibility Scorecard is attached, showing the measure was tested in more than three EHRs (Epic- University of Utah, CPRS- Veterans Affairs, Epic- University of Michigan). Most data required for this measure are readily available in a structured format, residing in fixed fields within the EHR systems tested. Lastly, the data are captured during the course of care, ensuring that the measure does not require additional steps or disrupt normal clinical workflows.
- Adjustments Based on Feasibility: The feasibility assessment informed the final measure design, indicating decisions made to adjust the measure in response to data availability and other practical considerations.
Limitations:
- The feasibility plan in the feasibility scorecard notes that "transfer from other hospital" is not currently tracked in a structured field in some EHRs. However, they plan to continue to assess the feasibility of search methods where the element is not encoded and evaluate improvements.
Rationale:
- This new measure meets all criteria for 'Met' due to its well-documented feasibility assessment, clear and implementable data collection strategy, ensuring practical implementation within the healthcare system.
The eCQM Feasibility Scorecard results indicate that the measure is well-supported by the current capabilities of the tested EHR systems in terms of data availability, accuracy, adherence to standards, and workflow integration. These factors collectively ensure that the measure can be implemented effectively and sustainably in a real-world healthcare setting.
Scientific Acceptability
Strengths:
- Data Sources and Dates: Data used for testing were sourced from EHRs during the period January 1, 2022 to June 30, 2024. These data are all collected routinely during usual clinical care through the process of inpatient hospitalizations. The 109 hospitals included in the analysis were characterized by being in the VA healthcare system which spans urban and rural areas.
- Accountable Entity Level Reliability: The developer conducted signal-to-noise reliability testing at the accountable entity-level. More than 90% of accountable entities meet the expected threshold of 0.6 with a minimum reliability of 0.302.
Limitations:
- None.
Rationale:
- The results demonstrate sufficient reliability at the accountable entity level.
Strengths:
- The developer assessed face validity, encounter-level validity, and accountable entity-level validity. For face validity, a TEP (N=8) demonstrated consensus with the statement "The measure as specified can be used to distinguish between better and worse quality hospitals." For encounter-level validity, based on 450 case reviews (using HMS data at the University of Michigan), the measure had high sensitivity and specificity for identifying excess antibiotic duration. For accountable-entity validity, the developer presented one association study (a weak correlation with inappropriate broad spectrum antibiotic measure) and one "modeled" association study (a quasi-experimental analysis demonstrating improvement over time).
Limitations:
- For face validity, the main limitation may be the size and composition of the TEP (N=8) which lessens the persuasive impact of the consensus statement. For encounter-level validity, the main limitation is the use of only one EHR vendor and the use of HMS data limited to the University of Michigan, which may have limited external validity for less resource intensive environments (some of the comments related to "time to clinical stability" reflect that concern). For accountable-entity validity, the main limitation is the lack of mechanism studies on the features of the mechanism complex (i.e. what worse performing entities do not do that better performing entities do). The "modeled" association study (change over time) does support the rule out non-causal connections and confounding.
Rationale:
- The validity testing results support a relatively strong inference of validity for the measure, confirming that the measure accurately reflects performance on quality or resource use and can distinguish good from poor performance. However, additional data element testing within at least two EHR vendors is needed.
Equity
Strengths:
- The developer referred to literature on health inequities, and reported that no differences by patient race, age, gender have been found related to inappropriate broad spectrum antibiotic therapy for community-acquired pneumonia.
Limitations:
- However, this content alone does not address the equity criterion. The developer should consider empirically testing for differences in performance scores across multiple sociocontextual variables (e.g., race, ethnicity, urbanicity/rurality, socioeconomic status (SES), gender, gender identity, sexual orientation, age).
Rationale:
- Though the developer provided some information on inequities, no empirical testing was completed.
Use and Usability
Strengths:
- Current Use: The measure is currently used in the Michigan Hospital Medicine Safety Consortium.
Performance Trends (improvement reported): The developer reports changes in performance from 2017 – 2020 using the chart-review based form of this measure, in which the overall performance increased. - Feedback Mechanism: Feedback is collected during tri-annual collaborative-wide meetings, and site-specific data reports allow hospitals to report any errors in classifications, or other abstraction errors. They also get expert feedback from Data Design and Publications Committee and Michigan Hospital Medicine Safety Consortium Hospital Experts/ representatives.
Limitations:
- Measure Updates: The developer could provide more detail on whether any feedback received led to changes in the measure specifications.
- Performance Trends (improvement reported): Although improvement data was reported, it was only for the chart-based version, not this eCQM.
Rationale:
- This new measure is currently used in at least one accountability application, and the measure provides actionable information for improvement. The developer describes approaches to collecting feedback on this measure, but more detail is needed on how that feedback led to changes in the measure specifications, especially for maintenance endorsement review. Although performance results for the chart-based version of the measure showed positive results, more data on performance trends for the eCQM is needed. The developer reports no unexpected findings.
Committee Independent Review
The developers address a…
Importance
Considering the gaps in current practice, the measure has solid merit for its uses. However, I have two concerns that the developers could address:
- Exclusion of patient voices and;
- Lack of stratification of patient population. Since CAP is a serious risk for pregnant women, it would have been ideal to access the measures performance and clinical outcomes among these patients.
Feasibility Acceptance
Comprehensive feasibility assessment and scorecard.
Scientific Acceptability
Using EHRs with a large sample from multiple health systems strengthens the scientific reliability of this measure.
However, the universal exclusion of pregnant women may limit the reliability of this measure.
I applaud the developer's effort in creating a workgroup of experts in developing the measures. However, the workgroup included only clinicians, not patients.
Also, the sample size (N=8) could have been more and included patients.
Equity
The measure did not meet equity standards. Even though the developers noted that CAP is prevalent among priority populations, including people from low socioeconomic status and other at-risk patients, these were not considered in the measure.
Additionally, as referenced elsewhere, the literature suggests that CAPS is a debilitating condition during pregnancy; the developers do not mention or address this population in their measure.
Use and Usability
The data presented suggests usability.
Summary
The developers address a critical gap. While important, the viewpoints of patients, particularly women, are missing from the development and clinical outcomes of the measures. As a result, their measure may not be generalized across all "adult" populations.
See comments above
Importance
Agree that direct patient input on the importance of the measure is needed.
Feasibility Acceptance
Agree with staff assessment.
Scientific Acceptability
Agree with staff assessment.
The staff assessment noted that data element validity testing should be conducted in 2+ EHRs. I don't see that in the guidebook.
Also, the staff assessment noted concerns about the TEP only including 8 members. Is there a minimum needed for the TEP? I didn't see that addressed in the guidebook.
Equity
Optional.
Use and Usability
Agree with staff assessment.
Summary
See above.
Small number of concerns that can be addressed in future
Importance
Agree with staff assessment that the patient perspective was not as well established as other perspectives on importance. However, the wealth of evidence from other perspectives and the comment from PFPS encouraging endorsement of the measure minimizes this concern for me. I think if the developer can build some evidence from patients ahead of next endorsement cycle, that would be beneficial.
Feasibility Acceptance
Agree with staff assessment. One public comment says that "It is uncommon for hospitals to have antibiotic stewardship programs that extend into post-discharge prescribing." Developer may want to explore feasibility in more diverse settings in the future, but I believe the criteria are met for endorsement now.
Scientific Acceptability
no concerns. Good evidence and explanations.
I have seen TEPs consisting of 8 individuals before so am not as concerned about this level of validity evidence, especially in light of empirical evidence that shows strong validity.
Equity
Empirical assessment was not done. Encourage developer to do empirical/significance testing in the future. OK for now as this is optional at this endorsement cycle.
Use and Usability
no concerns. For purposes of testing in more EHRs in future, will require use in other systems in the future.
Summary
Mostly agreed with staff assessment except I have less concern over the 8 TEP members in validity testing as this is not particularly unusual and the empirical evidence is strong. Agree with other reviewers and public commenters that testing in more EHRs in the future would be essential. See other fields for more detailed comments.
Impressed with performance improvements in implementation
Importance
Although evidence from the literature is limited, the two studies provided (62 and 63 in the reference list) make the case that patients are concerned about antibiotic overutilization, as well as the demonstrable relationship to clinical outcomes of importance to patients, particularly older adults. Two small-scale studies I found suggests an opportunity for patient engagement on the subject (https://pubmed.ncbi.nlm.nih.gov/26809477/; https://academic.oup.com/jacamr/article/2/3/dlaa046/5879268). As well, I note that PFPS US has commented in strong support of the measure.
Feasibility Acceptance
Concur with staff assessment; feasibility assessment has been completed.
Scientific Acceptability
Concur with staff assessment.
In this case, given the limited number of data elements in use and the strong confidence in feasibility, I don’t foresee any issues with validity when applied in a different EHR.
I would emphasize that it is regrettable that patients were not included in the TEP, but it is not sufficient to disqualify the measure’s validity.
Equity
Concur with staff assessment; it seems reasonable to test for the influence of socioeconomic factors.
Use and Usability
Concur with staff assessment. The measure is in use in an accountability application.
Summary
Implementation of this eCQM has shown a substantial and rapid performance improvement – extrapolating outwards to a national implementation is an exciting prospect for public health. Progressive year over year improvement in results in the implemented setting shows the measure has promise over the long term.
One key limitation is the inclusion of the patients’ perspective in the development of the measure, both in the research literature and as members of the expert panel. Patients might reasonably react to a change in the standard of care with concern about a cost savings at the expense of their health – although that isn’t the case here, and limited research suggests that patients are prepared to engage on the issue.
satisfactory
Importance
If patient input is required then there should be a description of patient input, which seems to be missing. However, it would be quite a surprise to discover that patients don't support
Feasibility Acceptance
Agree with preliminary assessment
Scientific Acceptability
agree with staff
Of course doing more testing with other EHRs and a larger TEP would be more compelling, but the information presented provides the impression of validity
Equity
agree with preliminary assessment
Use and Usability
agree with staff assessment
Summary
In totality, the information that has been submitted seems adequate to support endorsement with few caveats
CBE #4540e
Importance
Pneumonia is most common reason for IP Abx, and most common infectious mortality.
Feasibility Acceptance
Agree with Staff Preliminary Assessment.
Scientific Acceptability
Agree with Staff Preliminary Assessment.
Agree with Staff Preliminary Assessment.
Equity
Agree with Staff Preliminary Assessment.
Use and Usability
Agree with Staff Preliminary Assessment.
Summary
Support this measure. This is a great attempt to accomplish decreased days of antibiotic therapy for CAP. By making this an eCQM, the developers have created a simpler solution to capturing data that will drive lower length antibiotic days and decrease complications and side effects of protracted antibiotic therapy. They also have tackled a significantly "big dog" in the realm of antibiotic use and mortality. Improving outcomes around CAP has wide-reaching breadth to impact thousands of people yearly.
Public Comments
PDQM CBE 4540e : Excess Antibiotic Duration for Adult CAP
CDC supports measure 4540e
The Centers for Disease Control and Prevention (CDC) is strongly supportive of measures PDQM CBE 4540e and 4545e addressing the agents selection and duration of therapy for uncomplicated community acquired pneumonia. Over the past decade, the United States has made major advances in efforts to improve antibiotic use in hospitals as part of efforts to combat the public health crisis of antimicrobial resistance. Nearly all hospitals now have dedicated programs to improve antibiotic use (antibiotic stewardship programs), as required by the Centers for Medicare and Medicaid Services (CMS). The vast majority also now have access to risk adjusted benchmarks of the quantity of their antibiotic use through the CDC’s National Healthcare Safety Network Antibiotic Use Option. This reporting is also now required by CMS.
However, efforts to improve hospital antibiotic use remain challenged by an inability to quickly and easily assess the quality of antibiotic prescribing. Improving the quality of antibiotic use is the ultimate goal of antibiotic stewardship and we need measures that will help us target this goal directly. Studies have consistently shown that assessing the guideline concordance of antibiotic prescribing for common infections consistently leads to better use.
These measures represent a critical 1st step into a new era for improving hospital antibiotic use. They are targeting the single most common reason that antibiotics are prescribed in hospitals. In many hospitals, 50% or more of all antibiotic prescribing is for pneumonia. And because they are electronic, they will allow antibiotic stewardship programs to assess the quality of prescribing without having to conduct labor intensive chart reviews. As you can see from the data in the application, use of the manual versions of these measures led to significant improvements in prescribing.
CDC is currently working towards a pathway for hospitals to be able to report patient level antibiotic use data into the National Healthcare Safety Network through Fast Healthcare Interoperability Resources (FHIR). Patient level data will allow CDC to begin analyzing and reporting on antibiotic use quality in addition to quantity. To do that, we will, of course, need validated measures, like PDQM CBE 4540e and 4545e. CDC urges the panel to endorse these measures.
Comment on measure
PQM CBE 4540e: Excess Antibiotic Duration for Adult Hospitalized Patients with Uncomplicated Community-Acquired Pneumonia
The quality measure for reducing excess antibiotic duration in patients hospitalized with uncomplicated community-acquired pneumonia (CAP) is crucial for several reasons.
Impact to Patient Outcomes:
Feasibility of Measure Data Collection
Alignment with Other Quality Initiatives
Overall, implementing this quality measure is both impactful, aligned and feasible and can lead to significant improvements in patient outcomes and public health by promoting responsible antibiotic use and reducing the burden of antibiotic resistance.
Comments on CBE #4540e
The Society of Hospital Medicine (SHM) encourages the E&M Committee to consider the impact of the measure and potential unintended consequences or pressures created by the measure. It is uncommon for hospitals to have antibiotic stewardship programs that extend into post-discharge prescribing. This may be challenging to implement particularly for smaller, rural or less resourced hospitals. While SHM broadly supports the clinical concept of this measure, we also question whether it is best positioned as a hospital-level measure (as currently structured) or whether it may be more actionable as a provider-level measure.
SIDP Support For CBE 4540e
SIDP supports PQM endorsement of CBE 4540e to address excess antibiotic duration in the treatment of hospitalized patients with community acquired pneumonia. This measure would impact institutions of all sizes, as pneumonia is among the most common infectious indications for hospitalization. The risk of developing antimicrobial resistance increases by 7% with every additional day of antibiotic exposure. Increasing accountability for reasonable durations of therapy will decrease antimicrobial utilization and likely result in reduced rates of adverse drugs reactions such as C. difficile and rates of antimicrobial resistance. There are over 14 randomized clinical trials that report shorter courses of treatment for CAP are both safe and effective. Manual review of duration of therapy is time intensive and not feasible to measure for ongoing quality assurance and compliance to institutional guidelines. An electronic measure will allow antimicrobial stewardship programs to more efficiently and routinely track duration of therapy and target interventions accordingly.
DCASIP Supports 4540e
Electronic measures to better assess antimicrobial stewardship interventions are sorely needed. The proposed measures CBE 4540e and 4545e, which aim to electronically assess empiric antibiotic selection and treatment duration for adult patients hospitalized with uncomplicated community-acquired pneumonia (CAP), show significant progress in development of electronic clinical quality measures (eCQMs). These metrics are built on evidence-based practices to improve antibiotic durations for a leading indication for antibiotic use in US hospitals.
Successful programs to optimize treatment durations for uncomplicated CAP, such as those described by the Michigan Hospital Medicine Safety Consortium have been hard to replicate in other quality measurement programs because they relied upon labor-intensive manual data collection processes. Laborious manual data capture can detract from the important multifaceted efforts of antimicrobial stewardship programs (ASPs). The eCQMs developed and tested by the groups in Michigan and Utah have shown it is possible to capture similar data electronically in 3 systems: two health systems with Epic and the VA system. Endorsing these measures will allow more widespread development and testing of the feasibility of electronic capture of these data in more varied electronic health record (EHR) data systems. In our experience with electronic data reporting from >70 hospitals in the SE USA, we know how burdensome data collection requirements can be, especially for hospitals with minimal resources or IT support. Thus, we emphasize that it will be essential to engage EHR vendors as key partners in further deployment and implementation of these eCQMs as an important next phase of this work which will greatly benefit stewards to help improve quality of patient care.
The Duke Center for Antimicrobial Stewardship and Infection Prevention (DCASIP) supports the proposed measures CBE 4540e and 4545e. We are excited to see important next steps toward having electronic measures to support our stewardship work.
CBE 4540e and 4545e
PFPS US is strongly supportive of measures PDQM CBE 4540e and 4545e, which address the selection of agents and duration of therapy for uncomplicated community-acquired pneumonia. Over the past decade, significant strides have been made across the United States to improve antibiotic use in hospitals as part of broader efforts to combat the public health crisis of antimicrobial resistance. Most hospitals now have dedicated antibiotic stewardship programs (ASPs) to promote responsible antibiotic use, as required by the Centers for Medicare and Medicaid Services (CMS). Additionally, many hospitals have access to risk-adjusted benchmarks of their antibiotic use through the National Healthcare Safety Network Antibiotic Use Option, a reporting system also required by CMS.
Despite these advances, improving the quality of hospital antibiotic use remains a challenge. Current efforts are hindered by the difficulty of quickly and easily assessing the appropriateness of antibiotic prescribing. The ultimate goal of antibiotic stewardship is to ensure high-quality antibiotic use, and targeted measures are essential for achieving this goal. Research consistently shows that evaluating the guideline adherence of antibiotic prescribing for common infections leads to more effective and appropriate use.
Measures like PDQM CBE 4540e and 4545e are critical first steps in a new era of hospital antibiotic stewardship. These measures target pneumonia — the single most common reason for hospital antibiotic prescriptions. In some hospitals, pneumonia accounts for over 50% of all antibiotic prescribing. Importantly, since these measures are electronic, they will enable stewardship programs to assess prescribing quality without the need for labor-intensive chart reviews. Data from previous applications of manual versions of these measures demonstrate that they significantly improve prescribing practices.
PFPS US is committed to supporting pathways for hospitals to report patient-level antibiotic use data through Fast Healthcare Interoperability Resources (FHIR). Access to patient-level data will enable analysis and reporting on the quality, rather than just the quantity, of antibiotic use. However, achieving this requires validated measures like PDQM CBE 4540e and 4545e. PFPS US urges the panel to endorse these critical measures, which will play a pivotal role in advancing the quality of hospital antibiotic use nationwide.
Organization: PFPS US