Due to No Consensus.
This metric is designed to measure the critical care transport team's utilization of waveform capnography during critical care medical transport. Waveform capnography has evolved as the standard for the safe placement and maintenance of advanced airways (e.g., endotracheal tubes) in adult and pediatric patients. The metric specifically focuses on transported patients with advanced airways in whom continuous waveform capnography is appropriately used. This metric is stratified by age into the following three categories: neonatal (defined as infants <29 days), pediatric (defined as patients aged 29 days to <18 years), and adults (defined as age 18 or older). This metric is reported as “Percent of patients with advanced airways in whom waveform capnography was utilized." Transport programs track this metric for each applicable transport and report their average utilization percentage monthly.
Measure Specs
General Information
Based on the summative evidence, waveform capnography in adults, pediatric and neonatal care is well-supported in the out-of-hospital critical care transport domain for verification of tracheal tube placement within the trachea just as it is for in-hospital care.
Programs track local data using their organizations' established protocols, then abstract this data using EHR user-created reports with subsequent data validation by examining individual EHRs for inclusion/exclusion. Data are tabulated monthly and manually transferred into a web-based, data-secured platform called The GAMUT QI Collaborative Database. The numbers needed to calculate the metric are derived from the patient’s EHRs using the following fields: the presence of an advanced airway, the presence of waveform capnography use, and what age category defines the patient (neonatal, pediatric, or adult). Once calculated, the monthly metric is then entered into the GAMUT Quality Improvement Collaborative Database.
Numerator
Number of patient contacts during the calendar month with an advanced airway for whom waveform capnography is initiated and/or maintained throughout transport by the transport team, excluding patients with tracheostomies not requiring ventilatory support or patients that require high-frequency oscillatory ventilation.
There are no codes/value sets utilized in end users’ calculations of the metric, but we have included a dictionary referencing how the metric elements are defined.
This is our published numerator currently in use by the medical transport industry:
Number of patient contacts during the calendar month with an advanced airway for whom waveform capnography is initiated and/or maintained throughout transport by the transport team, excluding patients with tracheostomies not requiring ventilatory support or patients that require high-frequency oscillatory ventilation.
This numerator is categorized by age into the following 3 categories (neonatal defined as infants <29 days, pediatric defined as patients aged 29 days to <18 years, and adults defined as age 18 or older). Transport patient contact is defined as any instance a transport team contacts and provides care for a patient. This includes patients transported and those cared for without ultimately being transported (including those cared for who expire prior to transport). Waveform capnography is defined as a quantitative, graphical, and real-time measurement of the partial pressure of carbon dioxide in each exhalation. Advanced airway is defined as one of the following: Endotracheal Tube (ETT), Tracheal Tube (TT), Supraglottic Airway (SGA), or Cricothyrotomy. The numerator is a single number reported for each month.
Here is further clarification from resources provided to end-users that define and describe data elements in the database.
Transport patient contact is defined as any instance in which a transport team contacts and provides care for a patient. This includes patients transported and those cared for without ultimately being transported (including those cared for who expire prior to transport).
Waveform capnography is defined as a quantitative, graphical, and real-time measurement of the partial pressure of carbon dioxide in each exhalation. A patient is either on a machine measuring waveform capnography during medical transport or is not.
Neonatal age is defined as less than 29 days of age.
Pediatric age is defined as 29 days of age to 17 years of age.
Adult age is defined as 18 years of age and above.
Advanced airway is defined as one of the following: Endotracheal Tube (ETT), Tracheal Tube (TT), Supraglottic Airway (SGA), or Cricothyrotomy.
Calendar month is defined as occurring within the month of January, February, March, April, May, June, July, August, September, October, November, or December.
Tracheostomy is defined as a surgical opening in the neck created to establish an alternative airway by accessing the trachea. A patient either has a tracheostomy at the time of medical transport or does not.
Ventilatory support is defined as providing mechanical assistance for breathing in individuals with compromised respiratory function, aiding or completely taking over the breathing process. A patient is either receiving ventilatory support at the time of medical transport or is not.
High-Frequency Oscillatory Ventilation (HFOV) is defined as a mechanical ventilation strategy utilizing rapid, small breaths delivered at high frequencies with varying pressures provided by a specific machine designed solely for this purpose. A patient is either on a HFOV machine at the time of medical transport or is not.
Codes are not available because these are not automated. Metrics are calculated by hand and transferred into our database. If programs are getting automatic reports generated with applicable GAMUT useful data fields from their EHR programs, each is going to be different since we aren’t all on the same EHR.
Denominator
Number of patient contacts during the calendar month involving a patient with an advanced airway, excluding patients with tracheostomies not requiring ventilatory support or patients that require high-frequency oscillatory ventilation.
There are no codes/value sets utilized in end users’ calculations of the metric, but we have included a dictionary referencing how the metric elements are defined.
This is our published denominator currently in use by the medical transport industry:
Number of patient contacts during the calendar month involving a patient with an advanced airway, excluding patients with tracheostomies not requiring ventilatory support or patients that require high-frequency oscillatory ventilation.
This denominator is categorized by age into the following 3 categories (neonatal defined as infants <29 days, pediatric defined as patients aged 29 days to <18 years, and adults defined as age 18 or older). Transport patient contact is defined as any instance a transport team contacts and provides care for a patient. This includes patients transported and those cared for without ultimately being transported (including those cared for who expire prior to transport). Waveform capnography is defined as a quantitative, graphical, and real-time measurement of the partial pressure of carbon dioxide in each exhalation. Advanced airway is defined as one of the following: Endotracheal Tube (ETT), Tracheal Tube (TT), Supraglottic Airway (SGA), or Cricothyrotomy. The denominator is a single number reported for each month.
Here is further clarification from resources provided to end-users that define and describe data elements in the database.
Transport patient contact is defined as any instance in which a transport team contacts and provides care for a patient. This includes patients transported and those cared for without ultimately being transported (including those cared for who expire prior to transport).
Waveform capnography is defined as a quantitative, graphical, and real-time measurement of the partial pressure of carbon dioxide in each exhalation. A patient is either on a machine measuring waveform capnography during medical transport or is not.
Neonatal age is defined as less than 29 days of age.
Pediatric age is defined as 29 days of age to 17 years of age.
Adult age is defined as 18 years of age and above.
Advanced airway is defined as one of the following: Endotracheal Tube (ETT), Tracheal Tube (TT), Supraglottic Airway (SGA), or Cricothyrotomy.
Calendar month is defined as occurring within the month of January, February, March, April, May, June, July, August, September, October, November, or December.
Tracheostomy is defined as a surgical opening in the neck created to establish an alternative airway by accessing the trachea. A patient either has a tracheostomy at the time of medical transport or does not.
Ventilatory support is defined as providing mechanical assistance for breathing in individuals with compromised respiratory function, aiding or completely taking over the breathing process. A patient is either receiving ventilatory support at the time of medical transport or is not.
High-Frequency Oscillatory Ventilation (HFOV) is defined as a mechanical ventilation strategy utilizing rapid, small breaths delivered at high frequencies with varying pressures provided by a specific machine designed solely for this purpose. A patient is either on a HFOV machine at the time of medical transport or is not.
Codes are not available because these are not automated. Metrics are calculated by hand and transferred into our database. If programs are getting automatic reports generated with applicable GAMUT useful data fields from their EHR programs, each is going to be different since we aren’t all on the same EHR.
Exclusions
Patients with tracheostomies not requiring ventilatory support and patients requiring high-frequency oscillatory ventilation (HFOV) are excluded.
Codes are not available because these are not automated. Metrics are calculated by hand and transferred into our database. If programs are getting automatic reports generated with applicable GAMUT useful data fields from their EHR programs, each is going to be different since we aren’t all on the same EHR.
Denominator exclusions are handled by each institution's data manager during review of non-use cases before reporting aggregate totals monthly.
Here is further clarification from resources provided to end-users that define and describe data elements in the database.
Tracheostomy is defined as a surgical opening in the neck created to establish an alternative airway by accessing the trachea. A patient either has a tracheostomy at the time of medical transport or does not.
Ventilatory support is defined as providing mechanical assistance for breathing in individuals with compromised respiratory function, aiding or completely taking over the breathing process. A patient is either receiving ventilatory support at the time of medical transport or is not.
High-Frequency Oscillatory Ventilation (HFOV) is defined as a mechanical ventilation strategy utilizing rapid, small breaths delivered at high frequencies with varying pressures provided by a specific machine designed solely for this purpose. A patient is either on a HFOV machine at the time of medical transport or is not.
Codes are not available because these are not automated. Metrics are calculated by hand and transferred into our database. If programs are getting automatic reports generated with applicable GAMUT useful data fields from their EHR programs, each is going to be different since we aren’t all on the same EHR.
Measure Calculation
This measure is calculated separately for each age cohort (calculations are separate for neonatal, pediatric, and adult) and then together as an all-population proportion. To calculate the all-population rate, the sum of numerators (sum of applicable patients for all age groups) is divided by the denominator (sum of eligible patients for all age groups), then multiplied by 100 to yield a percent of appropriate waveform capnography use. To calculate the age-based cohort rate, the numerator meeting the age-defined cohort is divided by the denominator of the same age-defined cohort, then multiplied by 100 to yield a percent appropriate waveform capnography use for each age cohort.
This metric is categorized by age into the following 3 categories (neonatal defined as infants <29 days, pediatric defined as patients aged 29 days to <18 years, and adults defined as age 18 or older).
Data for this metric represents entire patient populations and is not a sampling. There is no minimum sample size; some organizations may have a numerator of zero for a given month. All participating programs submit monthly data in a web-based, data-secured platform. Data are entered in a program-level de-identified fashion (numerators and denominators), and then the database performs the calculations for each program to minimize human calculation errors.
Measure Record
Point of Contact
N/A
Heather Anthony
GAMUT Quality Improvement Collaborative
68 Ravenna Street #463
Hudson, OH 44236
United States
Hamilton Schwartz
GAMUT Quality Improvement Collaborative
68 Ravenna St #463
Hudson, OH 44236
United States
Importance
Evidence
Based on the summative evidence, waveform capnography in adults, pediatric and neonatal care is well-supported in the out-of-hospital critical care transport domain for verification of tracheal tube placement within the trachea just as it is for in-hospital care.
Although waveform capnography is used extensively for the safe monitoring of patients with advanced airways during critical care transport, this use outside of the hospital has not been the target of a large systematic review. In fact, typing “waveform capnography” into the National Library of Medicine’s PubMed website (pubmed.ncbi.nlm.nih.gov) and selecting for “Meta-Analysis” and “Systematic Review,” returns only two studies since 2016. One concerns waveform capnography’s use during procedural sedation in patients without advanced airways, and is therefore not applicable to our metric. The other (Hansel J, Law JA, Chrimes N, Higgs A, Cook TM. Clinical tests for confirming tracheal intubation or excluding oesophageal intubation: a diagnostic test accuracy systematic review and meta-analysis. Anaesthesia. 2023 Aug;78(8):1020-1030. doi: 10.1111/anae.16059. Epub 2023 Jun 16. PMID: 37325847.) is a systematic review of OTHER clinical techniques for ensuring safe advanced airway positioning, evaluating them against WAVEFORM CAPNOGRAPHY AS THE GOLD STANDARD. “In conclusion, the evidence exploring the performance characteristics of commonly used clinical tests to confirm tracheal intubation or exclude esophageal intubation is limited. The available data strongly suggest that clinical signs lack the discriminatory power to exclude esophageal intubation to a sufficient degree to ensure patient safety when capnography is not available or doubted. The esophageal detector device performs better than clinical examination, and in resource-limited environments with no access to capnography, may be sufficiently sensitive and specific to help guide decision-making. While appropriate interpretation of waveform capnography remains the reference standard for exclusion of esophageal intubation, the findings of our study have particular bearing on situations where capnography may not be available, or its findings are doubted.” Though there are no systematic reviews, there is a fair amount of literature on the technology’s use for the safe monitoring of patient’s respiratory status in the hospital setting and some in the inter-hospital transport setting. The transport medicine community feels the evidence for its use in the hospital is so compelling that it should be the standard of care when patients are being transported outside of the hospital.1 Indeed, this opinion has been established in multiple national Delphi consensus projects by experts in critical care and transport medicine.2,3 The selection of this performance metric by the GAMUT Improvement Collaborative is directly related to the existing evidence for its use in the literature and the results of the national Delphi consensus projects. The metric has already been in use for over seven years as an industry-accepted measure of quality (www.gamutqi.org). When organizations have waveform capnography equipment and trained personnel (structure) and use the waveform capnography equipment as intended (process; THIS METRIC), they will decrease the complications patients experience from ill-placed, unidentified advanced airways (outcome).
- Russotto V, Cook TM. Capnography use in the critical care setting: why do clinicians fail to implement this safety measure? Br J Anaesth. 2021;127(5):661-664. doi:10.1016/j.bja.2021.08.010
- Bigham MT, Schwartz HP, Ohio Neonatal/Pediatric Transport Quality Collaborative. Quality metrics in neonatal and pediatric critical care transport: a consensus statement. Pediatr Crit Care Med J Soc Crit Care Med World Fed Pediatr Intensive Crit Care Soc. 2013;14(5):518-524. doi:10.1097/PCC.0b013e31828a7fc1
- Schwartz HP, Bigham MT, Schoettker PJ, et al. Quality Metrics in Neonatal and Pediatric Critical Care Transport: A National Delphi Project. Pediatr Crit Care Med J Soc Crit Care Med World Fed Pediatr Intensive Crit Care Soc. 2015;16(8):711-717. doi:10.1097/PCC.0000000000000477
Measure Impact
Patients with advanced airways, like an endotracheal tube, require it to be located correctly for proper oxygenation and ventilation. During critical care transport, patients are moved from stretchers to beds and vice versa. They travel down bumpy roads and endure the turbulence of air flight. All of these put the patient's artificial airway at risk for dislodgement, affecting oxygenation and ventilation. Unappreciated improper oxygenation and ventilation leads to hypoxia and acidemia, which in turn leads to bradycardia and cardiac arrest. The evidence supports the impact of waveform capnography in improving patient safety, preventing adverse events, and potentially reducing healthcare costs associated with these adverse events1. It is a valuable monitoring tool in various clinical settings, including critical care transport medicine, and guidelines and research studies underscore its importance in clinical practice. The use of waveform capnography for verification of correctly placed advanced airways is a gold standard.2–5 It is used as a gold standard in several systematic reviews of the literature for evaluating the success of OTHER confirmatory techniques.6–8 The use of waveform capnography for ensuring correct placement of advanced airways is already standard clinical practice for many critical care transport teams and hospital-based intensive care units.9,10 The use of it for this purpose during critical care transport has not been the subject of a systematic review. However, the expert opinion of industry leaders and the current industry-wide use of this performance metric speaks to its impact on outcomes. Three separate transport-related consensus methodology projects have validated the importance of this gold-standard method of tracheal tube placement confirmation.11–13.
- Saunders R, Erslon M, Vargo J. Modeling the costs and benefits of capnography monitoring during procedural sedation for gastrointestinal endoscopy. Endosc Int Open. 2016 Mar;4(3):E340-51. doi: 10.1055/s-0042-100719. PMID: 27004254; PMCID: PMC4798929.
- Bhavani-Shankar K, Moseley H, Kumar AY, Delph Y. Capnometry and anaesthesia. Can J Anaesth. 1992;39(6):617-632. doi:10.1007/BF0300833
- Huang CH, Wei KH. Applications of capnography in airway management outside the operating room. Signa Vitae. Published online 2021. doi:10.22514/sv.2021.061
- Langhan ML, Ching K, Northrup V, et al. A Randomized Controlled Trial of Capnography in the Correction of Simulated Endotracheal Tube Dislodgement: CAPNOGRAPHY IN ENDOTRACHEAL TUBE DISLODGEMENT. Acad Emerg Med. 2011;18(6):590-596. doi:10.1111/j.1553-2712.2011.01090.x
- Stansell C, Cherry B. A Systematic Approach to Ventilator Management for the Pediatric Patient During Air Medical Transport. Air Med J. 2020;39(1):27-34. doi:10.1016/j.amj.2019.09.011
- Chou EH, Dickman E, Tsou PY, et al. Ultrasonography for confirmation of endotracheal tube placement: a systematic review and meta-analysis. Resuscitation. 2015;90:97-103. doi:10.1016/j.resuscitation.2015.02.013
- Hansel J, Law JA, Chrimes N, Higgs A, Cook TM. Clinical Tests for Confirming Tracheal Intubation or Excluding Oesophageal Intubation: A Diagnostic Test Accuracy Systematic Review and Meta-Analysis. Anaesthesia. 2023;78(8):1020-1030. doi:10.1111/anae.16059
- Das SK, Choupoo NS, Haldar R, Lahkar A. Transtracheal ultrasound for verification of endotracheal tube placement: a systematic review and meta-analysis. Can J Anaesth J Can Anesth. 2015;62(4):413-423. doi:10.1007/s12630-014-0301-z
- Monangi N, Hannah C, kwiecinski S, Mukthapuram SSN, Schwartz HP, Haberman B. Increasing Use of Waveform Capnography for Neonates During Interfacility Transport. Pediatrics. 2022;149(1 Meeting Abstracts February 2022):952.
- Fucikova H, Blatny J, Stingl J, Miletin J. Capnometry during neonatal transport—Mini review. Acta Paediatr. 2023;112(5):919-923. doi:10.1111/apa.16729
- Bigham MT, Schwartz HP, Ohio Neonatal/Pediatric Transport Quality Collaborative. Quality metrics in neonatal and pediatric critical care transport: a consensus statement. Pediatr Crit Care Med J Soc Crit Care Med World Fed Pediatr Intensive Crit Care Soc. 2013;14(5):518-524. doi:10.1097/PCC.0b013e31828a7fc1
- Schwartz HP, Bigham MT, Schoettker PJ, et al. Quality Metrics in Neonatal and Pediatric Critical Care Transport: A National Delphi Project. Pediatr Crit Care Med J Soc Crit Care Med World Fed Pediatr Intensive Crit Care Soc. 2015;16(8):711-717. doi:10.1097/PCC.0000000000000477
- Home: GAMUT Ground & Air Medical Quality in Transport. Accessed July 20, 2023. https://www.gamutqi.org/index.html#WHAT_IS_IT
The use of waveform capnography for demonstrating that an advanced airway is properly located has been identified as the gold standard by the American Heart Association and at least three systematic reviews/meta-analyses.2-4, 10 Before the creation of this metric by consensus methodology first in 2011 then revised in 2012, there was NO existing measure or quality improvement program to evaluate how transport programs mitigate the adverse events associated with unappreciated, dislodged advanced airway devices.7,8 In 2013, a national meeting of the Air Medical Physician Association and other industry leaders again engaged in modified Delphi technique, now for a third time, validating the importance of the use of waveform capnography for monitoring advanced airways as a measure of quality of care. This ultimately led to the metrics inclusion in the GAMUT metric set and support for its use by CAMTS (Commission on Accreditation of Medical Transport Systems).9 In the years since, a multitude of studies and articles have supported waveform capnography’s importance for confirming the appropriate placement of advanced airways in the out-of-hospital setting.1,5,6,11–19
- Huang CH, Wei KH. Applications of capnography in airway management outside the operating room. Signa Vitae. Published online 2021. doi:10.22514/sv.2021.061
- Chou EH, Dickman E, Tsou PY, et al. Ultrasonography for confirmation of endotracheal tube placement: a systematic review and meta-analysis. Resuscitation. 2015;90:97-103. doi:10.1016/j.resuscitation.2015.02.013
- Hansel J, Law JA, Chrimes N, Higgs A, Cook TM. Clinical Tests for Confirming Tracheal Intubation or Excluding Oesophageal Intubation: A Diagnostic Test Accuracy Systematic Review and Meta-Analysis. Anaesthesia. 2023;78(8):1020-1030. doi:10.1111/anae.16059
- Das SK, Choupoo NS, Haldar R, Lahkar A. Transtracheal ultrasound for verification of endotracheal tube placement: a systematic review and meta-analysis. Can J Anaesth J Can Anesth. 2015;62(4):413-423. doi:10.1007/s12630-014-0301-z
- Monangi N, Hannah C, kwiecinski S, Mukthapuram SSN, Schwartz HP, Haberman B. Increasing Use of Waveform Capnography for Neonates During Interfacility Transport. Pediatrics. 2022;149(1 Meeting Abstracts February 2022):952.
- Fucikova H, Blatny J, Stingl J, Miletin J. Capnometry during neonatal transport—Mini review. Acta Paediatr. 2023;112(5):919-923. doi:10.1111/apa.16729
- Bigham MT, Schwartz HP, Ohio Neonatal/Pediatric Transport Quality Collaborative. Quality metrics in neonatal and pediatric critical care transport: a consensus statement. Pediatr Crit Care Med J Soc Crit Care Med World Fed Pediatr Intensive Crit Care Soc. 2013;14(5):518-524. doi:10.1097/PCC.0b013e31828a7fc1
- Schwartz HP, Bigham MT, Schoettker PJ, et al. Quality Metrics in Neonatal and Pediatric Critical Care Transport: A National Delphi Project. Pediatr Crit Care Med J Soc Crit Care Med World Fed Pediatr Intensive Crit Care Soc. 2015;16(8):711-717. doi:10.1097/PCC.0000000000000477
- Home: GAMUT Ground & Air Medical Quality in Transport. Accessed July 20, 2023. https://www.gamutqi.org/index.html#WHAT_IS_IT
- Neumar RW, Otto CW, Link MS, et al. Part 8: adult advanced cardiovascular life support: 2010 American Heart Association Guidelines for Cardiopulmonary Resuscitation and Emergency Cardiovascular Care. Circulation. 2010;122(18 Suppl 3):S729-767. doi:10.1161/CIRCULATIONAHA.110.970988
- Scrivens A, Zivanovic S, Roehr CC. Is waveform capnography reliable in neonates? Arch Dis Child. 2019;104(7):711-715. doi:10.1136/archdischild-2018-316577
- Model-Clinical-Guideline-Prehospital-Airway-Management.pdf. Accessed October 27, 2023. https://nasemso.org/wp-content/uploads/Model-Clinical-Guideline-Prehospital-Airway-Management.pdf
- Fawke J, Wyllie J. Neonatal end-tidal carbon dioxide detection and waveform capnography. Anaesthesia. 2019;74(1):114-115. doi:10.1111/anae.14525
- Harris M, Lyng JW, Mandt M, et al. Prehospital Pediatric Respiratory Distress and Airway Management Interventions: An NAEMSP Position Statement and Resource Document. Prehosp Emerg Care. 2022;26(sup1):118-128. doi:10.1080/10903127.2021.1994675
- Rückoldt H, Marx G, Leuwer M, Panning B, Piepenbrock S. [Pulse oximetry and capnography in intensive care transportation: combined use reduces transportation risks]. Anasthesiol Intensivmed Notfallmed Schmerzther. 1998;33(1):32-36. doi:10.1055/s-2007-994207
- Fanara B, Manzon C, Barbot O, Desmettre T, Capellier G. Recommendations for the intra-hospital transport of critically ill patients. Published online 2010.
- Silvestri S, Ralls GA, Krauss B, et al. The Effectiveness of Out-of-Hospital Use of Continuous End-Tidal Carbon Dioxide Monitoring on the Rate of Unrecognized Misplaced Intubation Within a Regional Emergency Medical Services System. Ann Emerg Med. 2005;45(5):497-503. doi:10.1016/j.annemergmed.2004.09.014
- Khan SM, Lance MD, Elobied MAK. Transport of Critically Ill Patients – A Review of Early Interventions, Protocols, and Recommendations. Int J Health Sci Res. 2021;11(4):133-143. doi:10.52403/ijhsr.20210418
- Dassios T, Williams EE, Greenough A. Waveform capnography in neonatal intensive care: is it unreliable? Arch Dis Child - Fetal Neonatal Ed. 2021;106(2):228-228. doi:10.1136/archdischild-2020-321007
Waveform capnography is a valuable tool used during intubation and mechanical ventilation to monitor a patient's exhaled carbon dioxide levels.1,2 It helps gauge correct airway device placement, ventilation effectiveness, and early identification of potential complications. While the primary beneficiaries of waveform capnography are healthcare providers and caregivers, it importantly benefits patients by improving the safety and quality of care provided to them by allowing care providers to identify airway complications early and before major detrimental effects have occurred.3-6 To obtain input directly from patients, there would need to be patient satisfaction surveys or qualitative interviews with individuals who have undergone intubation and mechanical ventilation - there are none of these studies. Feedback from healthcare providers, such as physicians, anesthesiologists, and respiratory therapists, who routinely use waveform capnography during intubation, provide insights into the benefits observed in patient care.
- Huang CH, Wei KH. Applications of capnography in airway management outside the operating room. Signa Vitae. Published online 2021. doi:10.22514/sv.2021.061
- Das SK, Choupoo NS, Haldar R, Lahkar A. Transtracheal ultrasound for verification of endotracheal tube placement: a systematic review and meta-analysis. Can J Anaesth J Can Anesth. 2015;62(4):413-423. doi:10.1007/s12630-014-0301-z
- Monangi N, Hannah C, kwiecinski S, Mukthapuram SSN, Schwartz HP, Haberman B. Increasing Use of Waveform Capnography for Neonates During Interfacility Transport. Pediatrics. 2022;149(1 Meeting Abstracts February 2022):952.
- Fucikova H, Blatny J, Stingl J, Miletin J. Capnometry during neonatal transport—Mini review. Acta Paediatr. 2023;112(5):919-923. doi:10.1111/apa.16729
- Fawke J, Wyllie J. Neonatal end-tidal carbon dioxide detection and waveform capnography. Anaesthesia. 2019;74(1):114-115. doi:10.1111/anae.14525
- Harris M, Lyng JW, Mandt M, et al. Prehospital Pediatric Respiratory Distress and Airway Management Interventions: An NAEMSP Position Statement and Resource Document. Prehosp Emerg Care. 2022;26(sup1):118-128. doi:10.1080/10903127.2021.1994675
Equity
Equity
Our data is in aggregate. We do not report patient-level data. There is no way for us to look at the data for inequities in health care. In a pubmed.gov search of the literature, there are no studies examining disparities in the application of waveform capnography.
Feasibility
Feasibility
The feasibility of this metric has been assessed in three major ways. The first assessment was at the time of its selection during three large and national stakeholder consensus-determining events, which informed selection of waveform capnography use as a must-have for measuring the quality of critical care transport.1,2 Each metric selected at the time was rigorously evaluated by the stakeholders for its feasibility to collect from patients' care encounters. Waveform capnography was one of those metrics evaluated and deemed feasible. The second assessment was during a GAMUT end-user survey of current metric users in 2019. Participants overwhelmingly felt the waveform capnography use metric continued to be feasible to collect. Lastly, end-users are prompted regularly for input around problematic aspects of the current metric set (including waveform capnography use). There have been no signals to GAMUT (current stewards of this metric) that the data included in this metric is more challenging to collect than any other data gathered from an electronic medical record.
- Bigham MT, Schwartz HP, Ohio Neonatal/Pediatric Transport Quality Collaborative. Quality metrics in neonatal and pediatric critical care transport: a consensus statement. Pediatr Crit Care Med J Soc Crit Care Med World Fed Pediatr Intensive Crit Care Soc. 2013;14(5):518-524. doi:10.1097/PCC.0b013e31828a7fc1
- Schwartz HP, Bigham MT, Schoettker PJ, et al. Quality Metrics in Neonatal and Pediatric Critical Care Transport: A National Delphi Project. Pediatr Crit Care Med J Soc Crit Care Med World Fed Pediatr Intensive Crit Care Soc. 2015;16(8):711-717.
Because feedback has been positive, the metric continues to be in use and UNMODIFIED from its original state.
Proprietary Information
There are no fees associated with using the Waveform Capnography In Ventilated Patients metric. This metric is available to the public for free.
Scientific Acceptability
Testing Data
All data for testing was derived from three sources: the Electronic Health Records from participating GAMUT member organizations, the GAMUT Quality Improvement Collaborative Database, GAMUT Users Validity Survey, all over the time period 01-01-2014 through 12-31-2019. Participating programs DO NOT report patient-level data, so there is no way for us to examine the data for inequities in the application of waveform capnography. Age is the only patient characteristic collected, and that is only because there are separate numerators and denominators for each age group. Metrics are not stratified by gender or race, only by the categories of neonatal, pediatric, and adult. The numbers needed to calculate the metric are derived from the patient’s EHRs: the presence of an advanced airway, the presence of waveform capnography use, and what age category defines the patient (neonatal, pediatric, or adult). Once calculated, the monthly metric is then entered into the GAMUT Quality Improvement Collaborative Database. All aspects of reliability testing used the same data set derived from EHRs from participating GAMUT organizations in the GAMUT QI Collaborative Database over the time period of 01-01-2014 through 12-31-2019. Validity testing was accomplished using a SurveyMonkey® survey (GAMUT Users Validity Survey) of 34 GAMUT programs selected randomly from 150 to determine face validity of the GAMUT metric Waveform Capnography In Ventilated Patients. Respondents assessed questions using a Likert 1-5 scale.
There were no differences in the data used for testing whether this met muster as a quality metric.
The data are from all GAMUT programs and represent the transport industry as a whole -- neonatal, pediatric, and adult programs transporting patients by ambulance, helicopter, and airplane.
Total number of programs (entities) reporting data with overall transported neonatal patients
Neonatal Program Percentage of Total Patients Number of Programs
80 - 100 = 10
60 - 79 = 2
40 - 59 = 7
20 - 39 = 19
1-19 = 45
Total Neonatal Programs = 83
Total number of programs (entities) reporting data with overall transported pediatric patients
Pediatric Program Percentage of Total Patients Number of Programs
80 - 100 =11
60 - 79 =16
40 - 59 = 7
20 - 39 = 14
1-19 =75
Total Pediatric Programs =123
Total number of programs (entities) reporting data with overall transported adult patients
Adult Program Percentage of Total Patients Number of Programs
80 - 100 =63
60 - 79 =14
40 - 59 =5
20 - 39 =4
1-19 =14
Total Adult Programs =100
Number of Programs by Region of USA
Midwest = 112
Northeast = 40
South = 135
West = 75
Uknown = 3
Number of Programs Outside of United States/International
Australia = 5
Brazil = 1
Canada = 8
China = 4
Germany = 2
Ireland = 2
New Zealand = 1
Qatar = 1
Spain = 1
Thailand = 1
United Kingdom = 5
United States Virgin Islands = 1
Since 2014, within the GAMUT database, the Waveform Capnography in Ventilated Patients metric has been used for 80,072 adult patient contacts, 22,702 pediatric patient contacts, and 22,180 neonatal patient contacts. The contacts captured for this specific metric are patients who are critically ill, are being managed by transport programs, and have an advanced airway in place. Program compliance with appropriate waveform capnograph use is reported monthly in aggregate as the proportion of eligible patient encounters in which the technology was used.
Reliability
Interrater reliability was assessed for the two critical data elements, namely mechanical ventilation determination (Yes/No) and waveform capnography usage (Yes/No). We asked participating transport teams to generate lists of patients with advanced airways being mechanical ventilation. Each list contained consecutive patients. Patient data was collected until a minimum number of patients per site was reached to assess agreement within 20% based on assumed percentage agreement. The data elements were abstracted by four different transport nurses, managers, or directors at three different transport facilities. Percentage agreement that waveform capnography was used appropriately for each encounter was calculated and summarized for each of the three study sites and overall. A free marginal Kappa was employed to measure interrater reliability at each site and overall and 95% confidence intervals were determined.
The percentage agreement for mechanical ventilation determination was 100% for each site, resulting in a free marginal kappa overall of 1.00. This reliability statistic value is significantly greater than the 0 value that which would occur by chance (p<0.05). Waveform capnography overall had 89.3% percentage agreement of the 318 assessed cases with a corresponding free marginal Kappa value of 0.79 (95% CI 0.75 – 0.85). Again, this reliability statistic is significantly greater than that of chance alone (p<0.05). Kappa values for each site ranged from 0.72 – 1.00, with each demonstrating significantly greater values than would occur by chance (p<0.05).
Each of the two critical data elements overall demonstrated excellent interrater reliability using 100% of 318 instances across three separate programs during one calendar year. According to Fleiss’ (1981) interpretation of Kappa values indicating values greater than 0.75 are considered excellent. The abstractions of the two critical data elements were consistent and demonstrated that the waveform capnography metric may be reliably collected.
| | Overall | Minimum | Decile_1 | Decile_2 |
|---|---|---|---|---|
| Reliability | ||||
| Mean Performance Score | ||||
| N of Entities |
Validity
Thirty-four GAMUT programs were randomly selected from 150 to participate in a survey to determine face validity of the GAMUT metric Waveform Capnography In Ventilated Patients. The metric definition, numerator, and denominator were provided. Two questions were asked: 1) Do you agree that this metric definition, calculated using the numerator and denominator, provides a valid quality measure in patient contacts with advanced airways; and 2) Do you agree that this metric is reliable in that it provides an accurate, reproducible, and consistent result from one testing occasion to another? The 34 experts assessed each of the two questions using a Likert 1-5 scale.
For the 2 questions asked...#1) Do you agree that this metric definition, calculated using the numerator and denominator, provides a valid quality measure in patient contacts with advanced airways? 76.5% of respondents chose "very important" on the Likert scale. 11.8% of respondents chose "important" on the Likert scale. 11.8% of respondents chose "neutral," "unimportant," or "very unimportant." For #2) Do you agree that this metric is reliable in that it provides an accurate, reproducible, and consistent result from one testing occasion to another? 73.5% of respondents chose "very important" on the Likert scale. 17.6% chose "important" on the Likert scale. 8.8% of respondents chose "neutral," "unimportant," or "very unimportant."
Of 34 GAMUT programs surveyed, approximately 75% of them felt the metric was an important and valid quality measure for the field of critical care transport. "Strongly agree" was selected by (76.5%) of program managers/content experts.
Risk Adjustment
Not applicable
Use & Usability
Use
Usability
Programs must know how they perform on this metric, how they compare to peer organizations on this metric, and how they compare to the Achievable Benchmark of Care on this metric before deciding if they need to improve and how to approach that improvement.1 Programs will typically use strategies like the Model for Improvement or Six Sigma to map out the process, construct a key driver diagram, and prepare planned experimentation to enhance the use of waveform capnography for patients with advanced airways.2 There are published examples of programs improving their performance on this metric without tremendous obstacles.3
- Weissman NW, Allison JJ, Kiefe CI, et al. Achievable benchmarks of care: the ABCs of benchmarking. Journal of Evaluation in Clinical Practice. 1999;5(3):269-281. doi:10.1046/j.1365-2753.1999.00203.x
- API - Associates in Process Improvement - Home. Accessed October 27, 2023. https://www.apiweb.org/
- Monangi N, Hannah C, kwiecinski S, Mukthapuram SSN, Schwartz HP, Haberman B. Increasing Use of Waveform Capnography for Neonates During Interfacility Transport. Pediatrics. 2022;149(1 Meeting Abstracts February 2022):952.
Comments
Staff Preliminary Assessment
CBE# 4045 Staff Assessment
Importance
Strengths:
- Waveform capnography is the standard of care for ensuring safe advanced airway positioning in hospital, and has substantial support for its use in transport settings due to the higher risk of artificial airways becoming dislodged, impacting oxygenation and ventilation; this support includes three separate consensus statements among experts in critical care and transport medicine and a small number of empirical studies.
- Failure to identify airway complications early has several deleterious effects, including respiratory arrest, cardiac arrest, and consequences of overventilation (cerebral hypoperfusion) or hypoventilation (pulmonary hypertension, intracranial hypertension)). After the measure was submitted to Battelle, the developer added more information after its review of the staff assessment: The rate of hospital mortality of patients experiencing UE [unplanned extubation] can vary from 10% to 25%.
- After the measure was submitted to Battelle, the developer added measure performance ranges observed since program initiation in 2014: Overall, 89.2-95.6%; Pediatric, 87.5-94.1%; Adult, 94-97.8%.
- Developers claim the proposed measure is the first of its kind.
Limitations:
- The developer notes no systematic reviews of waveform capnography in transport exist.
- Variance in measure performance by program (e.g., IQR, deciles) was not provided beyond ranges. Ranges in performance provided may indicate limited room for improvement, especially for adults.
- Meaningfulness to patients has not been established directly in the literature, and developers do not report on a TEP; developers claim that feedback from clinicians regarding the benefits of waveform capnography in patient care provide important insights.
Rationale:
- Waveform capnography is the standard of care for ensuring safe advanced airway positioning in hospital, and support for using it in transport includes empirical studies and three consensus statements. A dislodged artificial airway impacts oxygenation and ventilation, and can result in respiratory and cardiac arrest, among other undesirable outcomes. The mortality rate among patients experiencing unplanned extubation ranges from 10% to 25% . Developers assert that this measure is the first of its kind.
- Measure performance since 2014 ranges from 89.2% to 95.6% (87.5-94.1% among pediatric patients and 94-97.8% among adults), and may have limited room for improvement for some groups.
- Meaningfulness to patients has not been assessed directly, although developers argue that clinician feedback regarding the benefits of waveform capnography for clinical care provides support.
Feasibility Acceptance
Strengths:
- Information required for the measure is collected in the routine course of care. There are no fees or licensing requirements for the measure.
- Developer refers to consensus events, an end-user survey, and requests for feedback from end users as evidence for feasibility of the measure, and concluded from these that effort to collect the required data was similar to required effort for other EHR-based data gathering activities for reporting.
- After the measure was submitted to Battelle, the developer added more information after its review of the staff assessment: Over a recent 12-month period, waveform capnography utilization data has been reported on 41,659 transported patients across 143 transport programs, which developers argue indicates high feasibility of the measure.
Limitations:
- Details regarding how feasibility was assessed in the survey (e.g., survey items, responses, sample) or in the consensus event (e.g., member, results of voting) were not provided.
- No automated codes exist for numerator or denominator; numbers of patients meeting criteria each month are abstracted and compiled by hand from EHRs.
Rationale:
- Information required for the measure is collected in the course of care and there are no fees or licensing requirements to use the measure.
- Data elements must be abstracted by hand from EHRs and compiled prior to submission. Developers refer to several feasibility assessments undertaken, including consensus events and an end-user survey that provide support for the measure's feasibility, but details on the content of the survey or how feasibility was established during consensus events was not provided.
Scientific Acceptability
Strengths:
- The measure is well-defined and precisely specified. Details for measure calculation were clear and thorough.
- For patient or encounter level reliability, the interrater reliability kappa value ranged from 0.72 to 1.00, exceeding the threshold of 0.4.
- Free marginal kappa overall of 1.00.
- All aspects of reliability testing used the same data set derived from EHRs from participating GAMUT organizations in the GAMUT QI Collaborative Database over the time period of 01-01-2014 through 12-31-2019.
Limitations:
- Tested reliability at only three different facilities.
Rationale:
- Measure score reliability testing (data element reliability) performed. The interrater reliability exceeded the accepted threshold of 0.4 in all cases.
- Interrater reliability was assessed at three different facilities, and the developer should consider accountable entity-level testing in the future, since the measure has been used since 2014.
Strengths:
- Metric is stratified by age group: neonatal (<29 days), pediatric (29 days to <18 years), adult (18+)
- Waveform capnography is the standard of care for ensuring safe advanced airway positioning in hospital, and is the gold standard used for evaluating other methods
- Face validity was established by a survey of 34 randomly selected program participants. Approximately 75% of respondents responded "strongly agree" to either of two questions, 1) agreement that the metric as defined provided a valid quality measure, and 2) agreement that the measure provides accurate, reproducible, and consistent results across testing occasions.
Limitations:
- For face validity testing, it is unclear who the participants were, if they included patients, and why 25% of the respondents did not agree.
Rationale:
- Waveform capnography is used as the gold standard for evaluating other methods for ensuring safe advanced airway placement. Face validity of the measure was established by a survey of programs using the measure, and approximately 75% of respondents strongly agreed that the metric as defined was a valid quality measure and that it provided accurate, reproducible, and consistent results. However, it is unclear who the participants were, if they included patients, and why 25% of the respondents did not agree. The committee should inquire about this from the developer.
Equity
Strengths:
N/A
Limitations:
Developer did not address this optional criterion.
Rationale:
Developer did not address this optional criterion.
Use and Usability
Strengths:
- Measure is currently in use within the GAMUT database since 2014 for internal QI, and QI with external benchmarking. The developer further indicates current or planned use for public reporting.
- Developer alludes to benchmarking as means for providers to identify the need to improve, common processes for organizations to implement a QI strategy (e.g., Six Sigma, Model for Improvement), and cite one source that asserts the technology can be implemented without significant obstacles (pediatric population)
Limitations:
- Developer's assertion that providers commonly use strategies such as Six Sigma for QI could use more attention - reference goes to a website containing little additional information.
Rationale:
- This new measure is currently in use. Developers assert that benchmarking will be useful for providers to identify improvement needs and cite evidence that the technology can be implemented without significant barriers. Since this measure has been in use since 2014, developer should consider reporting improvement/change in measure performance over time.
Summary
N/A
Committee Independent Review
Overall this quality measure…
Importance
The supplemental data on current rates of waveform capnography use were helpful in this regard.
Feasibility Acceptance
Agree with staff assessment.
Scientific Acceptability
Agree with PQM staff assessment, fairly limited study.
Agree with PQM staff assessment, follow up questions are needed, especially regarding why 25% of respondents did not agree. Validity study was fairly limited as well.
Equity
Agree with staff assessment.
Use and Usability
Agree with staff assessment.
Summary
Overall this quality measure looks to evaluate a priority area for this field. I would challenge the measure developers to help us understand if there is true variability in performance at this stage. It seems waveform capnography use is widespread and may already be in use in 90% or more of times when it is indicated. What prevents this measure from reaching 100%? Is it a gap among specific providers without reliable access to these devices? Does it relate to instances when the devices are malfunctioning? Or is it a documentation gap that may not be indicative of an actual performance gap (validity issue)?
Measure seems to be usable, needs further review
Importance
This measure seems to have substantial support regarding the importance of a way to ensure the proper placement of airway positioning during the transport, and the difficulty in ensuring that proper positioning during transport. Unfortunately, there is no scientific evidence that this is the only or gold standard process that supports safety in intubation of the patient during transport. In addition, it was not observed any discussion regarding the meaningfulness to patients, either via literature review or focus groups or panels. In addition, there is no discussion regarding the burden of the measure to those reporting versus the benefits. It is unclear the addition of this measure will support any gap in care while addressing any burden of reporting, since the measure performance has been reported to be over 89% across all population included.
Feasibility Acceptance
This measure seems to be easy to implement and collect the data, and there are no associated fees to implementing the measure. However, it is unknown to this reviewer what cost is associated with any equipment to obtain the waveform capnograpy report. In addition, the report has to be calculated by hand, which could introduce human errors in the calculation. It is unclear how the possibility of calculation errors will be decreased/elimination during implementation. There does not seem to be an established path to support routine data generationfrom electronic data sources.
Scientific Acceptability
Even though the measure seems to be simple in data collection, and reliability seems to be within expected range, it is unclear why testing was not done at a higher level, especially considering there was not a high level of scientific evidence provided to support the need for this measure to cover a gap in care.
In addition, only face validity was evaluated, via survey of 34 randomly selected program participants. This does not seem enough to support a measure that does not have established literature as evidence of a gap in care. Furthermore, the face validity done does not stratify the responses making it difficult to understand the importance established by the answers. It also does not discuss the reasons behind a high percentage of respondents do not agree.
Equity
Discussion of application of the measure on equity was not included in the developer's submission.
Use and Usability
This measure has been in use for several years. However, there is very little information regarding the measure being used in the past. Nonetheless, this measure seems to be simple in application and evaluation of results, with a calculation that is simple to interpret.
Summary
The measure regarding waveform capnography seems to be usable, and in fact, has been in use for several years. The measure components for numerator and denominator seem to be simple to attain and there does not seem to be a high burden of implementation. Nonetheless, there is no additional information provided on the measure website, and the calculation is made by hand. The measure developer does not address any process to decrease the risk of miscalculation. There also seems not be a gap in care established in the literature, and no other process in place that this measure compares to as a gold standard.
Wave form Capnography
Importance
Important new measure for quality of care and a standard of care
Feasibility Acceptance
Manual chart extract. Recommend codes be developed as part of the EMR
Scientific Acceptability
Only 3 facilities participated and recommend further testing with more facilities
Only 3 facilities participated and recommend further testing with more facilities
Equity
There should be no variation in equity since this is a standard of care
Use and Usability
GAUMT
2 critical data elements for this measure and should be built into the EMR
Summary
This is a standard of care, should be documented in the medical record. Data fields need to be developed for the EMR. This should be inplace for any inpatient transports as well as ground, rotor wing and fixed wing transports
this measure could be crucial
Importance
Staff assessment raises important points. The only thing I'd add is that if no systematic reviews of waveform capnography in transport exist, then how is it known that this issue needs scrutiny. Before a full scale investment, would a pilot project and/or period be a useful test to understand how widespread the failure to identify airway problems in transport is and to what extent is there indeed a lack of proper use of waveform capnography in transport
Feasibility Acceptance
seems that data is available
Scientific Acceptability
It seems that making only one adult category for those over the age of 18 might miss important distinctions. There are younger adults, middle aged adults, and older adults, and very old adults. Each group might have some unique features that might have an effect on accurate assessment and use of waveform capnography. Older adutls might have mutiple chronic conditions that impact accuracy and usage approaches.
Agree with staff - what was the input of the 25% who did not agree. That seems like a large precentage.
Equity
It seems that equity considerations were quickly dismissed. If an EHR record is part of this measure, then might some demographic data be pulled to help assess inequities?
Use and Usability
The absence of involvement of patient and family input might be leaving degrees of uncertainty that are not included in the approach towards undersatnding all the acotrs related to undersatnd, improve, and maintain measure improvements.
Summary
it depends on the extent and pervasiveness of the problem the measure addresses. And I would imagine, from having seen how complex it can be make process/protocol changes in healthcare, I think improvement approaches might require more attention. I also think the patient/family caregiver experience, especially during episodes of critical transitions needs to be taken into account and included in assessments of improvements.
NA
Importance
A key concern about waveform capnography during critical care transport is the limited systematic reviews or meta-analyses to establish a gold standard of care. Patient perspective is currently lacking as positive feedback is mainly reported by clinicians.
Though there are consensus statements supporting the anticipated outcomes, they are subjective and are at the bottom of the research pyramid to be deemed as evidence-based.
The developers state that the metrics are calculated by hand to be entered into their database and EHRs differ. Manually collecting data will lead to significant errors especially which worsen with the differences in EHR systems. Data modernization and quality assurance measures need to be in place.
Feasibility Acceptance
The developers did not describe the process and how feasibility was done. They state that feasibility was "rigorously evaluated by the stakeholders." What methods were employed and by which stakeholders?
Scientific Acceptability
What is the rationale for selecting a 20% percentage agreement?
The total number of randomly selected participants in less than half of the programs (34/150; 22.6%). As a result, the results do not reflect the programs and may not be generalized. Furthermore, only two questions were asked to establish face validity, which is concerning. Even though more than half of the participants reported positive sentiments, for a measure without scientific evidence, one would think that the negative sentiments would be further investigated. This begs whether the instrument included open-ended responses to gather qualitative opinions.
Another concern is the survey language. These are leading questions that would most likely elicit positive answers because of the verbs used. In addition, the survey contains scientific terms "valid," "denominator," or "numerator" that the average non-sciency person would understand. Unless, of course, these terms were explained.
Equity
The developer did not provide equity-related data because patient-level data were not collected. However, this should open the door for a conversation on the feasibility of collecting patient-level data, particularly given the limited evidence around its use for transportation care.
Use and Usability
The measure is pretty straightforward, but patient voices are lacking. As a patient advocate, and given the outcry for patient-centered care across the continuum of care, this is an area of concern.
Summary
NA
Waveform Capnography in Ventilated Patients During Transport
Importance
Developer noted that waveform capnography is the standard of care for ensuring safe advanced airway positioning in hospital, during transport and failure to identify airway complications early can have disastrous effects for the patient when not detected timely. Having reviewed patient events due to an improperly placed airway that was not detected timely, I agree that clinical care feedback provides the support how important this measure can be in preventing such events.
I would like to understand what is to be gained by reporting this measure. How will this benefit a facility and have sites that have implemented this reporting seen in increase in compliance/outcomes or were they already performing at a high standard and there has been little to no improvement?
Feasibility Acceptance
Although this seems to be a measure easily abstracted, I am concerned this is manual abstraction and not using the available EHR or mobile capabilities to interface/upload to be able to pull the data necessary for this reporting measure. In addition, I would like to better understand why the sites that have implemented this measure, have not been able to develop documentation in the EHR or use mobile capabilities to have the data available to limit manual abstraction and not introduce human errors.
Scientific Acceptability
Would like to understand the limitations of only 3 facilities testing when there are over 300 US programs. The is a very low sample size.
Agree w PQM staff assessment
Equity
Developer did not address this optional criterion.
Use and Usability
Agree with staff assessment and would like to see more on the commonly used Six Sigma QI strategies.
Summary
I would like to understand since there are over 300 programs in the US alone, what have been the benefits in those facilities where they have been monitoring this standard of care? Have they seen any improvements in compliance, have patient outcomes improved? Or are sites just monitoring a standard of care that is already at about 100%?
What is the effort made by sites to standardize this documentation in the EHR to facilitate reporting via electronic meds to reduce manual abstraction/burden and limit human error? Can a system not pull the codes for mechanical ventilation, or capnography documentation to assist in some of the abstraction?
Additional question: I was unable to identify why high-frequency oscillatory ventilation (HFOV) patients were excluded from the denominator. Are patients ever switched during transport on/off HFOV? Reason is do they need to be HFOV the entire time to be excluded or just at some point during the transport?
Need more details, but concerns can be addressed.
Importance
Meaningfulness to patients has not been established and there is limited evidence that the measure would be needed for improvement. Need more evidence in the future about whether the measure will lead to improvement.
Feasibility Acceptance
no data presented on the end-user survey results, although they claim the results were good.
Scientific Acceptability
I do not see testing of all data elements that are required for measure calculation. E.g., testing of data elements to calculate age, transport, Tracheostomy were not presented.
Can the developer confirm that the thirty-four GAMUT programs surveyed were also the ones using the metric? Need more details about the participants in the face validity testing to assess whether it was representative and whether the concerns about the measure were addressed.
Equity
Would have liked to see plans for how they may disaggregate data in the future to identify potential inequities.
Use and Usability
This new measure meets expectations. However, because external benchmarking was checked in the application, would like to see more details on this use specifically.
Summary
- Meaningfulness to patients not shown. Need more evidence about whether the measure will lead to improvement.
- For feasibility, no data presented on the end-user survey results.
- I do not see testing of data elements to calculate age, transport, Tracheostomy. These appear to be required to calculate the measure.
- Were the thirty-four GAMUT programs surveyed also the ones using the metric? Need more details about the participants in the face validity testing.
- Would have liked to see plans for how they may disaggregate data in the future to identify potential inequities.
- Would like to see more details on external benchmarking use specifically.
- Appreciate stratification of measure by age.
Interesting Measure
Importance
Appears to be of value for patient care, however, it is difficult to say how often harm occurs --- which may also be a reason to capture the information and report it publicly.
Feasibility Acceptance
Challenging without the use of codes. Not sure how feasible this is at this point.....and also concerned about reliability.
Scientific Acceptability
The issue of reliability has not been addressed except on a very small scale. Needs more testing sites to demonstrate reliability
The measure appears to be valid and helpful for patients and healthcare providers. It is a relatively new measure and needs more study and testing for validity and reliability. As the age groups were added in after measure submitted, I wonder if these are valid age groups. The adults grouped into one age group may need to be broken down.....plus, there may be chronic disease conditions that need to be considered for adults.
Equity
There were no equity measures. In reading the info provided by the submitted, it seems there might need to also be adjustment based upon road or air conditions as well -- environmental factors involved in transport.
Use and Usability
This could be a valuable measure --- but I am trying to get a sense of the variation in transports and if the info would be communicated to the public at transport organization level. Also, will patients be presented with a choice of transport companies/organizations based upon public data?
Summary
If a patient or family knew there was a variation in transport quality, would they have a choice to select a different transport organization? If this is a measure that will help drive improvement, then it is worth exploring more. I do think the equity issues need to be addressed as I am sure there are variations in underserved rural and even urban areas --- which sometimes relfects SES, race, ethnic, age variations. I also wonder if transport services are readily available to safety net hospitals --- or would the decision to transport be based upon the recipient hospital or facility.
I do not think patients…
Importance
This is an important measure to consider because it has a beneficial impact for patients/families. Hospitals are already addressing unplanned extubations and the same should be captured during transport.
They do not have direct feedback from patient/family experience. While some patients may not cognitively be able to provide feedback. It would still be nice to see patient reported outcomes or patient reported experiences.
It did not state in what manner (surveys or direct conversations) on the feedback they received from the healthcare providers.
It was good they broke down the populations (neonates, pediatrics and adults) they are unique populations for this measure.
Feasibility Acceptance
It is not an eCQM measure, but data can be extracted from the EHR.
There is a question of feasibility when trying to capture patient reported outcomes (PRO) or patient reported experience (PRE).
There are not barriers for proprietary information or licensing.
Scientific Acceptability
The geographic spread was good and not focused in one particular state or part of the country.
I think more data is needed.
The pediatric range is wide and I wonder if it should be broken down even more. There is a big disparity between young children and teenagers.
Equity
There was not equity information with this measure.
Use and Usability
This is a measure already in use just in a different setting.
Many organizations are already tracking unexpected extubations so usability is proven.
Summary
I do not think patients/families would look at the measure because it is not something they would think about or potentially consider.
I do believe it is important and is already being captured in the hospital. It would definitely benefit patient care.
CBE #4045
Importance
No systematic review available, consensus statements among critical care experts, logic model provided. Airway complications are associated with adverse outcomes. Agree with staff that more information on variation in performance would be useful.
Feasibility Acceptance
Measure is in use and therefore appears to be feasible to collect. Requires manual abstraction and interested to know if any viable path to an eCQM. Agree with staff assessment additional info on feasibility assessment with stakeholders would be beneficial.
Scientific Acceptability
Reliability – Results suggests the measure is reliable with 100% agreement for one of the data elements and Kappa value of 0.79 (95% CI 0.75 – 0.85) indicating substantial agreement for the other data element with less than 100% agreement. Sample size is limited. Only numerator/denominator elements assessed other data elements not tested.
Validity – Face validity question reasonable, unclear if measure results were provided with survey, results and interpretation don’t seem aligned and developer should clarify. Additional information needed on characteristics of survey participants.
Equity
Developer indicates data are not available to address this optional criterion.
Use and Usability
Measure appears to be in use in the GAMUT Quality Improvement Collaborative Database. Unclear what proposed accountability application is. Agree with staff assessment that additional information regarding performance over time would lend credibility that measure is actionable.
Summary
Overall additional information needed.
Waveform Capnography in Ventilated Patients:
Importance
Waveform capnography is the standard of care for ensuring safe advanced airway positioning in hospital, and support for using it in transport includes empirical studies and three consensus statements.
Feasibility Acceptance
Waveform capnography is the standard of care for ensuring safe advanced airway positioning in hospitals, and support for using it in transport includes empirical studies and three consensus statements.
Scientific Acceptability
There have been clinical studies to support the need for this practice. Having a measurement/documentation will help to show the value.
Having a measurement will help to show the value to the patient safe transport.
Equity
The measurement is inclusive of all patients who are transported with ventilators
Use and Usability
It is a standard of care in all settings.
Summary
This is or should be a standard of care to ensure the safe transport of patients who require artificial ventilation. A measurement will show the value and improve patient safety and quality of care.
Developers showed the…
Importance
the developers demonstrated the importance of using waveform capnography to recognize dislodgment of airways and to prevent serious complication to patients. The ratio of use in participating center was high so the likelihood for improvement is low. Are the developers' considering compelling more centers to participate?
Feasibility Acceptance
The developers state that the data id collected "manually". Each center collects the data and send it to a collecting center. this can create burden on centers. the data may be inconsistent.
Is there a plan to make data collection automatic.
Scientific Acceptability
No additional comments
No additional comments.
Equity
No equity information is provided by the developers.
Use and Usability
123 neonatal and pediatric programs participated in the project and 100 adults' program. It is not clear to me what the parentage of programs participating from all programs transporting patient in USA. and why other program may not participate. Are the programs participating are larger or more central and therefore it is easier for them to participate? Is compelling other programs to participate may cause burden?
Summary
Developers showed the importance of using capnography while transporting patient with airways. If the programs participating in the project represent most of transportation programs in USA, the measure is not needed because quality rates are already very high. If the group of participating programs are relatively small then the question is if the assumptions the developer states like the ease of collecting data etc. is correct.
need some more info mostly around scientific acceptability
Importance
Based on the submitted information, there is moderate reason to believe that this measure could lead to improved patient outcomes and that there remains a performance gap.
Feasibility Acceptance
This measure is already in routine use by a substantial number of GAMUT users, which is compelling evidence that data is available and able to be captures to permit use of this measure.
Scientific Acceptability
Reliability appears satisfactory, but the provided information doesn’t allow one to assess how reliability at the three facilities that participated in reliability testing is typical of all facilities that may report the measure. The measure steward should explain how those 3 sites are representative of the expected reliability more broadly.
Appears that measure is valid. Disagreement with validity was represented by those who responded “unimportant," or "very unimportant.” “Neutral” may be a concern too. Not sure why those 3 categories were lumped together. Need to know the expertise of the 34 respondents to be certain they are good judges of validity.
Equity
not addressed
Use and Usability
agree with preliminary assessment
Summary
more details about the experimental design would be helpful to interpret the testing results
Waveform Capnography in Ventilated Patients
Importance
Evidence linking CW with desired health outcomes is not provided. Data on opportunity for improvement are not provided. Data showing CW is meaningful to patients are not provided.
Feasibility Acceptance
Information used in the measure comes from electronic sources that reflect care provided. Measure is in use.
Scientific Acceptability
Overall reliability results were good. However, they were based on information from only three facilities and were not broken down by the 3 age groups.
Face validity assessment does not explicitly address whether performance scores resulting from the measure as specified can be used to distinguish good from poor quality. The assessment was systematic, but experts were not identified. More information is needed on how many disagreed and why.
Equity
Currently, other demographic data (beyond age) is not collected. However, at least some such data are available in EHRs. Performance for subpopulation groups may be of particular importance for this measure.
Use and Usability
This measure is already in use. While the developer does cite articles on how the measure performance can be improved, they do not explain/summarize.
Summary
Before endorsing this measure, the developer should provide additional information regarding evidence that links CW with desired health outcomes and some indication that CW is meaningful to patients. They should also provide performance data to show that there is opportunity for improvement. Finally, additional testing for reliability is needed, as is additional assessment of face validity.
Work in progress.
Importance
agree with staff assessment
Feasibility Acceptance
manual abstraction process and data to be compiled by age groups manually
Scientific Acceptability
agree with staff assessment
agree with assessment
Equity
not addressed
Use and Usability
agree with staff assessment
Summary
Work in progress.
Mind the gap
Importance
Although the clinical importance of the procedure is quite clearly demonstrated from the evidence provided by the developer, a performance gap has not been demonstrated. The range of performance scores is very tight, and distribution by deciles/quartiles is not provided.
Feasibility Acceptance
End-user survey data or citation not provided.
Scientific Acceptability
Although reliability scores are strong, as staff noted, reliability was only tested at three sites.
Details on face validity assessment insufficient – we need to understand the rationale for disagreement.
Equity
Not addressed.
Use and Usability
The measure is in use, and a plausible path to improvement for underperforming facilities (essentially just adopting the technology). Nevertheless, change in performance over time would be interesting, given nearly a decade of implementation.
Summary
The most important thing for the developer to demonstrate is a gap in performance, to support broader implementation of the measure.
Strong Measure
Importance
Waveform capnography is the standard of care for ensuring safe advanced airway positioning in hospitals, and support for using it in transport includes empirical studies and three consensus statements. A dislodged artificial airway impacts oxygenation and ventilation and can result in respiratory and cardiac arrest, among other undesirable outcomes. The mortality rate among patients experiencing unplanned extubation ranges from 10% to 25%. Developers assert that this measure is the first of its kind.
Feasibility Acceptance
The most significant challenge in this area is the data capture. Data for this measure are not routinely generated from electronic sources but rather must be manually abstracted by hand from the EHR. This is not so substantial a burden as to preclude measure acceptance, but it is a mild limiting factor.
Scientific Acceptability
The measure is well-defined and precisely specified. Details for measure calculation were clear and thorough.
Waveform capnography is the standard of care for ensuring safe advanced airway positioning in hospitals and is the gold standard used for evaluating other methods.
Equity
Developer did not address this optional criterion.
Use and Usability
Measure is currently in use within the GAMUT database since 2014 for internal QI, and QI with external benchmarking. The developer further indicates current or planned use for public reporting.
Summary
I believe this is a strong measure that will not cause a great reporting burden. This measure appears to be aligning with a workflow that is already in place. The requirement of manual data abstraction is the most significant limitation in my estimation. However, the data definitions are clean and simple which will ease data capture.
Comments on 4045
Importance
Waveform capnography is the standard of care for ensuring safe advanced airway positioning in hospital, and support for using it in transport includes empirical studies and three consensus statements. A dislodged artificial airway impacts oxygenation and ventilation, and can result in respiratory and cardiac arrest, among other undesirable outcomes. The mortality rate among patients experiencing unplanned extubation ranges from 10% to 25% .
Developers assert that this measure is the first of its kind.
Measure performance since 2014 ranges from 89.2% to 95.6% (87.5-94.1% among pediatric patients and 94-97.8% among adults), and may have limited room for improvement for some groups.
Feasibility Acceptance
Required data are routinely generated and used during care, required data are available in EHRs or other electronic sources, and the data collection strategy can be implemented.
Scientific Acceptability
Concerns that the denominator exclusions are for the database manager to decide (no standardization)
Face validity was done with current participants of the program, not an independent group of experts. Concerns with bias, as the participants already have some degree of "buy-in".
Equity
No patient-level data are available for an equity assessment.
Use and Usability
No details were provided on how the measure would be used in an accountability program.
Summary
Comments on 4045
Capnography CBE ID 4045
Importance
It is not clear what qualifies in numerator and what happens if only partially satisfied. (Initiated but not maintained. ) So if there is any tracing ever during transport that qualifies for the numerator? “A patient is either on a machine measuring waveform capnography during medical transport or is not.” But this does not determine for how long they are on it to qualify. The measure, if it will be binary (used not used) shoudl clearly define what is needed to qualified as a "yes" to be in teh numerator.
There is a summary of the lack of the evidence rather than a summary of the evidence supporting. Evidence supporting it will not only examine if it effectively verifies tracheal placement but would also evaluate the feasibility of use in the field. There would also be evidence that capnography improves outcomes hoepfully. There are three citations focused around consensus statements to support the use. There are other citations that focus on neonatal transport use.
There is not a clear relationship in the logic model or the evidence for importance for how “higher score” translates into better quality.
The developer reports there are no codes for this measure. There are likely Medcin (etc) codes that could better define the measure. If the data are extractable from an EHR without manual chart review then there are structured fields for the variables.
The developer suggests patient and provider surveys to demonstrate value. Given the state of the evidence, it may be easier to tie the use of capnography to clinical outcomes. Much of the described importance is about how a policy related to capnography makes people feel better in the abstract. The question for importance is does capnography make care or outcomes better or more efficient.
Feasibility Acceptance
Feasibility is demonstrated through normative evaluations from the consensus developers as well as the quality evaluators. There is no assessment if the tools for capnography are easily available to transport service providers and if they are sufficiently easy to implement for transport providers. In addition, there is no analysis of the challenges to data collection. The developer implies this is through the EHR but also says there are no structured codes for doing this.
Scientific Acceptability
Reliable testing is reported to evaluate if capnography was used “appropriately” the proposed measure purports to evaluate whether it is “initiated” or “maintained”. The interrater reliability should look at whether the reported data and the audited data were in agreement not whether the use of capnography was "appropriate".
Validity was assessed based on face validity. This was done through a survey or GAMUT programs. Are there other stakeholders beyond the quality organization that would determine validity?
No risk adjustment was done. There may be a benefit to risk adjust based on program characteristics like geographic location, ownerships status, volume.
Equity
This is an optional measure. The developer states there is no way to evaluate equity based on individual patient data. While that may be true, equity could be evaluated based on geographic location of provider, ownership status of provider, or characteristics of the population served. For example, is capnography the same for a NICU health system owned transport company as a rural ambulance service transferring between hospitals?
Use and Usability
There is no clear description of planned uses within usability. In other sections usability and use seems to be focused on consensus statements.
Summary
This is an interesting topic and may be valuable as a measure of quality. The current application is clinically persuasive about the consensus to use waveform capnography but has not sufficiently defined this as a reportable, measurable, and usable quality metric.
Important measure
Importance
This is an important measure to assess that the standard of care is met in safe airway management in transport. The measure is simple and straightforward and has been in use since 2014. The fact that it has high performance rates already may indicate that it has limited room for improvement, but it is important to have the standard in place.
Feasibility Acceptance
More details could be provided about feasibility testing.
Scientific Acceptability
Agree that more facilities could be included in reliability testing.
The interpretation of the survey results for face-value validity could be clarified.
Equity
Equity does not seem applicable since individual patient data are not collected.
Use and Usability
The measure has been in use without issue for the better part of a decade.
Summary
This is a straightforward measure on an important practice that has been field tested for many years. I recommend endorsement.
Ventilated patients
Importance
It is unclear of the importance of this measure to patients although it can be interpreted.
Feasibility Acceptance
Agree with staff assessment
Scientific Acceptability
In agreement with staff assessment
In agreement with staff assessment
Equity
Not addressed by the developer as this was optional.
Use and Usability
In agreement with staff assessment.
Summary
Agree overall
Public Comments