This measure estimates a hospital-level risk standardized mortality rate (RSMR) in adult patients without cardiogenic shock or cardiac arrest undergoing PCI. The outcome is defined as in-hospital mortality following a PCI procedure performed during the episode of care. Mortality is defined as death for any cause during the episode of care.
Measure Specs
- General Information(active tab)
- Numerator
- Denominator
- Exclusions
- Measure Calculation
- Supplemental Attachment
- Point of Contact
General Information
Measuring patient mortality following a percutaneous coronary intervention (PCI) provides valuable information on a facility’s performance that can be used for benchmarking against the national aggregate and other facilities with similar volumes. These data can assist those facilities with high mortality rates in engaging in quality improvement activities and can be used for clinical decision-making and inform patients as they make decisions regarding their health and healthcare.
The American College of Cardiology developed and iterated on several versions of a mortality model over the years and received endorsement for a measure using one that accounted for patients undergoing high-risk PCI in 2013. That measure was in use within the National Cardiovascular Data Registry (NCDR) CathPCI Registry since 2013[HB1] . Due to ongoing concerns that the most recent model did not adequately account for patients at extreme risk or facilities with lower volumes and therefore hospitals with larger numbers of these individuals would perform more poorly on mortality, ACC integrated additional variables (e.g., frailty, cardiovascular instability) into the CathPCI registry and subsequently developed a new hierarchical mortality model (Castro-Dominguez, 2021; Boyden, 2015; McCabe, 2016; Hannan, 2017). This model uses these new variables, accounts for case mix and hospital volume, and improves on our ability to predict the risk of mortality across different cohorts.
This hierarchical model includes variables that evaluate the patient’s level of consciousness following cardiac arrest and refractory cardiogenic shock, which may classify her or him as extreme risk. The updates to this measure now focus on those patients who do not fall within this risk category and therefore minimizes the risk of penalizing clinicians and facilities who are willing to provide care for these individuals who are more likely to have poorer outcomes unrelated to the care that they subsequently receive at the facility.
As a result of this updated model, facilities have access to data that better classify a patient’s mortality risk and allows in-depth analyses of the causes behind variations in mortality during or post PCI leading to the identification of best practices. In addition, detailed case reviews can identify clinicians with poorer performance for whom additional training or reduced caseloads could be considered. Active dissemination of those best practices and support to enable their adoption will improve outcomes and reduce variations in clinical practice. Improvements in the quality of care resulting from the evaluation of the risk for mortality, before and after implementing quality improvement interventions, can enable facilities to quantify their improved outcomes with respect to peri-procedural mortality and a reduction in cost associated with these events. Additionally, by putting the responsibility for improved quality in the hands of physicians and other healthcare providers, this updated risk standardized mortality measure engages the medical community around the common goal of better healthcare value.
References:
Boyden TF, Joynt KE, McCoy L, Neely ML, Cavender MA, Dixon S, Masoudi FA, Peterson E, Rao SV, Gurm HS. Collaborative quality improvement vs public reporting for percutaneous coronary intervention: A comparison of percutaneous coronary intervention in New York vs Michigan. Am Heart J. 2015 Dec;170(6):1227-33. doi: 10.1016/j.ahj.2015.09.006. Epub 2015 Sep 16. PMID: 26678645; PMCID: PMC6948714.
Castro-Dominguez YS, Wang Y, Minges KE, et al. Predicting In-Hospital Mortality in Patients Undergoing Percutaneous Coronary Intervention. J Am Coll Cardiol. 2021 Jul 20;78(3):216-229. doi: 10.1016/j.jacc.2021.04.067. Epub 2021 May 3. PMID: 33957239.
Hannan EL, Zhong Y, Cozzens K, Gesten F, Friedrich M, Berger PB, Jacobs AK, Walford G, Ling FS, Venditti FJ, King SB 3rd. The Impact of Excluding Shock Patients on Hospital and Physician Risk-Adjusted Mortality Rates for Percutaneous Coronary Interventions: The Implications for Public Reporting. JACC Cardiovasc Interv. 2017 Feb 13;10(3):224-231. doi: 10.1016/j.jcin.2016.10.040. PMID: 28183462.
McCabe JM, Waldo SW, Kennedy KF, Yeh RW. Treatment and Outcomes of Acute Myocardial Infarction Complicated by Shock After Public Reporting Policy Changes in New York. JAMA Cardiol. 2016 Sep 1;1(6):648-54. doi: 10.1001/jamacardio.2016.1806. PMID: 27463734.
National Cardiovascular Data Registryâ (NCDR) CathPCI Registry
Numerator
The outcome for this measure is in-hospital mortality following a PCI procedure performed during the episode of care.
The measure includes all deaths for any cause that occur during the hospital admission of the PCI procedure.
- All-cause in-hospital: Discharge status of deceased
Denominator
The target population for this measure is patients aged 18 years of age or older with a PCI procedure during the episode of care (excluding patients with pre-procedure cardiogenic shock or cardiac arrest).
To be included in the measure cohort, patients must meet the following inclusion criteria:
- Aged 18 or over (Note: Patients under 18 are not included in this measure as the NCDR CathPCI Registry only collects data on adult patients)
- Not transferred to another facility after the index procedure
Patients are included in one of four mutually exclusive categories in decreasing order of procedural urgency and mortality risk:
1) Cardiovascular instability (CVI) (includes hemodynamic instability, acute heart failure symptoms and ventricular arrhythmia in the absence of cardiogenic shock) without salvage
a) Cardiovascular instability type = hemodynamic instability (not cardiogenic shock) OR acute heart failure symptoms OR ventricular arrhythmia
AND NOT Cardiovascular instability type = cardiogenic shock
AND NOT PCI status = salvage
b) Cardiovascular instability type = acute heart failure symptoms
AND NOT Cardiovascular instability type = cardiogenic shock
AND NOT PCI status = salvage
c) Cardiovascular instability type = ventricular arrhythmia
AND NOT Cardiovascular instability type = cardiogenic shock
AND NOT PCI status = salvage
2) Emergency PCI without shock or CVI
a) PCI status = emergency
AND NOT Cardiovascular Instability Type = hemodynamic instability, acute heart failure symptoms, or ventricular arrhythmia or cardiogenic shock or refractory cardiogenic shock
3) Urgent PCI without shock or CVI
a) PCI status = urgent
AND NOT Cardiovascular Instability Type = hemodynamic instability, acute heart failure symptoms, or ventricular arrhythmia or cardiogenic shock or refractory cardiogenic shock
4) Elective PCI without shock or CVI
a) PCI status = elective
AND NOT Cardiovascular Instability Type = hemodynamic instability, acute heart failure symptoms, or ventricular arrhythmia or cardiogenic shock or refractory cardiogenic shock
Exclusions
This mortality measure excludes the following:
1) Patients in cardiogenic shock
2) Patients resuscitated from cardiac arrest that occurred either: 1) outside of the healthcare facility prior to arrival; 2) while being transferred to the facility; or 3) while at the facility
3) Patients transferred to another facility or “Extended care/transitional care unit/Rehab” or patients that left “against medical advice”
The measure has the following exclusions:
1) Patients in cardiogenic shock
a) Cardiovascular instability type = cardiogenic shock OR refractory cardiogenic shock
2) Patients resuscitated from cardiac arrest
a) Responsive or unresponsive patients with cardiac arrest that occurred:
Out of a healthcare facility prior to arrival,
OR while being transferred to a facility,
OR while at the facility
3) Transferred to another facility after the index procedure or “Extended care/transitional care unit /Rehab” or patients that left “against medical advice”
Measure Calculation
This measure uses predictive variables to estimate in-hospital mortality following PCI using a hierarchical risk model. The approach simultaneously models data at the patient and hospital levels to account for variance in patient outcomes within and between hospitals [Normand and Shahian, 2007]. At the patient level, it models the log-odds of mortality within 30 days of index admission using age, sex, selected clinical covariates, and a hospital-specific intercept. At the hospital level, it models the hospital-specific intercepts as arising from a normal distribution. The hospital intercept represents the underlying risk of a mortality at the hospital, after accounting for patient risk. The hospital-specific intercepts are given a distribution to account for the clustering (non-independence) of patients within the same hospital. If there were no differences among hospitals, then after adjusting for patient risk, the hospital intercepts should be identical across all hospitals.
The RSMR is calculated as the ratio of the number of “predicted” to the number of “expected” deaths at a given hospital. For each hospital, the numerator of the ratio is the number of deaths following a PCI procedure predicted on the basis of the hospital’s performance with its observed case mix, and the denominator is the number of deaths expected based on the nation’s performance with that hospital’s case mix.
This approach is analogous to a ratio of “observed” to “expected” used in other types of statistical analyses. It conceptually allows for a comparison of a particular hospital’s performance given its case mix to an average hospital’s performance with the same case mix. Thus, a lower ratio indicates lower-than-expected mortality rates or better quality, and a higher ratio indicates higher-than-expected mortality rates or worse quality. The “predicted” number of deaths (the numerator) is calculated by using the coefficients estimated by regressing the risk factors and the hospital-specific intercept on the risk of mortality. The estimated hospital-specific intercept is added coefficients multiplied by the patient characteristics. The results are transformed and summed over all patients attributed to a hospital to get a predicted value. The “expected” number of deaths (the denominator) is obtained in the same manner, but a common intercept using all hospitals in our sample is added in place of the hospital-specific intercept. The results are transformed and summed over all patients in the hospital to get an expected value. To assess hospital performance for each reporting period, we re-estimate the model coefficients using the years of data in that period.
This calculation transforms the ratio of predicted over expected into a rate that is compared to the national observed readmission rate. The hierarchical logistic regression models are described fully in the original methodology report posted on QualityNet [https://qualitynet.org/inpatient/measures/mortality/methodology].
References:
Normand S-LT, Shahian DM. 2007. Statistical and Clinical Aspects of Hospital Outcomes Profiling. Stat Sci 22(2): 206-226.
Krumholz H, Normand S, Galusha D, et al. Risk-Adjustment Models for AMI and HF 30-Day Mortality Methodology. 2005.
This measure is not stratified.
No minimum sample size is required.
Supplemental Attachment
Point of Contact
N/A
United States
Kathryn Goodwin
ACC
United States
Importance
Evidence
In-patient mortality following percutaneous coronary intervention is an important outcome and lower risk-standardized mortality rates (RSMRs) can be achieved if clinicians and facilities have the proper structures and processes in place. For example, there are opportunities to improve the health of individuals and of populations by better coordination of all aspects of medical care, and by assessing and responding to each individual's health risks. This goal of individualized care can be achieved for patients requiring coronary artery revascularization with the use of validated risk models that identify PCI related risk factors and accurately quantify the clinical and procedural risks such as the one used in this measure.
In addition, clinical acuity is a strong predictor of PCI procedural mortality. With inclusion of variables that further characterize clinical stability, the updated CathPCI Registry mortality model remains current and well-calibrated across the spectrum of PCI risk (Castro-Dominguez, 2021).
Treatment determinations that are supported by clinical guidelines must be made based on the individual patient’s risk factors and choice (Lawton, 2022; Virani, 2023). One such example is the appropriate use criteria (AUC) for coronary revascularization, which are tailored to the specific characteristics of individual patients. The evaluation of AUC covers broader array of specific conditions, sometimes hundreds for a given test or treatment decision, to encompass the majority of practice situations. Appropriateness relate to individual patient demographic characteristics, clinical history, risk scores, and/or symptoms and signs (Patel, 2017a; Patel 2017b). Another example is ensuring that the access site is appropriately selected as identified in a meta-analysis of nine studies of patients with ST-elevation myocardial infarction (STEMI) as this decision can significantly decrease death and other outcomes (e.g., major access site complications) (Mamas, 2012). The COMPLETE trial demonstrated that while FFR- and angiography-guided strategies did not affect PCI outcomes, mortality was positively impacted if the surgeon performed complete revascularization versus culprit-lesion-only (Bainey, 2020).
Facilities can also implement various structures and processes to further decrease the risk of inpatient deaths such as ensuring that there is adequate nurse staffing when caring for patients receiving PCI, particularly in ICUs and implementing the heart team approach (Kim, 2020; Masters, 2017).
Upon consideration of associated risk factors and evaluation of guidelines and appropriateness, and structures and processes that can be implemented by facilities, coronary artery reperfusion improves clinical outcomes for patients.
References
Bainey KR, Engstrøm T, Smits PC, Gershlick AH, James SK, Storey RF, Wood DA, Mehran R, Cairns JA, Mehta SR. Complete vs Culprit-Lesion-Only Revascularization for ST-Segment Elevation Myocardial Infarction: A Systematic Review and Meta-analysis. JAMA Cardiol. 2020 Aug 1;5(8):881-888. doi: 10.1001/jamacardio.2020.1251. PMID: 32432651; PMCID: PMC7240651.
Castro-Dominguez YS, Wang Y, Minges KE, et al. Predicting In-Hospital Mortality in Patients Undergoing Percutaneous Coronary Intervention. J Am Coll Cardiol. 2021 Jul 20;78(3):216-229. doi: 10.1016/j.jacc.2021.04.067. Epub 2021 May 3. PMID: 33957239.
Kim Y, Kim J. In-Hospital Mortality in Patients Receiving Percutaneous Coronary Intervention According to Nurse Staffing Level: An Analysis of National Administrative Health Data. Int J Environ Res Public Health. 2020;17(11):3799. Published 2020 May 27. doi:10.3390/ijerph17113799
Mamas MA, Ratib K, Routledge H, et al. Influence of access site selection on PCI-related adverse events in patients with STEMI: meta-analysis of randomised controlled trials. Heart. 2012 Feb;98(4):303-11. doi: 10.1136/heartjnl-2011-300558. Epub 2011 Dec 6. PMID: 22147900.
Lawton JS, Tamis-Holland JE, Bangalore S, et al. 2021 ACC/AHA/SCAI guideline for coronary artery revascularization: a report of the American College of Cardiology/American Heart Association Joint Committee on Clinical Practice Guidelines. J Am Coll Cardiol. 2022;79:e21-e129
Masters J, Morton G, Anton I, et al. Specialist intervention is associated with improved patient outcomes in patients with decompensated heart failure: evaluation of the impact of a multidisciplinary inpatient heart failure team. Open Heart. 2017 Mar 8;4(1):e000547. doi: 10.1136/openhrt-2016-000547. PMID: 28409010; PMCID: PMC5384462.
Patel MR, Calhoon JH, Dehmer GJ, Grantham JA, Maddox TM, Maron DJ, Smith PK. ACC/AATS/AHA/ASE/ASNC/SCAI/SCCT/STS 2016 Appropriate Use Criteria for Coronary Revascularization in Patients With Acute Coronary Syndromes: A Report of the American College of Cardiology Appropriate Use Criteria Task Force, American Association for Thoracic Surgery, American Heart Association, American Society of Echocardiography, American Society of Nuclear Cardiology, Society for Cardiovascular Angiography and Interventions, Society of Cardiovascular Computed Tomography, and the Society of Thoracic Surgeons. J Am Coll Cardiol. 2017 Feb 7;69(5):570-591. doi: 10.1016/j.jacc.2016.10.034. Epub 2016 Dec 21. PMID: 28012615.
Patel MR, Calhoon JH, Dehmer GJ, Grantham JA, Maddox TM, Maron DJ, Smith PK. ACC/AATS/AHA/ASE/ASNC/SCAI/SCCT/STS 2017 Appropriate Use Criteria for Coronary Revascularization in Patients With Stable Ischemic Heart Disease: A Report of the American College of Cardiology Appropriate Use Criteria Task Force, American Association for Thoracic Surgery, American Heart Association, American Society of Echocardiography, American Society of Nuclear Cardiology, Society for Cardiovascular Angiography and Interventions, Society of Cardiovascular Computed Tomography, and Society of Thoracic Surgeons. J Am Coll Cardiol. 2017 May 2;69(17):2212-2241. doi: 10.1016/j.jacc.2017.02.001. Epub 2017 Mar 10. Erratum in: J Am Coll Cardiol. 2018 Apr 13;: PMID: 28291663.
Virani SS, Newby LK, Arnold SV, et al. 2023 AHA/ACC/ACCP/ASPC/NLA/PCNA guideline for the management of patients with chronic coronary disease: a report of the American Heart Association/American College of Cardiology Joint Committee on Clinical Practice Guidelines. J Am Coll Cardiol. 2023;82:833-955.
Measure Impact
This measure was developed with input from a technical expert panel that includes patient and caregiver representation. As regaining and preserving health is likely the goal of most patients, in-hospital mortality as an outcome is easily understood and will help patients in understanding quality of care across facilities and in making decisions about where receive their care.
Performance Gap
The tables below illustrate the distribution of the risk-standardized mortality rates by two time periods of observation: 2022 (Table 1) and 2021 (Table 2 attached). Included is the mean score, entities (or hospitals), and total persons (or patients), all evaluated by decile of performance from data collected from abstractors and reported to the CathPCI Registry. As illustrated in Table 1, with the more contemporaneous data, the minimum RSMR was 0.36% whereas the maximum score was 2.82%, suggesting a wide gap in performance. Further, comparing those sites with the lowest and highest deciles of performance using the 2022 data, 0.57% vs. 1.40%, respectively, demonstrates over a 1% difference in mortality rates. While 1% may not appear to be a considerable gap, it translates to an additional 6,540 deaths per year, justifying the importance of capturing and reporting these data. The distribution of performance across the 2021 year of data yielded similar results.
| Overall | Minimum | Decile_1 | Decile_2 | Decile_3 | Decile_4 | Decile_5 | Decile_6 | Decile_7 | Decile_8 | Decile_9 | Decile_10 | Maximum | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Mean Performance Score | 0.0088 | 0.0036 | 0.0057 | 0.0067 | 0.0073 | 0.0077 | 0.0081 | 0.0085 | 0.0091 | 0.0098 | 0.0109 | 0.0140 | 0.0282 |
| N of Entities | 1669 | 1 | 166 | 167 | 167 | 167 | 167 | 167 | 167 | 167 | 167 | 167 | 1 |
| N of Persons / Encounters / Episodes | 654127 | 725 | 110417 | 71836 | 57018 | 46113 | 43149 | 59815 | 55082 | 63544 | 61616 | 85537 | 247 |
Equity
Equity
We examined variation in hospital performance for the measure based on overall performance, and stratified by subgroups of age, sex, race/ethnicity, and proportion of patients insured through Medicaid to identify if there were any meaningful differences in social risk.
In terms of the overall distribution, the median risk standardized rate of mortality was 0.83%, with an interquartile range of 0.73% to 0.98% (Table 14). Although most data are centered around the median, there is a slight right-skew to the mortality rates (Figure 11).
Subgroups
Across stratified analysis based on age, sex, patients insured through Medicaid, and proportion of non-white patients, we found significant overlap in the distribution of hospital performance, as detailed below.
Age
Hospitals (n=1,669) were stratified into quartiles by their proportion of patients over the age of 65 (median: 58.76%, IQR: 53.71% to 63.93%). Hospital performance was similar across hospitals stratified by quartile based on age (Table 15, Figure 12).
Sex
The median percent of female patients was 30.74% (IQR: 27.66% to 33.90%), and hospitals performed similarly in RSMR across quartiles of proportion female (Table 16, Figure 13).
Proportion on Non-White
The median proportion of non-white patients was 12.16% (IQR: 5.46% to 23.18%). Hospital performance across quartiles was similar regardless of the proportion of non-white patients treated, with median performance scores ranging from 0.82% (Q, lowest proportion of non-white patients) to 0.85% (Q4, highest proportion on non-white patients) (Table 17, Figure 14).
Insurance
Hospitals (n=1,669) were stratified into quartiles by their proportion of patients with Medicaid as the primary insurance (median: 11.60%, IQR: 6.80% to 17.41%). Hospital performance was similar across hospitals stratified by quartile based the proportion of patients with Medicaid insurance coverage (Table 18, Figure 15).
Feasibility
Feasibility
The data elements required to generate this measure are abstracted from a medical record by someone other than person obtaining original information (e.g., chart abstraction for quality measure or registry). All data elements are available in defined fields in electronic clinical data (e.g., clinical registry). This measure uses clinical data from the NCDR CathPCI Registry.
Availability:
Participating hospitals report patient demographics, medical history, risk factors, hospital presentation, initial cardiac status, procedural details, medications, laboratory values and in-hospital outcomes as the key activity of participating in the NCDR CathPCI registry. The majority of the 21 required data elements are routinely generated and acquired during the delivery of standard cardiac care to this patient population. Electronic extraction of data recorded as part of the procedure expedites data collection. This strategy offers point of care data collection and minimizes time and cost. Institutions can manually report using a free web-based tool or automate the reporting by using certified software developed by third-party vendors. The data elements required for this measure are readily available within the patient’s medical record or can be attained without undue burden by the hospital. Most data elements exist in a structured format within patient’s electronic health record.
Sampling:
There is no sampling of patient data allowed within the contractual terms of participation in the NCDR CathPCI Registry. The registry is designed to include 100 percent of consecutive adult patients who undergo PCI at participating institutions. Section 2.b of the NCDR Master Agreement with participants includes ‘Participant Responsibilities’: “b. Use of ACCF Data Set and ACCF-Approved Software. Participant will submit a data record on each patient who receives medical care and who is eligible for inclusion in the Registries in which Participant is participating under this Agreement.” Adult patients, ages 18 years and older, who undergo a diagnostic cardiac catheterization and/or PCI. Eligible diagnostic catheterizations are characterized by the passage of a catheter into the aortic root for pressure measurements and/or angiography, and can include Left Ventricle (LV) pressure measurements, LV angiography, coronary angiography, and coronary artery bypass angiography. Eligible PCI procedures include those that involve passage or attempted passage of a coronary device across one or more coronary lesions for purposes of increasing the intraluminal diameter of the vessel and/or restoring or improving circulation. Patients are selected for inclusion by reviewing existing medical records and no direct interaction with the patient is required outside of the normal course of care. There is no discrimination or bias with respect to inclusion on the basis of sex, race, or religion.
Patient confidentiality:
The CathPCI Registry dataset was created by a panel of experts using available ACC-AHA guidelines, data elements and definitions, and other evidentiary sources. Protected Health Information (PHI) as such term is defined by the Health Insurance Portability and Accountability Act of 1996 (HIPAA), such as social security number, is collected. The intent for collection of PHI is to allow for registry interoperability and the potential for future generation of patient-level drill downs in Quality and Outcomes Reports. Registry sites can opt out of transmitting direct identifiers to the NCDR, enabling inclusion of direct identifiers in the registry to be at the discretion of the registry participants themselves. When using the NCDR web-based data collection tool, direct identifiers are entered but a partition between the data collection process and the data warehouse maintains the direct identifiers separate from the analysis datasets. The minimum level of PHI transmitted to the ACCF when a participant opts out of submitting direct identifiers meets the definition of a Limited Data Set as such term is defined by HIPAA. All analyses are performed by contracted data analytic centers who conduct such analysis on a Limited Data Set.
Data collection within the NCDR conforms to laws regarding PHI Each participant signs a Business Associate Agreement with the ACCF permitting the use and disclosure of PHI. Patient confidentiality is of utmost concern with all metrics. The proposed measure does not currently include a patient survey. There is no added procedural risk to patients through involvement in the CathPCI Registry. No testing, time, risk, or procedures beyond those required for routine care are imposed. The primary risk associated with this measure is the potential for a breach of patient confidentiality. The ACCF has established a robust plan for ensuring appropriate and commercially reasonable physical, technical, and administrative safeguards are in place to mitigate such risks.
Data are maintained on secure servers with appropriate safeguards in place. The project team periodically reviews all activities involving PHI to ensure that such safeguards including standard operating procedures are being followed. The procedure for notifying the ACCF of any breach of confidentiality and immediate mitigation standards that need to be followed are communicated to participants. ACCF limits access to PHI , and to equipment, systems, and networks that contain, transmit, process or store PHI, to employees who need to access the PHI for purposes of performing ACCF’s obligations to participants who are in a contractual relationship with the ACCF. All PHI are stored in a secure facility or secure area within ACCF’s facilities which has separate physical controls to limit access, such as locks or physical tokens. The secured areas are monitored 24 hours per day, 7 days per week, either by employees or agents of ACCF, by video surveillance, or by intrusion detection systems.
Each participant who has access to the NCDR website must have a unique identifier. The password protected webpages have implemented inactivity time-outs. Encryption of wireless networks, data transmission and authentication of wireless devices containing NCDR Participant’s information is required. PHI may only be transmitted off of ACCF’s premises to approved parties, which shall mean: A subcontractor who has agreed to be bound by the terms of the Business Associate Agreement between the ACCF and the NCDR participant.
Overall there is no added procedural risk to patients through their hospital’s involvement in the CathPCI Registry. No testing, time, risk, or procedures beyond those required for routine care will be imposed.
Please see sections 1.10 and 3.1.
Proprietary Information
Scientific Acceptability
Testing Data
We used a clinical registry, the National Cardiovascular Data Registry (NCDR) for CathPCI Registry, to support this measure. This is a national quality improvement registry with >1,700 US hospitals participating. Some states and healthcare systems mandate participation. Rigorous quality standards are applied to the data and both quarterly and ad hoc performance reports are generated for participating centers to track and improve their performance.
Dates of testing for scientific acceptability: January 1, 2022 – December 31, 2022
Empirical validity was tested using endorsed measure 0964: CathPCI Discharge Medications Quality Composite for the corresponding time period, January 1, 2022 – December 31, 2022.
While all of the data used for testing was derived from the CathPCI Registry, the analysis to examine trends in data were conducted using 2021 and 2022 data to provide two time periods of observation.
Analyses that assessed reliability used 2021 and 2022 data and validity only used 2022 data.
Empirical validity was tested using data from the endorsed measure #0964 CathPCI Discharge Medications also using the 2022 data for comparison purposes.
We did not use a sample of data for this measure; rather, we include all available data meeting inclusion criteria for the 2022 calendar year (n=1669 hospitals). Please refer to Table 3 for a description of hospital characteristics.
For the updated derivation and validation of the mortality risk model, 654,127 patients undergoing PCI between 1/1/2022-12/31/2022 at 1,669 hospitals were included: 70% in the derivation cohort and a random 30% in the validation cohort. The median in-hospital mortality rate was 0.85%. A summary of these patients’ clinical characteristics stratified by validation and derivation cohorts are provided under Tables 4. A summary of these patients’ clinical characteristics focused on those predictor variables in the all-patients cohort excluding cardiac arrest/cardiogenic shock are provided under Table 5.
Reliability
We conducted two types of reliability testing.
First, we performed a signal-to-noise ratio (SNR) reliability test to assess the quality of a signal (mortality) by comparing the strength of the desired signal to the level of background noise present. A higher SNR indicates a stronger signal relative to the noise, which suggests better reliability and accuracy in signal detection and processing.
Second, we pursued a split sample methodology to assess the consistency or reliability of the measure by dividing it into two halves and comparing the results obtained from each half. We also included performance across 2021 and 2022 time frames for additional reliability testing. For the performance rates, raw rates were calculated, and a correlation coefficient was computed. The split samples were calculated during the same timeframe to mitigate confounding factors based on time differences. The cohort was split into two random samples to compare measure scores. The type of error tested by a split-sample reliability test is primarily related to the consistency or stability of measurements obtained from the measure. This test helps identify errors or sources of variability that may affect the reliability of the measurement process, ensuring that the measure results are trustworthy and replicable.
Signal to Noise analysis for the hospitals are noted under Table 6.
Results from the split sample methodology are available in the Tables 7-8 and Figures 1-4.
Signal to Noise Analysis:
The signal to noise ratio analysis measures the confidence levels in differentiating performance between hospitals. Our analyses (Table 6) found the median SNR was 0.95 and had a fairly narrow interquartile range of 0.91 and 0.97. These numbers demonstrate variability that is attributable to real differences in hospital quality as opposed to measurement error. Collectively, we believe that the data strongly support the reliability of the data elements used in the model.
(Reference: Landis J, Koch G, The measurement of observer agreement for categorical data, Biometrics, 1977;33:159-174.)
Split Sample Methodology:
The box and whisper plot of the distribution of hospital performance for the CathPCI In-Hospital Mortality Measure in 2021 and 2022 (Figures 1 and 3) show a similar distribution of use of the risk standardized mortality rates for both split samples. Figures 2 and 4 show the scatterplot of the distribution of hospital performance when assessed in randomly split samples. Overall hospital performance in one random sample was correlated with hospital performance in the other split sample (r=0.27821, 0.28253), for 2021 and 2022 respectively (Tables 7 and 8), which is consistent with a reliable measure.
Validity
Empirical analysis was tested by determining if hospitals performed similarly on the composite discharge medications measure and in-hospital mortality. The testing focused on construct validation which tested the hypothesis that following the provision of discharge medications for patients who underwent a PCI may lead to better short-term outcomes. This was achieved by examining the distribution and correlation of the discharge medications composite score and the in-hospital risk-standardized mortality rates (RSMR) for PCI. We used the endorsed CathPCI Discharge Medications composite measure (#0964). For this specific analysis, the study period was 1/1/2022 to 12/31/2022. A correlation coefficient was calculated to demonstrate the correlation between the mortality measure and the process of care measure. Missing data were very low (<1%) and imputation was used if data were missing. Of note, some hospitals (n=21) lacked data pertaining to scores for the CathPCI Discharge Medications measure, and thus were excluded from empirical analyses.
Table 9 and Figure 5 displays the results achieved from the empirical validity testing.
The median rate of delivering defect free care was 96.3% (IQR: 92.7% to 98.2%), and the median in-hospital mortality rate was 0.83% (IQR: 0.73% to 0.98%) (Table 9, Figure 5). There was a similar distribution of hospitals by volume across both measures. The negative correlation coefficient was significant and in the hypothesized direction, such that a higher group of patients receiving discharge medications was associated with lower mortality rates. In summary, the empirical validation demonstrates there is a relationship, albeit statistically a small one, between discharge medications and in-hospital mortality.
Risk Adjustment
Rationale: Appropriate risk adjustment is necessary to prevent potential risk-adverse behaviors that may negatively affect patients who are at highest risk, particularly those with cardiogenic shock and cardiac arrest, who may benefit the most from revascularization.
Approach to Risk Model Development: Prior to the development of the new model, the NCDR defined new variables that could potentially be incorporated into the new model. Specifically, as outlined in the Castro-Dominguez article (2021), the v5 data collection form (DCFv5) integrated a series of new variables that further characterize patients’ clinical status. For example, to better characterize cardiovascular instability, new variables included ventricular arrhythmias, acute heart failure symptoms, hemodynamic instability without cardiogenic shock, cardiogenic shock, and refractory cardiogenic shock (defined as persistent hypotension despite mechanical or pharmacologic vasopressor support). Additionally, a composite ordinal variable was created combining the components of cardiovascular instability with the procedural status, assigned into 6 mutually exclusive categories in decreasing order of procedural urgency and mortality risk: 1) salvage PCI or refractory shock; 2) cardiogenic shock (not refractory) without salvage; 3) cardiovascular instability [CVI] (includes hemodynamic instability, acute heart failure symptoms, and ventricular arrhythmia in the absence of shock) without salvage; 4) emergency PCI without shock or CVI; 5) urgent PCI without shock or CVI; and 6) elective PCI without shock or CVI.
The new frailty variable included in DCFv5 was based on the Canadian Study of Health and Aging clinical frailty scale. Patients were classified as nonfrail, intermediately frail (mild and moderate frailty), and severely frail (severe, severely frail, and terminally ill). Per the data definitions for DCFv5, frailty was based on the clinical condition prior to the start of the procedure, which could lead to patients presenting with cardiac arrest, cardiogenic shock, or salvage being coded as severely frail irrespective of their baseline status before admission. For purposes of the model, only those patients without cardiac arrest, shock, or undergoing salvage PCI were eligible to considered as severely frail and were compared with all other patients (non-severe frailty).
A new variable that captured level of consciousness at start of PCI in patients who have suffered cardiac arrest was also incorporated. Patients were categorized as unresponsive if they were not responsive to verbal or painful stimuli or if their level of consciousness was unable to be assessed (e.g., patients who are intubated and sedated). In addition, surgical evaluation prior to PCI was also integrated as new variable. Patients were considered to be a surgical turndown in those cases in which a cardiac surgical consult was obtained before engaging in PCI but surgery was not recommended. Aortic stenosis severity as an indication for cath lab visit was also a newly collected variable. The definitions for number of diseased vessels have been updated to include not only angiographically significant stenosis, but also fractional flow reserve and instantaneous wave-free ratio values indicative of ischemia. Finally, estimated glomerular filtration rate (GFR) was calculated based on the Chronic Kidney Disease Epidemiology Collaboration equation. Chronic kidney disease was classified according to latest guideline-recommended definition: stage 3a, GFR 45 to 60 ml/min/1.73 m2; stage 3b, GFR 30 to 44 ml/min/1.73 m2; stage 4, GFR 15 to 29 ml/min/1.73 m2; stage 5, GFR <15 ml/min/1.73 m2 or dialysis. The full definitions of the data elements in the registry are available on the NCDR website.
Correspondingly, the NCDR established a Risk-Standardized Mortality work group of physicians and research scientists to oversee model development and provide input on variable selection and considerations for the model. Candidate variables were screened and selected by the workgroup based on their clinical relevance, association with outcomes from prior research, and importance in model development.
For final variable selection, bootstrap analysis was performed. First, the development sample
was used to create 1,000 “bootstrap” samples. For each sample, we ran a logistic regression that included the candidate variables using stepwise selection method (entry = 0.0005, exit = 0.0001). We then calculated the percentage of times each of the variables was selected in each of the
1,000 samples. The variables that were selected in at least 70% of bootstrap samples were then included in the final model. All clinical variables that had been identified a priori as being clinically relevant met this threshold except patients turned down for surgery. Given that this variable represents a unique population that may be clustered at certain facilities and high-risk patients with limited treatment options, it was forced into the final model.
Reference:
Castro-Dominguez YS, Wang Y, Minges KE, et a., Predicting In-Hospital Mortality in Patients Undergoing Percutaneous Coronary Intervention. J Am Coll Cardiol. 2021 Jul 20;78(3):216-229. doi: 10.1016/j.jacc.2021.04.067. Epub 2021 May 3. PMID: 33957239.
Social risk factors were not used in this risk model for the following reasons:
While proxy variables could be considered, these were not believed to be relevant to an inpatient mortality model, in contrast to a longer-term outcome model where difficulties with access to care, affording medications or cardiac rehabilitation would be more important. Moreover, while it may be true that worse social risk factors might be associated with more severe illness at the time of presentation, we had direct access to detailed clinical variables describing the severity of illness and believe that incorporating such factors (e.g. severe frailty, clinical instability, LVEF, etc.) is a much more accurate means of stratifying risk. Accordingly, given the rich clinical data available through the NCDR CathPCI registry, social risk factors would not likely contribute much improvement to this particular risk model, which exhibits excellent goodness of fit.
Table 10 includes a description of hospital and patient characteristics. For example, this illustrates the number of patients that represent diverse race/ethnicity is 21.8%.
Please refer to Tables 11 & 12 for the specifications, coefficients, codes with descriptors and definitions. Additionally, the statistical risk model was developed accordingly: Graphical functions were evaluated for all continuous variables to test for a linear relationship with mortality. For non-linear relationships the variable was transformed using spline functions. Extreme values for continuous variables were set to outer limits based on clinical judgment. A multivariate logistic regression model linking mortality to the selected variables was fitted. To account for the natural clustering of observations within hospitals, a hierarchical logistic regression model was fitted, linking mortality to the selected variables with a hospital-specific random effect. Hospital-specific risk standardized mortality rates (RSMRs) for each hospital were calculated using the regression coefficients from the hierarchical model. RSMRs were obtained as the ratio of hospital-specific predicted mortality to the hospital-specific expected mortality, multiplied by the mortality rate in the study cohort. The expected number of deaths for each hospital was calculated by summing over the predicted mortality risks for all patients in the hospital using the average of all hospital specific intercepts, and the predicted number of deaths was calculated in the same manner but using an estimated intercept that is specific for that hospital. This ratio was then multiplied by the mortality rate in the study cohort to calculate the RSMR for that particular site.
The process for developing the model is described in section 4.4.2. above. Discrimination was assessed with the c-statistic and calibration was assessed by the slope of the predicted vs. observed risk.
The c-statistic is 0.88, which means that the probability that predicting the outcome is substantially better than chance. This method is used to compare the goodness of fit of logistic regression models. The range is between 0.5 to 1.0. A value of 0.5 indicates that the model is no better than chance at making a prediction of membership in a group and a value of 1.0 indicates that the model perfectly identifies those within a group and those not. Models are typically considered reasonable when the C-statistic is higher than 0.7. (Hosmer & Lemeshow, 2000).
The c-statistics for the original derivation and validation cohorts, as well as clinically important subgroups are provided under Table 13.
We believe that our mortality model performs exceedingly well in adjusting for patient characteristics present prior to the conduct of PCI and is able to discriminate well across a wide variety of important clinical subsets of patients. Moreover, there is substantial hospital variation before and after risk-adjusting patient characteristics. The distribution of hospitals’ O/E ratios show that there are some sites with excellent performance and others with mortality rates that are more than 2-fold greater than expected. These would be sites where substantial opportunities to improve patient safety likely exist.
Use & Usability
Use
Facility level of analysis/hospital in-patient
Usability
Performance results are provided as part of quarterly performance report which includes a rolling 4 quarters of data. These reports provide a detailed analysis of an individual institution ́s performance in comparison with the entire registry population from participating hospitals across the nation. Reports include an executive summary dashboard, at-a-glance assessments, and patient level drill-downs. Registry participants also have access to an outcome report companion guide which provides common definitions and detailed metric specifications to assist with interpretation of performance rates. This information along with the other process and outcome measures included in the CathPCI registry enable participants to identify interventions that will lead to improvement in in-patient mortality rates.
There are a number of methods used to educate and provide general support to registry participants.
These include the following:
• Registry Site Manager Calls are available for all NCDR participants. RSM calls are provided as a source of communication between NCDR and participants to provide a live chat Q and A session on a continuous basis.
• New User Calls are available for NCDR participants, and are intended for assisting new users with their questions.
• NCDR Annual Conference
The NCDR Annual Conference is a well-attended and energetic two-day program at which participants from across the country come together to hear about new NCDR and registry-specific updates. During informative general sessions, attendees can learn about topics such as transcatheter therapies, the NCDR dashboard, risk models, data quality and validation, and value-based purchasing. Attendees also receive registry updates and participate in advanced case studies covering such topics as Appropriate Use Criteria and outcomes report interpretation.
• Release notes (for outcomes reports)
• Clinical Support
The NCDR Product Support and Clinical Quality Consultant Teams are available to assist participating sites with questions Monday through Friday, 9:00 a.m. - 5:00 p.m. ET.
Health care facilities, physicians, data abstractors, registry steering committee members, and other stakeholders routinely provide feedback to the Registry support team via email or phone (i.e., SalesForce) Additional opportunities for detailed measure discussion can occur on bi-monthly registry site manager calls or annually at the in-person NCDR Quality Summit conference where registry management and physician leadership will explore the measure in detailed followed by an open Q&A session.
Feedback varies from detailed comments on the measure criteria, reflections and examples of how the model handles a health care facility’s specific patient population (i.e., high-risk vs stable), and general questions about how end-point decisions were made. When stakeholders fully understand the model and the logic used to create it, they have expressed it is valuable in helping to guide their quality-of-care improvement efforts.
Any criticism of the model or observational data on performance are escalated to the applicable ACC team(s) (i.e., Registry management, Science leadership, Data Analytic Center, etc.) for consideration. If the feedback represents an opportunity for model improvement, the Data Analytic Center is engaged to provide data insights. These data are reviewed by the Senior NCDR Leadership & Science Leadership team which may lead to updating the model or confirm the model is functioning as expected. If an adjustment is needed the change is approved and cascades to the various teams and implemented. The opportunity for these types of model refinements is always available. Feedback which represents a desire for a fundamental shift in model criteria or patient cohort will be assessed when the model is re-evaluated during a cyclical review process by a designated workgroup.
ACC outlined the reasons for the most recent update to the model in Section 1.10. In addition, we follow the processes outlined in Section 6.2.2 to ensure that the measure produces performance scores that drive improvement and inform patients and caregivers as they make decisions about their care.
In 2018, the CathPCI Registry was updated to include additional variables to better classify higher-risk patients. Along those changes, this measure was also updated to use a new hierarchical risk model , which was implemented in 2022 and is intended to better predict in-hospital mortality risk following PCI. While this measure using the previous model demonstrated improvement over time, additional data beyond the model’s first two years of implementation are needed to perform any new analyses (Cavender, 2015).
Reference:
Cavender MA, Joynt KE, Parzynski CS, Resnic FS, Rumsfeld JS, Moscucci M, Masoudi FA, Curtis JP, Peterson ED, Gurm HS. State mandated public reporting and outcomes of percutaneous coronary intervention in the United States. Am J Cardiol. 2015 Jun 1;115(11):1494-501. doi: 10.1016/j.amjcard.2015.02.050. Epub 2015 Mar 12. PMID: 25891991; PMCID: PMC6948713.
Our previous CathPCI Registry mortality models had many assets, however, they had been criticized for failing to accurately define risk among “extreme risk” patients, such as those with cardiogenic shock and those who have suffered cardiac arrest prior to PCI. This led to concerns about whether decision makers will adopt risk-averse patterns of patient care. ACC’s response to how these concerns were addressed were outlined in Section 1.10. Since the update of the model, we have not identified any unexpected findings but will continue to monitor any feedback that is received.
Comments
Staff Preliminary Assessment
CBE #0133 Staff Assessment
Importance
Strengths:
- As the developer states, mortality is an intrinsically material outcome to entities and persons
- The empirical data presented suggest that improvement in performance would have a significant impact on population health (perhaps a 20% reduction in mortality or 1,100 fewer deaths)
- The submission cites numerous clinical studies that speak to clinician decisions relevant to this outcome
Limitations:
- The measure is collected using a registry, which perhaps lessens importance for non-registry participants
- The developer mentions patient representation on the technical expert panel, but some sort of registry patient and family advisory council might better represent patient experiences and inform patient decision-making
- The developer does not explicit provide a rationale for using in-hospital vs. out-of-hospital mortality
Rationale:
- Overall, use of the measure is impactful for a material outcome, as evidenced by clinical studies and empirical data.
Feasibility Acceptance
Strengths:
- Data are collected under the National Cardiovascular Data Registry (NCDR) for CathPCI Registry
Limitations:
- The developer does not explicit provide a rational for using in-hospital vs. out-of-hospital mortality
- If the reason is the lack of availability of data on out-of-hospital mortality, then the developer should provide a discussion of the benefits and harms associated with the trade-off
Rationale:
- Overall, the data are available from a well-established registry with robust processes for data collection
Scientific Acceptability
Strengths:
- The measure is clear and well defined.
- Signal-to-noise reliability is above the threshold of 0.6 with a median of 0.95 and interquartile range of 0.91 and 0.97.
Limitations:
- Split-half reliability (ICC) was estimated to be 0.27821 and 0.28253 for years 2021 and 2022, respectively. These are below the threshold of 0.6
Rationale:
- Although the two methods of reliability (signal-to-noise and split-half) give different answers to whether reliability has been met, research has shown that split-half reliability can be misleading when measured for only one split. Signal-to-noise reliability is above the established thresholds for all entities.
Strengths:
- Developer provides a logic model with explicit mechanisms to increase the likelihood of the measure focus
- The logic model is well supported by numerous clinical studies (both association and mechanism)
Limitations:
- The developer supports the claim that the explicit mechanisms are responsible for the better or worse entity performance with an association study using 0964: CathPCI Discharge Medications Quality Composite
- However, the overlap between the mechanisms for the measure of interest and the composite measure are not explicated stated, and may be limited (which would explain the minimal correlation)
- Therefore the association study does not rule out other competing mechanisms that may be response for better or worse entity performance
Rationale:
- Overall, based on the strength of the body of clinical study evidence, the measure has a strong demonstration of the association between the entity and the measure focus
Equity
Strengths:
- The developer presents stratified analysis based on age, sex, patients insured through Medicaid, and proportion of non-white patients
- The results demonstrated minimal differences in performance for the tested subpopulations
Limitations:
- The absence of evidence of inequity is not evidence of absence of inequity
- Perhaps the reliance upon objective appropriate use criteria increases the plausibility of the equity claim
Rationale:
- Overall, accounting to a significant degree for clinical risk, there does not appear to be detectible differences in performance scores across subgroups
Use and Usability
Strengths:
- The developer cites quality improvement technical assistance associated with registry participation
Limitations:
- There is no explicit articulation of the resources and context that might facilitate or be a barrier to the way an entity may improve
Rationale:
- Overall, the measure is a component of a structured quality improvement process
- Use may be limited for those entities that do not participate in the registry, although the learnings from reported studies would enable dissemination
Committee Independent Review
Mortality for PCI Intervention Summary
Importance
There are opportunities to improve inpatient mortality following PCI through improved care coordination and assessment of patient specific health.
Feasibility Acceptance
Data is captured by registry and has strong process for collection.
Scientific Acceptability
Agree with staff assessment for reliability and split methods.
Agree with staff assessment
Equity
The measure developer evaluated stratified results and did not find significant differences.
Use and Usability
Agree with staff assessment
Summary
There are opportunities to improve inpatient mortality following PCI through improved care coordination and assessment of patient specific health. Data is captured by registry and has strong process for collection.
PCI Mortality
Importance
This measure is impactful and clearly associates performance on PCI with patient centered care/best practice.
Feasibility Acceptance
If the intent of the metric is to evaluate performance of the PCI team and short term complications this makes sense. If the goal is to look at longer term outcomes then this may include multiple variables which will impact overall performance.
Think it is reasonable to keep as in-hospital mortality if only evaluating PCI efficacy/safety.
Scientific Acceptability
Agree with Staff
Agree with Staff
Equity
Addresses areas for possible areas of inequity, and the performance of the metric does not appear to be different among the subgroups.
Use and Usability
Clear use as a QI performance metric. Clear interventions exist that would help participants improve quality of patient care.
Summary
Mortality associated with PCI seems to be an obvious performance metric and participation in this registry/reporting of the metric would lead to an improvement in patient care.
Measure Summary
Importance
Agree with staff assessment.
Feasibility Acceptance
Agree with staff assessment.
Scientific Acceptability
Agree with staff assessment.
Agree with staff assessment.
Equity
Agree with staff assessment.
Use and Usability
Agree with staff assessment.
Summary
Support this measure.
Public Comments
Public Comment Shared During May 29 Listening Session
Hello! This is Florence, patient partner. My comment is just directed toward the age limit. Just asking the question, why is there an age limit on the upper end?