Risk standardized rate of intra and post procedure bleeding for patients age 18 and over without cardiogenic shock or cardiac arrest undergoing PCI.
We have revised this model to no longer include “all patients” but instead exclude cardiogenic shock and cardiac arrest patients. The definition of bleeding was updated as reflected in the numerator statement. Transfusion criteria is now stratified by PCI indication with two separate Hgb thresholds (8 and 10 g/dL). We also added improved clarity on timing (post-procedure) for hemorrhagic stroke, tamponade, etc. The use of a mechanic ventricular support device no longer excludes the patient from the outcome measure.
Measure Specs
General Information
Hospitals should continue to report this updated risk standardized bleeding performance measure because it provides a foundation for bleeding reduction quality improvement initiatives that can improve both the procedure safety and outcomes of treatment with PCI for patients with coronary artery disease and heart attacks.
This risk of bleeding for patients following percutaneous coronary interventions (PCI) continues to varying degrees across hospitals in the US (Price et al., 2024). Intra- and post-procedure bleeding is the most common non-cardiac complication of PCI, leading to adverse patient outcomes (e.g. increased morbidity and mortality, prolonged length of stay and costs) and, importantly, can be modified using bleeding avoidance strategies such as radial arterial access (Amin, 2022; Heidary Moghadam, 2024; Ndrepepa, 2013; Rao, 2025; Vora, 2016). Facility-focused quality improvement strategies are effective in reducing these bleeding events. For example, a recent hospital-based quality improvement campaign led by the American College of Cardiology (ACC) resulted in reductions in bleeding events across all hospitals, but rates were further reduced for those that implemented the toolkit and quality improvement strategies (Price, 2024). The use of risk-stratification is the first step towards identifying patients at highest risk to appropriately allocate bleeding avoidance strategies, and reduce unnecessary costs associated with bleeding events and prolonged length of stays. (Price et al., 2024; Capodanno et al., 2022)
The measure currently under review for endorsement maintenance reflects an updated bleeding model that aligns with updated clinical guidelines recommendations. The ACC developed and iterated on a bleeding model over the years for use within the National Cardiovascular Data Registry (NCDR) CathPCI Registry and received initial endorsement in 2014. Due to recent findings from the Myocardial Ischemia and Transfusion (MINT) trial, NCDR recognized that the current bleeding model required update. Specifically, this trial examined the impact of a restrictive (cutoff for transfusion of hemoglobin less than 8 g/dL) versus liberal transfusion strategy (cutoff for transfusion of hemoglobin less than 10 g/dL) on two outcomes (myocardial infarction or death within 30 days of the procedure) and even though the results were not statistically significant, the findings indicated that the more liberal transfusion strategy had more short-term clinical benefit than the alternative restrictive approach (Carson, 2023). The trial’s results are reflected in the updated clinical recommendations in the 2025 guideline on the management of patients with acute coronary syndromes (Rao, 2025). Specifically, clinical guidance now encourages blood transfusion for those individuals with acute coronary syndrome and a hemoglobin less than or equal to 10 g/dL as opposed to the previous assumption that transfusing a patient with that range of hemoglobin meant that a patient was bleeding and therefore had an adverse event. This change has been reflected in what is considered a bleeding event for the measure’s outcome.
Both this measure and the PCI in-hospital mortality measure (CBE #133) were revised to address the concerns voiced by the medical community that the model did not adequately account for patients at extreme risk or facilities with lower volumes and therefore hospitals with larger numbers of these individuals would perform more poorly on these outcomes. ACC integrated additional variables (e.g., frailty, cardiovascular instability) into the CathPCI registry’s dataset and subsequently developed a new hierarchical mortality model for mortality. The same methodology was applied to this bleeding model. This hierarchical model includes variables that identify if a patient is experiencing cardiogenic shock or is status post resuscitated cardiac arrest. The updates to the bleeding measure now focus on those patients who do not fall within this risk category and therefore minimizes the risk of penalizing clinicians and facilities who are willing to provide care for these individuals who are more likely to have poorer outcomes unrelated to the care that they subsequently receive at the facility.
As a result of this updated model, facilities have access to data that better classify a patient’s bleeding risk and allows in-depth analyses of the causes behind variations in bleeding during or post PCI leading to the identification of best practices. In addition, detailed case reviews can identify clinicians with poorer performance for whom additional training or reduced caseloads could be considered. Active dissemination of those best practices and support to enable their adoption will improve outcomes and reduce variations in clinical practice. Improvements in the quality of care resulting from the evaluation of the risk of bleeding, before and after implementing quality improvement interventions, can enable facilities to quantify their improved outcomes and a reduction in cost associated with these events. Additionally, by putting the responsibility for improved quality in the hands of physicians and other healthcare providers, this updated risk standardized bleeding measure engages the medical community around the common goal of better healthcare value.
References:
Amin AP, Frogge N, Kulkarni H, et al. The bleeding risk treatment paradox at the physician and hospital level: Implications for reducing bleeding in patients undergoing percutaneous coronary intervention. Am Heart J. 2022;243:221-231. doi:10.1016/j.ahj.2021.08.021
Capodanno, D., Bhatt, D.L., Gibson, C.M. et al. Bleeding avoidance strategies in percutaneous coronary intervention. Nat Rev Cardiol 19, 117–132 (2022). https://doi.org/10.1038/s41569-021-00598-1
Carson JL, Brooks MM, Hébert PC, et al. Restrictive or Liberal Transfusion Strategy in Myocardial Infarction and Anemia. N Engl J Med. 2023;389(26):2446-2456. doi:10.1056/NEJMoa2307983
Heidary Moghadam R, Mohammadi A, Salari N, Ahmed A, Shohaimi S, Mohammadi M. The prevalence of bleeding after percutaneous coronary interventions: A systematic review and meta-analysis. Indian Heart J. 2024;76(1):16-21. doi:10.1016/j.ihj.2024.01.009
Inohara T, Kohsaka S, Spertus JA, et al. Comparative Trends in Percutaneous Coronary Intervention in Japan and the United States, 2013 to 2017. J Am Coll Cardiol. 2020;76(11):1328-1340. doi:10.1016/j.jacc.2020.07.037
Ndrepepa G, Neumann FJ, Richardt G, et al. Prognostic value of access and non-access sites bleeding after percutaneous coronary intervention. Circ Cardiovasc Interv. 2013;6(4):354-361. doi:10.1161/CIRCINTERVENTIONS.113.000433
Price AL, Amin AP, Rogers S, et al. Implementation of a Multidimensional Strategy to Reduce Post-PCI Bleeding Risk. Circ Cardiovasc Interv. 2024;17(3):e013003. doi:10.1161/CIRCINTERVENTIONS.123.013003
Rao SV, O'Donoghue ML, Ruel M, et al. 2025 ACC/AHA/ACEP/NAEMSP/SCAI Guideline for the Management of Patients With Acute Coronary Syndromes: A Report of the American College of Cardiology/American Heart Association Joint Committee on Clinical Practice Guidelines. Circulation. 2025;151(13):e771-e862. doi:10.1161/CIR.0000000000001309
Vora AN, Peterson ED, McCoy LA, et al. The Impact of Bleeding Avoidance Strategies on Hospital-Level Variation in Bleeding Rates Following Percutaneous Coronary Intervention: Insights From the National Cardiovascular Data Registry CathPCI Registry. JACC Cardiovasc Interv. 2016;9(8):771-779. doi:10.1016/j.jcin.2016.01.033
National Cardiovascular Data Registry (NCDR®) CathPCI Registry®
The details listed here are repeated in the Feasibility section of this application.
The data elements required to generate this measure are, to the best of our knowledge, abstracted from the electronic medical record by someone other than person obtaining original information (e.g., chart abstraction for quality measure or registry). All data elements are available in defined fields in electronic clinical data (e.g., clinical registry). This measure uses clinical data from the NCDR CathPCI Registry. This measure has been in use for many years and as a result, while the CathPCI Registry continues to monitor the feasibility and data collection burden of this measure, minimal changes to how the data are collected and reported have been required in recent years. We outline the general process used by any hospital reporting to an NCDR registry below.
Availability:
Participating hospitals report patient demographics, medical history, risk factors, hospital presentation, procedural details, medications, laboratory values and in-hospital outcomes as the key activity of participating in the NCDR registries. All of the required data elements for this measure are routinely generated and acquired during the hospitalization. Electronic extraction of data recorded as part of the procedure expedites data collection. This strategy offers point of care data collection and minimizes time and cost. Institutions can manually report using a free web-based tool or automate the reporting by using certified software developed by third-party vendors.
Numerator
Patients 18 years of age and older with a post-PCI bleeding event as defined below
Post-PCI bleeding defined as any ONE of the following:
- Bleeding event within 72 hours
- Hemorrhagic stroke post procedure
- Tamponade post procedure
- Transfusion PCI = yes AND pre-procedure hemoglobin greater than 8 g/dL AND PCI Indication = New onset angina ≤2 months or stable angina or CAD without ischemic Symptoms.
- Transfusion PCI = yes AND pre-procedure hemoglobin greater than 10 g/dL AND PCI Indication in STEMI, NSTEMI or unstable angina.
- Absolute Hgb decrease of >= 4g/dL from pre-PCI to post-PCI
Bleeding defined as any ONE of the following:
- Bleeding event within 72 hours post PCI
- Hemorrhagic stroke within 72 hours post PCI
- Tamponade within 72 hours post PCI
- Blood transfusion AND pre-procedure hemoglobin greater than 8 g/dL AND PCI Indication = New onset angina ≤2 months or stable angina or CAD without ischemic symptoms.
- Blood transfusion AND pre-procedure hemoglobin greater than 10 g/dL AND PCI Indication in STEMI, NSTEMI or unstable angina.
- Absolute hemoglobin level decrease of >= 4g/dL from pre-PCI to post-PCI
Note:
• All data element numbers listed above are included in the attached data dictionary which includes more detailed definitions for the above elements.
• The measure includes risk standardization to account for differences in case mix across hospitals, thus the ratio determined by the numerator and denominator are modified based upon the adjustment.
Denominator
Patients 18 years of age and older with a PCI procedure performed during admission (excluding cardiogenic shock and cardiac arrest)
The following patients are included in the denominator:
- Patients 18 years of age or older
- Patients undergoing PCI during the episode of care
- Initial PCI procedures for patients who underwent multiple PCI procedures during the episode of care (subsequent PCIs during a single episode of care excluded).
Note: All data element numbers listed above are included in the attached data dictionary, which includes more detailed definitions for the above elements.
Exclusions
This measure excludes the following:
- Patients who died within 24 hours of the PCI procedure.
- Patients who have CABG during the episode of care.
- Patients with cardiogenic shock.
- Patients resuscitated from cardiac arrest that occurred either: 1) outside of the healthcare facility prior to arrival; 2) while being transferred to the facility; or 3) while at the facility and prior to PCI.
- Subsequent PCI procedures (when the patient has more than one PCI during the episode of care).
The measure has the following exclusions:
- Patients who died (10105) within 24 hours (10101) of the PCI procedure (7000).
- Patients who have CABG during the episode of care (10030/10031).
- Patients with cardiogenic shock prior to the PCI (7415) [Cardiovascular instability type = cardiogenic shock OR refractory cardiogenic shock]
- Patients resuscitated from cardiac arrest (7400) that occurred either: 1) outside of the healthcare facility prior to arrival; 2b while being transferred to the facility; or 3) while at the facility and prior to the PCI
- Subsequent PCI procedures (7050) when the patient has had more than one during the episode of care.
Note: All data element numbers listed above are included in the attached data dictionary which includes more detailed definitions for the above elements.
At the facility level, all data submissions must pass the data quality and completeness reports to be included. Of note we used imputation for some variables with missing values. In the NCDR data quality program, all key variables in the risk model have a high “inclusion” criteria, meaning that when a hospital submits data, they need to have a high level of completeness (>95%) for those variables. If they are not able to meet the criteria in our data quality program, they do not receive risk-adjusted outcomes for any of the records they submitted for that quarter. Because the high-threshold for inclusion is present, the impact of imputation on hospital-specific rates is minimal but enables a more complete assessment of hospital performance.
Measure Calculation
- Remove hospitals who fail data quality and completeness reports as outlined in the NCDR Data Quality Program (further discussed in the Testing Supplement)
- Remove hospitals who do not have at least one patient with a pre-PCI or post-PCI hemoglobin value.
- Remove subsequent PCIs during the same admission (if the patient had more than one PCI procedure during that episode of care).
- Remove patients who did not have a PCI (Patient admissions with a diagnostic catheterization only during that episode of care)
- Remove patients who died on the same day of the procedure
- Remove patients who had CABG during the episode of care
- Remove patients with pre-procedure hemoglobin <8 g/dL patients (severely anemic) who did not also have a documented bleeding event other than transfusion were not counted in the numerator if they received a transfusion.
- Calculate the measure using weight system based on predictive variables as outlined in the accompanying testing documents and supplemental materials.
This measure uses predictive variables to estimate in-hospital bleeding following PCI using a hierarchical risk model. The approach simultaneously models data at the patient and hospital levels to account for variance in patient outcomes within and between hospitals [Normand and Shahian, 2007; Krumholz H, Normand S, Galusha D, et al., 2005]. At the patient level, it models the log-odds of bleeding within 30 days of index admission using age, sex, selected clinical covariates, and a hospital-specific intercept. At the hospital level, it models the hospital-specific intercepts as arising from a normal distribution. The hospital intercept represents the underlying risk of a bleed at the hospital, after accounting for patient risk. The hospital-specific intercepts are given a distribution to account for the clustering (non-independence) of patients within the same hospital. If there were no differences among hospitals, then after adjusting for patient risk, the hospital intercepts should be identical across all hospitals.
The risk standardized bleeding rate (RSBR) is calculated as the ratio of the number of “predicted” to the number of “expected” bleeds at a given hospital. For each hospital, the numerator of the ratio is the number of bleeds following a PCI procedure predicted based on the hospital’s performance with its observed case mix, and the denominator is the number of bleeds expected based on the nation’s performance with that hospital’s case mix. A series of denominator exclusions (summarized in Table 8. “Development of the Study Cohort” of section 5.1.3) were applied to define the study sample during a rolling four quarter period. For this sample, patients were excluded if their discharges were not between January 1, 2021 and December 31, 2022 or if they did not undergo percutaneous coronary intervention (PCI) during their admission. Furthermore, only the index PCI procedure was included. Cases were further excluded if in-hospital death occurred within 24 hours of the procedure. Patients who underwent coronary artery bypass (CABG) during the same hospitalization were removed. Additionally, patients presenting with resuscitated cardiac arrest or cardiogenic shock on admission, as well as those who underwent “salvage PCI” were excluded.
This approach is analogous to a ratio of “observed” to “expected” used in other types of statistical analyses. It conceptually allows for a comparison of a particular hospital’s performance given its case mix to an average hospital’s performance with the same case mix. Thus, a lower ratio indicates lower-than-expected bleeding rates or better quality, and a higher ratio indicates higher-than-expected bleeding rates or worse quality. The “predicted” number of bleeds (the numerator) is calculated by using the coefficients estimated by regressing the risk factors and the hospital-specific intercept on the risk of mortality. The estimated hospital-specific intercept is added coefficients multiplied by the patient characteristics. The results are transformed and summed over all patients attributed to a hospital to get a predicted value. The “expected” number of bleeds (the denominator) is obtained in the same manner, but a common intercept using all hospitals in our sample is added in place of the hospital-specific intercept. The results are transformed and summed over all patients in the hospital to get an expected value. To assess hospital performance for each reporting period, we re-estimate the model coefficients using the years of data in that period.
This calculation transforms the ratio of predicted over expected into a rate that is compared to the national observed rates. The hierarchical logistic regression models are described fully in the original methodology report posted on QualityNet [https://qualitynet.org/inpatient/measures/mortality/methodology].
References:
Normand S-LT, Shahian DM. 2007. Statistical and Clinical Aspects of Hospital Outcomes Profiling. Stat Sci 22(2): 206-226.
Krumholz H, Normand S, Galusha D, et al. Risk-Adjustment Models for AMI and HF 30-Day Mortality Methodology. 2005.
The measure is not stratified.
No minimum sample size is required.
Supplemental Attachment
Measure Record
Point of Contact
N/A
Katie Goodwin
Washington, DC
United States
Kathryn Goodwin
American College of Cardiology
Washington, DC
United States
Importance
Evidence
Evidence specific to the measure updates include recent findings from the Myocardial Ischemia and Transfusion (MINT) trial, which examined the impact of a restrictive (hemoglobin cutoff for transfusion of <7-8 g/dL) versus liberal transfusion strategy (hemoglobin cutoff for transfusion of <10 g/dL) on two outcomes – myocardial infarction or death within 30 days of the procedure, proved integral to the recent update to this measure as well as the 2025 guideline on the management of patients with acute coronary syndromes (Rao, 2025). While the results from the trial were not statistically significant, they indicated that a liberal strategy may have more short-term clinical benefit than the alternative restrictive approach (Carson, 2023). As a result, clinical guidance was updated to encourage blood transfusion for those individuals with acute coronary syndrome and a hemoglobin <10 g/dL as opposed to the previous assumption that transfusing a patient meant that a patient was bleeding and therefore had an adverse event. This change has been reflected in what is considered a bleeding event for inclusion in this measure.
Additionally, evidence exists to demonstrate that facilities can implement various structures and processes to further decrease the risk of bleeding following a PCI. As noted in section 1.10, a recent hospital-based quality improvement campaign led by the ACC led to reductions in bleeding events across all hospitals, but rates were further reduced for those that implemented the toolkit and quality improvement strategies (Price, 2024). The use of this risk-adjusted measure when paired with quality improvement activities can improve clinical outcomes for patients. Additionally, the 2025 ACC/AHA Guidelines for the treatment of patients with acute coronary syndrome (ACS) recommended bleeding avoidance strategy of radial access for PCI as a class 1, A recommendation. The class of recommendation (COR) reflects the magnitude of benefit over risk and corresponds to the strength of the recommendation. Class I recommendations are strong and indicate that the treatment, procedure, or intervention is useful and effective and should be performed or administered for most patients under most circumstances. The level of evidence (LOE) of A indicates high-quality evidence from more than one random controlled clinical trial or a meta-analysis of random controlled trials. [In patients with ACS undergoing PCI, a radial approach is preferred to a femoral approach to reduce bleeding, vascular complications, and death. (Class 1, LOE A)]. (Rao, et al., 2025)
Scientific and clinical evidence continues to support the ongoing use of this outcome measure. Individuals in the United States received more than 600,000 percutaneous coronary interventions (PCIs) in 2017 and roughly 60% of these procedures were elective (Inohara, 2020). Intra- and post-operative bleeding following PCI is an important outcome and lower rates can be achieved if clinicians and facilities use evidence-based criteria to assess a patient’s bleeding risk and implement effective strategies to reduce its occurrence. Implementation of the criteria and treatments decrease the likelihood of death, longer hospital stays and increase costs (Amin, 2022; Heidary Moghadam, 2024; Ndrepepa, 2013; Rao, 2025; Vora, 2016). One study using the CathPCI Registry data of more than 3 million procedures from 2004 to 2011 determined that post PCI bleeding events were associated with increased risk of in-hospital mortality, with an estimated 12.1% of deaths related to bleeding complications (Chhatriwalla, 2013).
For many years, data from clinical trials have been analyzed to identify the baseline characteristics that can predict bleeding. The most recent effort was by the Academic Research Consortium for High Bleeding Risk (ARC-HBR), which was published in 2019 (Urban, 2019). The 20 clinical criteria on which consensus across multiple clinical experts was reached ensure that a consistent definition on individuals at high risk of bleeding will be used in clinical trials and is used to define high risk patients within the CathPCI registry and this measure.
Published trials and observational studies have found that specific processes of care, including the use of radial arterial access (Jhand, 2021), mechanical closure devices when femoral access is used (Kreutz, 2022), and bivalirudin for anticoagulation (Al-Abdouh, 2024), are associated with lower risks of bleeding. All these processes resulted in the reduction of complications and death.
References:
Al-Abdouh A, Mhanna M, Jabri A, et al. Bivalirudin versus unfractionated heparin in patients with myocardial infarction undergoing percutaneous coronary intervention: A systematic review and meta-analysis of randomized controlled trials. Cardiovasc Revasc Med. 2024;61:52-61. doi:10.1016/j.carrev.2023.10.014
Amin AP, Frogge N, Kulkarni H, et al. The bleeding risk treatment paradox at the physician and hospital level: Implications for reducing bleeding in patients undergoing percutaneous coronary intervention. Am Heart J. 2022;243:221-231. doi:10.1016/j.ahj.2021.08.021
Carson JL, Brooks MM, Hébert PC, et al. Restrictive or Liberal Transfusion Strategy in Myocardial Infarction and Anemia. N Engl J Med. 2023;389(26):2446-2456. doi:10.1056/NEJMoa2307983
Chhatriwalla AK, Amin AP, Kennedy KF, et al. Association between bleeding events and in-hospital mortality after percutaneous coronary intervention. JAMA. 2013;309(10):1022-1029. doi:10.1001/jama.2013.1556
Heidary Moghadam R, Mohammadi A, Salari N, Ahmed A, Shohaimi S, Mohammadi M. The prevalence of bleeding after percutaneous coronary interventions: A systematic review and meta-analysis. Indian Heart J. 2024;76(1):16-21. doi:10.1016/j.ihj.2024.01.009
Jhand A, Atti V, Gwon Y, et al. Meta-Analysis of Transradial vs Transfemoral Access for Percutaneous Coronary Intervention in Patients With ST Elevation Myocardial Infarction. Am J Cardiol. 2021;141:23-30. doi:10.1016/j.amjcard.2020.11.016
Kreutz RP, Phookan S, Bahrami H, et al. Femoral Artery Closure Devices vs Manual Compression During Cardiac Catheterization and Percutaneous Coronary Intervention. J Soc Cardiovasc Angiogr Interv. 2022;1(5):100370. Published 2022 Jun 29. doi:10.1016/j.jscai.2022.100370
Inohara T, Kohsaka S, Spertus JA, et al. Comparative Trends in Percutaneous Coronary Intervention in Japan and the United States, 2013 to 2017. J Am Coll Cardiol. 2020;76(11):1328-1340. doi:10.1016/j.jacc.2020.07.037
Ndrepepa G, Neumann FJ, Richardt G, et al. Prognostic value of access and non-access sites bleeding after percutaneous coronary intervention. Circ Cardiovasc Interv. 2013;6(4):354-361. doi:10.1161/CIRCINTERVENTIONS.113.000433
Price AL, Amin AP, Rogers S, et al. Implementation of a Multidimensional Strategy to Reduce Post-PCI Bleeding Risk. Circ Cardiovasc Interv. 2024;17(3):e013003. doi:10.1161/CIRCINTERVENTIONS.123.013003
Rao SV, O'Donoghue ML, Ruel M, et al. 2025 ACC/AHA/ACEP/NAEMSP/SCAI Guideline for the Management of Patients With Acute Coronary Syndromes: A Report of the American College of Cardiology/American Heart Association Joint Committee on Clinical Practice Guidelines. Circulation. 2025;151(13):e771-e862. doi:10.1161/CIR.0000000000001309
Urban P, Mehran R, Colleran R, et al. Defining high bleeding risk in patients undergoing percutaneous coronary intervention: a consensus document from the Academic Research Consortium for High Bleeding Risk. Eur Heart J. 2019;40(31):2632-2653. doi:10.1093/eurheartj/ehz372
Vora AN, Peterson ED, McCoy LA, et al. The Impact of Bleeding Avoidance Strategies on Hospital-Level Variation in Bleeding Rates Following Percutaneous Coronary Intervention: Insights From the National Cardiovascular Data Registry CathPCI Registry. JACC Cardiovasc Interv. 2016;9(8):771-779. doi:10.1016/j.jcin.2016.01.033
Measure Impact
This measure was developed with input from a technical expert panel that included patient and caregiver representation. As regaining function and preserving and improving quality of life is the goal of most patients, decreasing the rate of bleeding during or after PCI is easily understood and will help patients in understanding the quality of care provided by facilities and in making decisions about where to receive their care.
Between April 11, 2025, and May 10, 2025, this measure and updated risk model underwent peer review and public comment during which ACC members, NCDR participants, patient advocacy groups, healthcare systems, private payors and other healthcare professionals had the opportunity to review and comment on the methodology and construct before ACC final approval for use. The distribution list used for this comment contains over 1200 names. In addition, ACC social media outlet and the NCDR participant webpage provides public access to the comment questions and content. Forty-nine reviewers provided 72 individual comments and scored the associated questionnaire. While most respondents are anonymous, patient support groups (such as Mended Hearts Mended Hearts Non-profit Organization) patient centered research systems (such as https://www.pcori.org/| PCORI) have equal opportunity to provide feedback and comment.
Performance Gap
Table 1 below illustrates the distribution of the risk-standardized bleeding rates during the two-year observation period between 2021 and 2022. Included is the mean score, entities (or hospitals), and total encounters (or admissions), all evaluated by decile of performance from data collected from abstractors and reported to the CathPCI Registry. As illustrated in Table 1, the minimum RSBR was 0.55% whereas the maximum score was 26.7%, suggesting a wide gap in performance. Further, comparing those sites with the lowest and highest deciles of performance, 0.10% vs. 3.34%, respectively, demonstrates more than a 3% difference in bleeding rates. While 3% may not appear to be a considerable gap, it translates to an additional 39,380 bleeds per year, justifying the importance of capturing and reporting these data.
| Overall | Minimum | Decile_1 | Decile_2 | Decile_3 | Decile_4 | Decile_5 | Decile_6 | Decile_7 | Decile_8 | Decile_9 | Decile_10 | Maximum | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Mean Performance Score | 0.0186 | 0.0055 | 0.0104 | 0.0128 | 0.0142 | 0.0156 | 0.0169 | 0.0181 | 0.0195 | 0.0215 | 0.0241 | 0.0334 | 0.2668 |
| N of Entities | 1704 | 1 | 170 | 170 | 171 | 170 | 171 | 170 | 171 | 170 | 171 | 170 | 1 |
| N of Persons / Encounters / Episodes | 1312961 | 1483 | 164404 | 132126 | 127997 | 110913 | 114943 | 115963 | 130561 | 121578 | 144116 | 150360 | 475 |
Care Gaps
Closing Care Gaps
We attributed social risk factors at the hospital-level for the purpose of this analysis. We used Medicaid insurance status as the economic indicator of social risk. We also examined age, sex, and race/ethnicity to determine if there were differences in these demographic indicators of social risk. Analyses of differences by subgroup were based on registry data procured from years 2021-22.
In terms of the overall distribution, the median risk standardized rate of bleeding was 1.74%, with an interquartile range of 1.43% to 2.16%. There is a right skew to the bleeding rates (Figure 1).
See attached Tables and Figures document in 7.1:
Tables 2-6
Figures 1-5
Feasibility
Feasibility
The data elements required to generate this measure are abstracted from a medical record by someone other than person obtaining original information (e.g., chart abstraction for quality measure or registry). All data elements are available in defined fields in electronic clinical data (e.g., clinical registry). This measure uses clinical data from the NCDR CathPCI Registry. This measure has been in use for many years and as a result, while the CathPCI Registry continues to monitor the feasibility and data collection burden of this measure, minimal changes to how the data are collected and reported have been required in recent years. We outline the general process used by any hospital reporting to an NCDR registry below.
Availability:
Participating hospitals report patient demographics, medical history, risk factors, hospital presentation, procedural details, medications, laboratory values and in-hospital outcomes as the key activity of participating in the NCDR registries. All of the required data elements for this measure are routinely generated and acquired during the hospitalization. Electronic extraction of data recorded as part of the procedure expedites data collection. This strategy offers point of care data collection and minimizes time and cost. Institutions can manually report using a free web-based tool or automate the reporting by using certified software developed by third-party vendors.
Sampling:
There is no sampling of patient data allowed within the contractual terms of participation in the NCDR registries. Section 2.b of the NCDR Master Agreement with participants includes ‘Participant Responsibilities’: “b. Use of ACCF Data Set and ACCF-Approved Software. Participant will submit a data record on each patient who receives medical care and who is eligible for inclusion in the Registries in which Participant is participating under this Agreement.” Patients are selected for inclusion by reviewing existing medical records and no direct interaction with the patient is required outside of the normal course of care. There is no discrimination or bias with respect to inclusion on the basis of sex, race, or religion.
This measure was developed and designed to be used across other organizations and by other measure implementers. The fee and licensing information include below is specific to NCDR program requirements:
The NCDR provides evidence-based solutions for cardiologists and other medical professionals committed to excellence in cardiovascular care. NCDR hospital participants receive confidential benchmark reports that include access to measure macro specifications and micro specifications, the eligible patient population, exclusions, and model variables (when applicable). In addition to hospital sites, NCDR Analytic and Reporting Services provides consenting hospitals’ aggregated data reports to interested federal and state regulatory agencies, multi-system provider groups, third-party payers, and other organizations that have an identified quality improvement initiative that supports NCDR-participating facilities. Lastly, the ACCF also allows for licensing of the measure specifications outside of the Registry.
Measures that are aggregated by ACCF and submitted for endorsement are intended for public reporting and therefore there is no charge for a standard export package. However, on a case-by-case basis, requests for modifications to the standard export package will be available for a separate charge.
Each NCDR institution signs a Participant Agreement with the American College of Cardiology Foundation (“ACCF”) including a Business Associate Agreement and Data Use Agreement. The NCDR requires the collection of protected health information as such term is defined by the Health Insurance Portability and Accountability Act of 1996 as amended (“HIPAA”). Submission of Protected Health Information is considered permissible as a healthcare operations disclosure not requiring a HIPAA authorization from individuals. Consistent with the requirements of HIPAA, ACCF has designed a comprehensive security program that protects the confidentiality, integrity and availability of protected health information through the implementation of administrative, physical, and technical safeguards. ACCF’s security program was designed using the NIST Cybersecurity Framework. ACCF periodically conducts an independent control assessment to confirm alignment with the HIPAA Security Rule and NIST Cybersecurity Framework. This measure does not include a patient survey. There is no added procedural risk to patients through involvement in the NCDR and no testing, time, risk, or procedures beyond those required for routine care are imposed.
The measure has been in use for many years and as a result, while the CathPCI Registry continues to monitor the feasibility and data collection burden of this measure, the most significant set of changes to how the data are collected and reported was the revisions to improve the capture of cardiogenic shock and cardiac arrest in 2018. ACC dedicated significant time and resources developing education and guidance to ensure that hospitals were able to accurately capture these data. Minimal changes have been made to this registry since then.
Proprietary Information
This measure is feasible to report and utilize as demonstrated by the more than 1,600 hospitals currently submitting data for this measure. Outside of participation in NCDR, there are no licensing or fees associated with this specific measure.
Scientific Acceptability
Testing Data
We used the National Cardiovascular Data Registry for CathPCI Registry. This is a national quality improvement registry with over 1700 participating US hospitals. Participation is largely voluntary though some states and healthcare systems mandate participation. Rigorous quality standards are applied to the data and both quarterly and ad hoc performance reports are generated for participating centers to track and improve their performance.
01/2021–12/2022
None.
See attached tables and figures document in 7.1 table 7 & 8.
See attached tables and figures document in 7.1 tables 9 & 10.
Reliability
Performance Measure Score (Signal-to-Noise):
ACCF performed the signal-to-noise analysis on the same cohort of individuals as noted under Section 5.1. For the signal-to-noise analysis, we followed the methodology as outlined in a Rand Corporation technical report by John L Adams. The document is available at the following URL (https://www.rand.org/content/dam/rand/pubs/technical_reports/2009/RAND_…). This approach uses a beta-binomial model that assumes the physician’s score is a binomial random variable conditional on the physician’s true value that comes from a beta distribution. The beta distribution is a very flexible distribution on the interval from 0 to 1 and can have any mean within the interval and can be skewed left or right or even U-shaped. It is the most common distribution for probabilities on the 0-1 interval. A higher SNR indicates a stronger signal relative to the noise, which suggests better reliability and accuracy in signal detection and processing.
Second, we pursued a split sample methodology to assess the consistency or reliability of the measure by dividing it into two halves and comparing the results obtained from each half. For the performance rates, raw rates were calculated, and a correlation coefficient was computed. The split samples were calculated during the same timeframe to mitigate confounding factors based on time differences. The cohort was split into two random samples to compare measure scores. The type of error tested by a split-sample reliability test is primarily related to the consistency or stability of measurements obtained from the measure. This test helps identify errors or sources of variability that may affect the reliability of the measurement process, ensuring that the measure results are trustworthy and replicable.
See tables 11 & 12 and figures 6 & 7 in the attached tables & figures document in 7.1.
Signal to Noise Analysis:
The signal to noise ratio analysis measures the confidence levels in differentiating performance between hospitals. Our analyses found the median SNR was 0.97 and had a fairly narrow interquartile range of 0.95 and 0.98. These numbers demonstrate variability that is attributable to real differences in hospital quality as opposed to measurement error. Collectively, we believe that the data strongly support the reliability of the data elements used in the model.
(Reference: Landis J, Koch G, The measurement of observer agreement for categorical data, Biometrics, 1977;33:159-174.)
Split Sample Methodology:
The box and whisper plot of the distribution of hospital performance for the model shows a similar distribution of use of the risk standardized bleeding rates for both split samples. Figures 7 (in the attached tables and figures document in 7.1) shows the scatterplot of the distribution of hospital performance when assessed in randomly split samples. Overall hospital performance in one random sample was correlated with hospital performance in the other split sample (r= 0.70943, P<0.0001), which is consistent with a reliable measure.
| | Overall | Minimum | Decile_1 | Decile_2 | Decile_3 | Decile_4 | Decile_5 | Decile_6 | Decile_7 | Decile_8 | Decile_9 | Decile_10 | Maximum |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Reliability | 0.9458 | 0.9886 | 0.9751 | 0.9651 | 0.9507 | 0.9296 | 0.8987 | 0.9170 | 0.9438 | 0.9550 | 0.9586 | 0.9640 | 0.9654 |
| Mean Performance Score | 0.0186 | 0.0055 | 0.0104 | 0.0128 | 0.0142 | 0.0156 | 0.0169 | 0.0181 | 0.0195 | 0.0215 | 0.0241 | 0.0334 | 0.2668 |
| N of Entities | 1704 | 1 | 170 | 170 | 171 | 170 | 171 | 170 | 171 | 170 | 171 | 170 | 1 |
| N of Persons / Encounters / Episodes | 1312961 | 1483 | 164404 | 132126 | 127997 | 110913 | 114943 | 115963 | 130561 | 121578 | 144116 | 150360 | 475 |
Validity
We performed 2 different strategies for assessing the validity of this measure. First, we underwent a rigorous process for establishing the face validity of the measure. Because it is a clinically meaningful outcome, we sought to make sure that a broad range of experts and clinicians concurred that this was a clinically important outcome measure. Second, we hypothesized that it would be associated with other clinically important outcomes and sought to establish the predictive validity of the measure. These are described in more detail below:
Systematic Assessment of Face Validity of the Performance Measure:
Bleeding remains one of the most common non-cardiac complications of PCI. It is a serious adverse consequence and, most importantly, is modifiable. The 2011 ACC/AHA guidelines provide for a Level IC recommendation for the assessment of bleeding prior to PCI. This is grounded in the realization that there are several strategies, such as radial approaches and the use of bivalirudin, that can be applied to mitigate the risk of bleeding, particularly in high-risk patients. The first bleeding risk model was published in 2009 (Circ Cardiovasc Intervent. 2009;2:222-229) and the update was published in 2013 (JACC Cardiovasc Intervention, 2013;6:897-904).
Content validity of this outcome – and the specific definition used in defining a bleeding event – was achieved by the specialized expertise of those individuals who developed this model as well as the structured discussions that the group conducted. For this particular topic those individuals who were involved in identifying the key attributes and variables for this risk model were leaders and experts in the field of interventional cardiology. Multiple conference calls were held to both define a bleeding event and to examine and vet the risk model. These individuals within specific committees and workgroups are noted below:
NCDR Science and Quality Oversight Committee— an ACC leadership oversight committee that serves as the primary resource for crosscutting scientific and quality of care methodological issues – ensured the data dictionaries and metrics are consistent across registries. They also reviewed and approved the methodology and results of the bleeding outcome and model.
These members included:
John C. Messenger, MD, FACC (Chair); David M. Shahian, MD, FACC; Thomas T Tsai, MD, MSC; Charles A. Henrikson, MD, MPH; Jeff Jacobs, MD, FACC; John R. Windle, MD, FACC; Amit Amin, MD; John W. M. Moore, MD, FACC; Deepak L. Bhatt, MD, MPH, FACC; Jeffrey Westcott, MD, FACC; Gregory M. Marcus, MD FACC; David J. Slotwiner, MD, FACC; Jeptha P. Curtis, MD, FACC; John Spertus, MD, FACC; Matthew T. Roe, MD, FACC; and Frederick A. Masoudi, MD, MSPH, FACC
NCDR Clinical SubWorkgroup was a designated workgroup that oversaw the initial NQF application. Prior to submission, the group ensured there was variation in care, disparities data, and that the measure is a true reflection of quality care at a particular site and can also be used to improve quality.
Dr. Jeptha Curtis (chair), Dr. Frederick Masoudi, Dr. John Rumsfeld, Dr. David Malenka, and Dr. Issam Moussa.
NCDR Registry Steering Committee provided strategic direction for the Registry and ensures the measures submitted to NQF met key criterion such as reliability, feasibility, and that there is compelling evidence base behind the development and implementation of this measure. Dr. Issam D. Moussa (chair), Dr. Kirk N. Garratt, Dr. Lloyd W. Klein, Dr. Kendrick A. Shunk, Dr. Samir R. Kapadia, Dr. Robert N. Piana, Dr. Roxana Mehran, Dr. Frederic S. Resnic, Dr. Aaron D. Kugelmass, Dr. Sunil V. Rao, Dr. W. Douglas Weaver, and Dr. John C. Messenger.
The NCDR Metrics and Reporting Methodology (MRM) Subcommittee of the Science and Quality Oversight Committee, reviews for re-endorsement and a data analytic center is involved in evaluating data, providing corresponding analysis/interpretation of data. The review included guidance and oversight from both NCDR’s Chief Science Officer (Frederick Masoudi) and chair of MRM (Jeptha Curtis).Lastly the 16 member NCDR Management Board and 31 member ACCF Board of Trustees originally approved these measures for submission to NQF.
In addition, the NCDR provides an open comment period (typically between 15 and 30 days) for: 1) all registry data set version changes, 2) new registry version measures and 3) significant changes/additions to registry version metrics/measures, including risk models and appropriate use criteria. The open comment period engages key registry shareholders (i.e., physicians and clinical care team members and hospital or practice representatives) as well as other external stakeholders (i.e., hospitals, physicians, payers, regulators, consumers, purchasers, etc.) Comments submitted are considered for modification of the version change. NCDR staff and members involved in developing the measures and reports receive all the comments submitted including the name of the individual and organization submitting comment. The NCDR determines which comments to incorporate into modifications and the internal timeline for any modifications. No formal response is provided back to individuals submitting comments through this process. The NCDR may choose to provide a report of comments received and decisions made regarding the various feedback to a broader audience.
Beyond the inherent content validity of this process, we have data showing that the bleeding risk score is highly actionable – a critical feature for moving beyond quality assessment to quality improvement. For example, a comparative effectiveness analysis of bivaluridin use by bleeding risk suggested that bivalirudin was preferentially used in low-risk patients (NNT=224) and least often used in patients at high risk for bleeding (NNT=43; JAMA 2010;303(21):2156-2164). At Saint Luke’s Mid America Heart Institute, the original bleeding model was executed prior to non-emergent PCI in all patients undergoing the procedure. Not only was the ‘risk-treatment’ paradox reversed, but the bleeding rate at that institution decreased by 40% (J Am Coll Cardiol 2013;61: 1847–52). More recently, a 9-center study of providing pre-procedural bleeding risks demonstrated a fully-adjusted 44% lower odds of bleeding when the models were used (BMJ, 2015;350:h1302). The ultimate validity of the model is that the use of the model to target therapy improves outcomes strongly supports the appropriateness and capacity of this model to measure and improve quality.
Empirical Validity:
To further underscore the importance of the bleeding measure, we examined the association of bleeding rates, by quintiles, with in-hospital mortality rates. We hypothesized that hospitals having a higher bleeding complication rate would also have higher rates of in-hospital mortality. Both of bleeding and mortality are important signals of quality.
References:
- Mehta SK, Frutkin AD, Lindsey JB, et al. Bleeding in patients undergoing percutaneous coronary intervention: the development of a clinical risk algorithm from the National Cardiovascular Data Registry. Circulation: Cardiovascular Interventions. 2009;2(3):222–229. doi: 10.1161/CIRCINTERVENTIONS.108.846741
- Rao SV, McCoy LA, Spertus JA, Krone RJ, Singh M, Fitzgerald S, Peterson ED. An updated bleeding model to predict the risk of post-procedure bleeding among patients undergoing percutaneous coronary intervention: a report using an expanded bleeding definition from the National Cardiovascular Data Registry CathPCI Registry. JACC Cardiovasc Interv. 2013 Sep;6(9):897-904. doi: 10.1016/j.jcin.2013.04.016. PMID: 24050858.
- Marso SP, Amin AP, House JA, Kennedy KF, Spertus JA, Rao SV, Cohen DJ, Messenger JC, Rumsfeld JS; National Cardiovascular Data Registry. Association between use of bleeding avoidance strategies and risk of periprocedural bleeding among patients undergoing percutaneous coronary intervention. JAMA. 2010 Jun 2;303(21):2156-64. doi: 10.1001/jama.2010.708. PMID: 20516416.
- Rao SC, Chhatriwalla AK, Kennedy KF, Decker CJ, Gialde E, Spertus JA, Marso SP. Pre-procedural estimate of individualized bleeding risk impacts physicians' utilization of bivalirudin during percutaneous coronary intervention. J Am Coll Cardiol. 2013 May 7;61(18):1847-52. doi: 10.1016/j.jacc.2013.02.017. Epub 2013 Mar 7. PMID: 23500304.
- Spertus JA, Decker C, Gialde E, Jones PG, McNulty EJ, Bach R, Chhatriwalla AK. Precision medicine to improve use of bleeding avoidance strategies and reduce bleeding in patients undergoing percutaneous coronary intervention: prospective cohort study before and after implementation of personalized bleeding risks. BMJ. 2015 Mar 24;350:h1302. doi: 10.1136/bmj.h1302. PMID: 25805158; PMCID: PMC4462518.
See attached tables and figures document in 7.1 table 14 and figure 8.
Face-Validity:
As described above, we undertook an extensive effort to establish the definition and utility of risk-adjusted bleeding as a quality metric. These included an expert team developing the model, a group of experts, the Strategic Oversight Committee, overseeing the work and reporting of the measure – including ascertaining its alignment with both ACC/AHA PCI Guidelines and the Society of Coronary Angiography and Intervention’s (SCAI’s) 2016 Expert Consensus Statement – and an NCDR Oversight Group for NQF measures. It further underwent public comment and approval by the NCDR Management Board of the ACC’s Board of Trustees. Beyond these traditional ascertainments of its face validity, we further leveraged evidence that the prospective use of the model was associated with a substantial reduction in bleeding after PCI, clearly demonstrating the model to serve as a means for improving the safety of PCI.
Empirical Validity:
The model’s empirical validity was tested to assess the correlation between two outcome measures: risk-standardized bleeding and risk-standardized mortality (a recently endorsed measure (CBE 0133)). There was a small, but positive correlation between the two measures, such that higher bleeding rates were correlated with higher in-hospital mortality rates (r=0.223). This is the signal that is clinically sensible.
Risk Adjustment
A hierarchical logistic regression model was created for this model, with the relevant variables and odds ratios posed below in Table 15 (5.4.2a.). The data definitions are available on the NCDR website (https://cvquality.acc.org/NCDR-Home/registries/hospital-registries/cath… ). The beta coefficients and covariance matrix are available from NCDR upon request.
We believe that social factors did not need to be included as variables in risk-adjustment for peri-procedural bleeding after PCI. This was predicated on the feasibility of patient-level social factors. The belief that the consequence of adverse social factors (e.g. leading to greater rates of obesity, hypertension, smoking or other comorbidities) would be directly captured by our rich clinical data, and that the short duration of follow-up (72 hours, during which the patient was hospitalized), would negate potential barriers to healthcare access and treatment that might be more relevant with longer-term outcomes. Accordingly, we feel that in this model of in-hospital risk-adjusted bleeding rate, given the rich clinical data available through the NCDR CathPCI registry, that social risk factors, which are not readily available, would not likely improve this particular risk model.
There was an extensive process to develop the face and content validity of the measure. To ensure our model achieved its purported goals, an expert panel was assembled to assess inclusion criteria, definitions, and risk-adjustment methodology. After settling on the outcome definition and candidate variables, categorical variables were summarized as frequencies and percentages and compared with Pearson chi-squared tests. Continuous variables were summarized as medians (interquartile range) and compared using Wilcoxon rank-sum tests. Ordinal variables were tested using a chi-square test based on the rank of the group mean score. The study population was then randomly split into a development sample consisting of 70% of PCI procedures and a validation sample consisting of the remaining 30% of admissions.
Due to high rates of missing data and possible biased data as a result, the following variables were forced out of the model: stress test results; Seattle Angina Questionnaire (SAQ) results; Rose Dyspnea Scale (RDS) results; assessment of chest pain symptoms; new antiarrhythmic therapy. Desire to exclude variables possibly related to physician choice and decision-making, as opposed to intrinsic patient-level risk, led to forcing out the following variables from the model: concomitant peripheral intervention, peripheral angiogram, heart biopsy, or procedure type not listed; arterial access site; arterial cross over; venous access; multivessel procedure type; pre-procedure hemoglobin; procedure medications. Of note, the original model selected the following variables into the risk model: right heart catheterization, mitral valve or percutaneous replacement of aortic valve using fluoroscopic guidance, insertion of temporary cardiac pacemaker, and arch aortogram. To make the model more parsimonious, we combined concomitant procedures into one yes/no variable. The following variables without clear clinical meaning were forced out: BMI unknown; GFR unknown; heart rate unknown; Troponin I unknown; Troponin T unknown; Ejection Fraction unknown; Systolic Blood Pressure unknown; Closure method not documented. Finally, to account for various metrics of clinical instability and procedural status, a “clinical instability” composite variable was created to reduce compounding effects of the following multiple associated variables: cardiogenic shock; ventricular support; pharmacologic vasopressor support; mechanical ventricular support; level of consciousness; STEMI; PCI status; hypothermia induced; hypothermia induction timing. Instead, the following composite variables of “clinical instability” were created: elective PCI without any cardiovascular instability; urgent PCI without any cardiovascular instability; emergency PCI without any cardiovascular instability; any cardiovascular instability without salvage PCI; cardiogenic shock (not refractory) without salvage PCI; refractory shock or salvage PCI; other. Ultimately, at the discretion of the workgroup members, no variables were identified that required “forcing in” to the model.
Stepwise selection logistic regression was used on 1,000 bootstrapped samples from the development cohort. The final model included variables that were selected in at least 70% of the bootstrapped samples and those identified a priori due to clinical relevance.
The C-statistic was used to describe the discrimination of the model. All statistical analyses were performed using SAS software (version 9.3, SAS Institute, Cary, NC).
See attached tables and figures document in 7.1 table 16.
As described above, bivariate analyses were done to identify candidate variables that differed significantly between those with and without a clinically important bleeding event. Multivariable, hierarchical logistic regression analyses were then performed to retain those with a statistically significant association with bleeding (p<0.05 for each). Table 10 Predicted Probability of Bleeding in Section 5.1.4 demonstrates the difference between those with and without bleeding events.
See attached document in 5.4.4a. tables 17 & 18.
We developed the model in the 70% derivation set and tested its discrimination and calibration (using both the Hosmer-Lemeshow test and the slope of the predicted vs. observed risk).
The c-statistic is 0.772 for the model, which means that the probability that predicting the outcome is better than chance. This method is used to compare the goodness of fit of logistic regression models. The range is between 0.5 to 1.0. A value of 0.5 indicates that the model is no better than chance at making a prediction of membership in a group and a value of 1.0 indicates that the model perfectly identifies those within a group and those not. Models are typically considered reasonable when the C-statistic is higher than 0.7. (Hosmer & Lemeshow, 2000).
See attached document in 5.4.5a. table 19 and figures 9-12.
We believe this model performs very well, accounting for patient characteristics present prior to the conduct of PCI and discriminating within important clinical subsets of patients. Moreover, there is substantial hospital variation before and after risk-adjustment. The distribution of institutional predicted to expected (P/E) ratios identifies some sites with excellent performance and others with rates of bleeding that are 80% or greater than expected. These would be sites where substantial opportunities to improve patient safety likely exist.
Use & Usability
Use
The CathPCI Registry® assesses the characteristics, treatments and outcomes of cardiac disease patients who receive diagnostic catheterization and/or percutaneous coronary intervention (PCI) procedures.
Geographic area is an estimated 90% of all US based cardiac cath labs. CathPCI Registry specific participants are around 1,800. Total patient records are over 22 million.
Facility level of analysis/hospital in-patient.
Usability
Performance results are provided as part of quarterly performance report, which includes a rolling 4 quarters of data. These reports provide a detailed analysis of an individual institution's performance in comparison with the entire registry population from participating hospitals across the nation. Reports include an executive summary dashboard, at-a-glance assessments, and patient level drill-downs. Registry participants also have access to an outcome report companion guide which provides common definitions and detailed metric specifications to assist with interpretation of performance rates. This information along with the other process and outcome measures included in the CathPCI registry enable participants to identify interventions that will lead to improvement in in-patient mortality rates.
There are a number of methods used to educate and provide general support to registry participants.
These include the following:
• Registry Site Manager Calls are available for all NCDR participants. RSM calls are provided as a source of communication between NCDR and participants to provide a live chat Q and A session on a continuous basis.
• New User Calls are available for NCDR participants and are intended for assisting new users with their questions.
• NCDR Annual Conference
The NCDR Annual Conference is a well-attended and energetic two-day program at which participants from across the country come together to hear about new NCDR and registry-specific updates. During informative general sessions, attendees can learn about topics such as transcatheter therapies, the NCDR dashboard, risk models, data quality and validation, and value-based purchasing. Attendees also receive registry updates and participate in advanced case studies covering such topics as Appropriate Use Criteria and outcomes report interpretation.
• Release notes (for outcomes reports)
• Clinical Support
The NCDR Product Support and Clinical Quality Consultant Teams are available to assist participating sites with questions Monday through Friday, 9:00 a.m. - 5:00 p.m. ET.
Health care facilities, physicians, data abstractors, registry steering committee members, and other stakeholders routinely provide feedback to the Registry support team via email or phone (i.e., SalesForce). Additional opportunities for detailed measure discussion can occur on bi-monthly registry site manager calls or annually at the in-person NCDR Quality Summit conference where registry management and physician leadership will explore the measure in detailed followed by an open Q&A session.
Feedback varies from detailed comments on the measure criteria, reflections, and general questions about how end-point decisions were made. When stakeholders fully understand the measure, they have expressed it is valuable in helping to guide their quality-of-care improvement efforts. Feedback following the changes to this measure has been generally positive.
Any criticism of the observational data on performance is escalated to the applicable ACC team(s) (i.e., Registry management, Science leadership, Data Analytic Center) for consideration. If the feedback represents an opportunity for measure improvement, the Data Analytic Center is engaged to provide data insights. These data are reviewed by the Senior NCDR Leadership & Science Leadership team which may lead to updates. If an adjustment is needed the change is approved and cascades to the various teams and implemented. The opportunity for these types of refinements is always available.
As mentioned earlier, this model is used in performance improvement and quality initiatives to reduce the risk of patients undergoing PCI at many hospitals across the country. With the refinement of the cohort, to exclude patients status post cardiac arrest or experiencing cardiogenic shock, the outcome performance identifies those at lower absolute risk and highlights any bleeding event experienced. This should sharpen hospitals focus on the patients who experience any bleeding and allow them to initial quality improvement programs to reduce the risk. While this measure before undergoing recent refinement demonstrated improvement over time, additional data are needed post implementation to perform any new analyses.
ACC determined that the measure required updates including revising the criteria used to define a bleeding event and the model to accurately define risk among “extreme risk” patients, such as those with cardiogenic shock and those who have suffered cardiac arrest prior to PCI. Additional detail justifying these revisions were outlined in Sections 1.10 and 2.2. We have not identified any unexpected findings since these changes were implemented but will continue to monitor any feedback that is received.
Comments
Staff Preliminary Assessment
CBE #2459 Staff Preliminary Assessment
Importance
Strengths
- A clear logic model is provided, depicting the relationship between inputs (e.g., quality improvement activities and tools and technology integration), activities (e.g., implementation of quality improvement activities and tools and technology integration and delivery of patient education materials), and desired outcomes (increased clinician and patient awareness and attention to use of interventions; reduced intra- and post-operative bleeding rates following percutaneous coronary intervention (PCI); and reduction in complications). This model demonstrates how the measure’s implementation will lead to the anticipated outcomes.
The measure is supported by a comprehensive literature review, including systematic reviews with high evidence quality and clinical practice guidelines with evidence grading of strong/high quality empirical studies, demonstrating a clear net benefit in terms of improved outcomes and reduced costs for patients over the age of 18 with post-PCI bleeding events.
The problem this measure addresses presents a significant burden for patients with individuals in the United States receiving more than 600,000 percutaneous coronary interventions (PCIs), with about 60% being elective.
Data from the CathPCI Registry from 2021-2022 show a performance gap, with decile ranges from 0.10% to 3.34% indicating variation in measure performance. This 3 percentage point difference translates to an additional 39,380 bleeds per year.
Description of patient input supports the conclusion that the measured outcome is meaningful with at least moderate certainty. Patient input was obtained through technical expert panel (TEP) during measure development. Patient advocacy groups, such as Mended Hearts Non-profit Organization, were given opportunity to provide inputs through peer review and public comment.
Limitations
- None Identified.
Rationale
- This maintenance measure meets all criteria for ‘Met’ for importance due to the significance of the problem it addresses, its robust evidence base, a documented performance gap, and well-articulated logic model, making it essential for addressing bleeding rates for patients undergoing PCI.
Closing Care Gaps
Strengths
- The measure’s performance was empirically tested across all identified subgroup variables including insurance status, age, sex, and race/ethnicity.
The analysis revealed possible differences in performance scores by insurance status, age, sex, and race/ethnicity. For example, hospitals with the highest proportion of Medicaid recipients had higher median bleeding rates (1.83%) compared to hospitals with the lowest rate of Medicaid recipients (1.70%).
Limitations
- The developer did not explain why these variables were included in the analysis, they did not include a description of methodology and step-by-step approach to employ the analysis, and they did not present significance testing for differences, which limits the comprehensiveness of the assessment.
The developer does not provide an interpretation of the results.
The developer did not provide recommended actions entities can take to close care gaps.
Rationale
- While the developer attempted to assess gaps in care across various subgroups, no significant differences were identified, and the developer did not provide a rationale as to why.
Feasibility Assessment
Strengths
- All required data elements are routinely generated during care delivery, and required elements are available from digital or electronic sources.
The developer described how changes to the specifications impact data structure and availability.
The developer described how any feasibility issues they found informed the final measure specifications.
The developer described how all required data elements can be collected without risk to patient confidentiality as this measure does not include a patient survey. American College of Cardiology Foundation (ACCF) has safeguards in place to protect patient information including a Business Associate Agreement and Data Use Agreement with each National Cardiovascular Data Registry (NCDR) institution and ACCF’s security program.
Limitations
- The developer described the costs specific to NCDR program requirements; however, they did not mention any barriers encountered during implementation. The submission could be strengthened by including discussion around barriers/burdens or providing reasoning as to why none are identified.
Potential fees, licensing, or other requirements to use any aspect of the measure are not fully explained, which could hinder widespread adoption and implementation. There are mentioned costs, including a fee for participation in NCDR and separate charges for case-by-case requests for modifications to the standard export package; however, estimated costs are not provided.
Rationale
- The maintenance measure is rated ‘Not Met, but Addressable’ for feasibility due to a lack of discussion/explanation of implementation burdens and barriers and potential concerns regarding associated fees. The submission could be strengthened by including discussion around barriers/burdens or providing reasoning as to why none are identified and providing estimated costs that could be expected associated with participation in NCDR.
Scientific Acceptability
Strengths
- None identified.
Limitations
- Data used for reliability testing were sourced from the National Cardiovascular Data Registry for CathPCI Registry during the two-year period of January 2021 to December 2022. The entities included in the analysis were characterized by over 1.3 million patients in 1,704 entities, but there may be gaps in overall representativeness of this two-year dataset relative to the period of performance of the measure which the developer states is a rolling 4 quarters (or 1 year) of data, specifically the reliability of the measure is over-estimated.
The developer conducted signal-to-noise reliability testing at the accountable entity-level using a beta-binomial method that is not appropriate for a risk-adjusted measure. The developer also conducted split-half reliability testing. The developer reported an overall ICC of 0.68 but did not demonstrate that at least 70% of the accountable entities have a reliability>0.6 for the one-year period of performance.
Rationale
- This maintenance measure is rated as ‘Not Met, but Addressable’ for reliability because the reliability testing may be insufficient, indicating potential issues with the consistency and accuracy of the results across different settings and populations. However, the identified limitations are deemed addressable, as the developer may consider providing estimates of entity level reliability adjusted for the period of performance based on the split-half method. By addressing this issue, there is potential to enhance the reliability.
Strengths
- Data sources used for validity analysis are adequately described and include National Cardiovascular Data Registry for CathPCI Registry during the period January 2021 to December 2022. Data used for testing are from 1,196,456 patients in 1,704 hospital, and hospitals are diverse in terms of size, region, location, funding type.
- The developer reported on results from a face validity assessment and accountable entity-level (“measure score”) validity testing at the level(s) for which the measure is specified. For face validity, they described their process for collecting expert input on a range of topics, including defining bleeding events, identifying appropriate variables for the risk model, and identifying interventions. For empiric validity testing, they chose to correlate the measure score with a conceptually related endorsed measure, In-Hospital Risk Standardized Mortality for Percutaneous Coronary Intervention (CBE #0133), reasoning that hospitals with higher rates of PCI bleeding events would also have higher rates of in-hospital, PCI-related mortality. They reported a correlation coefficient of 0.22 (significance not reported), in line with the hypothesis.
- The developer conducted statistical risk adjustment, based on a conceptual model, selecting risk factors that are present at the start of care and have a significant correlation with the outcome. The developer reported a c-statistic of 0.77, indicating good model discrimination.
Limitations
- The developer did not perform all of the required elements of validity testing for this respecified maintenance measure. Specifically, they did not present results of data element validity testing for the respecified measure, nor a rationale for not doing so.
- With respect to face validity assessment, it is not clear from the submission how recently this assessment was performed or whether it addressed the respecified version of the measure. In addition, the number and proportion of experts consulted who agreed the measure was a true reflection of quality was not reported.
- For empirical validity testing, the developer did not describe the type of correlation analysis used or report the results of significance testing. In Table 14 in the supplemental attachment where the correlation is reported, it is unclear what values the columns labeled 'HVOL' contain. In addition, in the results validity results section the developer referred to Figure 8 in the supplemental attachment, but did not describe the analysis performed, explain what the figure shows, or interpret the findings in the figure. Figure 8 appears to show the results from a regression analysis of the relationship between the measure score and the PCI mortality measure, but no clear relationship is apparent in the scatterplot or regression line, perhaps due to the vertical scale of the plot.
- The measure's uncertain reliability does not support an inference of validity, because it may indicate that observed relationships are not real.
- The developer did not provide evidence demonstrating variation in the prevalence of risk factors across accountable entities.
Rationale
- This maintenance measure is rated as ‘Not Met But Addressable’ for validity because the validity testing results partially support an inference of validity for the measure, suggesting that the measure somewhat accurately reflects performance on quality and can distinguish good from poor performance to a limited extent.
- The risk adjustment methods used are appropriate and demonstrate that risk factors contribute to unique variation in the outcome. The model performance is acceptable.
Use and Usability
Strengths
- The measure is currently used in the CathPCI Registry.
The developer notes that updates to the measure specifications were made to address feedback received by the medical community that the model did not adequately account for patients at extreme risk or facilities with lower volume. To address this feedback, ACC developed a new hierarchical model that incorporates additional risk variables, such as frailty and cardiovascular instability. CBE #2459 specifically excludes patients experiencing cardiogenic shock or is status post resuscitated cardiac arrest, minimizing the risk of penalization for hospitals and facilities who are willing to provide care to individuals that are likely to have poorer outcomes. This information was provided in section 1.10 Measure Rationale.
The developer provided a summary of how accountable entities can use the measure results to improve performance. Specifically, hospital-based quality improvement campaigns and activities paired with the use of CBE #2459 can improve outcomes for patients.
The developer noted that stakeholders can provide feedback to the Registry support team via email or phone (i.e., SalesForce). Additionally, detailed discussions can occur on bimonthly registry site manager calls or annually at the in-person NCDR Quality Summit conference where registry management and physician leadership will explore the measure in detail followed by an open Q&A.
The developer reported no unexpected findings.
Limitations
- The developer provided a rationale, stating that due to the refinement of the cohort to exclude patient status post cardiac arrest or experiencing cardiogenic shock, additional data are needed post implementation to perform any new analyses. Before undergoing recent refinement, there was demonstrated improvement over time.
Rationale
- This maintenance measure is rated ‘Not Met, but Addressable’ for use and usability because while the measure is actively used in at least one accountability application, the submission lacks new data to assess progress on improvement following recent refinement to the measure specifications.
Committee Independent Review
Support
Importance
Based on my understanding the information seems reasonable
Closing Care Gaps
Based on my understanding the information seems reasonable
Feasibility Assessment
Based on my understanding the information seems reasonable
Scientific Acceptability
Based on my understanding the information seems reasonable
Based on my understanding the information seems reasonable
Use and Usability
Based on my understanding the information seems reasonable
Summary
I am in support of the measure based on my understanding of the information provided.
Support-Bleeding Rate for Patients Undergoing PCI-2459
Importance
I agree that hospitals should continue to report this updated measure to provide "a foundation for bleeding reduction quality improvement". It is noted that intra- and "post-procedure bleeding is the most common non-cardiac complication of PCI". The exclusion criteria of patients " who died within 24 hours of the PCI procedure" is concerning as they could have died to bleeding complications. I appreciate the updated comparison of transfusion for hemoglobin n of <7-8 g/dL vs. <10 g/dL) "on two outcomes – myocardial infarction or death within 30 days of the procedure". I support the updated parameters for transfusion for "those individuals with acute coronary syndrome and a hemoglobin <10 g/d".
Closing Care Gaps
Social risk included Medicaid insurance status. In addition other factors included "age, sex, and race/ethnicity". Regarding performance gaps, it was determined that there "are big differences in bleeding rates between hospitals".
Feasibility Assessment
It is noted that this "is a maintenance measure, and the measure specifications have not changed". In addition, all "required data elements for this measure are routinely generated and acquired during the hospitalization".
Scientific Acceptability
It is noted that for reliability, they "performed the signal-to-noise analysis on the same cohort".
For validity, it was noted that for content validity of this outcome and defining a bleeding event "was achieved by the specialized expertise of those individuals who developed this model as well as the structured discussions that the group conducted". Regarding risk adjustment, it was interesting to note that "social factors did not need to be included as variables in risk-adjustment for peri-procedural bleeding after PCI". I agree with not excluding patients with mechanical assistance.
Use and Usability
This measure will be used for Public reporting and quality improvement.
Summary
This is yet another important measure for best outcomes. Consideration should also be given for patients with bleeding disorders with preventive protocols. Again, patients who die within 1 day following the procedure should not be excluded if due to a bleeding event.
Continue use with some needed details
Importance
Updated maintenance measure to algin with clinical guideline recommendations regarding blood transfusions for those individuals with acute coronary syndrome and a hemoglobin less than or equal to 10 g/dL.
Post PCI bleeding events were associated with increased risk of in-hospital mortality, with an estimated 12.1% of deaths related to bleeding complications. Analysis of performance gap demonstrates variability in hospital performance, with a 3% difference in bleeding rates, equating to an additional 39,380 bleeds per year.
Closing Care Gaps
Considered economic and demographic factors including insurance status, age, sex, and race/ethnicity as indicators of social risk in the models. Results showed differences in facility-level performance by several key indicators, however, the interpretation of these results and how they factors into care gaps is not provided.
Feasibility Assessment
All data elements are available in defined fields in electronic clinical data (e.g., clinical registry). All of the required data elements for this measure are routinely generated and acquired during the hospitalization. The developers have created education and guidance to ensure that hospitals are able to accurately capture these data. There is no charge for a standard export package. More than 1,600 hospitals are currently submitting data for this measure. There is no mention of whether any of these hospitals have reported barriers to use.
It is a proprietary measure with associated fees. The developers note that participation in NCDR is required and has an associated fees. Custom reporting modifications will also incur a fee. However, the costs are not specified..
Scientific Acceptability
The developers used a reliability assessment approach that is not suitable for a measure that is risk adjusted. Thus, while they report a median signal to noise ratio of 0.97 with a fairly narrow interquartile range of 0.95 and 0.98, it is likely that their approach is overstating the measure reliability. Likewise, while they reported a overall score of 0.68 in the split half reliability assessment, they failed meet the minimum thresholds for at least 70% of facilities obtaining a score of 0.60
Multipronged effort to establish metric content validity including an expert team developing the model, a group of experts and Strategic Oversight Committee. Demonstrated small, but positive correlation between higher bleeding rates and higher in-hospital mortality rates (r=0.223). Also have data showing that the bleeding risk score is highly actionable with demonstrated reduction in bleeding after PCI.
The measure excludes patients who die within 24 hours of PCI, those who undergo CABG during the care episode and who die from hemorrhagic complications. These remove the highest risk patients at highest risk of post PCI bleeding-related harm, which could limit the measure validity by underestimating true bleeding rates.
The rationale that social risk factors are captured through clinical data for the risk adjustment modeling is flawed. Social risk factors not just affect the presence of clinical factors but also their severity as well as associated factors such as frailty and inflammation that could impact bleeding risk.
Use and Usability
Several reports of performance, companion guide, and outreach including a conference and clinical support, user calls etc. as well as a data analytic center to review feedback. However, the developers do not include any new data in the maintenance application to demonstrate changes in improvements after the recent measure modifications.
Summary
Need additional details on exclusions of high risk patients that could compromise measure validity and how clinical factors are sufficient substitutes for social factors in risk adjustment models.
Risk Standardized Bleeding Rate for Patients Undergoing PCI
Importance
The measure relies its arguments heavily on registry data.
Concerns: 1) The exclusion of patients such as CABGs and deaths needs further justification. It is particularly concerning that the measure excludes "Patients who died within 24 hours of the PCI procedure" and "Patients who have CABG during the episode of care".
2) I didn't see any justification provided on why adopting this measure would add value to better outcome when patient outcomes through CathPCI Registry may already be available or enhanced.
Closing Care Gaps
No convincing argument that using this measure would close a care gap as long as risk-adjusted PCI outcome measures continue to be available.
Feasibility Assessment
No concern here.
Scientific Acceptability
The exclusion of social factors in favor of indirect risk indicators such as obesity needs better explanation.
Concerns on the risk-adjustment:
- The exclusion of variables because of missing information understandably may lead to bias. However, no effort is presented to address the missing value problem through statistical means. If too many variables have missing values, what would it means to the importance of the measure as a whole across hospitals?
- The exclusion of variables due to the "Desire to exclude variables possibly related to physician choice and decision-making" casts doubt on the measure as a whole.
Use and Usability
See my concerns above on the measure's construction.
Summary
This measure needs a closer review for the value it provides in light of CathPCI and other related measures.
Needs more details
Importance
Clearly an important topic.
Closing Care Gaps
The authors just need a more details description as noted in the staff comments.
Feasibility Assessment
Not certain that this is feasible in its current state.
Scientific Acceptability
Agree with staff comments regarding acceptability reliability
Agree with staff comments regarding acceptability validity
Use and Usability
As with feasibility, the usability is unclear.
Summary
Overall, this is an important topic which would benefit from this type of measure. The authors just need to address validity, feasibility, and usability concerns.
potentially support endorsement
Importance
no comments
Closing Care Gaps
Needs additional rationale from developer that can be provided during committee meeting
Feasibility Assessment
has been in use and do not see how minor changes would significantly impact feasibility
Scientific Acceptability
agree with staff comments
agree with staff comments; also agree with public comment and would like to understand developer's rationale for excluding patients dying with 24hours of PCI - are they captured in another measure?
Use and Usability
no comments
Summary
this measure has been in use and just seems to require developer to provide some additional information/justification/explanation
Patient Perspective
Importance
NA
Closing Care Gaps
NA
Feasibility Assessment
NA
Scientific Acceptability
NA
NA
Use and Usability
NA
Summary
I have been experiencing this before and after the last several years of my many surgeries and it has keep me safe and alive. If there is better and more up-to-date information, we should be using it in our measures.
Patient Perspective
Importance
NA
Summary
This measure meets the points needed from the patient’s perspective.
Needed measure - needs clarification on exclusions & feasability
Importance
The benefits are clearly stated.
Closing Care Gaps
Would like to see a deeper dive into the ways to improve care gaps and also how the addition of the exclusion for death doesn't affect this care gap. What if the death within 24 hours was from bleeding?
Feasibility Assessment
Described a fee but did not mention what it was or ways the facilities can work with the fee.
Scientific Acceptability
Agree with Battelle's feedback.
No results of data element validity testing for the respecified measure.
Use and Usability
-
Summary
I agree this measure is beneficial. There are certain items in the statistical analysis that need clarification, as well as why excluding all deaths within 24 hours without further analysis of cause of death.
Evaluation
Importance
This measure addresses an important patient area and provides strong evidence for the continued endorsement to address bleeding rates.
Closing Care Gaps
Addressed but rationale was not adequately provided in the approach among the identified groups.
Feasibility Assessment
All required data elements are routinely captured during care delivery and are available from digital or electronic sources. The American College of Cardiology Foundation (ACCF) maintains appropriate safeguards to protect patient information. However, potential fees, licensing requirements, and other obligations associated with using this measure are not clearly detailed, which may limit broader adoption. No estimated cost ranges are provided, creating uncertainty for implementing organizations.
Scientific Acceptability
The developer used a method for risk adjustment that was not appropriate. The developer reported an overall ICC of 0.68 but did not demonstrate that at least 70% of the accountable entities have a reliability>0.6 for the one-year period of performance as expected.
The developer did not perform all of the required elements of validity testing for this respecified maintenance measure. Specifically, they did not present results of data element validity testing for the respecified measure, nor a rationale for not doing so.
Use and Usability
The measure is currently in use in the CathPCI registry. Additional data is necessary for additional refinement and to demonstrate improvement based on the updated specifications.
Summary
This is an important measure but I would recommend the developer address the areas identified for continued endorsement.
PCI bleeding
Importance
Addressed
Closing Care Gaps
addressed
Feasibility Assessment
addressed
Scientific Acceptability
addressed
addressed
Use and Usability
addressed
Summary
measure aligned with other key stakeholder groups
overview
Importance
I agree with the staff preliminary assessment
Closing Care Gaps
I agree with the staff preliminary assessment.
Feasibility Assessment
I agree with the staff preliminary assessment
Scientific Acceptability
I agree with the staff preliminary assessment.
I agree with the staff preliminary assessment
Use and Usability
I agree with the staff preliminary assessment.
Summary
nothing to add
Important
Importance
clear approach, thank you for addressing the previously submitted concerns
Closing Care Gaps
This is optional, correct?
The details appeared to further detail the problem, but not offer approaches to address and close the care gaps.
Feasibility Assessment
Agree with evaluations from staff regarding fees, costs, details.
Scientific Acceptability
Align with staff comments.
Agree with staff comments, also reflecting on the submitted public comments regarding the potential over-reach of data category inclusion/exclusion. Will rely on connected-practice MD community for further refinement needs.
Use and Usability
Assumes the public comment is resolved for this scoring.
The staff assessment mentions the prior comments about patients at extreme risk and if remembering that discussion, this was a significant area of concern for those in the field due to the conflict in cath standards of care for some patient populations.
Summary
The staff assessment mentions the prior comments about patients at extreme risk and if remembering that discussion, this was a significant area of concern for those in the field due to the conflict in cath standards of care for some patient populations.
(May be addressed during discussion for use/usability)
Public Comments
Risk Standardized Bleeding Rate for Patients Undergoing PCI
I am unable to find or even think of any justification for these two exclusions:
Patients who died due to a major hemorrhagic complication - which often occurs within 24 hours of the PCI - should be included in the measure!
So should patients who require CABG to treat a bleeding complication of the PCI (e.g., coronary dissection with tamponade).
The denominator exclusions for cardiogenic shock and cardiac arrest seem reasonable because these are preoperative findings rather than postoperative complications. The latter do not represent appropriate denominator exclusions.