The measure estimates a hospital-level risk-standardized complication rate (RSCR) associated with elective primary THA and/or TKA procedures for Medicare patients (Fee-for-Service [FFS] and Medicare Advantage [MA]) aged 65 and older. The outcome (complication) is defined as any one of the specified complications occurring from the date of index admission to up to 90 days after the index admission. Complications are counted in the measure only if they occur during the index hospital admission or during a readmission. The complication outcome is a dichotomous (yes/no) outcome; if a patient experiences one or more of these complications in the applicable time period, the complication outcome for that patient is counted in the measure as a ''yes.'’
Measure Specs
General Information
The goal of this measure is to improve patient outcomes by providing patients, physicians, hospitals, and policymakers with information about hospital-level, risk-standardized complication rates following primary elective THA and/or TKA. This re-specified hospital-level risk-standardized complication rate (RSCR) following elective primary total hip arthroplasty (THA) and/or total knee arthroplasty (TKA) measure (hereafter “THA/TKA Complications”) captures complications following a primary elective (inpatient only) THA/TKA procedure in hospitals. Measurement of patient outcomes allows for a broad view of quality of care that encompasses more than what can be captured by individual process-of-care measures. The re-specified measure now includes both Medicare Advantage (MA) and Fee-for-Service (FFS) beneficiaries. Expanding the measure’s cohort to include both MA beneficiaries in the Centers for Medicare & Medicaid Services (CMS) hospital outcome measures helps ensure that hospital quality is measured across all Medicare beneficiaries and not just the FFS population.
According to CMS, Medicare/Medicaid Part B National Summary, in 2019 the annual volume of primary TKA was 480,958 and that of primary THA was 262,369 with forecasts suggesting an increase in demand for procedures due to gains in post-surgery care, aging population, and increasing osteoarthritis (Singh et al., 2019). Complications following a THA/TKA can vary in frequency and drive the overall cost leading to a substantial burden on both the patient and the healthcare system (Schwarzkopf et al., 2019). Improving complex and critical aspects of care, such as communication between providers, rapid response to complications, patient safety, and coordinated transitions to the outpatient environment, all contribute to better patient outcomes.
The THA/TKA Complications measure addresses a priority area for outcomes measurement. It is an outcome that is likely attributable to care processes and is an important outcome for patients. Measuring and reporting complication rates inform healthcare providers about opportunities to improve care, strengthen incentives for quality improvement, and ultimately improve the quality of care received by Medicare patients. This measure also provides patients with information that could guide their choices. Furthermore, the measure increases transparency for consumers and has the potential to lower healthcare costs associated with complications following this common elective procedure.
References
Schwarzkopf, R., Behery, O. A., Yu, H., Suter, L. G., Li, L., & Horwitz, L. I. (2019). Patterns and costs of 90-day readmission for surgical and medical complications following total hip and knee arthroplasty. The Journal of arthroplasty, 34(10), 2304-2307.
Singh, J. A., Yu, S., Chen, L., & Cleveland, J. D. (2019). Rates of total joint replacement in the United States: future projections to 2020–2040 using the national inpatient sample. The Journal of rheumatology, 46(9), 1134-1140.
Medicare fee-for-service (FFS) claims and Medicare Advantage (MA) encounters, in addition to Medicare administrative data, are used to derive all components of the measure.
Numerator
The outcome for this measure is any one of the specified complications occurring during the index admission to 90 days after the index admission. Complications are counted in the measure only if they occur during the index hospital admission or during a readmission. The complication outcome is a dichotomous (yes/no) outcome. If a patient experiences one or more of these complications in the applicable time period, the complication outcome for that patient is counted in the measure as a "yes."
The specific complications captured by this measure are:
- Acute myocardial infarction (AMI) during the index admission or a subsequent inpatient admission that occurs within seven days from the start of the index admission;
- Pneumonia or other acute respiratory complication during the index admission or a subsequent inpatient admission that occurs within seven days from the start of the index admission;
- sepsis/septicemia/shock during the index admission or a subsequent inpatient admission that occurs within seven days from the start of the index admission;
- surgical site bleeding or other surgical site complication during the index admission or a subsequent inpatient admission within 30 days from the start of the index admission;
- pulmonary embolism during the index admission or a subsequent inpatient admission within 30 days from the start of the index admission;
- death during the index admission or within 30 days from the start of the index admission;
- mechanical complication during the index admission or a subsequent inpatient admission that occurs within 90 days from the start of the index admission; or
- periprosthetic joint infection/wound infection or other wound complication during the index admission or a subsequent inpatient admission that occurs within 90 days from the start of the index admission.
The outcome for the THA/TKA Complications measure is dichotomous (yes for any complication(s); no for no complications). Therefore, if a patient experiences one or more of the complications outlined in Section 1.14, the outcome variable will get coded as a "yes". Complications are counted in the measure only if they occur during the index hospital admission (and are not present on admission) or during a readmission.
The complications captured in the numerator (see Section 1.14) are identified during the index admission OR associated with a readmission up to 90 days post-date of index admission, depending on the complication.
The follow-up period for complications from the date of the index admission is as follows:
- The follow-up period for acute myocardial infarction (AMI), pneumonia, and sepsis/septicemia/shock is seven days from the date of index admission because these conditions are more likely to be attributable to the procedure if they occur within the first week after the procedure. Additionally, analyses indicated a sharp decrease in the rate of these complications after seven days.
- Death, surgical site bleeding, and pulmonary embolism are followed for 30 days following admission because clinical experts agree these complications are still likely attributable to the hospital performing the procedure during this period and rates for these complications remained elevated until roughly 30 days post admission.
- The measure follow-up period is 90 days after admission for mechanical complications and periprosthetic joint infection/wound infection. Experts agree that mechanical complications and periprosthetic joint infection/wound infections due to the index THA/TKA occur up to 90 days following the index admission.
- The measure counts all complications occurring during the index admission regardless of when they occur. For example, if a patient experiences an AMI on day 10 of the index admission, the measure will count the AMI as a complication, although the specified follow-up period for AMI is seven days. Clinical experts agree with this approach, as such complications likely represent the quality of care provided during the index admission.
The measure does not count complications in the complications outcome that are coded as present on admission (POA) during the index admission; this prevents identifying a condition as a complication of care if it was present on admission for the THA/TKA procedure.
For a full list of codes defining complications, see the Data Dictionary attached in Section 1.13. The 2024 THA/TKA Measure Dictionary tab “HKComp Outcome Inclusion” provides the specific codes used to identify complications.
Denominator
The cohort includes admissions for patients that meet all of the following inclusion criteria:
1. Enrolled in Medicare (Fee-for-Service [FFS] or Medicare Advantage[MA]) for the 12 months prior to the date of admission and during the index admission;
2. Aged 65 or older;
3. Having a qualifying elective primary THA/TKA procedure.
A qualifying elective primary THA/TKA procedure is defined as a procedure without any of the following:
- Fracture of the pelvis or lower limbs coded in the principal or secondary discharge diagnosis fields on the index admission claim (Note: Periprosthetic fractures must be additionally coded as present on admission [POA] in order to disqualify a THA/TKA from cohort inclusion, unless exempt from POA reporting.);
- A concurrent partial hip or knee arthroplasty procedure;
- A concurrent revision, resurfacing, or implanted device/prosthesis removal procedure;
- Mechanical complication coded in the principal discharge diagnosis field on the index admission claim;
- Malignant neoplasm of the pelvis, sacrum, coccyx, lower limbs, or bone/bone marrow or a disseminated malignant neoplasm coded in the principal discharge diagnosis field on the index admission claim; or,
- Transfer from another acute care facility for the THA/TKA.
The cohort includes admissions for patients who meet all of the following inclusion criteria below.
- Enrolled in Medicare fee-for-service (FFS) and/or MA for the 12 months prior to the date of admission; and enrolled in FFS or MA during the index admission;
- Rationale: The 12-month prior enrollment criterion ensures that the comorbidity data used in risk adjustment can be captured from inpatient, outpatient, and physician claims data for up to 12 months prior to the index admission, to augment the index admission claim itself.
- Aged 65 or older;
- Rationale: Medicare beneficiaries younger than 65 are not included in the measure because they are considered to be too clinically distinct from Medicare beneficiaries who are 65 or older.
- Having a qualifying elective primary THA/TKA procedure. Elective primary THA/TKA procedures are defined as those procedures without any of the following:
- Fracture of the pelvis or lower limbs coded in the principal or secondary discharge diagnosis fields on the index admission claim (Note: Periprosthetic fractures must be additionally coded as present on admission (POA) in order to disqualify a THA/TKA from cohort inclusion, unless exempt from POA reporting.);
- Rationale: Patients with fractures have higher mortality, complication, and readmission rates, and the procedures are typically not elective.
- A concurrent partial hip or knee arthroplasty procedure;
- Rationale: Partial arthroplasty procedures are primarily done to treat fractures and are typically performed on older patients who are frailer and have more comorbid conditions
- A concurrent revision, resurfacing, or implanted device/prosthesis removal procedure;
- Rationale: Revision procedures may be performed at a disproportionately small number of hospitals and are associated with higher mortality, complication, and readmission rates. Resurfacing procedures are a different type of procedure involving only the joint’s articular surface and are typically performed on younger, healthier patients. Elective procedures performed on patients undergoing removal of implanted device/prosthesis procedures may be more complicated.
- Mechanical complication coded in the principal discharge diagnosis field on the index admission claim;
- Rationale: A complication coded as the principal discharge diagnosis suggests the procedure was more likely the result of a previous procedure and indicates the complication was POA. These patients may require more technically complex arthroplasty procedures and may be at increased risk for complications, particularly mechanical complications
- Malignant neoplasm of the pelvis, sacrum, coccyx, lower limbs, or bone/bone marrow or a disseminated malignant neoplasm coded in the principal discharge diagnosis field on the index admission claim;
- Rationale: Patients with these malignant neoplasms are at increased risk for complications, and the procedure may not be elective.
- Transfer from another acute care facility for the THA/TKA.
- Rationale: The THA/TKA Complications measure does not include admissions for patients transferred to the index hospital, as they likely do not represent elective THA/TKA procedures (see additional information below).
- Fracture of the pelvis or lower limbs coded in the principal or secondary discharge diagnosis fields on the index admission claim (Note: Periprosthetic fractures must be additionally coded as present on admission (POA) in order to disqualify a THA/TKA from cohort inclusion, unless exempt from POA reporting.);
The measure uses International Classification of Diseases, Tenth Revision, Clinical Modification/Procedure Coding System (ICD-10-CM and ICD-10-PCS) codes on claims to define a THA/TKA procedure and to identify a THA/TKA procedure as non-elective or non-primary (and disqualify the admission from cohort inclusion). Codes that define the cohort can be found in the “HKComp Cohort Inclusions” tab in the Data Dictionary.
Exclusions
This measure excludes index admissions for patients:
- Without at least 90 days post-discharge enrollment in Medicare Fee-for-Service (FFS) or Medicare Advantage (MA);
- Discharged against medical advice (AMA);
- With more than two THA/TKA procedure codes during the index hospitalization;
- With a principal diagnosis code of COVID-19 (ICD-10-CM code U07.1) or with a secondary diagnosis code of COVID-19 coded as present on admission (POA)on the index admission claim.
This measure excludes index admissions for patients:
- Without at least 90 days post-discharge enrollment in Medicare Fee-for-Service (FFS )and/or Medicare Advantage (MA);
- Rationale: The 90-day complication outcome cannot be assessed in this group since claims data are used to determine whether a complication of care occurred.
- Discharged against medical advice (AMA);
- Rationale: Providers did not have the opportunity to deliver full care and prepare the patient for discharge.
- With more than two THA/TKA procedure codes during the index hospitalization;
- Rationale: Although clinically possible, it is highly unlikely that patients would receive more than two elective THA/TKA procedures in one hospitalization. Coding in such cases may reflect a coding error.
- With a principal diagnosis code of COVID-19 (ICD-10-CM code U07.1) or with a secondary diagnosis code of COVID-19 coded as POA on the index admission claim.
- Rationale: COVID-19 patients are removed from the THA/TKA cohort in response to the COVID-19 PHE, and to maintain alignment with the THA/TKA Complications measure included in the FY 2025 Hospital VBP Program.
After the above exclusions are applied, the measure randomly selects one index admission per patient per time period for inclusion in the cohort so that each episode of care is mutually independent with the same probability of the outcome. Additional admissions within that time period are excluded.
Measure Calculation
The measure estimates hospital-level risk-standardized complication rate (RSCRs) following elective primary THA/TKA using a hierarchical logistic regression model. In brief, the approach simultaneously models data at the patient and hospital levels to account for variance in patient outcomes within and between hospitals (Normand and Shahian, 2007). At the patient level, it models the log odds of a complication occurring within 90 days of the index admission using age, sex, selected clinical covariates, and a hospital-specific intercept. At the hospital level, it models the hospital-specific intercepts as arising from a normal distribution. The hospital intercept represents the underlying risk of a complication at the hospital, after accounting for patient risk. The hospital-specific intercepts are given a distribution to account for the clustering (non-independence) of patients within the same hospital. If there were no differences among hospitals, then after adjusting for patient risk, the hospital intercepts should be identical across all hospitals.
The RSCR is calculated as the ratio of the number of “predicted” to the number of “expected” admissions with a complication at a given hospital, multiplied by the national observed complication rate. For each hospital, the numerator of the ratio is the number of complications within 90 days predicted on the basis of the hospital’s performance with its observed case mix, and the denominator is the number of complications expected based on the nation’s performance with that hospital’s case mix. This approach is analogous to a ratio of “observed” to “expected” used in other types of statistical analyses. It conceptually allows for a comparison of a particular hospital’s performance given its case mix to an average hospital’s performance with the same case mix. Thus, a ratio of less than one indicates lower-than-expected complication rates or better quality, and a ratio of greater than one indicates greater-than-expected complication rates or worse quality.
The “predicted” number of admissions with a complication (the numerator) is calculated by using the coefficients estimated by regressing the risk factors and the hospital-specific intercept on the risk of having an admission with a complication. The estimated hospital-specific intercept is added to the sum of the estimated regression coefficients multiplied by the patient characteristics. The results are log-transformed and summed over all patients attributed to a hospital to get a predicted value. The “expected” number of admissions with a complication (the denominator) is obtained in the same manner, but a common intercept using all hospitals in our sample is added in place of the hospital-specific effect. The results are log-transformed and summed over all patients in the hospital to get an expected value. The predicted over-expected ratio is then multiplied by the national observed rate to calculate the risk-standardized complication rate. To assess hospital performance for each performance period, we re-estimate the model coefficients using the years of data in that period.
This calculation transforms the ratio of predicted over expected into a rate that is compared to the national observed complication rate. The hierarchical logistic regression models are described fully in the original methodology report posted on QualityNet: https://www.qualitynet.org/inpatient/measures/complication/methodology.
References
Normand S-LT, Shahian DM. 2007. Statistical and Clinical Aspects of Hospital Outcomes Profiling. Stat Sci 22(2): 206-226.
This measure is not stratified.
The measure does not have a minimum sample size.
Supplemental Attachment
Point of Contact
Not applicable.
Raquel Myers
7500 Security Boulevard
Windsor Mill, MD 21244
United States
Amy Moyer
Yale-New Haven Health Services Corporation/Center for Outcomes Research and Evaluation
195 Church St., 5th Floor
New Haven, CT 06510
United States
Importance
Evidence
THA and TKA are among the most commonly performed orthopedic surgeries worldwide aimed at relieving pain and restoring function in patients with severe arthritis and joint damage. According to the American Academy of Orthopedic Surgeons (AAOS), between 2012 and 2022 more than 1.6 million total knee arthroplasties and over 1 million total hip arthroplasties were performed in the United States (AAOS Report, 2023). Based on 2000-to-2014 data, one study predicted that annual THA volumes are projected to grow to 635,000 procedures a year by 2030 and TKA volumes to 1.26 million procedures a year by 2030 (Sloan et al., 2018).
Complications following a THA/TKA can vary in frequency. Periprosthetic joint infections can range from 0.2% to 2% in THA and TKA including for the Medicare population. (Bourget-Murray et al.,2023; Bozic et al., 2014; Kurtz et al., 2018). Venous thromboembolism (VTE), including deep vein thrombosis and pulmonary embolism, is reported to occur in about 0.6% to 3.0% of patients (Simon et al., 2023). Reported 90-day death rates after THA or TKA were 0.65% (95% confidence interval (CI) 0.50 to 0.81) and 0.39% (95% CI 0.32 to 0.49), respectively (Berstock et al., 2014; Berstock et al., 2018). Rates for septicemia range from 0.1%, during the index admission to 0.3%, 90 days following discharge for primary TKA (Bozic et al., 2014). In the data submitted for this THA/TKA Complications measure (representing 272,164 procedures performed between January 1, 2022, and December 31, 2023), hospital-level, risk-standardized complication rates ranged from 1.5% to 8.7%.
Since THAs and TKAs are commonly performed and costly procedures, it is imperative to address quality of care to improve complication rates. There is evidence that complications following a THA/TKA procedure can be impacted by improvements in critical aspects of care, such as communication between providers, prevention of response to complications, patient safety, and coordinated transitions to the outpatient environment (Bozic et al., 2014; Kurtz et al., 2018). Several studies indicate that by implementing quality improvement efforts including multidisciplinary care teams, pre- and post-counseling and education around THA/TKA procedures, and shared decision-making, hospitals can reduce complications and overall associated costs (Kulshestra et al., 2022; Wong et al., 2022; Sorensen et al., 2019). Since THAs and TKAs are commonly performed and costly procedures, it is imperative to address quality of care.
Addressing complications can also mitigate the economic challenges associated with these common orthopedic procedures. Medicare expenditures for THA and TKA are expected to increase by almost 50% by 2030, driven by the growing number of procedures and rising costs (Singh et al., 2019). This projection underscores the importance of implementing strategies to manage and reduce costs, such as improving surgical outcomes and postoperative care for both individual patients and the healthcare system (Pagano et al., 2018; Liu and Young, 2020). The variation in complication rates across hospitals indicates there is room for quality improvement and targeted efforts to reduce these complications could result in better patient care and potential cost savings (Navathe et al, 2017; Cyriac et al., 2016; Borza et al., 2019). Measurement of patient outcomes allows for a comprehensive view of quality of care that reflects complex aspects of care such as communication between providers and coordinated transitions to the outpatient environment. These aspects are critical to patient outcomes and are broader than what can be captured by individual process of care measures. Please see Section 6.2.1 for additional evidence for hospital-level interventions that impact THA/TKA Complication rates.
References
American Association of Orthopaedic Surgeons. American Academy of Orthopaedic Surgeons 2023 Annual Report. In https://issuu.com/aaos1/docs/aaos_2023_annual_report?fr=sMDJhYjcwNjgyMTI.
Bozic, K. J., Grosso, L. M., Lin, Z., Parzynski, C. S., Suter, L. G., Krumholz, H. M., ... & Drye, E. E. (2014). Variation in hospital-level risk-standardized complication rates following elective primary total hip and knee arthroplasty. JBJS, 96(8), 640-647
Berstock, J. R., Beswick, A. D., Lenguerrand, E., Whitehouse, M. R., & Blom, A. W. (2014). Mortality after total hip replacement surgery: A systematic review. Bone & Joint Research, 3(6), 175-182. https://doi.org/10.1302/2046-3758.36.2000288
Berstock, J. R., Beswick, A. D., López-López, J. A., Whitehouse, M. R., & Blom, A. W. (2018). Mortality after total knee arthroplasty: A systematic review of incidence, temporal trends, and risk factors. The Journal of Bone and Joint Surgery. American Volume, 100(12), 1064-1070. https://doi.org/10.2106/JBJS.17.00630
Bourget-Murray, J., Bansal, R., Soroceanu, A., Piroozfar, S., Railton, P., Johnston, K., ... & Powell, J. (2021). Assessment of risk factors for early-onset deep surgical site infection following primary total hip arthroplasty for osteoarthritis. Journal of Bone and Joint Infection, 6(9), 443-450.
Bozic KJ, Grosso LM, Lin Z, et al. Variation in hospital-level risk-standardized complication rates following elective primary total hip and knee arthroplasty. J Bone Joint Surg Am. 2014;96(8):640‐647. doi:10.2106/JBJS.L.01639.
Borza T, Oerline MK, Skolarus TA, et al. Association Between Hospital Participation in Medicare Shared Savings Program Accountable Care Organizations and Readmission Following Major Surgery. Ann Surg. 2019;269(5):873‐878. doi:10.1097/SLA.0000000000002737.
Cyriac, James MD; Garson, Leslie MD; Schwarzkopf, Ran MD; Ahn, Kyle MD; Rinehart, Joseph MD; Vakharia, Shermeen MD, MBA; Cannesson, Maxime MD, PhD; Kain, Zeev MD, MBA. Total Joint Replacement Perioperative Surgical Home Program: 2-Year Follow-Up, Anesthesia & Analgesia: July 2016 - Volume 123 - Issue 1 - p 51-62 doi: 10.1213/ANE.0000000000001308.
Kulshrestha, V., Sood, M., Kumar, S., Sood, N., Kumar, P., & Padhi, P. P. (2022). Does risk mitigation reduce 90-day complications in patients undergoing total knee arthroplasty?: A cohort study. Clinics in Orthopedic Surgery, 14(1), 56.
Kurtz, S. M., Lau, E. C., Son, M. S., Chang, E. T., Zimmerli, W., & Parvizi, J. (2018). Are we winning or losing the battle with periprosthetic joint infection: trends in periprosthetic joint infection and mortality risk for the medicare population. The Journal of arthroplasty, 33(10), 3238-3245.
Liu, J. L., & Young, A. M. (2020). Preoperative optimization and its effect on outcomes in total joint arthroplasty. Orthopedic Clinics of North America, 51(2), 215-223. https://doi.org/10.1016/j.ocl.2019.12.003
Navathe AS, Troxel AB, Liao JM, et al. Cost of Joint Replacement Using Bundled Payment Models. JAMA Intern Med. 2017;177(2):214–222. doi:10.1001/jamainternmed.2016.8263.
Pagnano, M. W., & Hanssen, A. D. (2018). Enhanced recovery after surgery (ERAS) protocols for total knee arthroplasty: A systematic review. Knee Surgery, Sports Traumatology, Arthroscopy, 26(2), 450-457. https://doi.org/10.1007/s00167-017-4542-2
Sloan, M., Premkumar, A., & Sheth, N. P. (2018). Projected volume of primary total joint arthroplasty in the US, 2014 to 2030. JBJS, 100(17), 1455-1460.
Simon, S. J., Patell, R., Zwicker, J. I., Kazi, D. S., & Hollenbeck, B. L. (2023). Venous thromboembolism in total hip and total knee arthroplasty. JAMA Network Open, 6(12), e2345883-e2345883.
Sorensen, L., Idemoto, L., Streifel, J., Williams, B., Mecklenburg, R., & Blackmore, C. (2019). A multifaceted intervention to improve the quality of care for patients undergoing total joint arthroplasty. BMJ open quality, 8(3), e000664.
Wong, W., Bridges, C., Serebin, M., Gordon, A., Jones, S., Ebert, T., & Scheidt, K. (2022). A quality improvement project to decrease length of Stay after total hip and total knee arthroplasty surgery at a veteran affairs academic medical center. Perioperative Care and Operating Room Management, 26, 100230
Measure Impact
Measuring and reporting complication rates to inform patients, providers, and hospitals can help improve care, strengthen incentives for quality improvement, and improve the quality of care received by Medicare patients. As many providers may not be monitoring their patients’ complications after surgery, they may underestimate adverse events, suggesting the need for better measurement to drive quality improvement. In addition, for THA and TKA, most patients have sufficient time to consider their options and understand the quality differences between hospitals. Therefore, both patients and providers benefit from this outcome measure – a broad, patient-centered outcome that reflects procedure-specific complications among patients undergoing THA/TKA.
Complications following surgery have been shown to impact broader patient outcomes such as quality of life. For example, a 2019 study found that patients who experience postoperative complications were more likely to report lower quality of life in comparison to patients who did not experience complications, and the magnitude of the impact was similar to other variables such as increased age and the presence of comorbidities (Woodfield et al., 2019). The relationship between post-surgical complications and reduced quality of life has been confirmed in multiple more recent studies (Downey et al., 2023; Kauppila et al,.2020). Importantly, individuals typically undergo elective THA/TKA procedures to improve their quality of life, however for patients with complications after TKA, complications have been shown to be significantly associated with lower quality of life (Askari, et al., 2024).
There are currently no other national, publicly reported measures of THA/TKA complications, which underscores the measurement gap that would exist without this measure. In addition, this measure complements the existing, nationally-reported (all-cause, unplanned) THA/TKA Readmission measure, but provides more granular information about rates of specific complications. Thus, this measure addresses an important quality measurement area and enhances the information available to patients choosing among hospitals. Furthermore, providing outcome rates to hospitals makes visible meaningful quality differences and incentivizes improvement.
References
Askari A, Mohammadpour M, Jabalameli M, Naeimipoor N, Goodarzy B, Jafari B, Rashidi H, Mousazadeh F, Rajei M, Khazanchin A, Bahardoust M, Hassanzadeh M. Predictors of health-related quality of life after total knee arthroplasty: a case-control study. Sci Rep. 2024 Jun 19;14(1):14176. doi: 10.1038/s41598-024-65042-z. PMID: 38898136; PMCID: PMC11187171.
Downey CL, Bainbridge J, Jayne DG, Meads DM. Impact of in-hospital postoperative complications on quality of life up to 12 months after major abdominal surgery. Br J Surg. 2023 Aug 11;110(9):1206-1212. doi: 10.1093/bjs/znad167. PMID: 37335925; PMCID: PMC10416679.
Kauppila, J. H., Johar, A., & Lagergren, P. (2020). Postoperative Complications and Health-related Quality of Life 10 Years After Esophageal Cancer Surgery. Annals of surgery, 271(2), 311–316. https://doi.org/10.1097/SLA.0000000000002972
Woodfield J, Deo P, Davidson A, Chen TY, van Rij A. Patient reporting of complications after surgery: what impact does documenting postoperative problems from the perspective of the patient using telephone interview and postal questionnaires have on the identification of complications after surgery? BMJ Open. 2019 Jul 9;9(7):e028561. doi: 10.1136/bmjopen-2018-028561. PMID: 31289081; PMCID: PMC6615906.
Performance Gap
Table 1, Table 2, and Figure 2 (in the attachment) show that there is meaningful variation in the distribution of measure scores (risk-standardized complication visit rates or RSCRs) using the most recent testing data (January 1, 2022 - December 31, 2023). As shown in Table 1, RSCRs range from 1.47% to 8.79%; the median is 3.50%; the 10th percentile is 2.91% and the 90th percentile is 4.42%. The best performer (1.47%) has a risk-standardized complication rate that is 72.40% better than the median (3.50%); the worst performer (8.79%) has a risk-standardized complication rate that is 2.5 times or 151% worse than the median (3.50%).
We provide further evidence of variation by calculating and interpreting the median odds ratio (Merlo, et al., 2006). The median odds ratio, in this context, calculates the odds of the outcome (complications) if the same patient were treated at a higher-risk hospital compared with a lower-risk hospital. The mean odds ratio for the THA/TKA Complications measure is 2.21, indicating that a patient has 2.2 times the odds (or 121% greater risk) of a complication if they were initially admitted to a hospital for THA/TKA procedure at a high-risk facility compared to a lower-risk facility indicating meaningful variation in performance.
References
Merlo J., Chaix, B., Ohlsson, H., Beckman, A., Johnell, K., Hjerpe, P., Råstam, L., & Larsen, K. (2006). A brief conceptual tutorial of multilevel analysis in social epidemiology: Using measures of clustering in multilevel logistic regression to investigate contextual phenomena. Journal of Epidemiology and Community Health, 60(4), 290-297. https://doi.org/10.1136/jech.2004.029454
| Overall | Minimum | Decile_1 | Decile_2 | Decile_3 | Decile_4 | Decile_5 | Decile_6 | Decile_7 | Decile_8 | Decile_9 | Decile_10 | Maximum | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Mean Performance Score | 3.60 | 1.47 | 2.57 | 3.06 | 3.26 | 3.39 | 3.47 | 3.53 | 3.68 | 3.89 | 4.19 | 4.96 | 8.79 |
| N of Entities | 3,124 | 1 | 312 | 312 | 313 | 312 | 313 | 312 | 313 | 312 | 313 | 312 | 1 |
| N of Persons / Encounters / Episodes | 272,164 | 1,072 | 106,482 | 30,500 | 17,776 | 13,147 | 8,589 | 9,316 | 24,331 | 18,378 | 18,134 | 25,511 | 112 |
Equity
Equity
Please see the "All Tables and Figures THA TKA Complications" attachment for all tables and figures that are referred to within this submission, unless otherwise specified.
Social Risk Factor Summary
Overall, and consistent with the literature, we found that there is a low proportion of patients with social risk factors undergoing THA/TKA procedures. While patients with social risk factors (Dual Eligibility (DE)/high Area Deprivation Index (ADI)) who do undergo THA/TKA have higher unadjusted outcomes, however, there is almost no impact on measure scores when adding each variable to the risk adjustment model. This suggests that the clinical risk variables in the model already account for most of the risk associated with these two social risk factors. We also found that the model performs well for patients with DE and high ADI separately. CMS has, therefore, decided not to adjust this quality measure for social risk factors. However, with respect to its use in a payment program (THA/TKA Complications measure is being proposed for the Hospital Value-Based Program [HVBP]), CMS has finalized a new payment adjustment methodology to account for health equity; starting in 2026, HVBP will allow hospitals to earn up to 10 bonus points to their Total Payment Score, accounting for the hospital’s proportion of patients with dual eligibility (CMS, 2023).
Social Risk Factor Testing
To understand the impact of social risk factors on the THA/TKA Complications measure, we assessed the following: prevalence of each social risk factor (DE, high ADI) among hospitals, association with the unadjusted outcome, model calibration for patients with each social risk factor, and impact on measure scores. Each analysis is described in more detail below. All analyses used the CY2022 dataset (one year of Medicare Advantage (MA)+Fee-for-Service (FFS) data, January 1, 2022-December 31, 2022).
For the THA/TKA Complications measure, the median hospital proportion of patients (among hospitals with at least 25 admissions, n= 1,270 hospitals) was 5.13% for the DE variable and 6.78% for the high ADI variable (see Table 10 in the attachment).
As shown in Table 11 (see attachment), unadjusted complication rates for patients with DE or high ADI were higher than for patients without either social risk factor (mean unadjusted rates for DE vs non-DE, 4.0% vs 3.4%; high ADI vs. low ADI, 4.0% vs 3.3%).
We also examined the model calibration for each social risk factor to determine if the risk model (without including either social risk factor) performs well for patients with each social risk factor (see Figure 4A and 4B and Figure 5A and 5B in the attachment). The results show that the model is well-calibrated for both sets of patients.
To understand the impact of each variable on the THA/TKA Complications measure score, we calculated measure scores with and without each social risk factor and then calculated the differences in measure scores and the correlation between measure scores (Table 12 in the attachment). Results show that measure scores calculated with and without social risk factors are highly correlated (correlation coefficient 1.000 and 0.989 for DE and high ADI, respectively), and differences between measure scores are very small.
References
Centers for Medicare & Medicaid Services. (2023, August 28). Medicare Program; Hospital Inpatient Prospective Payment Systems for Acute Care Hospitals and the Long-Term Care Hospital Prospective Payment System and Policy Changes and Fiscal Year 2024 Rates; Quality Programs and Medicare Promoting Interoperability Program Requirements for Eligible Hospitals and Critical Access Hospitals; Rural Emergency Hospital and Physician-Owned Hospital Requirements; and Provider and Supplier Disclosure of Ownership; and Medicare Disproportionate Share Hospital (DSH) Payments: Counting Certain Days Associated With Section 1115 Demonstrations in the Medicaid Fraction. Federal Register, 88(58640), 58640-59438. https://www.federalregister.gov/documents/2023/08/28/2023-16252
Feasibility
Feasibility
The THA/TKA Complications measure is a claims-based measure, and the measure score is calculated automatically from claims data which are routinely generated during the delivery of care. This measure does not require additional data collection by facilities, which eliminates the reporting burden for hospitals. Since data are collected through standard claims submissions and processed by CMS, this measure does not introduce implementation challenges.
CMS monitors feedback from the public and measured entities through CMS’s Q&A portal on QualityNet; there have been no concerns about feasibility or burden related to this measure. There are no concerns about patient confidentiality because the measure is based on claims data submitted by facilities to CMS, and CMS then uses that data for both payment and calculation of the measure score.
We did not perform an analysis of missing data for the measure because it is based on a 100% sample of paid, final action claims submitted by facilities for payment. To ensure complete claims, we allow at least 3 months of time between accessing the data and the end of the performance period.
Because this is a claims-based measure there is no burden on the facility and no feasibility concerns; rates are automatically calculated by CMS based on claims data submitted by facilities for payment.
Proprietary Information
Scientific Acceptability
Testing Data
For most of the testing in this submission, we used two years of Medicare (Fee-for-Service (FFS) and Medicare Advantage (MA)) data (January 1, 2022-December 31, 2023). Descriptions of the data used for testing are outlined in Table 3 ( "All Figures and Tables THA TKA Complications" attachment, pages 3-5).
Please see Section 4.1.4 for details. Differences in data used for testing are outlined in Table 3 ("All Figures and Tables THA TKA Complications" attachment, pages 3-5).
Characteristics of measured entities differ depending on the dataset. Please see Table 3 ("All Figures and Tables THA TKA Complications" attachment, pages 3-5).
The datasets, dates, number of measured hospitals, and number of admissions used in each type of testing are in Table 3 (see "All Figures and Tables THA TKA Complications" attachment, pages 3-5).
For most measure testing, we used Medicare data from January 1, 2022-December 30, 2023. These datasets also include data on each patient for the 12 months prior to the index admission and contain facility inpatient and Medicare Enrollment Database (EDB) data.
Reliability
We tested facility-level measure score reliability using the signal-to-noise method, using the formula presented by Adams and colleagues (Adams et al., 2010; Yu et al., 2013). Specifically, for each facility, we calculate reliability using the hospital intercept estimated through the random intercept of the hierarchical logistic regression model, which is the quality signal, as follows:
Reliability=(σ_(facility-to-facility)^2)/(σ_(facility-to-facility)^2+ (σ_(facility error variance)^2)/n)
Where facility-to-facility variance is estimated from the hierarchical logistic regression model, n is equal to each facility’s observed case size, and the facility error variance is estimated using the variance of the logistic distribution (pi^2/3). This calculation provides a valid reliability score because the hospital intercept (random effects) of the hierarchical logistic regression model is highly correlated with the standardized readmission ratio (SRR), which is used to calculate the measure score (the SRR is multiplied by a constant [the national unadjusted readmission rate] to derive the risk-standardized readmission rate or RSRR). The correlation between the hospital intercept and the SRR for this measure is 0.999 (p<.0001). As stated in Section 1.18, the hospital intercept represents the underlying risk of a readmission at the hospital, after accounting for patient risk. The hospital-specific intercepts are given a distribution to account for the clustering (non-independence) of patients within the same hospital. If there were no differences among hospitals, then after adjusting for patient risk, the hospital intercepts should be identical across all hospitals. Please see the formula provided in the attachment in Section 1.18a for additional details.
References
Adams J, Mehrota, A, Thoman J, McGlynn, E. (2010). Physician cost profiling – reliability and risk of misclassification. NEJM, 362(11): 1014-1021.
Yu, H, Mehrota, A, Adams J. (2013). Reliability of utilization measures for primary care physician profiling. Healthcare, 1, 22-29.
Please see the "All Tables and Figures THA TKA Complications" attachment for all tables and figures that are referred to within this submission, unless otherwise specified.
We calculated signal-to-noise reliability for all hospitals in the testing sample (n= 3,124) and hospitals with at least 25 cases (n= 1,777), the volume threshold for public reporting of measure scores on Care Compare (Table 4 in the attachment). We used two years of data for the analysis (CY2022/2023). For hospitals with at least 25 cases, the median reliability score was 0.784, ranging from 0.545 to 0.997. The 25th and 75th percentiles were 0.623 and 0.784, respectively. Table 5 (in the attachment) provides the Battelle-required table of measure scores within deciles of reliability.
Using two years of performance data, signal-to-noise reliability ranged from 0.545- 0.997 with a median of 0.784 for facilities with at least 25 procedures. Most hospitals with at least 25 procedures fall above the minimum signal-to-noise reliability threshold of >=0.6.
| | Overall | Minimum | Decile_1 | Decile_2 | Decile_3 | Decile_4 | Decile_5 | Decile_6 | Decile_7 | Decile_8 | Decile_9 | Decile_10 | Maximum |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Reliability | 0.776 | 0.545 | 0.567 | 0.625 | 0.674 | 0.720 | 0.766 | 0.808 | 0.844 | 0.882 | 0.918 | 0.961 | 0.997 |
| Mean Performance Score | 3.59 | 3.57 | 3.65 | 3.75 | 3.72 | 3.75 | 3.78 | 3.71 | 3.68 | 3.57 | 3.38 | 2.87 | 2.16 |
| N of Entities | 1,777 | 33 | 180 | 182 | 170 | 176 | 181 | 181 | 175 | 178 | 178 | 176 | 1 |
| N of Persons / Encounters / Episodes | 259,441 | 825 | 4,946 | 6,363 | 7,350 | 9,522 | 12,416 | 15,988 | 19,840 | 27,949 | 42,677 | 112,390 | 6,172 |
Validity
Please see the "All Tables and Figures THA TKA Complications" attachment for all tables and figures that are referred to within this submission, unless otherwise specified.
For this submission, we provide evidence of validity through face validity conducted during measure development, and empiric validity through examining associations of THA/TKA Complications measure scores with volume. Below we describe the methodology; the results are provided in the subsequent section.
Face Validity Using Technical Expert Panel
During measure re-specification (which included an expansion of the cohort to include Medicare Advantage patients and reselection of risk variables) we re-assessed the THA/TKA Complications measure’s face validity with a technical expert panel (TEP) of national experts and stakeholder organizations. We systematically assessed the face validity of the measure score as an indicator of quality by soliciting TEP members’ agreement with the following statement (via an online survey): “The risk-standardized complication rate obtained from the measures as specified can be used to distinguish between better and worse quality hospitals.”
The TEP is comprised of clinicians, health services researchers, statisticians, patients, patient advocates/caregivers, health insurance representatives, and hospital administrators (see Table 6 in the attachment for additional details on panel members).
The survey offered participants six response options ranging from “strongly disagree” to “strongly agree” on a six-point scale:
- 1=Strongly disagree
- 2=Moderately disagree
- 3=Somewhat disagree
- 4=Somewhat agree
- 5=Moderately agree
- 6=Strongly agree
Empiric Validity
To demonstrate empirical validity for the THA/TKA Complications measure, we assessed the measure score’s correlation with the quality construct of volume.
Volume may be considered a quality construct in that it is a proxy for underlying quality-related factors. Hospitals with higher operative volume for THA/TKA procedures may have surgeons and staff with greater experience and repeated performance of evidence-based processes, allowing for the development and implementation of standardized care delivery, which in turn can reduce errors and delays in care and improve outcomes. Higher volume hospitals may also invest in more specialized support systems and training (Jha, 2015) and may also have more resources to invest in quality improvement and participation in formal quality improvement registries, such as the American Joint Replacement Registry (Hedge et al., 2023).
There is evidence that surgical complication rates for providers (both surgeons and hospitals) decline with increasing THA and TKA volume (Sibley et al., 2017; Murphy et al., 2019; Courtney et al., 2018). Thus, we assessed the validity of the measure by examining the relationship between volume and the measure score for hospitals. To establish validity, we expect THA/TKA measure scores to be correlated with case volume at the hospital level. We hypothesized that there would be a weak to moderate, negative relationship between hospital admission volume and THA/TKA Complications measure scores, with higher volumes associated with better (lower) THA/TKA Complications measure scores.
These concepts are discussed in Section 2.2 and Section 6.2.1 and are shown in the logic model (Figure 1 in the attachment).
References
Courtney M, Frisch N, Bohl D, Della Valle C. Improving Value in Total Hip and Knee Arthroplasty: The Role of High Volume Hospitals. The Journal of Arthroplasty. 2018;33(1):1-5. https://doi.org/10.1016/j.arth.2017.07.040.
Hegde V, Stambough JB, Levine BR, Springer BD. Highlights of the 2022 American Joint Replacement Registry Annual Report. Arthroplast Today. 2023 Apr 25;21:101137.
Murphy WS, Cheng T, Lin B, Terry D, Murphy SB. Higher Volume Surgeons Have Lower Medicare Payments, Readmissions, and Mortality After THA. Clin Orthop Relat Res. 2019;477(2):334‐341. doi:10.1097/CORR.0000000000000370.
Sibley R, Charumbhumi, V, Hutzler L, Paoli A, Bosco, J. Joint Replacement Volume Positively Correlates With Improved Hospital Performance on Centers for Medicare and Medicaid Services Quality Metrics. The Journal of Arthroplasty. 2017;32(5):1409-1413. https://doi.org/10.1016/j.arth.2016.12.010.
Please see the "All Tables and Figures THA TKA Complications" attachment for all tables and figures that are referred to within this submission, unless otherwise specified.
Face Validity
A total of 12 TEP members responded to the face validity survey. 11 of 12 TEP members (91.7%) agreed (strongly, moderately, somewhat) with the face validity statement; 1 of 12 TEP members (8.3%) moderately disagreed.
The specific counts for each voting category are shown below:
- Strongly agree: 4 (33.3%)
- Moderately agree: 6 (50%)
- Somewhat agree: 1 (8.3%)
- Somewhat disagree: 0 (0.0%)
- Moderately disagree: 1 (8.3%)
- Strongly disagree: 0 (0.0%)
Empiric Validity
Table 7 (see attachment) shows THA/TKA risk-standardized complications rates (RSCRs) within deciles of hospital admission volume (cohort volume). RSCRs decline with increasing admission volume, starting with the 7th decile. RSCRs are the lowest in the highest-volume decile (within hospitals with the highest admission volume). The overall Pearson’s correlation coefficient between RSCRs and hospital volume is -0.25 (p<.0001). The correlation coefficient between THA/TKA RSCRs and hospital volume for hospitals with at least 200 procedures is -0.138 (p<.0001), and for hospitals with less than 200 procedures is -0.099 (p<.51).
The validity of this measure is supported by three sources of evidence: (1) empiric validity testing supporting a volume/outcome relationship, (2) strong face validity, and (3) evidence of improvement (in the currently reported measure).
Our empiric validity testing examined the quality construct of volume, based on literature supporting a volume/outcome relationship between THA/TKA volume and complications. We found a significant association in the expected strength and direction between THA/TKA hospital volume and the THA/TKA Complications measure: higher THA/TKA hospital volume was significantly associated with lower THA/TKA Complications measure scores (better performance). We also saw that this relationship significant at higher hospital volume (>=200 procedures), but not lower volumes (<200) which is also consistent with the literature.
The validity of this respecified THA/TKA Complications measure is further supported by updated, strong face validity results during measure re-specification (91.7% agreement of face validity from the Technical Panel Experts (TEP).
Finally, analyses based on the current publicly reported Fee-for-Service (FFS)-only THA/TKA Complications measure show that there has been improvement in measure scores over time (see Section 6.2.4) in a setting of quality improvement efforts (see Section 6.2.1).
Taken together, these results support the validity of the THA/TKA Complications measure.
Risk Adjustment
The goal of risk adjustment is to adjust for case-mix differences across the hospitals. Risk adjustment supports fair and accurate comparison of outcomes across measured entities by including an adjustment for factors such as age, comorbid diseases, and indicators of patient frailty, which are clinically relevant and have relationships with the outcome.
In pursuing an approach that best leverages the data and analytical advancements since initial measure development, we developed and evaluated a framework to use individual ICD-10 codes for risk adjustment. The main advantage of leveraging ICD-10 codes in place of the prior method (that used an ICD-10 grouper, CMS’s Condition Categories, or CCs) is the ability to address the clinical heterogeneity found in the broadly defined CCs. Our previous research indicates that the model performance of the mortality measures is significantly improved by using individual codes instead of CCs (Krumholz et al., 2019).
Selection of Clinical Risk Variables
For candidate risk variables, we included all secondary ICD-10 codes documented as present-on-admission (POA) during the index admission (except for the palliative care code of Z51.5, which, effective October 1, 2021, was considered POA-exempt), and both principal and secondary ICD-10 codes in the 12 months prior to admission from any inpatient, outpatient, and professional provider claims. We also considered the principal discharge diagnosis code for the index admission. In addition, we considered age, frailty, sex, an indicator for whether the admission was Medicare Advantage (MA) vs. Fee-for-Service (FFS), and other non-individual-ICD variables in the existing publicly reported THA//TKA Complications measure. The variable selection of individual ICD codes mainly relied on data-driven methodologies involving three key steps: 1) pre-processing, 2) evaluating association with outcome, and 3) consideration of associations between other non-individual-code variables, including frailty, with the outcome.
In pre-processing, we screened and included index and history (pre-index) codes if their prevalence exceeded 0.5% and 2.5%, respectively. Further, co-occurring index and pre-index codes with Pearson correlation coefficients greater than 0.8 were combined into one risk variable. Finally, pairs of identical index and pre-index ICD-10 codes with similar odds ratios that acted in the same direction (where the difference in association with the outcome, measured by odds ratio (OR), was less than 0.2) were merged.
In the second step, we included the remaining candidate variables including age in a multivariable logistic regression model that underwent variable selection through 1,000 iterations of bootstrapping. We selected variables that were statistically significantly associated with outcomes (p<0.05) in at least 70% of the bootstrapped samples. Additional variables were added if there was a resulting increase in c-statistic of at least 0.0005 for each additional variable or an increase of at least 0.005 for including additional variables within the next 5% of bootstrapped samples (e.g. moving from 70% to 65%). Lastly, we included other non-individual-ICD variables from the current FFS-only THA/TKA Complications measure if the regression coefficients were statistically significant when added to the models.
Lastly, based on evidence from the literature, expert input, guidance from the consensus-based entity for measure endorsement, the Assistant Secretary for Planning and Evaluation (ASPE, 2020), input from other stakeholders, as well as prior testing results, we included a claims-based indicator of frailty that was developed for CMS’s Multiple Chronic Conditions measure (Yale New Haven Health Services Corporation/Center for Outcomes Research & Evaluation (YNHHSC/CORE), 2019) in the final model for all measures. We did not include sex as a variable since sex can be considered a socio-demographic variable. After variable selection, we also added into the model the history of coronavirus disease 2019 (COVID-19) variable to be consistent with current CMS policy.
For the combined MA and FFS cohort, the risk adjustment model was updated to include an MA indicator (versus FFS) as a main effect. This was to adjust for the generally higher prevalence of comorbidities in the MA cohort, especially among the pre-index variables that were derived from services in the outpatient setting (e.g. physician visits).
Social Risk Factors
Although some recent literature evaluates the relationship between patient social risk factors and the complication outcome, few studies directly address causal pathways or examine the role of the hospital in these pathways (Trivedi et al., 2014; Buntin et al., 2017; Borza et al., 2019). Moreover, the current literature examines a wide range of conditions and risk variables with no clear consensus on which risk factors demonstrate the strongest relationship with complication.
The social risk factors that have been examined in the literature can be categorized into three domains: (1) patient-level variables, (2) neighborhood/community-level variables, and (3) hospital-level variables.
Patient-level variables describe the characteristics of individual patients and include the patient’s race and ethnicity, income, or education level. For example, Black and Hispanic patients have been shown to experience higher rates of postoperative complications, longer lengths of stay, and more frequent non-home discharges (Rudisill et al., 2023; Usiskin and Misra, 2022; Brown, Paisner, and Sassoon, 2022)). These disparities are often influenced by socioeconomic factors such as lower income or education levels, which can limit access to healthcare services, preventive care, or rehabilitation (Alvarez et al., 2022). Additionally, barriers such as language proficiency or health literacy may further exacerbate these challenges and contribute to poorer outcomes and higher rates of complications for patients with social risk factors (Suleiman et al., 2021).
Neighborhood/community-level variables use information from sources such as the American Community Survey as either a proxy for individual patient-level data or to measure environmental factors. Studies using these variables use one-dimensional measures such as median household income or composite measures such as the AHRQ-validated SES index score (Blum et al., 2014; Courtney et al., 2016; Martsolf et al., 2016; White et al., 2018). Some of these variables may include the local availability of clinical providers (Herrin et al., 2015; Herrin et al., 2016).
Hospital-level variables measure attributes of the hospital which may be related to patient risk. Examples of hospital-level variables used in studies are ZIP code characteristics aggregated to the hospital level or the proportion of Medicaid patients served in the hospital (Gilman et al., 2014; Joynt et al., 2013; Jha et al., 2013; Xu et al., 2018).
The conceptual relationship, or potential causal pathways by which these possible social risk factors influence the risk of complication following an acute illness or major surgery, like the factors themselves, are varied and complex. There are at least four potential pathways that are important to consider:
- Patients with social risk factors may have worse health at the time of hospital admission. Patients who have lower income/education/literacy or unstable housing may have a worse general health status and may present for their hospitalization or procedure with a greater severity of underlying illness. These social risk factors, which are characterized by patient-level or neighborhood/community-level (as proxy for patient-level) variables, may contribute to worse health status at admission due to competing priorities (for example, restrictions based on job), lack of access to care (e.g., geographic, cultural, or financial), or lack of health insurance. Given that these risk factors all lead to worse general health status, this causal pathway should be largely accounted for by current clinical risk adjustment.
- Patients with social risk factors may receive care at lower-quality hospitals. Patients of lower income, lower education, or unstable housing have inequitable access to high-quality facilities, in part because such facilities may be less likely to be found in geographic areas with large populations of poor patients. Thus, patients with low income may be more likely to be seen in lower-quality hospitals, which can explain the increased risk of complications following hospitalization.
- Patients with social risk factors may receive differential care within a hospital. The third major pathway by which social risk factors may contribute to complications risk is that patients may not receive equivalent care within a facility. For example, patients with social risk factors such as lower education may require differentiated care (e.g. provision of lower literacy information – that they do not receive).
- Patients with social risk factors may experience worse health outcomes beyond the control of the healthcare system. Some social risk factors, such as income or wealth, may affect the likelihood of complications without directly affecting health status at admission or the quality of care received during the hospital stay. For instance, while a hospital may make appropriate care decisions and provide tailored care and education, a lower-income patient may have a worse outcome post-discharge due to competing financial priorities that don’t allow for adequate recuperation or access to needed treatments, or a lack of access to care outside of the hospital.
We developed and used the conceptual framework described below to identify potential social risk factors. We analyzed two well-studied social risk factors that could best be operationalized in data, outlined below. We note that this measure already adjusts for age and note that the risk model already accounts for patient comorbidities which may differ among patients with social risk factors.
Dual-Eligible (DE) Status
Dual eligibility for Medicare and Medicaid is available at the patient level in the Medicare Master Beneficiary Summary File. The eligibility threshold for Medicare beneficiaries aged 65 or older considers both income and assets. For the dual-eligible (DE) indicator, there is a body of literature demonstrating differential health care and health outcomes among beneficiaries (ASPE, 2020).
Area Deprivation Index (ADI)
While we previously used the AHRQ SES variable in these types of analyses, we now use the validated ADI (Forefront Group, 2023). We made this change to align with other CMS work on social risk factors that now use the ADI. We describe the ADI variable below.
The ADI, initially developed by the Health Resources & Services Administration, is based on 17 measures across four domains: income, education, employment, and housing quality (Kind et al., 2018; Singh, 2003).
The 17 components are listed below:
- Population aged ≥ 25 y with < 9 y of education, %
- Population aged ≥ 25 y with at least a high school diploma, %
- Employed persons aged ≥ 16 y in white-collar occupations, %
- Median family income, $
- Income disparity
- Median home value, $
- Median gross rent, $
- Median monthly mortgage, $
- Owner-occupied housing units, % (homeownership rate)
- Civilian labor force population aged ≥16 y unemployed, % (unemployment rate)
- Families below the poverty level, %
- Population below 150% of the poverty threshold, %
- Single-parent households with children aged < 18 y, %
- Households without a motor vehicle, %
- Households without a telephone, %
- Occupied housing units without complete plumbing, % (log)
- Households with more than 1 person per room, % (crowding)
ADI scores were derived using the beneficiary’s 9-digit ZIP Code of residence, which is obtained from the Master Beneficiary Summary File and is linked to 2017-2021 US Census/American Community Survey (ACS) data. In accordance with the ADI developers’ methodology, an ADI score is calculated for the census block group corresponding to the beneficiary’s 9-digit ZIP Code using 17 weighted Census indicators. Raw ADI scores were then transformed into a national percentile ranking ranging from 1 to 100, with lower scores indicating lower levels of disadvantage and higher scores indicating higher levels of disadvantage. Percentile thresholds established by the ADI developers were then applied to the ADI percentile to dichotomize neighborhoods into more disadvantaged (high ADI areas=ranking equal to or greater than 85) or less disadvantaged areas (Low ADI areas= ranking of less than 85).
References
Alvarez, P. M., McKeon, J. F., Spitzer, A. I., Krueger, C. A., Pigott, M., Li, M., & Vajapey, S. P. (2022). Socioeconomic factors affecting outcomes in total knee and hip arthroplasty: a systematic review on healthcare disparities. Arthroplasty, 4(1), 36.
Blum AB, Egorova NN, Sosunov EA, et al. Impact of socioeconomic status measures on hospital profiling in New York City. Circulation Cardiovascular quality and outcomes 2014; 7:391-7.
Buntin MB, Ayanian JZ. Social Risk Factors and Equity in Medicare Payment. New England Journal of Medicine. 2017;376(6):507-510.
Courtney M, Huddleston J, Iorio R, Markel D. Socioeconomic Risk Adjustment Models for Reimbursement Are Necessary in Primary Total Joint Arthroplasty. July 2016; 32(1):1-5. https://doi.org/10.1016/j.arth.2016.06.050.
Department of Health and Human Services, Office of the Assistant Secretary of Planning and Evaluation (ASPE). Second Report to Congress: Social Risk Factors and Performance in Medicare’s Value-based Purchasing Programs. 2020; https://aspe.hhs.gov/system/files/pdf/263676/Social-Risk-in-Medicare%E2%80%99s-VBP-2nd-Report.pdf. Accessed July 2, 2020.
Gilman M, Adams EK, Hockenberry JM, et al. California safety-net hospitals likely to be penalized by ACA value, readmission, and meaningful-use programs. Health Aff (Millwood). Aug 2014; 33(8):1314-22.
Herrin J, Kenward K, Joshi MS, Audet AM, Hines SJ. Assessing Community Quality of Health Care. Health Serv Res. 2016 Feb;51(1):98-116. doi: 10.1111/1475-6773.12322. Epub 2015 Jun 11. PMID: 26096649; PMCID: PMC4722214.
Herrin J, St Andre J, Kenward K, Joshi MS, Audet AM, Hines SC. Community factors and hospital readmission rates. Health Serv Res. 2015 Feb;50(1):20-39. doi: 10.1111/1475-6773.12177. Epub 2014 Apr 9. PMID: 24712374; PMCID: PMC4319869.
Jha AK, Orav EJ, Epstein AM. Low-quality, high-cost hospitals, mainly in South, care for sharply higher shares of elderly black, Hispanic, and medicaid patients. Health affairs 2011; 30:1904-11.
Joynt KE, Jha AK. Characteristics of hospitals receiving penalties under the Hospital Readmissions Reduction Program. JAMA. Jan 23 2013; 309(4):342-3.
Kind AJH, Buckingham W. Making Neighborhood Disadvantage Metrics Accessible: The Neighborhood Atlas. New England Journal of Medicine, 2018. 378: 2456-2458. DOI: 10.1056/NEJMp1802313. PMCID: PMC6051533. AND University of Wisconsin School of Medicine Public Health. 2023 Area Deprivation Index v4.0. Downloaded from https://www.neighborhoodatlas.medicine.wisc.edu/.
Krumholz HM, Coppi AC, Warner F, et al. Comparative effectiveness of new approaches to improve mortality risk models from Medicare claims data. JAMA Network Open. 2019;2(7):e197314-e197314.
Martsolf G, Barrett M, Weiss A, Kandrack R, Washington R, Steiner C, Mehrotra A, SooHoo N, Coffey R. Impact of Race/Ethnicity and Socioeconomic Status on Risk-Adjusted Hospital Readmission Rates Following Hip and Knee Arthroplasty, The Journal of Bone and Joint Surgery. 2016;98(16):1385-1391. https://doi.org/10.2106/JBJS.15.00884.
Rudisill, S. S., Varady, N. H., Birir, A., Goodman, S. M., Parks, M. L., & Amen, T. B. (2023). Racial and ethnic disparities in total joint arthroplasty care: A contemporary systematic review and meta-analysis. The Journal of Arthroplasty, 38(1), 171-187.e18. https://doi.org/10.1016/j.arth.2022.08.006
Singh, G. K. (2003). Area deprivation and widening inequalities in US mortality, 1969–1998. American Journal of Public Health. 93(7), 1137–1143. https://doi.org/10.2105/ajph.93.7.1137
Suleiman, L. I., Tucker, K., Ihekweazu, U., Huddleston, J. I., & Cohen-Rosenblum, A. R. (2022). Caring for diverse and high-risk patients: Surgeon, health system, and patient integration. The Journal of Arthroplasty, 37(8), 1421–1425. https://doi.org/10.1016/j.arth.2022.04.077
Trivedi AN, Nsa W, Hausmann LR, et al. Quality and equity of care in U.S. hospitals. The New England journal of medicine 2014; 371:2298-308.
Upfill-Brown, A., Paisner, N., & Sassoon, A. (2023). Racial disparities in post-operative complications and discharge destination following total joints arthroplasty: A national database study. Archives of Orthopaedic and Trauma Surgery, 143(4), 2227–2233. https://doi.org/10.1007/s00402-022-04485-3
Usiskin, I., & Misra, D. (2022). Racial disparities in elective total joint arthroplasty for osteoarthritis. ACR Open Rheumatology, 4(4), 306–311. https://doi.org/10.1002/acr2.11399
White, R.S., Sastow, D.L., Gaber-Baylis, L.K. et al. Readmission Rates and Diagnoses Following Total Hip Replacement in Relation to Insurance Payer Status, Race and Ethnicity, and Income Status. J. Racial and Ethnic Health Disparities 5, 1202–1214 (2018). https://doi.org/10.1007/s40615-018-0467-0.
Xu HF, White RS, Sastow DL, Andreae MH, Gaber-Baylis LK, Turnbull ZA. Medicaid insurance as primary payer predicts increased mortality after total hip replacement in the state inpatient databases of California, Florida and New York. J Clin Anesth. 2017;43:24‐32. doi:10.1016/j.jclinane.2017.09.008.
Table 8 (see "All Figures and Tables THA TKA Complications" attachment) shows the risk variable frequencies and odds ratios for the final risk variables selected by the process described in Section 4.4.2. Risk variables are also provided within the attached data dictionary. Results related to social risk factors can be found in Section 5 (Equity).
Table 8 (see "All Figures and Tables THA TKA Complications" attachment) shows the risk variable frequencies and odds ratios for the final risk variables selected by the process described in Section 4.4.2.
Model Performance Testing Methods
To assess model performance, we assessed model discrimination, calibration, and overfitting. To assess discrimination, we computed two discrimination statistics, the c-statistic and predictive ability (Table 9 in the attachment). These analyses used the CY2022 dataset (derivation and validation). For calibration, we provide calibration (risk-decile) plots (see Figure 4A and 4B in the attachment) for CY2022 and CY2023 data.
The c-statistic is the probability that predicting the outcome is better than chance, which is a measure of how accurately a statistical model can distinguish between a patient with and without an outcome.
Predictive ability measures the ability to distinguish high-risk subjects from low-risk subjects; therefore, for a model with good predictive ability, we would expect to see a wide range in observed outcomes between the lowest and highest deciles of predicted outcomes. To calculate the predictive ability, we calculated the range of mean observed THA/TKA Complications between the lowest and highest predicted deciles of THA/TKA Complications probabilities.
For model calibration, we assessed calibration plots, with mean predicted and mean observed outcomes plotted against deciles of predicted outcomes. The closer the predicted outcomes are to the observed outcomes, the better calibrated the model is. We provide results for CY2022 (derivation sample) and CY2023 (validation sample).
In addition, we assess model calibration through risk-decile plots, one year for CY2022 data and CY2023 data. In addition, we provide an analysis of overfitting. Overfitting refers to the phenomenon in which a model accurately describes the relationship between predictive variables and outcome in the development dataset but fails to provide valid predictions in new patients. Estimated calibration values of γ0 close to 0 and estimated values of γ1 close 1 provide evidence of good calibration of the model
Model Performance Testing Results
Please see Table 9 and Figures 3A and 3B in the attachment "All Figures and Tables THA/TKA Complications " (pages 9-10) for the model testing results. The results are also described below.
The c-statistic was 0.671 in the derivation sample, and 0.663 in the validation sample (Table 9). Predictive ability ranged from 1.18%-8.65% in the derivation sample, and 1.34%-8.36% in the validation sample. Risk decile plots show that higher deciles of the predicted outcomes are associated with higher observed outcomes in both CY2022 and CY2023 data (Figures 3A and 3B). Overfitting results are shown in Table 9.
Discrimination
The c-statistic of 0.671 in the development sample, and 0.663 in the validation sample indicate excellent model discrimination. The model indicated a wide range between the lowest decile and highest decile, indicating the ability to distinguish high-risk subjects from low-risk subjects.
Calibration
Higher deciles of the predicted outcomes are associated with higher observed outcomes, which show a good calibration of the model. The predictive ability of the model shows a wide range between the lowest decile and highest decile, indicating the ability to distinguish high-risk subjects from low-risk subjects. Higher deciles of the predicted outcomes are associated with higher observed outcomes, which show good calibration of the model. As seen in the calibration plots for CY2022 and CY2023, higher deciles of predicted outcomes are associated with higher observed outcomes. This indicates good calibration of the model in data from two time periods.
Over-fitting (γ0, γ1)
If γ0 is substantially far from zero and γ1 is far from one, there is potential evidence of over-fitting. Our testing results show that while γ1 is almost one, γ0 is further from zero than expected. However, overfitting in the context of risk-standardization does not reflect the actual use of the measure because, with each update of data for any performance period, the model variable coefficients are recalculated, therefore, we are never calculating results with “new” data using the coefficients from the prior year.
Overall Interpretation
Interpreted together, our diagnostic results demonstrate the risk-adjustment model adequately controls for differences in patient characteristics (case mix).
Use & Usability
Use
The level of analysis is facility level. The care setting is short-term acute care hospitals.
The level of analysis is facility level. The care setting is short-term acute care hospitals.
Usability
Improving complication rates following Total Hip Arthroplasty (THA) and Total Knee Arthroplasty (TKA) involves a multifaceted approach focusing on preoperative, intraoperative, and postoperative strategies. Preoperatively, optimizing patient health through rigorous preoperative screening and comorbidity management can significantly reduce complications (Liu et al., 2020). Intraoperatively, employing advanced surgical techniques and technologies, such as computer-assisted navigation and minimally invasive approaches, has been shown to decrease complication rates (Biau et al., 2019). Postoperatively, enhancing recovery protocols, including early mobilization and comprehensive rehabilitation, helps prevent complications such as deep vein thrombosis and joint stiffness (Pagnano et al., 2018). Additionally, adherence to infection prevention measures, such as antibiotic prophylaxis and sterile techniques, is crucial for minimizing the risk of periprosthetic infections (Kurtz et al., 2021). Collectively, these strategies contribute to better outcomes and reduced complication rates for THA and TKA patients.
A patient-centered approach that includes preoperative education, optimization of health, and adherence to rehabilitation protocols is crucial. Educating patients about their roles in minimizing risks is crucial; studies show that preoperative education can significantly enhance patient outcomes by ensuring better understanding and adherence to postoperative care (Kurtz et al., 2018).
A timely health evaluation and efficient preoperative care for patients might facilitate the early identification of potential risk factors, such as untreated comorbidities. Research shows that timely and efficient action to manage conditions such as diabetes and obesity can significantly lower infection rates and enhance overall health outcomes for THA/TKA patients (Antonelli & Chen, 2019; Giorgino et al., 2021; Ahmad et al., 2022). Another aspect of providing high-quality care is using evidence-based recommendations in perioperative care, such as implementing standardized care protocols for pain management and mobility improvement. For instance, enhanced recovery after surgery (ERAS) protocols, which place a strong emphasis on early mobilization and multimodal analgesia, have been associated with shorter hospital stays and reduced complication rates for THA/TKA patients (Kaye et al., 2019; Li et al., 2019; Choi et al., 2022).
Additionally, recent efforts such as the Comprehensive Care for Joint Replacement (CJR) Model are designed to improve care for Medicare patients undergoing hip and knee replacements. With the goal of improving the quality and efficiency of care for Medicare patients undergoing hip and knee replacements, the CJR model offers incentives to hospitals to encourage preoperative optimization and coordinated care throughout the surgical episode. The model emphasizes timely care of comorbid diseases and promotes the implementation of evidence-based procedures to lower complications, hospital readmissions, and overall costs by linking financial incentives to patient outcomes (CMS, 2023). In addition, to encouraging provider collaboration, this integrated approach ensures patients receive comprehensive care from the time of pre-surgery through recovery.
Another component of the care continuum that affects complication rates after THA/TKA procedures is clear and effective patient communication to ensure that patients are adequately prepared for discharge (You et al., 2024). Teaching patients how to take care of their wounds, manage their medications, and identify early warning signs of complications like infection or thromboembolic events are all part of comprehensive discharge planning. Research evidence suggests that the adoption of structured discharge protocols can significantly lower readmission rates and enhance patient outcomes (Gonçalves-Bradley et al., 2016; Becker et al., 2021).
Improving communication between healthcare professionals involved in care transitions is also significant in lowering postoperative complications [ Patel & Bechmann, 2023]. Communication breakdowns that occur during the transition from surgical teams to inpatient care and rehabilitation might result in information being lost, including changes to drug schedules or unresolved postoperative issues. Research shows that using electronic health records (EHR) with clear, standardized documentation of the patient's condition and treatment plan enhances the continuity of care between the nursing, surgical, and rehabilitation teams (Stoicea et al., 2018). Bardram and Houben (2018) show that EHRs contain collaborative affordances that enable collaborative action and workflow among different actors. The authors identify four collaborative affordances: portability (to navigate health records between locations), co-located access (to support simultaneous access), shared overview (to collectively build a shared information overview), and mutual awareness (to maintain mutual awareness of the work’s progress), all of which improve provider-to-provider communication and care transitions. Multidisciplinary team meetings also ensure that all medical professionals agree with the patient's status and plans for discharge, which enhances the patient's recovery and lowers the risk of complications (Nag et al., 2024).
Finally, hospitals can use resources provided by CMS to help improve the drivers of hospital visits. To support quality improvement, CMS shares reports with measured entities that include measure results benchmarked against the state and nation (hospital-specific reports [HSRs]). These reports include, among other details, the principal diagnosis code associated with the hospitalization, which allows hospitals to tie their quality improvement efforts to the specific reasons for rehospitalization that are occurring. CMS gives hospitals 30 days to preview their results and submit questions before public reporting on the data catalog on Data.cms.gov (also known as the Provider Data Catalog) and Medicare.gov (also known as Care Compare) websites in July.
References
Ahmad, M. A., Ab Rahman, S., & Islam, M. A. (2022). Prevalence and risk of infection in patients with diabetes following primary total knee arthroplasty: A global systematic review and meta-analysis of 120,754 knees. Journal of Clinical Medicine, 11(13), 3752. https://doi.org/10.3390/jcm11133752
Antonelli, B., & Chen, A. F. (2019). Reducing the risk of infection after total joint arthroplasty: Preoperative optimization. Arthroplasty, 1, Article 4. https://doi.org/10.1186/s42836-019-0005-8
Becker, C., Zumbrunn, S., Beck, K., Vincent, A., Loretz, N., Müller, J., Amacher, S. A., Schaefert, R., & Hunziker, S. (2021). Interventions to improve communication at hospital discharge and rates of readmission: A systematic review and meta-analysis. JAMA Network Open, 4(8), e2119346. https://doi.org/10.1001/jamanetworkopen.2021.19346
Biau, D. J., Tournoux, C., & Nizard, R. (2019). Computer-assisted navigation in total knee arthroplasty: A systematic review and meta-analysis. Journal of Bone and Joint Surgery, 101(5), 438-445. https://doi.org/10.2106/JBJS.18.00635
Choi, Y. S., Kim, T. W., Chang, M. J., Kang, S. B., & Chang, C. B. (2022). Enhanced recovery after surgery for major orthopedic surgery: A narrative review. Knee Surgery & Related Research, 34, 8. https://doi.org/10.1186/s43019-022-00137-3
Centers for Medicare & Medicaid Services (CMS). (2023). Comprehensive Care for Joint Replacement Model. CMS.gov. https://www.cms.gov/medicare/medicare-fee-for-service-payment/sharedsavingsprogram/cjr
Giorgino, F., Bhana, S., Czupryniak, L., Dagdelen, S., Galstyan, G. R., Janež, A., Lalić, N., Nouri, N., Rahelić, D., Stoian, A. P., & Raz, I. (2021). Management of patients with diabetes and obesity in the COVID-19 era: Experiences and learnings from South and East Europe, the Middle East, and Africa. Diabetes Research and Clinical Practice, 172, 108617. https://doi.org/10.1016/j.diabres.2020.108617
Gonçalves-Bradley, D. C., Lannin, N. A., Clemson, L. M., Cameron, I. D., & Shepperd, S. (2016). Discharge planning from hospital. Cochrane Database of Systematic Reviews, 2016(1), CD000313. https://doi.org/10.1002/14651858.CD000313.pub5
Kaye, A. D., Urman, R. D., Rappaport, Y., Siddaiah, H., Cornett, E. M., Belani, K., Salinas, O. J., & Fox, C. J. (2019). Multimodal analgesia as an essential part of enhanced recovery protocols in the ambulatory settings. Journal of Anaesthesiology Clinical Pharmacology, 35(Suppl 1), S40–S45. https://doi.org/10.4103/joacp.JOACP_51_18
Kurtz, S., Ong, K., & Lau, E. (2018). Preoperative education and its effect on outcomes in total joint arthroplasty: A systematic review. Journal of Arthroplasty, 33(4), 1146-1152. https://doi.org/10.1016/j.arth.2017.10.030
Kurtz, S., Ong, K., Lau, E., et al. (2021). International utilization of infection prevention strategies in total hip and knee arthroplasty. Clinical Orthopaedics and Related Research, 479(3), 642-650. https://doi.org/10.1097/CORR.0000000000001204
Li, J., Zhu, H., & Liao, R. (2019). Enhanced recovery after surgery (ERAS) pathway for primary hip and knee arthroplasty: Study protocol for a randomized controlled trial. Trials, 20, 599. https://doi.org/10.1186/s13063-019-3706-8
Liu, J. L., & Young, A. M. (2020). Preoperative optimization and its effect on outcomes in total joint arthroplasty. Orthopedic Clinics of North America, 51(2), 215-223. https://doi.org/10.1016/j.ocl.2019.12.003
Nag, D. S., Swain, A., Sahu, S., Sahoo, A., & Wadhwa, G. (2024). Multidisciplinary approach toward enhanced recovery after surgery for total knee arthroplasty improves outcomes. World Journal of Clinical Cases, 12(9), 1549–1554. https://doi.org/10.12998/wjcc.v12.i9.1549
Pagnano, M. W., & Hanssen, A. D. (2018). Enhanced recovery after surgery (ERAS) protocols for total knee arthroplasty: A systematic review. Knee Surgery, Sports Traumatology, Arthroscopy, 26(2), 450-457. https://doi.org/10.1007/s00167-017-4542-2
Patel, P. R., & Bechmann, S. (2023). Discharge planning. In StatPearls. StatPearls Publishing. https://www.ncbi.nlm.nih.gov/books/NBK539858/
Stoicea, N., Magal, S., Kim, J. K., Bai, M., Rogers, B., & Bergese, S. D. (2018). Post-acute transitional journey: Caring for orthopedic surgery patients in the United States. Frontiers in Medicine, 5, 342. https://doi.org/10.3389/fmed.2018.00342
You, S., Li, N., Guo, M., & Ji, H. (2024). Are patients ready for discharge from the hospital after fast-track total knee arthroplasty? A qualitative study. PLoS One, 19(5), e0303935. https://doi.org/10.1371/journal.pone.0303935
Stakeholders can submit questions and issues to CMS through an online tool (Q&A tool) available to the public on QualityNet. CMS responds to each question submitted by stakeholders. Through the Q&A tool, stakeholders have asked for assistance with their questions including interpreting their facility’s patient-level data, understanding measure specifications (inclusion, exclusion, risk adjustment), and interpretation/clarification of results.
Each year, we review and consider issues raised through the Q&A or in the literature related to this measure, and those issues are considered by measure and clinical experts. Any issues that warrant additional analytic work due to potential changes in the measure specifications are addressed as a part of annual measure reevaluation. If small changes are indicated after additional analytic work is complete, those changes are usually incorporated into the measure in the next measurement period. If the changes are substantial CMS may propose the changes through rulemaking and adopt the changes only after CMS receives public comment on the changes and finalizes those changes in rulemaking.
Each year we also review and consider changes to HCPCS and ICD-10 codes that are then incorporated into the measure. Those code set files are made available to the public on QualityNet.
As noted, the Medicare Advantage (MA) population was an update to this measure’s cohort. MA beneficiary enrollment has been rapidly expanding as a share of Medicare beneficiaries. In 2023, nearly 51% of the eligible Medicare beneficiaries — or 30.8 million people — were covered by MA plans (Ochieng et al., 2023) The Congressional Budget Office projects that by 2030, 62% of beneficiaries will be covered by MA plans (Congressional Budget Office, 2022).
The inclusion of MA beneficiaries has several important benefits for the reliability and validity of the hospital outcome measures. The addition of MA beneficiaries to FFS significantly increases the size of the measure’s cohort, which enhances the reliability of the measure scores, leads to more hospitals receiving results, and increases the chance of identifying meaningful differences in quality for some low-volume hospitals (Kyanko et al., 2024). This update is currently undergoing the MUC 2024 submission process.
References
Kyanko K, Sahay KM, Wang Y, et al. Incorporating Medicare Advantage Admissions Into the CMS Hospital-Wide Readmission Measure. JAMA Netw Open. 2024;7(6):e2414431. doi:10.1001/jamanetworkopen.2024.14431
Ochieng N, Freed M, Biniek JF, Damico A, Neuman T. Medicare Advantage in 2023: Enrollment
Update and Key Trends. Accessed January 31, 2024. https://www.kff.org/medicare/issuebrief/medicare-advantage-in-2023-enrollment-update-and-key-trends/
Congressional Budget Office. Congressional Budget Office Baseline Projections May 2022. Accessed January 31, 2024. https://www.cbo.gov/system/files/2022-05/51302-2022-05-medicare.pdf
The THA/TKA Complications measure described in this CBE submission has been extensively updated, with cohort expansion (adding in Medicare Advantage (MA) patients), and risk variable re-selection. In addition, the performance period has been changed from three years to two years. Because of those changes, we are unable to compare the performance of this updated THA/TKA Complications measure with the performance of the previous measure.
We provide, however, a comparison of the distribution of risk-standardized complication rates (RSCRs) from the Fee-for-Service (FFS)-only measure, across different time periods. Table 13 (in the attachment) shows when comparing the distribution of risk-standardized scores across four time periods of the version of the FFS-only measure, hospital-level THA/TKA Complications risk-standardized complications rates (RSCRs) generally decrease across performance periods. This improvement in both RSCRs can also be seen from a density plot of performance over time (see Figure 6 in the "All Figures and Tables THA TKA Complications" attachment).
There have been no unexpected findings, negative or positive, during the implementation of this measure, including unintended impacts on patients.
Comments
Staff Preliminary Assessment
CBE #1550 Staff Assessment
Importance
Strengths:
- A logic model is provided, depicting the relationships between hospital inputs, which are largely processes (e.g., reconciling medications, enrolling patients in peri-operative care programs, providing patient education), and desired outcomes (e.g., reducing complications risk, improved health status and quality of life, decreased recovery time).
- The measure is supported by literature, noting various complications following THA/TKA. The developer posits that due to the estimated increase in these costly procedures, it is important to address quality of care to improve complication rates.
- As noted in the logic model, the developer provides evidence of interventions/actions that entities can do to mitigate complications and associated costs.
- Data from Medicare claims (01/01/2022 – 12/31/2023) show a decile range from 2.57% to 4.96%, indicating variation in measure performance. The developer further calculated a median odds ratio of 2.21, indicating that a patient has 2.2 times the odds (or 121% greater risk) of a complication if they received a THA/TKA procedure at a high-risk hospital compared to a lower-risk hospital.
- The developer states that patients will benefit from this measure, as it will provide important that quality performance information that patients can consider when choosing a hospital for these procedures.
- The developer further cites evidence, which found lower quality of life experienced by patients with postoperative complications compared with those who did not experience complications.
Limitations:
- Despite the range in performance scores by decile, roughly half of the overall population reside in the first three deciles, which have the highest performance scores. The concentration of patients in the higher performance deciles implies that there is less variation in care quality among the providers being measured, and most are performing well. However, the developer posits that there is still room for continued improvement.
Rationale:
- This maintenance measure is supported by a logic model that clearly depicts hospital activities that can lead to desired outcomes, one of which is mitigating complications associated with THA/TKA procedures. The developer further summarizes evidence of how this measure focus area is meaningful to patients. The measure is supported by extensive literature on the need to improve care quality in this area. However, with half of patients are in the higher performance deciles, this implies that there is less variation in care quality among the providers being measured, and most are performing well.
Feasibility Acceptance
Strengths:
- As a claims-based measure, the measures uses electronic data sources, and the developer posits that the measure is calculated automatically as a normal part of routine care.
- The developer notes that they did not conduct an analysis of missing data as the measure is based on a 100% paid, final medical claims.
- The measure does not require additional data collection by facilities, which eliminates the reporting burden for hospitals.
- The measure is not proprietary in any way.
Limitations:
- None.
Rationale:
- The measure utilizes electronic data sources (e.g., claims), automatically generated within routine care, requires no additional data collection, and is non-proprietary, enhancing its accessibility and applicability.
Scientific Acceptability
Strengths:
- Data Sources and Dates: Data used for testing were sourced from CY2022-CY2023 Medicare Fee-for-Service and Medicare Advantage data and includes facilities with at least 25 cases. The reliability has been calculated using 2 years of data, and so the reliability results are only applicable if the measure is also calculated using two years of data.
- Accountable Entity-Level Reliability: The developer conducted signal-to-noise reliability testing at the accountable entity-level using the hospital intercepts from the hierarchical logistic regression model.
Limitations:
- The developer notes that there is a high correlation (0.999) between the hospital intercepts and the final measure (the RSRR). However, the within hospital variance of the calculated RSRR may not scale with the variance of the logistic distribution that is used to calculate the reliability. So although more than 70% of accountable entities meet the expected threshold of 0.6, this estimate of reliability may not reflect the reliability of the RSRR.
Rationale:
- There are potential issues with the accuracy of the results using the current reliability metrics. However, the identified limitations are deemed addressable, as the developer may consider performing additional reliability testing such as split-half reliability. By addressing this issue, there is potential to enhance the reliability.
- Recommended Actions to Address Reliability Concerns: Refine Reliability Testing Methods: Adjust the reliability testing methods or analytic approaches to improve the assessment the reliability of the final measure.
Strengths:
- The developer provided face validity evidence from a TEP (N=13) in response to the question "The risk-standardized complication rate obtained from the measures as specified can be used to distinguish between better and worse quality hospitals." 91.7% agreed (strongly, moderately, somewhat). The developer also provided accountable entity evidence, which demonstrated correlations with procedure volume, positing that volume is a proxy for underlying quality-related factors. The developer states that hospitals with higher procedural volumes tend to have more experienced staff and standardized care processes, which can lead to fewer errors, delays, and improved outcomes. Research supports that surgical complication rates decrease with increasing procedure volume, and this relationship was used by the developer to validate the THA/TKA Complications measure. The developer expected a negative correlation between hospital volume and complication rates, indicating that higher volumes are associated with better outcomes. Overall, correlation for hospitals with ≥200 procedures was -0.138 (p<.0001); for hospitals with <200 procedures was -0.099 (p<.51).
- The developer conducted statistical risk adjustment, based on a conceptual model, selecting risk factors that have a significant correlation to the outcome. The developer also explored social risk factors, such as dual eligibility and the Area Deprivation Index. The developer did not include these in the final models due to the minimal impact these social risk factors have on the measure scores and to the model overall. The developer reported a c-statistic of 0.663, indicating acceptable model discrimination.
Limitations:
- Causal claims based on association (correlation) studies alone are prone to bias (i.e., confounding due to a common cause cannot be ruled-out). Additional support from mechanism studies that confirm the existence of a suitable (plausible) mechanism capable of accounting for the observed correlation would strengthen the causal claim. Otherwise, statements about the relative magnitude of the observed correlations and whether those magnitudes are greater or lesser than what one might anticipate are difficult to evaluate. The absence of material variation among entities in the Importance Table also does not substantiate validity.
Rationale:
- For validity, the developer conducted both face and empiric validity testing. Face validity was assessed by a technical expert panel of 12 members from diverse backgrounds, with 91.7% affirming the measure's ability to differentiate hospital quality. Empiric testing involved correlating the measure score with hospital procedure volume, supported by research indicating that higher volumes correlate with better outcomes due to more experienced staff and standardized processes. The results showed a significant negative correlation between hospital volume and complication rates, particularly in hospitals with over 200 procedures, validating the measure's effectiveness and the hypothesis that higher volumes lead to fewer complications. The risk adjustment methods used are appropriate with and acceptable model performance.
Equity
Strengths:
- The developer evaluated two social risk factors, dual eligibility status (DE) and high Area Deprivation Index (ADI). The developer identified that “unadjusted complication rates for patients with DE or high ADI were higher compared to patients without either social risk factor (mean unadjusted rates for DE vs non-DE, 4.0% vs 3.4%; high ADI vs. low ADI, 4.0% vs 3.3%).”
- THA/TKA Complications measure scores are highly correlated whether social risk factors like DE and high ADI are included or not (correlation coefficients of 1.000 and 0.989, respectively), with only minimal differences in the scores. This indicates that these social risk factors have negligible impact on the measure scores.
Limitations:
- The developer provides rationale as to why DE and high ADI are not included in the risk adjustment model as the inclusion of these social risk factors did not significantly impact the measure scores. The analyses showed that the clinical risk variables already included in the model sufficiently account for the risks associated with DE and high ADI. Additionally, the model was found to perform well for patients with these social risk factors, indicating that the existing model variables effectively capture the variance in outcomes related to these factors.
- Despite these findings, the results do not show how the scores differ across these social risk factors or others the developer identified. Specifically, the developer noted in the risk adjustment section that Black and Hispanic patients have higher rates of postoperative complications. Race was not explored in this analysis, which may be a future consideration for this domain.
Rationale:
- The analysis of the THA/TKA Complications measure showed that including social risk factors like DE and high ADI does not significantly change the scores, indicating that existing risk model variables sufficiently account for these risks. However, the analysis did not explore how scores vary across different social risk factors, such as race, which is known to affect complication rates, suggesting an area for future consideration for this domain.
Use and Usability
Strengths:
- This updated measure has a planned used in the Hospital Inpatient Quality Reporting (HIQR) and the Hospital Value-Based Purchasing (HVBP) programs to replace the fee-for-service-only measure.
- The developer provides a summary of how accountable entities can use the measure results to improve performance. Specifically, hospitals can implement multifaceted strategies that focus on preoperative optimization, advanced surgical techniques, and rigorous postoperative protocols, which have been shown to reduce complication rates.
- Hospitals can also conduct preoperative education and health optimization to improve on the measure, or leverage technology and communication, such as employing electronic health records and ensuring clear patient communication.
- Lastly the developer states that hospitals may also benefit from more structured care models, like the Comprehensive Care for Joint Replacement (CJR) Model, which incentivizes coordinated care and adherence to evidence-based practices.
- Stakeholders can use the Q&A tool on QualityNet to submit questions to CMS, seeking assistance with interpreting patient-level data, understanding measure specifications, and clarifying results. CMS responds to each inquiry submitted through this tool.
- CMS annually reviews and addresses stakeholder feedback and literature on the measure, making necessary updates through expert reevaluation and rulemaking after public comment.
- The measure also incorporates yearly updates to HCPCS and ICD-10 codes and includes the expanding Medicare Advantage population, enhancing cohort size and measure reliability, currently under review in the MUC 2024 process.
- The developer states that since the measure has been significantly updated, including the addition of Medicare Advantage patients, revised risk variables, and a shortened performance period from 2 to 3 years, they are unable to compare the performance with the fee-for-service (FFS)-only measure. However, historical data from the FFS-only measure show a general decrease in hospital-level risk-standardized complication rates over different time periods (mean scores decreasing from 3.3 in 2010-2013 data to 2.4 in 2020 – 2021 data).
- The developer reports no unexpected findings (positive or negative) with the implementation of the measure.
Limitations:
- Although there is a process for collecting and considering feedback, the developer does not mention whether feedback received led to any changes in the measure specifications.
Rationale:
- For maintenance, the current Medicare FFS-only measure is actively used in the HIQR and hospital value-based purchasing (HVBP) programs, with plans to incorporate Medicare MA patients. There is a clear feedback approach. However, it is unclear what, if any, feedback received led to any changes in the measure specifications. The developer also provides evidence of actions hospitals can take to improve measure performance, such as preoperative optimization, advanced surgical techniques, and rigorous postoperative care. Hospitals can also benefit from structured care models such as the Comprehensive Care for Joint Replacement (CJR) Model. Lastly, with recent changes to the measure cohort, the developer provides evidence of improvements in the FFS-only version of the measure over time and reports no unexpected findings due to the measure’s use.
Committee Independent Review
Endorsement Should Be Removed from CBE 1550
Importance
The developer did not report how many hospitals would be classified as different from the national average using the proposed measure. The Measure Update Report for the current version of the measure indicates that the measure was only able to determine that 27 hospitals out of 3,257 hospitals performed “Better Than the National Rate,” and only 8 performed “Worse Than the National Rate.” 1,498 hospitals had too few cases (less than 25) to make a determination of whether the hospital was better or worse than average.
Moreover, it does not appear that the developer made any effort to examine the hospitals identified as “Better” or “Worse” than the national rate to assess whether this was really a valid classification based on characteristics of the patients and factors such as how many patients were treated in emergency departments or observation stays rather than through inpatient admissions or what proportion of patients received surgery on an outpatient vs inpatient basis.
If so few hospitals can be classified as anything other than “no different than expected,” this measure is unlikely to encourage improvements in the quality of care. On the other hand, the weaknesses in the measure methodology could cause some facilities to be inappropriately labeled as “worse than expected” because of the kinds of patients they treat.
Feasibility Acceptance
The measure is calculated using claims data, and many of the weaknesses in the measure could also be corrected through the use of information available in claims data.
Scientific Acceptability
The signal-to-noise reliability reported by the developer was less than 0.5 for almost half of hospitals, which is far too low to justify using this measure for public reporting or other accountability purposes. The poor reliability of the measure is not surprising given the small number of patients receiving inpatient hip and knee surgeries in most hospitals and the small variation in complication rates between hospitals. Based on the developer’s response to public comments, it appears that even after excluding low-volume hospitals, reliability is still unacceptably low for one-fourth of hospitals. It is likely that the probability of misclassification would be high given the low signal-to-noise reliability for so many hospitals, but the developer did not report misclassification probabilities. The developer also did not report year-to-year reliability, even though multiple years of data were available to enable that assessment.
This is not a valid measure of outcomes because it only counts inpatient admissions to a hospital, not observation stays, emergency department visits, or visits to a physician’s office for treatment of complications of surgery. Hospitals experiencing shortages of inpatient beds that have to treat patients in the ED will have lower “readmission” rates as a result. Because observation stays are not included, the measure definition creates a perverse incentive to treat patients with complications in observation stays rather than admitting them as inpatients. No analysis was provided regarding the variation across hospitals in the proportion of patients treated in other settings, even though that could be a primary reason for variation in inpatient admissions.
The developer has indicated that exclusion of outpatient treatments is intended to “incentivize” treatment in outpatient settings, but the purpose of this measure should be to assess the quality of care, not to provide financial incentives to treat complications in particular ways.
In addition, differences in complication rates under this measure could be affected by variations in the proportion of patients who receive outpatient surgery as well as by the quality of care delivered to the patients receiving inpatient surgery, yet there is no variable in the risk adjustment model that assesses the proportion of Medicare beneficiaries in the community receiving inpatient vs. outpatient surgery.
Equity
The analysis of social risk factors reported by the developer demonstrated that lower income individuals have higher complication rates. The developer claims that addition of low income variables to the risk adjustment model had “almost no impact on measure scores,” but no data are provided to document this. The median difference in scores and the correlation analysis presented in Table 12 are not valid ways to determine whether a risk adjustment variable should be included in the model; if they were, the same analyses should have been provided for all of the variables in the model, not just those the developer wanted to exclude. The developer says that the decision not to adjust for social risk factors was made by the Centers for Medicare and Medicaid Services (CMS), not by the developer.
Use and Usability
Since this is not a valid measure of the outcomes of care for patients receiving hip or knee surgery, it does not provide useful information for improving the quality of care. In fact, use of the measure could mislead patients about where to receive care and could discourage hospitals from treating patients who are at higher risk of complications due to limited access to community services.
Summary
Endorsement should be removed from this measure. It is not a valid measure of the outcomes of care since it only measures complications treated during an inpatient admission to the hospital, not complications treated during an observation stay or in an emergency department. Moreover, the measure only assesses the quality of care for the subset of patients who receive inpatient hip or knee surgery, and rankings of hospital performance will be affected by the proportion of the total patients in the community who receive inpatient vs. outpatient surgery as well as by the quality of care delivered to the patients receiving inpatient surgery. The risk adjustment methodology inappropriately rewards hospitals with a higher proportion of Medicare Advantage patients, and there is no adjustment for a patient’s income, level of home support, or access to outpatient care after discharge. This is not a reliable measure of quality due to the small number of inpatient hip and knee surgeries at most hospitals and the small variation in complication rates across hospitals. Continued use of the measure could mislead patients about where to receive surgery and could discourage hospitals from treating patients who are at higher risk of complications.
Not met, I disagree with staff assessment
Importance
Agree with staff assessment
Feasibility Acceptance
Agree with staff assessment
Scientific Acceptability
Agree with staff assessment
Agree with staff assessment
Equity
A fuller understanding of SDOH factors is very important in a treatment option required by many individuals with SDOH
Use and Usability
Agree with staff assessment
Summary
While I agree with the staff that this measure seems to be technically valid with some concerns addressed, I believe that three factors make this unacceptable from the patient perspective.
- The lack of SDOH information fully integrated into the measure.
- The limitation to hospital performed procedures. A major decision for both the clinician and the patient is where to do these procedures. With an increasing number of these procedures performed in outpatient surgery centers their inclusion is essential to patients’ use of the measure.
- Failure to include urgent care centers and non-routine office visits significantly reduces the viability of this measure. As a patient, if I have to visit an urgent care facility due to a complication or if I have to go to the doctor’s office to address a complication even if the issue is addressed in these settings without hospital admission, I have experienced the complication and payers are paying for it to be treated.
Do Not Support
Importance
Agree with prelim assessment
Feasibility Acceptance
Agree with prelim assessment
Scientific Acceptability
Agree with prelim assessment
Agree with prelim assessment
Equity
Agree with prelim assessment
Use and Usability
Agree with prelim assessment
Summary
A lot of concerns need to be addressed, recommend revisiting concerns to address and then bring forward again
no additional comments
Importance
agree with staff review
Feasibility Acceptance
agree with staff review
Scientific Acceptability
agree with staff review
agree with staff review
Equity
agree with staff review
Use and Usability
agree with staff review
Summary
no additional comments
Should not be endorsed in its current form
Importance
The measure targets a critical clinical and financial concern.
Feasibility Acceptance
technically feasible as written (claims based)
Scientific Acceptability
Met
Face validity via expert review is a strength. Exclusion of SDOH is a problem
Equity
Excluding social risk factors from the risk adjustment model limits the measure's ability to address disparities, and potentially penalizes hospitals serving disadvantaged populations.
Use and Usability
Excluding outpatient procedures and non-inpatient complications limits the measure's utility as more procedures and care shift to outpatient settings.
Summary
Important clinical issue and measure feasibility is high. However, the inpatient vs outpatient concern needs to be addressed, the potential equity impact needs to be addressed, and most importantly the path from how hospitals would use these data to drive QI to improve outcomes needs to be clearly laid out with evidence. "identify areas for quality improvement, such as preoperative optimization, patient education, and standardized care processes seems very vague. For example, how specifically is "patient education" going to be identified as an improvement opportunity from this metric? What specific intervention targeting patient eduction has evidence that it improves readmission or other patient outcomes.
Public Comments
1550 Hospital-Level, Risk-Standardized Complication Rate (RSCR)
The American Medical Association (AMA) believes that a case minimum of above 25 individuals must be required as a part of endorsement of this measure. This minimum would ensure that the measure’s minimum reliability is close to 0.7, which is what we believe should be the standard for endorsed measures.
Response to AMA Public Comment (THA/TKA Complications)
Thank you for your comment. The public reporting threshold is part of measure implementation. We note that the measure score is calculated for all hospitals, and then the results are published publicly only for hospitals that perform at least 25 procedures during the performance period.
Endorsement Should Be Removed From This Measure
Endorsement should be removed from this measure. The measure only assesses the quality of care for the subset of patients receiving hip or knee surgery during an inpatient admission, and rankings of hospital performance will be affected by the proportion of patients who receive outpatient surgery as well as by the quality of care delivered to the patients receiving inpatient surgery. It is not a valid measure of the outcomes of care since it only measures complications treated during an inpatient admission to the hospital, not complications treated during an observation stay or in an emergency department. The risk adjustment methodology inappropriately rewards hospitals with a higher proportion of Medicare Advantage patients, and there is no adjustment for a patient’s income, level of home support, or access to outpatient care after discharge. This is not a reliable measure of quality due to the small number of surgeries at most hospitals and the small variation in complication rates across hospitals. Continued use of the measure could mislead patients about where to receive surgery and could discourage hospitals from treating patients who are at higher risk of complications.
Problems with the Numerator and Denominator
The measure developer provided no information at all about the proportion of patients who were treated for complications in other settings after discharge, and no analysis was provided regarding the variation across hospitals in the proportion of patients treated in other settings, even though that variation could be the primary reason for variation in inpatient admissions for complications. The developer has indicated that exclusion of outpatient treatments from hospital readmission and complication measures is intended to “incentivize” treatment in outpatient settings, but the purpose of this measure should be to assess the quality of care, not to provide financial incentives to treat complications in particular ways.
Problems with the Risk Adjustment Methodology
Poor Reliability of the Measure
The signal-to-noise reliability reported by the developer was less than 0.5 for almost half of hospitals, which is far too low to justify using this measure for public reporting or other accountability purposes. Misclassification probabilities are a more appropriate way of assessing reliability for a measure which is intended to classify hospitals. The developer did not report misclassification probabilities (even though the references they cite for calculating reliability describe how to do so), and it is likely that the probability of misclassification would be high given the low signal-to-noise reliability for so many hospitals.
The poor reliability of the measure is not surprising given the small number of patients receiving inpatient hip and knee surgeries in most hospitals and the small variation in complication rates between hospitals. As shown in Table 2, on average there are about 87 patients receiving hip or knee surgery at each hospital, and the complication rates for 70% of the hospitals differ by less than 1% (3.06% in Decile 2 and 3.89% in Decile 8). With 87 patients, a hospital’s complication rate could change from 3.4% to 4.6% (a change from Decile 4 to Decile 9) because of a single patient being readmitted to the hospital. As a result, many hospitals could easily have a “high” complication rate one year and a “low” rate the next. Unfortunately, this cannot be assessed because the developer did not report year-to-year reliability, even though multiple years of data were available to enable that assessment.
Lack of Business Case for Using the Measure and Undesirable Effects of Doing So
The developer did not report how many hospitals would be classified as different from the national average using the proposed measure. However, the Measure Update Report for the current version of the measure (which uses data only for Original Medicare beneficiaries) indicates that the measure was only able to determine that 27 hospitals out of 3,257 hospitals performed “Better Than the National Rate,” and only 8 performed “Worse Than the National Rate.” 1,724 hospitals were classified as “No Different Than the National Rate,” and 1,498 had too few cases (less than 25) to make a determination of whether the hospital was better or worse than average.
Moreover, it does not appear that the developer made any effort to examine the hospitals identified as “Better” or “Worse” than the national rate to assess whether this was really a valid classification based on characteristics of the patients and factors such as how many patients were treated in emergency departments or observation stays rather than through inpatient admissions or what proportion of patients received surgery on an outpatient vs inpatient basis.
If only one percent of hospitals can be classified as anything other than “no different than expected,” this measure is unlikely to encourage improvements in the quality of care delivery. On the other hand, the weaknesses in the measure methodology could cause some facilities to be inappropriately labeled as “worse than expected” because of the kinds of patients they treat. As a result, use of the measure could mislead patients and create an undesirable incentive for hospitals to avoid treating patients who are at higher risk of complications.
Response to CHQPR Public Comment: THA/TKA Complications
Developer’s Summary Response:
We thank the commenter for the time taken to provide feedback on the THA/TKA Complications measure. The THA/TKA Complications measure recently expanded to include Medicare Advantage patients, is a valid, reliable, and patient-centered measure of complications that are so serious that they require inpatient hospitalization. The measure focuses on the highest-risk patients (those that undergo THA/TKA procedures in the inpatient setting) and aims to reduce complications and exposure of patients to additional risks in the high-cost hospital inpatient setting. The measure has been thoughtfully developed to allow for fair comparisons for hospitals with different case mix and limits the public reporting of measure scores to hospitals with sufficiently high volume while providing all hospitals with detailed, admission-level data to help identify areas for quality improvement. Importantly, measure scores have improved over time, in the setting of quality improvement efforts. We respond to each of the commenter’s concerns below.
Cohort:
Comment: Exclusion of Patients Receiving Outpatient Surgery. When this measure was created, hip and knee replacement surgeries for Medicare beneficiaries could only be performed on an inpatient basis.
Response: The measure captures only inpatient procedures for several reasons (1) CMS measures are limited by their programmatic use. Therefore, CMS measures inpatient care paid for through the Inpatient Prospective Payment System (IPPS) in the Inpatient Quality Reporting (IQR) Program (HRRP, HVBP), and care paid for through the Outpatient Prospective Payment System (OPPS) in the Hospital Outpatient Quality Reporting (OQR), and so on. (2) Inpatient procedures, while less common today, are typically performed on the highest-risk patients, who also have higher complication rates. Therefore, even from a measurement perspective, it is logical to focus a measure on the highest-risk and the highest-cost setting. CMS captures outpatient THA/TKA procedures through several other measures, including HOPD Surgery, ASC Orthopedic Surgery, and the not-yet-implemented THA/TKA PRO-PM measure (one for each of the two outpatient settings, again for statutory reasons – OQR and ASCQR).
Comment: Exclusion of Patients During Their First Year of Medicare Eligibility. The measure only includes patients who were enrolled in Medicare for the 12 months prior to the date of admission to the hospital. This means that all new Medicare beneficiaries are inappropriately excluded from the measure.
Response: Enrollment in Medicare (FFS or MA) in the prior 12 months ensures that the comorbidity data used in risk adjustment can be captured from inpatient, outpatient, and physician claims data for up to 12 months prior to the index admission, to augment the index admission claim itself. This is a highly appropriate inclusion criterion.
Outcome
Comment: The measure only counts inpatient admissions to a hospital, not observation stays, emergency department visits, or visits to a physician’s office for treatment of complications of surgery.
Response: The focus of the THA/TKA measure is complications that are so serious they require an inpatient hospital readmission. By emphasizing inpatient admissions, the THA/TKA Complications measure focuses on the events most relevant to patient safety, outcomes, and healthcare costs. This is a deliberate focus on the most severe and costly of the post-discharge hospital-based acute care events. CMS does, however, have broader, condition- and procedure-specific measures that capture days in acute care that include inpatient admissions, ED visits, and observation stays (the Excess Days in Acute Care [EDAC] measures), and has developed but not yet implemented one for THA/TKA procedures.
The commenter requested information about utilization of care in other settings (ED visits and observation stays). Our results show that by far, the largest proportion of care (about 70%) is represented by post-procedural inpatient days in care. We provide information from the THA/TKA EDAC measure, which examines all-cause days in acute care following a THA/TKA procedure and therefore differs in the outcome from the THA/TKA Complications measure because it does not limit the outcome to specific complications. Using one year of data (January 1, 2022-December 30,2022) and including both Medicare Advantage and Medicare fee-for-service admissions, inpatient admissions account for 40.0 of 56.3 (71.0%) of the total (average) unadjusted days in acute care after an (inpatient) THA/TKA procedure, compared with 6.72 observation days (11.9%) and 12.6 ED days (16.0%).
We note that the THA/TKA Complications measure is not in HRRP; it is in the Inpatient Quality Reporting (IQR) Program, and the Hospital Value-Based Purchasing Program (HVBP).
Risk Adjustment
Comment: The commenter expressed concern about the decision to not adjust for low income status and had concerns about the types of empirical analyses and results provided by the developer.
Response: Our results show that patients with social risk factors have higher rates of unadjusted outcomes (Table 11) but that when we adjust for social risk factors, measure scores are almost identical when calculated with and without each social risk factor (correlation coefficients of 1.0 and 0.090, p<0.0001). Those results are provided both in the narrative and in Table 12 (in the attachment of figures and tables). The calibration plots in Figures 4 and 5 show good model calibration in patients with DE and high ADI (vs. patients without social risk factors). There is little impact of adjustment on social risk factors likely (at least in part) because there is an access issue for patients with social risk factors. For example, for condition-specific measures, the proportion of DE patients may be as high as 25%; for this measure, the proportion of DE patients is about 5%. The results support not adjusting for social risk in this measure. We note that the developer is a CMS contractor, and therefore, CMS makes all final decisions.
Comment: Inappropriate Adjustment for Type of Insurance. In addition to adjusting for clinical characteristics of patients, the prediction model includes a variable for whether the patient has coverage through a Medicare Advantage plan rather than Original Medicare. The developer states that this is to “adjust for the generally higher prevalence of comorbidities in the MA cohort.” However, the odds ratio for the MA variable reported in Table 8 is positive, indicating that it would result in a higher risk score for an MA patient with the same comorbidities as a patient with Traditional (“FFS”) Medicare. This inappropriately rewards a hospital for having a higher proportion of MA patients.”
Response: Because MA status is associated with the outcome and differs systematically across key risk variables, it may be acting as a confounder, and lack of adjustment for MA status could bias the results. While the effect size of the MA indicator is small (odds ratio of 1.04) including it in the model ensures accuracy and fairness in adjusting for case mix between hospitals. In addition, overall, our model testing results suggest that the model is adjusting fairly for case mix across hospitals.
References
Joiner KA, Lin J, Pantano J. Upcoding in Medicare: where does it matter most? Health Econ Rev. 2024 Jan 2;14(1):1. doi: 10.1186/s13561-023-00465-4. PMID: 38165452; PMCID: PMC10759668.
Comment: Inadequate Assessment of Model Fit. The measure developer did not provide sufficient information to judge how well the risk adjustment model controls for patient characteristics affecting complications.
Response: Yale/CORE provided all of the model testing required for CBE endorsement, including c-statistic, predictive ability, overfitting, and calibration plots. Figure 3 shows good model calibration for both 2022 and 2023 data. The calibration plots in Figures 3A (2022) and 3B (2023) are correct but are on a different scale. The point of the 2022 and 2023 results is not to compare them to each other, but rather to compare the observed values to the predicted values within the same year.
We understand the reviewers’ concerns about overfitting, but several elements of this model mitigate those risks. First, the hierarchical structure accounts for clustering and naturally applies shrinkage to extreme values, which prevents overfitting at the hospital level. The higher-than-expected gamma zero likely reflects the relatively low outcome rate and the fact that testing (at the time) was limited by data availability to one year of data, which was split into two datasets to have a development and a validation dataset. Importantly, the slope (gamma 1) is consistent across datasets suggesting the relationship between predictors and outcomes generalizes well. In addition, our other testing results (c-statistic, predictive ability, and calibration plots) performed on both development and validation datasets show good model performance on unseen data. The commenter noted that we did not provide the Hosmer-Lemeshow test; this is because of the known limitations of this test [1], and we instead provide overfitting and risk-decile plots.
References
Reliability
Comment: The signal-to-noise reliability reported by the developer was less than 0.5 for almost half of hospitals, which is far too low to justify using this measure for public reporting or other accountability purposes.
Response: The results cited by the commenter for all hospitals, even those that do not meet the minimum admission volume for public reporting. As shown in Table 4, more than 75% of hospitals have signal-to-noise reliability above the Battelle threshold of 0.6. Reliability was calculated using two years of data (CY2022/2023); as this is a two-year measure, we do not currently have data beyond 2023 to calculate reliability with more recent data. The commenter can view the publicly available reliability results for the prior submission of this measure (the FFS-only measure) which shows comparable reliability results for hospitals with at least 25 cases. Due to the statistical approach used to calculate measure scores, small facilities with low outcome rates will be pulled to the middle of the distribution of measure scores (https://www.cms.gov/medicare/quality-initiatives-patient-assessment-instruments/hospitalqualityinits/downloads/statistical-issues-in-assessing-hospital-performance.pdf). This calculation method avoids the situation suggested by the commenter where small facilities would swing from a low score to a high score with just one or two outcome events. In addition, facilities below the public reporting threshold for the volume of procedures are not included in public reporting (although they do still receive their measure score and claims-level detail). We further note that we provide the types of testing required by Battelle.
Comment: The commenter states that the business case for this measure is lacking and that the developer did not report how many hospitals would be classified as different from the national average using the proposed measure.
Response: Classification into performance categories is an implementation step, and this measure has not yet been implemented. Furthermore, the classification approach (95% confidence intervals) is very stringent and is a measure of statistical differences rather than clinically meaningful differences. We note that the business case for this measure has been established as evidenced by the improvement already shown across several performance periods (please see Figure 6 in the attachment of tables and figures) which has also been published [1]. In addition to receiving a measure score, hospitals receive detailed patient-level information that allows them to pinpoint underlying quality issues to address as part of their improvement efforts.
References