When the measure returns for maintenance, the committee would like to see:
- Explore systemic differences in ED admission/readmission rates and the potential impact on the clinician-group’s ability to improve
The 30-day Hospital-Wide, All-Cause Unplanned Readmission (HWR) Rate for the Merit-based Incentive Payment System (MIPS) Groups measure is a risk-standardized readmission rate for beneficiaries age 65 or older who were hospitalized and experienced an unplanned readmission for any cause to a short-stay acute-care hospital within 30 days of discharge. The measure attributes readmissions to up to three MIPS participating clinician groups, as identified by their Medicare Taxpayer Identification Number (TIN), and assesses each group’s readmission rate. The measure reports a single summary risk-adjusted readmission rate (RARR) derived from the volume-weighted results of 5 different models, one for each of the following specialty cohorts based on groups of discharge condition categories or procedure categories: surgery/gynecology; general medicine; cardiorespiratory; cardiovascular; and neurology, each of which will be described in greater detail below. This re-specified clinician group measure replaced the All-Cause Readmission (ACR) measure previously used in the Quality Payment Program (QPP).
Measure Specs
- General Information(active tab)
- Numerator
- Denominator
- Exclusions
- Measure Calculation
- Supplemental Attachment
- Point of Contact
General Information
The Hospital-Wide 30-Day, All-Cause, Unplanned Readmission Rate for the Merit-Based Incentive Payment System (MIPS) Eligible Clinician Group (MIPS HWR) measure addresses unplanned readmissions at the clinician group level for Medicare Fee-For-Service (FFS) beneficiaries aged 65 or older. The measure is risk adjusted and based on administrative claims. This measure is a re-specified version of the hospital-level measure, Hospital-Wide All-Cause, Unplanned Readmission (consensus-based entity [CBE] #1789) and related to the Hybrid Hospital-Wide All-Cause, Unplanned Readmission measure (CBE #2879e) that is under review in this same endorsement cycle (Spring 2024). The MIPS HWR measure has the same cohort, outcome, and claims-based risk variables as CBE #2879e and promotes a systems-level approach by clinicians and a focus on high-risk conditions, such as chronic obstructive pulmonary disease (COPD) and heart failure.
Hospital readmission, for any reason, is disruptive to patients and caregivers, costly to the healthcare system, and puts patients at additional risk of hospital-acquired infections and complications. In 2018, there were 3.8 million readmissions with an average cost of $15,200, or a total projected cost of about $58 billion (Weiss & Jiang, 2021). Readmissions are also a major source of patient and family stress and may contribute substantially to loss of functional ability, particularly in older patients.
Some readmissions are unavoidable, but others may result from poor quality of care, inadequate coordination of care, or lack of effective discharge planning and transitional care. Interventions aimed at improving care at discharge, including improving communication, medication reconciliation, and ensuring timely follow up following discharge, have been shown to be effective at reducing readmission rates (Becker et al., 2021; Morkisch et al., 2020; Kripalani et al., 2014; De Oliveira et al., 2021; Anderson et al., 2022). (Please see section 6.2 for a discussion of interventions to reduce readmissions.)
There has been a nation-wide focus on reducing post-discharge readmissions for many years, and while progress has been made, the overall national readmission rate remains high, with a 30-day readmission ranging from 11 percent of admission for surgery/gynecology, to 17 percent for cardiovascular admissions (based on CMS data in FFS patients between July 1, 2022-June 30, 2023). Furthermore, readmission rates vary widely across institutions and clinician groups (risk-standardized 30-day readmission rates for more than 97,000 clinician groups ranged from about 9 percent to 27 percent with recent data). Both the high baseline rate and the variability across institutions speak to the need for a quality measure to prompt more concerted and widespread action.
The Centers for Medicare & Medicaid Services (CMS) is applying this measure to MIPS and continuing to attribute outcomes to clinician groups, because reducing avoidable readmissions is a key component in the effort to promote more efficient, high-quality care. Physician groups have the capability to influence unplanned readmission outcomes through interventions such as appropriate medication reconciliation at discharge, reduction of infection risk, and ensuring proper outpatient follow-up. Current performance on this measure indicates a substantial need to reduce the expected rate and reduce variation across eligible physician groups. As an administrative claims measure, there is no separate reporting burden.
References
Anderson, A., Mills, C. W., Willits, J., Lisk, C., Maksut, J. L., Khau, M. T., & Scholle, S. H. (2022). Follow-up Post-discharge and Readmission Disparities Among Medicare Fee-for-Service Beneficiaries, 2018. Journal of general internal medicine, 37(12), 3020–3028. https://doi.org/10.1007/s11606-022-07488-3
Becker, C., Zumbrunn, S., Beck, K., Vincent, A., Loretz, N., Müller, J., Amacher, S. A., Schaefert, R., & Hunziker, S. (2021). Interventions to Improve Communication at Hospital Discharge and Rates of Readmission: A Systematic Review and Meta-analysis. JAMA network open, 4(8), e2119346. https://doi.org/10.1001/jamanetworkopen.2021.19346
De Oliveira G, Castro-Alves L, Kendall M, McCarthy R. Effectiveness of Pharmacist Intervention to Reduce Medication Errors and Health-Care Resources Utilization After Transitions of Care: A Meta-analysis of Randomized Controlled Trials. Journal of Patient Safety. 2021; 17 (5): 375-380. doi: 10.1097/PTS.0000000000000283.
Kripalani, S., Theobald, C. N., Anctil, B., & Vasilevskis, E. E. (2014). Reducing hospital readmission rates: current strategies and future directions. Annual review of medicine, 65, 471-485.
Morkisch, N., Upegui-Arango, L. D., Cardona, M. I., van den Heuvel, D., Rimmele, M., Sieber, C. C., & Freiberger, E. (2020). Components of the transitional care model (TCM) to reduce readmission in geriatric patients: a systematic review. BMC geriatrics, 20(1), 345. https://doi.org/10.1186/s12877-020-01747-w
Weiss, A. J., & Jiang, H. J. (2021). Overview of clinical conditions with frequent and costly hospital readmissions by payer, 2018. HCUP Statistical Brief #278. Agency for Healthcare Research and Quality. https://www.hcup-us.ahrq.gov/reports/statbriefs/sb278-Conditions-Frequent-Readmissions-By-Payer-2018.pdf
Medicare FFS claims data Part A and B and Medicare Enrollment Database.
This is a claims-based measure and the measure score is calculated automatically from 100% final-action claims; claims data are routinely generated during the delivery of care. We did not encounter any difficulties with respect to data feasibility, reliability, or validity.
Numerator
The outcome for this measure is any unplanned readmission to a non-federal, short-stay, acute-care or critical access hospital within 30 days of discharge from an eligible index admission. Planned readmissions are not counted in the outcome. In the case of multiple readmissions during the 30-day period, only one of the readmissions is counted for the outcome. If a patient is readmitted to the same hospital on the same calendar day of discharge for the same condition as the index admission, the measure considers the patient to have had one single continuous admission (that is, one index admission). However, if the condition is different from the index admission, this is considered a readmission in the measure.
The measure attributes the outcome (readmission) to up to three clinician groups to account for the reality that multiple healthcare roles can influence readmissions. The following three types of clinician groups are included in the multiple attribution approach: Discharge Clinician Group, Primary Inpatient Care Provider Group, and Outpatient Primary Care Physician Group.
The outcome for the MIPS HWR measure is an unplanned readmission to a non-federal, short-stay, acute-care or critical access hospital within 30 days of discharge from an index admission. Readmissions during the 30-day period that are considered planned or follow a planned readmission are not counted in the outcome. In the case of multiple readmissions during the 30-day period, only one of the readmissions is counted for the outcome. If a patient is readmitted to the same hospital on the same calendar day of discharge for the same condition as the index admission, the measure considers the patient to have had one single continuous admission (that is, one index admission). However, if the condition is different from the index admission, this is considered a readmission in the measure.
Numerator Time Window: The outcome is defined as an unplanned readmission within 30 days of discharge from an index admission.
Planned Readmission Algorithm
The Planned Readmission Algorithm is a set of criteria for classifying readmissions as planned among the general Medicare population using Medicare administrative claims data. The algorithm identifies admissions that are typically planned and may occur within 30 days of discharge from the hospital.
The Planned Readmission Algorithm has three fundamental principles:
- A few specific, limited types of care are always considered planned (obstetric delivery, transplant surgery, maintenance chemotherapy/immunotherapy, rehabilitation).
- A non-acute readmission for a procedure that is typically scheduled in advance is considered planned.
- Admissions for acute illness or for complications of care is never considered planned.
The algorithm was developed in 2011 as part of the Hospital-Wide Readmission measure. In 2013, CMS applied the algorithm to its other readmission measures.
The Planned Readmission Algorithm and associated code tables can be found in the attached data dictionary.
Attribution of the Outcome
There are three eligible clinician groups for attribution: (1) the Primary Inpatient Care Provider, (2) the Discharge Clinician, and (3) the Outpatient Primary Care Physician.
- Primary Inpatient Care Provider: All patient-facing claims for the patient filed during the stay are identified and totaled by clinicians identified on each claim; the admission is attributed to the clinician with the greatest charges billed. The cost of charges billed (as opposed to number of charges) better reflects the appropriate clinician, especially for the surgical specialty cohort. The identified primary inpatient care provider may also be the discharge clinician.
- Discharge Clinician: Identified by Current Procedural Terminology (CPT®) code 99238 or 99239 within the last three days of admission OR CPTs 99231, 99232, 99233 billed on the last day of admission. If none of these codes are found, a Discharge Clinician is not assigned.
- Outpatient Primary Care Physician: The clinician who provides the greatest number of claims for primary care services during the 12 months prior to the hospital admission date.
Eligible clinician groups are defined by grouping eligible clinicians who use the same Taxpayer Identification Number (TIN). Index admissions are attributed to a clinician group by each of these rules. Though an admission may be attributed to three distinct eligible clinician groups, it will often be the case that two or even all three of the above listed roles for a given patient are filled by clinicians assigned to the same clinician group. In the case of multiple assignments of an admission to the same eligible clinician group, each admission is included only once when measuring the eligible clinician group. Importantly, this implies that while there are three different rules for attribution, these are not distinguished when measuring clinician group performance. While a clinician group can have admissions attributed to them in multiple capacities – for instance, a clinician from the same group may be both a Discharge Clinician for some patients and a Primary Inpatient Care Provider for others – all attributed admissions are used to construct a single score for that eligible clinician group.
Denominator
Eligible index admissions include acute care hospitalizations for Medicare Fee-for-Service (FFS) beneficiaries age 65 or older at non-federal, short-stay, acute-care, or critical access hospitals that were discharged during the performance period. Beneficiaries must have been enrolled in Medicare FFS Part A for the 12 months prior to the date of admission and 30 days after discharge, discharged alive, and not transferred to another acute care facility. Admissions for all principal diagnoses are included unless identified as having a reason for exclusion. A hospitalization that counts as a readmission for a prior stay may also count as a new index admission if it meets the criteria for an index admission.
A list of inclusion criteria is shown below. Eligible index admissions include acute care hospitalizations for Medicare FFS beneficiaries aged 65 or older at non-federal, short-stay, acute-care or critical access hospitals that were discharged during the performance period. Beneficiaries must have been enrolled in Medicare FFS Part A for the 12 months prior to the date of admission and 30 days after discharge, discharged alive, and not transferred to another acute care facility. Admissions for all principal diagnoses are included unless identified as having a reason for exclusion. A hospitalization that counts as a readmission for a prior stay also may count as a new index admission if it meets the criteria for an index admission.
Index admissions are assigned to one of five specialty cohorts—surgery/gynecology, medicine, cardiorespiratory, cardiovascular, and neurology—based on diagnoses and procedure codes on the claim mapped to Agency for Healthcare Research and Quality (AHRQ) Clinical Classifications Software (CCS). CCS categories that define each cohort are shown in the attached data dictionary.
Inclusion criteria:
Admissions are eligible for inclusion in the measure if the:
- Patient is 65 or older
Rationale: Younger Medicare patients represent a distinct population with dissimilar characteristics and outcomes. - Patient survives index admission
Rationale: Patients who die during the initial admission cannot be readmitted. - Patient is not transferred to another hospital
Rationale: In an episode of care in which the patient is transferred between hospitals, responsibility for the readmission is assigned to the final discharging hospital. Therefore, intermediate admissions within a single episode of care are not eligible for inclusion. - Patient is continuously enrolled in FFS Medicare Part A or B for the 12 months prior to the index admission.
Rationale: This is necessary to ensure complete data for risk adjustment and attribution.
Exclusions
From the cohort we exclude admissions for patients who were:
- Discharged against medical advice.
- Hospitalized in a prospective payment system (PPS)-exempt cancer hospital.
- Hospitalized primarily for medical treatment of cancer.
- Hospitalized primarily for a psychiatric disease.
- Hospitalized for “rehabilitation care or fitting of prostheses and adjustment devices” (CCS 254).
- Not able to be attributed to a clinician group.
- Not continuously enrolled in Medicare FFS Part A or B for at least 30 days following discharge from the index admission.
- With a principal or a secondary diagnosis code of COVID-19 coded as present on admission on the index admission claim.
From the cohort, we exclude admissions for which:
- Patients were discharged against medical advice
Rationale: Clinicians have limited opportunity to implement high quality care. - Admissions for patients is to a PPS-exempt cancer hospital
Rationale: These hospitals care for a unique population of patients that cannot reasonably be compared to the patients admitted to other hospitals. - Admissions are primarily for medical treatment of cancer
Rationale: These admissions have a very different mortality and readmission profile compared to the rest of the Medicare population (higher rates of planned readmissions and higher rates of competing mortality), and outcomes for these admissions do not correlate well with outcomes for other admissions. Patients with cancer who are admitted for other diagnoses or for surgical treatment of their cancer remain in the measure. - Admissions are primarily for psychiatric disease
Rationale: Patients admitted principally for psychiatric treatment are typically cared for in separate psychiatric centers which are not comparable to acute care hospitals. See data dictionary for excluded CCSs. - Admissions are for “rehabilitation care; fitting of prostheses and adjustment devices” (CCS 254)
Rationale: These admissions are not typically admitted to an acute care hospital for acute care. - Patients cannot be attributed to a clinician group
Rationale: Only patients assigned to eligible clinician groups should be included in the measure. - Admissions are without 30 days of Post-Discharge Enrollment
Rationale: The 30-day readmission outcome cannot be assessed in patients who do not maintain enrollment for at least 30 days following discharge. - There is a principal or a secondary diagnosis code of COVID-19 coded as present on admission on the index admission claim
Rationale: COVID-19 patients are removed from the acute myocardial infarction (AMI) cohort in response to the COVID-19 Public Health Emergency and to align with other CMS measures.
Note that a readmission within 30 days will also be eligible as an index admission if it meets all other eligibility criteria. This allows the MIPS HWR measure to capture repeated admissions for the same patient, whether with the same clinician(s) or not. Since there are few patients with multiple admissions in the same year and in the same specialty cohort, we chose to treat multiple admissions as statistically independent.
Measure Calculation
Index admissions are identified as described above in the denominator section.
Specialty Cohorts
The MIPS HWR measure uses an algorithm identical to that of the hospital level measure (CBE #1789) to group index admissions into subgroups for risk adjustment. The measure aggregates the ICD-10 principal diagnosis and all procedure codes of the index admission into clinically coherent groups of conditions and procedures (condition categories or procedure categories) using the AHRQ CCS. There are more than 200 mutually exclusive AHRQ condition categories, most of which are single, homogenous diseases such as pneumonia or AMI. Some are aggregates of conditions, such as “other bacterial infections.” There are also more than 200 mutually exclusive procedure categories. Using these AHRQ CCS procedure and condition categories, the measure assigns each index hospitalization to one of five mutually exclusive specialty cohorts: surgery/gynecology, cardiorespiratory, cardiovascular, neurology, and medicine. The rationale behind this organization is that conditions typically cared for by the same team of clinicians are expected to experience similar added (or reduced) levels of readmission risk.
Step 1. The measure first assigns admissions with qualifying AHRQ procedure categories to the Surgery/Gynecology Cohort. This cohort includes admissions likely cared for by surgical or gynecological teams.
Step 2. The measure then sorts admissions into one of the four remaining specialty cohorts based on the AHRQ diagnosis category of the principal discharge diagnosis.
The Cardiorespiratory Cohort includes several condition categories with very high readmission rates such as pneumonia, COPD, and heart failure. These admissions are combined into a single cohort because they are often clinically indistinguishable, and patients are often simultaneously treated for several of these diagnoses.
The Cardiovascular Cohort includes condition categories, such as AMI, that in large hospitals might be cared for by a separate cardiac or cardiovascular team.
The Neurology Cohort includes neurologic condition categories, such as stroke, that in large hospitals might be cared for by a separate neurology team.
The Medicine Cohort includes all non-surgical patients who were not assigned to any of the other cohorts.
The full list of the specific diagnosis and procedure AHRQ CCS categories used to define the specialty cohorts are provided in the attached data dictionary.
Risk Adjustment
Risk adjustment is done separately for each specialty cohort using a logistic regression model with 30-day readmission as the outcome. Risk adjusters in each model are identical to those used in the specialty cohorts for the claims-only hospital level measure (CBE #1789) and include the CCS for the principal diagnosis. The full list of risk adjusters can be found in the data dictionary.
Measure Score
Because the same admission may be attributed to more than one unique Eligible Clinician group, we could not apply the method used by the existing hospital-level HWR measure (CBE#1789) to construct risk standardized readmission rates. Instead, we adopted a method that, while requiring an assumption of independence across entities, allowed us to account for correlation within entity. Please see the measure score attachment for equations and details needed to calculate the measure score.
This measure is not stratified.
This measure is not based on a sample. There is no minimum sample size required to calculate the measure score.
Supplemental Attachment
Point of Contact
Not applicable
Raquel Myers
Centers for Medicare & Medicaid Services (CMS)
7500 Security Boulevard
Windsor Mill, MD 21244
United States
Amy Moyer
Yale New Haven Health Services Corporation/Center for Outcomes Research and Evaluation
195 Church Street (Fifth Floor)
New Haven , CT 06510
United States
Importance
Evidence
Studies have shown readmissions to be related to quality of care, and that interventions have been able to reduce 30-day readmission rates. It is reasonable to consider an all-condition readmission rate as a quality measure (Desai et al. 2016; Wasfy et al. 2017; Zuckerman et al. 2016; Boccuti et al. 2017). The variation in readmission rates across hospitals indicates room for quality improvement. Targeted efforts to reduce these readmissions can result in better patient care and potential cost savings.
In general, randomized controlled trials have shown that improvement in the following areas can directly reduce readmission rates: quality of care during the initial admission; improvement in communication with patients, their caregivers and their clinicians; patient education; pre-discharge assessment; more support for patient self-management; improved disease monitoring processes; and coordination of care after discharge (Auerbach et al. 2016; Ravn-Nielsen et al., 2018; Becker et al., 2021). Evidence that hospitals have been able to reduce readmission rates through these quality-of-care initiatives illustrates the degree to which hospital practices can affect readmission rates. Successful randomized trials have reduced 30-day readmission rates by 20-40% (Feltner et al., 2014; Kamermayer et al., 2017; Wasfy et al., 2017; McWilliams et al., 2019; De Oliveira et al., 2021). A randomized controlled trial (RCT) by the Project RED (Re-Engineered Discharge), implemented an intervention in which a nurse was assigned to each patient as a discharge advocate, responsible for patient education, follow-up, medication reconciliation, and preparing individualized discharge instructions sent to the patient’s primary care provider. Additionally, there was a follow-up phone call from a pharmacist within four days of discharge. This RCT demonstrated a 30 percent reduction in 30-day readmissions (Patel et al., 2018).
As noted, many of the strategies and best practices used by hospitals to reduce risk of readmissions can also be adopted by groups of clinicians, which clinician groups can influence during the inpatient stay and during care transition (Adams et al. 2014; Sanchez et al. 2015). Importantly, a recent study found that timely follow up after a hospitalization is associated with readmission rates and for some conditions and patient populations accounts for most of the risk of readmission (Anderson et al., 2022). The MIPS HWR measure allows joint attribution for three clinician groups that provide care for patients inside and outside of the hospital prior to discharge and therefore are in positions to influence patients’ risk of readmission. In conclusion, the literature supports those interventions, especially those utilizing clinicians, have been able to reduce 30-day readmission rates for a variety of specific conditions; and the observed high and variable clinician-level readmission rates.
References
Adams, C. J., et al. (2014). Implementation of the Re-Engineered Discharge (RED) toolkit to decrease all-cause readmission rates at a rural community hospital. Qual Manag Health Care 23(3): 169-177.
Anderson, A., Mills, C. W., Willits, J., Lisk, C., Maksut, J. L., Khau, M. T., & Scholle, S. H. (2022). Follow-up Post-discharge and Readmission Disparities Among Medicare Fee-for-Service Beneficiaries, 2018. Journal of general internal medicine, 37(12), 3020–3028. https://doi.org/10.1007/s11606-022-07488-3
Auerbach, A.D., S. Kripalani, E.E. Vasilevskis, N. Sehgal, P.K. Lindenauer, J.P. Metlay, G. Fletcher, G.W. Ruhnke, S.A. Flanders, C. Kim, M.V. Williams, L. Thomas, V. Giang, S.J. Herzig, K. Patel, W.J. Boscardin, E.J. Robinson, and J.L. Schnipper. Preventability and Causes of Readmissions in a National Cohort of General Medicine Patients. JAMA Intern Med, 2016. 176(4): 484-93.
Becker, C., Zumbrunn, S., Beck, K., Vincent, A., Loretz, N., Müller, J., ... & Hunziker, S. (2021). Interventions to improve communication at hospital discharge and rates of readmission: a systematic review and meta-analysis. JAMA Network Open, 4(8), e2119346-e2119346.
Boccuti C, Casillas G. Aiming for Fewer Hospital U-turns: The Medicare Hospital Readmission Reduction Program. 2017; https://www.kff.org/medicare/issue-brief/aiming-for-fewer-hospital-u-turns-the-medicare-hospital-readmission-reduction-program/. Accessed December 18, 2017.
Bowles KH, Hanlon A, Holland D, Potashnik SL, Topaz M. Impact of discharge planning decision support on time to readmission among older adult medical patients. Prof Case Manag. 2014; 19(1): 29–38. doi:10.1097/01.PCAMA.0000438971.79801.7a
De Oliveira G, Castro-Alves L, Kendall M, McCarthy R. Effectiveness of Pharmacist Intervention to Reduce Medication Errors and Health-Care Resources Utilization After Transitions of Care: A Meta-analysis of Randomized Controlled Trials. Journal of Patient Safety. 2021; 17 (5): 375-380. doi: 10.1097/PTS.0000000000000283.
Desai NR, Ross JS, Kwon JY, et al. Association Between Hospital Penalty Status Under the Hospital Readmission Reduction Program and Readmission Rates for Target and Non-Target Conditions. JAMA. 2016; 316(24): 2647-2656.
Feltner, C., Jones, C. D., Cené, C. W., Zheng, Z. J., Sueta, C. A., Coker-Schwimmer, E. J., ... & Jonas, D. E. (2014). Transitional care interventions to prevent readmissions for persons with heart failure: a systematic review and meta-analysis. Annals of internal medicine, 160(11), 774-784.
Kamermayer A, Leasure A, Anderson L. The Effectiveness of Transitions-of-Care Interventions in Reducing Hospital Readmissions and Mortality. Dimensions of Critical Care Nursing. 2017; 36 (6): 311-316. doi: 10.1097/DCC.0000000000000266.
Kripalani, S., Theobald, C. N., Anctil, B., & Vasilevskis, E. E. (2014). Reducing hospital readmission rates: current strategies and future directions. Annual review of medicine, 65, 471-485.
Leppin AL, Gionfriddo MR, Kessler M, et al. Preventing 30-day hospital readmissions: a systematic review and meta-analysis of randomized trials. JAMA Intern Med. 2014;174(7): 1095-1107.
McWilliams, A., Roberge, J., Anderson, W. E., Moore, C. G., Rossman, W., Murphy, S., ... & Furney, S. (2019). Aiming to improve readmissions through Integrated hospital transitions (AIRTIGHT): a pragmatic randomized controlled trial. Journal of General Internal Medicine, 34, 58-64.
Ravn-Nielsen, L. V., Duckert, M. L., Lund, M. L., Henriksen, J. P., Nielsen, M. L., Eriksen, C. S., & Hallas, J. (2018). Effect of an in-hospital multifaceted clinical pharmacist intervention on the risk of readmission: a randomized clinical trial. JAMA internal medicine, 178(3), 375-382.
Patel PH, Dickerson KW. Impact of the Implementation of Project Re-Engineered Discharge for Heart Failure patients at a Veterans Affairs Hospital at the Central Arkansas Veterans Healthcare System. Hosp Pharm. 2018;53(4):266‐271. doi:10.1177/0018578717749925.
Sanchez, G. M., et al. (2015). Revisiting Project Re-Engineered Discharge (RED): The Impact of a Pharmacist Telephone Intervention on Hospital Readmission Rates. Pharmacotherapy 35(9): 805-812.
Wasfy JH, Zigler C, Choirat C, Wang Y, Dominici F, Yeh RW. Readmission rates after passage of the hospital readmissions reduction program: A pre–post analysis. Annals of Internal Medicine. 2017; 166(5): 324-331.
Zuckerman RB, Sheingold SH, Orav EJ, Ruhter J, Epstein AM. Readmissions, Observation, and the Hospital Readmissions Reduction Program. New England Journal of Medicine. 2016; 374(16): 1543-1551.
Measure Impact
Hospital readmission, for any reason, is disruptive to patients and caregivers, costly to the healthcare system and policy holders, and puts patients at additional risk of hospital-acquired infections and complications. Readmissions are also a major source of patient and family stress and may contribute substantially to loss of functional ability and independence, particularly in older patients. We interviewed patients and caregivers for a Technical Expert Panel (TEP) related to readmissions; patients and caregivers shared their stories of frustration, confusion, and suffering, as they or their loved ones faced unexpected returns to the hospital after discharge. In our interviews they cited experiences such as return to the hospital following exacerbation of a condition caused by changes in medication after discharge, returns to the hospital due to infection after an inpatient procedure, and other signs of poor coordination of care including insufficient communication from providers and hospital staff. In addition, a recent study that examined perspectives of patients on readmission compared patients’ perspectives on the causes and preventability of and underlying reasons for a readmission with those of registered nurses (RNs) and physicians (Smeraglio et. al., 2019). Interestingly, the study found that compared with physicians, patients were more likely to identify a readmission as preventable, and patients were more likely than physicians to identify system issues as an underlying reason for their readmission (58% of cases vs 2%, respectively). Furthermore, RNs and patients had similar assessments as to the preventability of their readmission.
References
Smeraglio A, Heidenreich PA, Krishnan G, Hopkins J, Chen J, Shieh L. Patient vs provider perspectives of 30-day hospital readmissions. BMJ Open Qual. 2019 Jan 7;8(1):e000264. doi: 10.1136/bmjoq-2017-000264. PMID: 30687798; PMCID: PMC6327873.
Performance Gap
We conducted analyses using a full year of Medicare data from January 1, 2021 through November 30, 2021 (n=4,698,018 patients) included in the final index cohort. We report the RARRs for eligible clinician groups with at least one attributed index admission.
The data show evidence of performance variation across the 97,027 clinician groups. Performance scores range from 8.98% to 26.80% with a median of 15.7%. The distribution shows meaningful variation: the worst performing clinician group (RARR of 26.8%) is performing about 77 percent worse than the median (15.17%), and the best performing clinician group (8.98%) is performing about 41 percent better than the median (see Table 1 in the attachment).
| Overall | Minimum | Decile_1 | Decile_2 | Decile_3 | Decile_4 | Decile_5 | Decile_6 | Decile_7 | Decile_8 | Decile_9 | Decile_10 | Maximum | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Mean Performance Score | 15.32% | 8.98% | 13.62% | 14.56% | 14.87% | 15.03% | 15.13% | 15.21% | 15.40% | 15.77% | 16.24% | 17.41% | 26.80% |
| N of Entities | 97,027 | 9,702 | 9,703 | 9,703 | 9,703 | 9,702 | 9,703 | 9,703 | 9,703 | 9,703 | 9,702 | ||
| N of Persons / Encounters / Episodes | 7,827,902 | 1,743,816 | 913,352 | 454,325 | 259,144 | 200,014 | 227,763 | 936,480 | 780,991 | 1,123,538 | 1,188,479 |
Equity
Equity
At the patient level, we know that patients with social risk factors (present in our conceptual model) may have higher unadjusted outcomes, but differences vary depending on the social risk factor. For example, using data from Jan 1, 2021-Nov 30, 2021, we found that across the entire cohort, patients with dual eligibility (DE) have an unadjusted hospital-wide readmission rate of 19.88%, compared with 14.74% for patients without DE; the difference is less marked for the high ADI variable (17.05% vs 15.3%) (see attachment, Table 13). At the cohort level we also see differences; for the medicine cohort, patients with DE have an unadjusted hospital-wide readmission rate of 20.81%, compared with 16.97% for patients without DE. In contrast, however, for patients with high ADI unadjusted rates are closer to rates for patients without high ADI (18.93% vs 17.48%, respectively). See Table 13 in the attachment for results for each specialty cohort.
To examine the impact of social risk factors on measure scores, we first examined correlations (Pearsons) between measure scores calculated with and without either social risk factor and found that correlations were near 1 (0.994, p<.0001 and 0.981, p<.0001, for DE and ADI, respectively). Second, to determine the impact on clinician groups with the highest proportion of patients with social risk, we examined the association between the clinician-group proportion of patients with each social risk factor and measure scores, focusing on the quartile with the highest proportion of patients with social risk factors (Figures 9 and 10, in attachment). We found that there is a very weak but significant correlation (r=0.018, p=.005) between the clinician-group proportion of patients with DE and the measure score for the top (fourth) quartile, and a similarly weak but significant correlation for patients in the top (fourth) quartile for the high ADI variable (r=0.036, p<.0001).
Our results discussed above show that measure scores calculated with and without social risk factors are highly correlated, and that there is only a very weak association between measure scores and the clinician-group proportion of patients with social risk. We concluded therefore, that there is minimal impact of social risk factors on measure scores, supporting the decision to not include social risk factor variables in the risk model during measure development. While these measures are not adjusted for either social risk factor, CMS allows MIPS-eligible providers to obtain “MIPS complex bonus points” on their payment scores that are based on the medical complexity and proportion of patients with dual eligibility (https://qpp-cm-prod-content.s3.amazonaws.com/uploads/2309/2022%20Complex%20Patient%20Bonus%20Fact%20Sheet.pdf)
Feasibility
Feasibility
This is a claims-based measure, and the measure score is calculated automatically from claims data which are routinely generated during the delivery of care. No data are collected by facilities; therefore, this measure imposes no burden on measured entities, and no implementation effort. CMS monitors feedback from the public and measured entities; there have been no concerns about burden related to this measure. There are no concerns about patient confidentiality because the measure is based on claims data submitted by facilities to CMS, and CMS then uses that data for both payment and calculation of the measure score.
We did not perform an analysis of missing data for the measure because it is based on a 100% sample of paid, final-action claims submitted by facilities for payment. To ensure complete claims, we allow at least three months of time between accessing the data and the end of the performance period.
Because this is a claims-based measure, there is no burden on the clinician group; measure scores are automatically calculated by CMS based on claims data submitted by clinician groups for payment.
Proprietary Information
Scientific Acceptability
Testing Data
Testing was performed using final-action Medicare Part A and B claims data for Medicare FFS beneficiaries, as well as the Medicare Enrollment Database (EDB) for demographic information.
The dates of data used for these analyses are January 1, 2021 – November 30, 2021 for the cohort performance period; data through December 31, 2021 are used to identify the outcome.
The datasets, dates, number of measured entities, and number of admissions used in each type of testing are as follows:
1. Measure Development and Initial Testing
For measure development and testing, we used two years (July 2015 – June 2017) of Medicare administrative claims data. The dataset contained Medicare inpatient and outpatient claims data and Medicare enrollment information. We randomly split the first year of data into two equal samples: Dataset A1 (July 2015 – June 2016) was used as the “development sample” and Dataset A2 (July 2015 – June 2016) was used as the “validation sample.” We used the second year of the data in addition to an extra year of data, Dataset B, as a temporal validation sample set (July 2016 – June 2017).
Dataset A1 (Development sample; July 2015 – June 2016):
Number of admissions = 3,234,836
Number of clinician groups = 117,788
Patient Descriptive Characteristics: mean age = 78.21 years; % female = 56.29
Dataset A2 (Validation sample; July 2015 – June 2016):
Number of admissions = 3,233,925
Number of clinician groups = 117,693
Patient Descriptive Characteristics: mean age = 78.21 years; % female = 56.33
Dataset B (Temporal validation sample; July 2016 – June 2017):
Number of admissions = 6,411,508
Number of clinician groups = 124,311
Patient Descriptive Characteristics: mean age = 78.19 years; % female = 56.05
For initial model testing
The measure development and validation samples were used to assess statistical model testing.
Datasets A1, A2, and Dataset B were used for patient level model testing.
For testing of measure risk adjustment
We used two years of data from July 2015 – June 2017 (Dataset A1/A2/B Combined).
2. Endorsement maintenance testing
The dataset below was used for updated testing for endorsement maintenance, including reliability and validity testing.
2024 Endorsement Maintenance Testing Dataset:
Dates of data: January 1, 2021 – November 30, 2021
Number of admissions = 4,698,018
Number of admissions in each specialty cohort:
Cardiorespiratory = 406,819
Cardiovascular = 488,965
Medicine = 2,345,332
Neurology = 328,388
Surgery/Gynecology = 1,128,514
Number of clinician groups = 97,027
Patient Descriptive Characteristics: mean age = 78.16 years; % female = 54.51
Eligible Clinician Groups participating in MIPS are the measured entities; Medicare FFS beneficiaries aged 65 or older are included. Eligible Clinician Groups are identified by aggregating National Provider Identifier (NPI)-TIN pairs with a common TIN. The number of measured entities (Eligible Clinician Groups) varies by testing type (see prior question, 4.1.2, for details).
The number of admissions/patients varies by testing type (see 4.1.2 for details). Testing includes all beneficiaries that meet inclusion/exclusion criteria described previously, attributed to the Eligible Clinician Group during the performance period.
Reliability
Measure Score Reliability: Clinician Group-Level Reliability
We estimated the clinician group signal-to-noise reliability for each of the five specialty cohorts that are combined to produce the overall measure result. This is “unit” reliability, that is, the reliability with which individual units (here, clinician groups) are measured. Because signal-to-noise reliability is based on model parameters, it is only meaningful to calculate it at the level of the specialty cohort; however, according to Rudner, the reliability of an aggregated scale is bounded below by the reliability of the least reliable component and will generally be greater than the most reliable component of the component scales are positively correlated. For each cohort, we use the formula presented by Adams et al, to calculate provider-level signal-to-noise reliability.
For each measured entity (eligible clinician group) we calculated the ratio of t2/(t2 + s2), using the value t2 (defined in the measure score calculation section) and estimating s2 using on bootstrapping. We summarized the distribution of these values for each of the five specialty cohorts for all clinician groups (with at least one attributed patient) and for clinician groups with at least 200 attributed patients (the public reporting threshold).
We note that we did not calculate split-sample reliability due to the number of clinician groups and the difficulty in obtaining the two similar samples that are required for the analysis to provide meaningful results.
References
Adams J, Mehrota, A, Thoman J, McGlynn, E. (2010). Physician cost profiling – reliability and risk of misclassification. NEJM, 362(11): 1014-1021.
Rudner, L (2001) ‘Informed test component weighting’, Educational Measurement: Issues and Practice, 20, pp. 16-19
Please see Table 3 in the attached document that describes the cohort-level reliability results. The results are also listed here for clinician groups with at least 200 patients.
Cohort: Mean (Minimum-Maximum)
Cardiorespiratory: 0.90 (0.82-0.98)
Cardiovascular: 0.88 (0.79-0.99)
Medicine: 0.86 (0.66-1.00)
Neurology: 0.91 (0.85-0.99)
Surgical: 0.84 (0.49-0.99)
Because the MIPS HWR measure has five specialty cohorts we were not able to use the Battelle-supplied table shown below. Please see Table 3 in the main attachment for the distribution of signal-to-noise reliability at the cohort level for each of the five specialty cohorts
Updated signal to noise reliability testing suggests that a minimum number of attributed patients of 200 provides acceptable reliability based on Batelle’s endorsement standards. Reliability is slightly lower for the Surgery cohort however since the specialty cohorts are combined to create the MIPS HWR measure score and because the other specialty cohorts all have a minimum reliability of >0.6, it is unlikely that the one specialty cohort would unduly impact the overall reliability of the measure.
Validity
Measure validity is demonstrated through empirical validity testing, and by systematic assessment of measure face validity via a TEP of national experts and stakeholder organizations.
Empirical Validity Testing
To provide additional validation of the measure, we assessed how well it correlated with hospital quality. Since the outcome depends on hospital processes, including coordination of care during the stay and during transition from the hospital, we anticipate the readmission rate for a provider would be consistent with the quality of the hospital where most of their attributed patients are discharged. As measures of hospital quality, we used the CMS Hospital Overall Star Ratings and Hospital Star Ratings readmission group scores. To assess consistency, we plotted the distribution of the measure score, Risk Adjusted Readmission Rate (RARR) over: a) Overall Star Rating (1-5) and b) quintiles of the Star Rating Readmission domain score. We hypothesized a weak-to-moderate negative correlation; better performance on the RARR (lower scores) is hypothesized to be associated with a greater number of stars (better performance) and associated with better Star Rating Standardized Readmission group scores. When the hospital-level HWR measure is removed from the Readmission group score, we expect to still see a weak but significant positive association. We expect a very weak, negative association with the Hospital Star Rating category, as the Hospital Overall Star Ratings is (in addition to readmission measures) comprised of many hospital-based measures (including mortality, hospital-acquired infections, and timely and effective care) that may not be attributable to (in the same causal pathway) as care provided by clinician groups captured by the MIPS HWR measure.
Face Validity
We convened a TEP to provide input and feedback during measure development from a group of recognized experts in relevant fields. To convene the TEP, we released a public call for nominations and selected individuals to represent a range of perspectives, including clinicians, patients, and individuals with expertise in quality improvement and performance measurement. We held three structured TEP conference calls consisting of a presentation of key issues, our proposed approach, and relevant data, followed by an open discussion among TEP members. We made modifications to the measure attribution based on TEP feedback on the measure.
TEP members were asked to assess the face validity of the final measure specification by confidentially responding to two questions:
The risk-standardized readmission rates obtained from the MIPS HWR measure as specified:
- Are valid and useful measures of MIPS Eligible Clinicians and MIPS Eligible Clinician Group quality of care.
- Will provide MIPS Eligible Clinicians and MIPS Eligible Clinician Groups with information that can be used to improve their quality of care.
TEP members were asked to report their agreement with each statement on a 6-point scale, representing a range from “strongly disagree” to “strongly agree.”
Empiric Validity
As hypothesized, the MIPS HWR measure was weakly and negatively associated with both the Star Rating category (-0.093, p<.0001) and the readmission group score (-0.144, p<.0001); as expected, the relationship became weaker when the hospital-level HWR measure was removed from the readmission group score (-0.119, p<.0001) (see Table 5 in the attachment). Box plots of these relationships show a consistent linear trend with regard to the relationship, with declining median MIPS HWR measure scores (better performance) with each increasing Star Rating category (1-5) (better performance), and similarly with quintiles of increasing Readmission Group Scores (see Figures 2 and 3, in the attachment). In addition, we know that at the hospital level, the HWR measure score is associated with the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) Care Transition, and HCAHPS Discharge Information performance scores (r=-0.324, p<.0001; r=-0.324, p<.0001, respectively) in the expected strength and direction.
Face Validity as Determined by a TEP
Validity was assessed by the TEP. Of 19 TEP members asked to complete a survey regarding validity and usability of the measure, 17 responded.
The majority of respondents, 12/17 or 70 percent, agreed that the MIPS HWR measure scores were valid and useful, and the same proportion agreed that the measure would provide information that could be used to improve the quality of care.
The TEP supported attribution to multiple clinician groups. The TEP considered the fact that readmission is a multi-factorial outcome and that multiple clinicians may play important roles in providing appropriate care, making good decisions and practical recommendations, and intentionally planning care transitions. These actions, while only part of the overall success of a care transition, are necessary to promote the best possible outcome. In this context, most TEP members agreed that patient outcomes should not be attributed to a single clinician but should rather be attributed to multiple clinicians responsible for the patient during and after the inpatient stay. Some TEP members noted the multiple attribution algorithm addressed the current reality of patient care, as opposed to being optimistic in how care should be coordinated. Some TEP members also felt that clinicians in the hospital were best able to drive system changes. TEP members were also supportive of attributing to the outpatient provider as they felt it was in part the responsibility of the outpatient provider to ensure the patient does not return to the hospital with remnant issues from their inpatient stay.
Among those who disagreed, the primary concern was that factors which led to increased risk of readmission were beyond the control of any single eligible clinician or clinician group. This concern drove the adoption of ‘multiple’ attribution, in which no single eligible clinician is solely responsible for a readmission outcome; this attribution approach also has the potential to incentivize collaboration within the hospital and across the care system, further aligning the measure with the attribution.
Empirical Validity Testing
The results of the external validity testing confirm our hypothesis that MIPS HWR measure scores were associated, in the expected strength and direction, with the comparator measures (Star Rating category, Readmission Group Scores). These results provide meaningful external validation of the current measure.
Validity as Assessed by External Groups and by TEP
The results demonstrate TEP agreement with overall face validity of the measure as specified, and the main concern that the TEP voiced in their feedback (multiple attribution) was addressed after the face validity vote. However, the attribution rules adopted for this measure were subsequently reviewed and supported by the TEP members as well as several clinical expert workgroup members.
Overall, the survey indicates support of the validity and usability of the measure.
Risk Adjustment
To harmonize with the existing hospital-level HWR measure (CBE #1789) the same claims-based risk factors were adopted for this clinician-group measure. As described below, we first considered adjustment for clinical conditions and then examined additional risk imparted by social risk factors after the potential for greater disease burden is included in the risk model. We believe that this is consistent with the evidence that people who experience greater social risk are more likely to have more disease burden compared with those who do not. We describe both approaches below. Please see Section 1.18 for equations used to derive the measure score that include risk adjustment.
Approach to Variable Selection:
To harmonize with the existing hospital level measure (CBE #1789) the same risk factors were adopted. These were originally selected from a pool comprised of 30 variables drawn from previous readmission measures (acute myocardial infarction, heart failure, pneumonia, hip and knee arthroplasty, and stroke) and 11 additional CMS-CCs that were determined on a clinical basis to be relevant to an all-condition measure. Using data from the index admission and any admission in the prior 12 months, standard logistic regression models were run for every discharge condition category with the full set of candidate risk adjustment variables. The final set of comorbid risk variables were selected for the hospital-level HWR measure (CBE #1789) based on the following principles:
- Exclude risk variables that were statistically significant for very few condition categories, given that they would not contribute much to the overall models.
- Exclude risk variables that behaved in clinically incoherent ways.
- Exclude risk variables that were predominantly protective when it was felt this protective effect was not clinically reasonable but more likely reflected coding factors.
- Risk variables that were clinically coherent and carried similar risks across condition categories were grouped. For example, we combined coronary artery disease (CCs 83-84) with cerebrovascular disease (CCs 98, 99, and 103).
- Risk variables that had been combined in previous CMS publicly reported measures were examined, and in one instance separated.
Complications occurring during hospitalization are not comorbid illnesses, may reflect hospital quality of care, and therefore should not be used for risk adjustment. Hence, conditions that may represent adverse outcomes due to care received during the index hospital stay are not included in the risk-adjusted model.
Service Mix Adjustment
The MIPS HWR measure includes many different discharge condition categories that differ in their baseline readmission risks. In addition, hospitals differ in their relative distribution of these condition categories (service mix). To adjust for service mix, the measure uses an indicator variable for the discharge condition category in addition to risk variables for comorbid conditions. The models include a condition-specific indicator for all condition categories with sufficient volume (defined as those with more than 1,000 admissions nationally in a given year for Medicare FFS data) as well as a single indicator for conditions with insufficient volume in each model.
Social Risk Factors
Social risk factors that we identified for examination were based on a review of literature, conceptual pathways, and feasibility. Below we describe the conceptual pathways by which social risk factors may influence 30-day readmission.
Studies indicate that social risk variables can be associated with increased risk of readmission across multiple major illnesses and conditions (Joynt-Maddox et al., 2019, Aseltine RH, et al., 2015; Odonkor CA, et al., 2015; Herrin J, et al., 2015; Gu Q, et al., 2014, Kangovi S, et al., 2012; Iloabuchi TC, 2014; Beck AF, et al., 2012; Hu J, 2014; Nagasako EM, et al., 2014; Joynt, KE, et al., 2013). While some studies found there may not be a significant effect on hospital-level profiling (Blum AB, et al., 2014), others have found that risk adjustment for social risk factors has an impact, in particular for safety-net hospitals (Joynt-Maddox et al., 2019). However, others have found that social risk factors may be capturing clinical complexity (Cher, Bay, Hoffman, & Sheetz, 2020, Pandey et al., 2020, Sterling et al., 2022) or hospital-level characteristics (Brown et al., 2015), or that differences in hospital outcomes by race and neighborhood income may be systemic, rather than localized within particular hospitals (Downing et al., 2018).
Although some recent literature evaluates the relationship between patient social risk and the readmission outcome, few studies directly address causal pathways or examine the role of the hospital in these pathways. Moreover, the current literature examines a wide range of conditions and risk variables with no clear consensus on which risk factors demonstrate the strongest relationship with readmission. The social risk factors that have been examined in the readmission literature can be categorized into three domains: (1) patient-level variables, (2) neighborhood/community-level variables, and (3) hospital-level variables.
Patient-level variables describe characteristics of individual patients and include the patient’s income or education level (Imran, Rawal, Botre, & Patil, 2022). Neighborhood/community-level variables use information from sources such as the American Community Survey (ACS) as either a proxy for individual patient-level data or to measure environmental factors. Studies using these variables use one dimensional measures such as median household income or composite measures such as the Area Deprivation Index (ADI) (Kind & Buckingham, 2018; University of Wisconsin School of Medicine and Public Health, 2023; Powell, Sheehy, & Kind, 2023; Singh, 2003). Some of these variables may include the local availability of clinical providers (Herrin et al., 2015, 2016). Hospital-level variables measure attributes of the hospital which may be related to patient risk. Examples of hospital-level variables used in studies are ZIP code characteristics aggregated to the hospital level or the proportion of Medicaid patients served in the hospital (Gilman M, et al., 2014; Joynt KE and Jha AK, 2013).
The conceptual relationship, or potential causal pathways by which these possible social risk factors influence the risk of readmission following an acute illness or major surgery, like the factors themselves, are varied and complex. There are at least four potential pathways that are important to consider. While the MIPS HWR measure attributes readmissions to clinician groups, it still captures quality care provided by clinicians within the hospital system. Thus, the conceptual framework used to understand how social risk factors play a role in how hospitals influence readmissions can also be used to understand how social risk factors play a role in how clinician groups at the hospital can influence readmissions.
- Patients with social risk factors may have worse health at the time of hospital admission. Patients who have lower income/education/literacy or unstable housing may have a worse general health status and may present for their hospitalization or procedure with a greater severity of underlying illness (Owens et al., 2022). These social risk factors, which are characterized by patient-level or neighborhood/community-level (as proxy for patient-level) variables, may contribute to worse health status at admission due to competing priorities (restrictions based on job, lack of childcare, etc.), lack of access to care (geographic, cultural, or financial), or lack of health insurance. Given that these risk factors all lead to worse general health status, this causal pathway should be largely accounted for by current clinical risk-adjustment.
- Patients with social risk factors may receive care at lower quality hospitals. Patients of lower income, lower education, or unstable housing may have inequitable access to high quality facilities, in part, because such facilities are less likely to be found in geographic areas with large populations of impecunious patients (Nwana et al., 2022; Fahrenbach et al., 2020). Thus, patients with low income may be more likely to be seen in lower quality hospitals, which can contribute to increased risk of readmission following hospitalization.
- Patients with social risk factors may receive differential care within a hospital. The third major pathway by which social risk factors may contribute to readmission risk is that patients may not receive equivalent care within a facility (Downing et al., 2018). Alternatively, patients with social risk factors such as lower education may require differentiated care – e.g., provision of lower literacy information – that they do not receive.
- Patients with social risk factors may experience worse health outcomes beyond the control of the health care system. Some social risk factors, such as income or wealth, may affect the likelihood of readmission without directly affecting health status at admission or the quality of care received during the hospital stay. For instance, while a hospital may make appropriate care decisions and provide tailored care and education, a lower-income patient may have a worse outcome post-discharge due to competing economic priorities or a lack of access to care outside of the hospital. In addition, there is evidence that differences in hospital-level readmission rates based on patients’ race and neighborhood income may be systemic, rather than localized within particular hospitals (Downing et al., 2018).
These proposed pathways are complex to distinguish analytically. They also have different implications on the decision to risk adjust or not. We, therefore, first assessed if there was evidence of a meaningful effect on the risk model to warrant efforts to distinguish among these pathways. Based on this model, the following social risk variables were considered:
• Dual-eligible Status
Dual eligibility for Medicare and Medicaid is available at the patient level in the Medicare Master Beneficiary Summary File. The eligibility threshold for aged 65 or older Medicare patients considers both income and assets. For the dual-eligible (DE) indicator, there is a body of literature demonstrating differential health care and health outcomes among beneficiaries (ASPE, 2020).
• Area Deprivation Index
While we previously have used the AHRQ social risk variable in these types of analyses, we now use the validated ADI (Forefront Group, 2023). We made this change to align with other CMS work on social risk factors that now uses the ADI. We describe the ADI variable below.
The ADI, initially developed by Health Resources & Services Administration, is based on 17 measures across four domains: income, education, employment, and housing quality (Kind et al., 2018; Singh, 2003).
The 17 components are listed below:
- Population aged ≥ 25 y with < 9 y of education, %
- Population aged ≥ 25 y with at least a high school diploma, %
- Employed persons aged ≥ 16 y in white-collar occupations, %
- Median family income, $
- Income disparity
- Median home value, $
- Median gross rent, $
- Median monthly mortgage, $
- Owner-occupied housing units, % (home ownership rate)
- Civilian labor force population aged ≥16 y unemployed, % (unemployment rate)
- Families below poverty level, %
- Population below 150% of the poverty threshold, %
- Single-parent households with children aged < 18 y, %
- Households without a motor vehicle, %
- Households without a telephone, %
- Occupied housing units without complete plumbing, % (log)
- Households with more than 1 person per room, % (crowding)
ADI scores were derived using the beneficiary’s 9-digit ZIP Code of residence, which is obtained from the Master Beneficiary Summary File, and is linked to 2017-2021 US Census/ACS data. In accordance with the ADI developers’ methodology, an ADI score is calculated for the census block group corresponding to the beneficiary’s 9-digit ZIP Code using 17 weighted Census indicators. Raw ADI scores were then transformed into a national percentile ranking ranging from 1 to 100, with lower scores indicating lower levels of disadvantage and higher scores indicating higher levels of disadvantage. Percentile thresholds established by the ADI developers were then applied to ADI percentile to dichotomize neighborhoods into more disadvantaged (high ADI areas=ranking equal to or greater than 85) or less disadvantaged areas (low ADI areas=ranking of less than 85).
References
Aseltine RH, Jr., Yan J, Gruss CB, Wagner C, Katz M. Connecticut Hospital Readmissions Related to Chest Pain and Heart Failure: Differences by Race, Ethnicity, and Payer. Connecticut medicine. 2015;79(2):69-76.
Beck AF, Simmons JM, Huang B, Kahn RS. Geomedicine: area-based socioeconomic measures for assessing risk of hospital reutilization among children admitted for asthma. American journal of public health. 2012;102(12):2308-2314.
Blum AB, NN Egorova, E. A. Sosunov, A. C. Gelijns, E. DuPree, A. J. Moskowitz, A. D. Federman, D. D. Ascheim and S. Keyhani. "Impact of Socioeconomic Status Measures on Hospital Profiling in New York City." Circ Cardiovasc Qual Outcomes 7, no. 3 (2014): 391-7.
Brown, J. R., Chang, C. H., Zhou, W., MacKenzie, T. A., Malenka, D. J., & Goodman, D. C. (2014). Health system characteristics and rates of readmission after acute myocardial infarction in the United States. Journal of the American Heart Association, 3(3), e000714. https://doi.org/10.1161/JAHA.113.000714
Cher, B., Bay, R., Hoffman, G. J., & Sheetz, K. H. (2020). Association of Medicaid eligibility with surgical readmission among Medicare beneficiaries. JAMA Network Open, 3(6), Article e207426. https://doi.org/10.1001/jamanetworkopen.2020.7426
Department of Health and Human Services, Office of the Assistant Secretary of Planning and Evaluation (ASPE). Second Report to Congress: Social Risk Factors and Performance in Medicare’s Value-based Purchasing Programs. 2020; https://aspe.hhs.gov/system/files/pdf/263676/Social-Risk-in-Medicare%E2%80%99s-VBP-2nd-Report.pdf.
Downing, N. S., Wang, C., Gupta, A., Wang, Y., Nuti, S. V., Ross, J. S., Bernheim, S. M., Lin, Z., Normand, S. T., & Krumholz, H. M. (2018). Association of Racial and Socioeconomic Disparities With Outcomes Among Patients Hospitalized With Acute Myocardial Infarction, Heart Failure, and Pneumonia: An Analysis of Within- and Between-Hospital Variation. JAMA network open, 1(5), e182044. https://doi.org/10.1001/jamanetworkopen.2018.2044
Eapen ZJ, McCoy LA, Fonarow GC, Yancy CW, Miranda ML, Peterson ED, Califf RM, Hernandez AF. Utility of socioeconomic status in predicting 30-day outcomes after heart failure hospitalization. Circ Heart Fail. May 2015; 8(3):473-80.
Fahrenbach, J., Chin, M. H., Huang, E. S., Springman, M. K., Weber, S. G., & Tung, E. L. (2020). Neighborhood Disadvantage and Hospital Quality Ratings in the Medicare Hospital Compare Program. Medical care, 58(4), 376–383. https://doi.org/10.1097/MLR.0000000000001283
Gilman M, Adams EK, Hockenberry JM, Wilson IB, Milstein AS, Becker ER. California safety-net hospitals likely to be penalized by ACA value, readmission, and meaningful-use programs. Health Aff (Millwood). Aug 2014; 33(8):1314-22.
Gu Q, Koenig L, Faerberg J, Steinberg CR, Vaz C, Wheatley MP. The Medicare Hospital Readmissions Reduction Program: potential unintended consequences for hospitals serving vulnerable populations. Health services research. 2014;49(3):818-837.
Hardy RJ, Thompson SG. A Likelihood Approach to Meta-analysis with random effects. Statistics in medicine. 1996;15:619-629.
Herrin, J., Kenward, K., Joshi, M. S., Audet, A. M., & Hines, S. J. (2016). Assessing community quality of health care. Health Services Research, 51(1), 98-116. https://doi.org/10.1111/1475-6773.12322
Herrin, J., St Andre, J., Kenward, K., Joshi, M. S., Audet, A. M., & Hines, S. C. (2015). Community factors and hospital readmission rates. Health Services Research, 50(1), 20-39. https://doi.org/10.1111/1475-6773.12177
Horwitz L, Partovian C, Lin Z, et al. Hospital-Wide All-Cause Unplanned Readmission Measure: Final Technical Report. 2012; https://www.qualitynet.org/dcs/ContentServer?c=Page&pagename=QnetPublic%2FPage%2FQnetTier4&cid=1228772504318
Hu J, Gonsahn MD, Nerenz DR. Socioeconomic status and readmissions: evidence from an urban teaching hospital. Health affairs (Project Hope). 2014; 33(5):778-785.
Iloabuchi TC, Mi D, Tu W, Counsell SR. Risk factors for early hospital readmission in low-income elderly adults. Journal of the American Geriatrics Society. 2014;62(3):489-494.
Imran, A., Rawal, M. D., Botre, N., & Patil, A. (2022). Improving and promoting social determinants of health at a system level. The Joint Commission Journal on Quality and Patient Safety, 48(8), 376-384.
Jha AK, Orav EJ, Epstein AM. Low-quality, high-cost hospitals, mainly in South, care for sharply higher shares of elderly black, Hispanic, and Medicaid patients. Health Affairs 2011; 30:1904-11.
Joynt KE, Jha AK. Characteristics of hospitals receiving penalties under the Hospital Readmissions Reduction Program. JAMA. Jan 23, 2013; 309(4):342-3.
Joynt, K. E., E. J. Orav and A. K. Jha. "Thirty-Day Readmission Rates for Medicare Beneficiaries by Race and Site of Care." JAMA 305, no. 7 (2011): 675-81.
Joynt Maddox, K. E., Reidhead, M., Hu, J., Kind, A. J. H., Zaslavsky, A. M., Nagasako, E. M., & Nerenz, D. R. (2019). Adjusting for social risk factors impacts performance and penalties in the hospital readmissions reduction program. Health services research, 54(2), 327–336. https://doi.org/10.1111/1475-6773.13133
Kangovi S, Grande D, Meehan P, Mitra N, Shannon R, Long JA. Perceptions of readmitted patients on the transition from hospital to home. Journal of hospital medicine. 2012;7(9):709-712.
Keenan PS, Normand SL, Lin Z, et al. An administrative claims measure suitable for profiling hospital performance on the basis of 30-day all-cause readmission rates among patients with heart failure. Circulation 2008;1(1):29-37.
Kim H, Ross JS, Melkus GD, Zhao Z, Boockvar K. Scheduled and unscheduled hospital readmissions among patients with diabetes. The American journal of managed care. 2010;16(10):760-767.
Kind, A. J. H., & Buckingham, W. (2018). Making neighborhood disadvantage metrics accessible: The Neighborhood Atlas. New England Journal of Medicine, 378, 2456-2458. https://doi.org/10.1056/NEJMp1802313
Krumholz HM, Brindis RG, Brush JE, et al. 2006. Standards for Statistical Models Used for Public Reporting of Health Outcomes: An American Heart Association Scientific Statement From the Quality of Care and Outcomes Research Interdisciplinary Writing Group: Cosponsored by the Council on Epidemiology and Prevention and the Stroke Council Endorsed by the American College of Cardiology Foundation. Circulation 113: 456-462.
Krumholz HM, Lin Z, Drye EE, et al. An administrative claims measure suitable for profiling hospital performance based on 30-day all-cause readmission rates among patients with acute myocardial infarction. Circulation 2011;4(2):243-52.
Mitchell SE, Sadikova E, Jack BW, Paasche-Orlow MK. Health literacy and 30-day post discharge hospital utilization. Journal of health communication. 2012;17 Suppl 3:325-338.Nagasako EM, Reidhead M, Waterman B, Dunagan WC. Adding socioeconomic data to hospital readmissions calculations may produce more useful results. Health affairs (Project Hope). 2014;33(5):786-791
Nwana, N., Chan, W., Langabeer, J., Kash, B., & Krause, T. M. (2022). Does hospital location matter? Association of neighborhood socioeconomic disadvantage with hospital quality in US metropolitan settings. Health & place, 78, 102911. https://doi.org/10.1016/j.healthplace.2022.102911
Normand S-LT, Shahian DM. 2007. Statistical and Clinical Aspects of Hospital Outcomes Profiling. Stat Sci 22 (2): 206-226.
Odonkor CA, Hurst PV, Kondo N, Makary MA, Pronovost PJ. Beyond the Hospital Gates: Elucidating the Interactive Association of Social Support, Depressive Symptoms, and Physical Function with 30-Day Readmissions. American journal of physical medicine & rehabilitation / Association of Academic Physiatrists. 2015;94(7):555-567
Owens PL, Liang L, Barrett, ML, Fingar, KR. Comorbidities Associated with Adult Inpatient Stays, 2019. Agency for Healthcare Research and Quality, Healthcare Utilization Project Statistical Brief #303, December 2022. https://hcup-us.ahrq.gov/reports/statbriefs/sb303-Comorbidities-Adult-Hospitalizations-2019.jsp
Pandey, A., Keshvani, N., Khera, R., Lu, D., Vaduganathan, M., Joynt Maddox, K. E., Das, S. R., Kumbhani, D. J., Goyal, A., Girotra, S., Chan, P., Fonarow, G. C., Matsouaka, R., Wang, T. Y., & de Lemos, J. A. (2020). Temporal Trends in Racial Differences in 30-Day Readmission and Mortality Rates After Acute Myocardial Infarction Among Medicare Beneficiaries. JAMA cardiology, 5(2), 136–145. https://doi.org/10.1001/jamacardio.2019.4845
Pope, G., et al., Principal Inpatient Diagnostic Cost Group Models for Medicare Risk Adjustment. Health Care Financing Review, 2000. 21(3):26.
Powell, W. R., Sheehy, A. M., & Kind, A. J. (2023). The area deprivation index is the most scientifically validated social exposome tool available for policies advancing health equity. Health Affairs Forefront.
Singh, G. K. (2003). Area deprivation and widening inequalities in US mortality, 1969–1998. American Journal of Public Health, 93(7), 1137-1143. https://doi.org/10.2105/ajph.93.7.1137
Sterling, M. R., Ringel, J. B., Pinheiro, L. C., Safford, M. M., Levitan, E. B., Phillips, E., Brown, T. M., Nguyen, O. K., & Goyal, P. (2022). Social Determinants of Health and 30-Day Readmissions Among Adults Hospitalized for Heart Failure in the REGARDS Study. Circulation. Heart failure, 15(1), e008409. https://doi.org/10.1161/CIRCHEARTFAILURE.121.008409
University of Wisconsin School of Medicine and Public Health. (2023). Area Deprivation Index v4.0. Retrieved from https://www.neighborhoodatlas.medicine.wisc.edu/
Please see the excel file entitled MIPSHWR_Surg_Risk_Variable_ORS_Frequencies for the frequency and odds ratios for each risk variable in the final risk model across patients in the 2024 endorsement maintenance testing dataset.
Tables 6, 7, and 8 (see attachment) show the proportion of social risk factors identified in the conceptual model for the MIPS HWR measure. Across the entire cohort, the proportion of patients with the DE and ADI variables is 14.66% and 12.11%, respectively. Across the specialty cohorts, the proportion of patients with DE ranges from 9.41% in the Surgery/Gynecology cohort, to 17.74% in the Cardiorespiratory cohort; the proportion of patients with high ADI ranges from 10.61% in the Surgery/Gynecology cohort to 14.68% in the Cardiorespiratory cohort. At the clinician group level, the median proportion of patients with DE was 5.6% and for patients with high ADI was 0% (Table 9).
Case Mix Adjustment
Please see a list of all risk variables along with risk variable frequencies and odds ratios is available in the file entitled MIPSHWR_Surg_Risk_Variable_ORS_Frequencies. We note that a present-on-admission algorithm is used in the risk-adjustment methodology to identify condition category (CC)-defined risk-adjustment variables from secondary diagnosis codes on the index admission claim.
Service Mix Adjustment
In addition to the case mix variables shown below, the MIPS HWR measure is also adjusted for service mix. The measure includes many different discharge condition categories that differ in their baseline readmission risks. In addition, hospitals differ in their relative distribution of these condition categories (service mix). To adjust for service mix, the measure uses an indicator variable for the discharge condition category in addition to risk variables for comorbid conditions. The models include a condition-specific indicator for all CCs with sufficient volume (defined as those with more than 1,000 admissions nationally in a given year for Medicare FFS data) as well as a single indicator for conditions with insufficient volume in each model.
To assess model performance, we computed three summary statistics to demonstrate model discrimination and calibration, and we provide risk-decile plots to demonstrate additional evidence of calibration. We describe the approach and results below; related tables and figures are in the attachment.
Discrimination statistics (c-statistic and predictive ability)
1. C-statistic: Area under the receiver operating characteristic (ROC) curve (the c-statistic) indicates the probability that predicting the outcome is better than chance, which is a measure of how accurately a statistical model can distinguish between a patient with and without an outcome. Our results show that c-statistics ranged from 0.63-0.68 across specialty cohorts (see Table 10 in the attachment), similar to the c-statistics submitted for initial CBE endorsement, and in line with readmission-type measures. We also list the c-statistic for each specialty cohort below:
Cardiorespiratory: 0.64
Cardiovascular: 0.66
Medicine: 0.65
Neurology: 0.63
Surgery/Gynecology: 0.68
2.Predictive ability: Discrimination in predictive ability measures the ability to distinguish high-risk subjects from low-risk subjects; therefore, we want to see a wide range between the lowest decile and highest decile. Predictive ability for each cohort (see Table 11 in the attachment) is sufficiently wide across cohorts and is similar to results provided to support initial CBE endorsement. Below we also list the predictive ability (lowest decile %-highest decile %) for each cohort.
Cardiorespiratory: 7.9 - 34.5
Cardiovascular: 5.1 - 27.9
Medicine: 7.6 - 33.2
Neurology: 6.1 - 25.2
Surgery/Gynecology: 3.0 - 26.2
Calibration statistics (overfitting)
3. Over-fitting refers to the phenomenon in which a model accurately describes the relationship between predictive variables and outcome in the development dataset but fails to provide valid predictions in new patients. Estimated calibration values of γ0 close to 0 and estimated values of γ1 close 1 provide evidence of good calibration of the model.
We used Datasets A1, A2, and B for this analysis, which were performed during initial measure development. Our results show a calibration value of close to 0 at one end and close to 1 to the other end indicating good calibration of the model (Table 12, attachment).
CORE notes that after initial measure development we do not re-test our risk models for overfitting using a dataset that is external to the testing sample. In our risk models, coefficients are updated each time the measure is calculated; we refit the model with new data each time the measure is calculated. Therefore, random statistical fluctuations in model coefficients across repeated reporting cycles are part of the overall random error in the facility performance estimates.
Risk-decile Plots
Risk decile plots assess calibration of the risk model by examining the alignment between predicted and observed outcomes across deciles of predicted risk. Risk-decile plots for MIPS HWR (Figures 4-8 in the attachment) show good calibration across each specialty cohort.
We describe the approach to risk variable selection in Section 4.4.4a, including the rationale for including or not including risk factors in the final model. We note that this clinician-group measure uses the same risk variables as the hospital-level measure, from which the clinician-group measure was respecified. The risk model was developed originally without social risk factors; in Section 5.1 we provide updated incremental testing on the impact of social risk factors on measure scores.
In this section we provide the interpretation of the risk model testing results shown in section 4.4.5.
The following results demonstrate that the risk-adjustment model adequately controls for differences in patient characteristics:
Discrimination Statistics
The calculated c-statistics ranged from 0.63 to 0.68 across specialty cohorts; these values indicate good model discrimination across the cohort models. The models also predicted a wide range between the lowest decile and highest decile for each cohort and dataset, indicating the ability to distinguish high-risk subjects from low-risk subjects.
Calibration Statistics
The calibration values which are consistently close to 0 at one end and close to 1 for all specialty cohorts and datasets, indicating good calibration of the models.
Risk Decile Plots
Higher deciles of the predicted outcomes are associated with higher observed outcomes, which show a good calibration of the model. This plot indicates excellent discrimination of the model and good predictive ability.
Overall Interpretation
Interpreted together, our diagnostic results demonstrate the risk-adjustment model adequately controls for differences in patient characteristics (case mix).
Use & Usability
Use
Level of analysis is clinician group; care setting is inpatient and clinic/outpatient.
Usability
The outcome of unplanned hospital visits following discharge from an inpatient admission is a widely accepted measure of care quality. The MIPS HWR measure provides the opportunity to improve the quality of care and to lower rates of adverse events that result in unplanned readmission after an inpatient stay.
There are evidence-based interventions that can reduce readmission rates. These interventions often address inadequate transitions of care, including patient education at discharge and coordination of outpatient care. For example, a 2021 systematic review that analyzed 60 trials, including 19 randomized controlled trials, concluded, in agreement with prior systematic reviews, that interventions that focus on communication at discharge were statistically significantly associated with lower rates of hospital readmissions (Becker et al., 2021). Within the 19 trials,10 focused on medication counselling, and six focused on patient education about their condition: the other three focused on other specific communication strategies. A 2022 systematic review found that post-discharge care including home care, telephone, and/or clinic visits resulted in lower rates of readmission compared with “usual care” for cardiac patients (Chauhan & McAlister, 2022). A systematic review published in 2023 pooled the results from 73 different studies to compare transitional care interventions with different levels of complexity and their impact on improving outcomes and found that low- and medium-complexity interventions were the most effective at reducing 30-day readmissions (Tyler et al., 2023). Study authors found that compared with usual care, readmission rates were reduced by 18 percent to 55 percent for these types of interventions. Complexity was categorized by the number of components of the intervention, and the number of stages of the hospitalization that the intervention was implemented. Finally, CMS has published a guide for hospitals, aimed at leadership, staff, and clinicians, which outlines effective strategies for reducing readmissions and reducing disparities. Strategies covered in the guide include: ensuring that patients understand discharge instructions and have appropriate follow-up visits, improving accessibility (transportation) for post-discharge care, ensuring patients have a primary care provider, starting post-discharge visit planning early in the discharge process, ensuring transfer of information to the post-discharge provider, and strategies to address language barriers and low health literacy (CMS Office of Minority Health, 2024).
The MIPS HWR measure is part of the Merit-Based Incentive Payment System (MIPS) a pay-for-performance program. This measure addresses an important quality measurement area and enhances the information available to patients about where to seek care. Furthermore, providing outcome rates to clinician groups and the public makes visible meaningful quality differences and incentivizes improvement.
References
Becker, C., Zumbrunn, S., Beck, K., Vincent, A., Loretz, N., Müller, J., Amacher, S. A., Schaefert, R., & Hunziker, S. (2021). Interventions to Improve Communication at Hospital Discharge and Rates of Readmission: A Systematic Review and Meta-analysis. JAMA network open, 4(8), e2119346. https://doi.org/10.1001/jamanetworkopen.2021.19346
Chauhan, U., & McAlister, F. A. (2022). Comparison of Mortality and Hospital Readmissions Among Patients Receiving Virtual Ward Transitional Care vs Usual Post discharge Care: A Systematic Review and Meta-analysis. JAMA network open, 5(6), e2219113. https://doi.org/10.1001/jamanetworkopen.2022.19113
CMS Office of Minority Health (2024). Guide for Reducing Disparities in Readmissions. Accessed April 23, 2024; https://www.cms.gov/about-cms/agency-information/omh/downloads/omh_readmissions_guide.pdf
Tyler, N., Hodkinson, A., Planner, C., Angelakis, I., Keyworth, C., Hall, A., Jones, P. P., Wright, O. G., Keers, R., Blakeman, T., & Panagioti, M. (2023). Transitional Care Interventions From Hospital to Community to Reduce Health Care Use and Improve Patient Outcomes: A Systematic Review and Network Meta-Analysis. JAMA network open, 6(11), e2344825. https://doi.org/10.1001/jamanetworkopen.2023.44825
Throughout the measure development process, we obtained expert and stakeholder input through regular discussions with external clinical consultants, consulting our national TEP, and holding a 30-day public comment period. CORE clinicians, as well as several clinical experts, met regularly to discuss all aspects of measure development, including the cohort, outcome definition, risk adjustment and attribution rules.
In addition to the clinical consultations and in alignment with CMS MMS guidance, we convened a TEP to provide input and feedback during measure development from a group of recognized experts in relevant fields. To convene the TEP, we released a public call for nominations and selected individuals to represent a range of perspectives, including clinicians, patients, and individuals with expertise in quality improvement and performance measurement. We held three structured TEP conference calls consisting of a presentation of key issues, our proposed approach, and relevant data, followed by open discussion among TEP members.
Following measure implementation, CMS gathers stakeholder feedback through CMS’s QPP Q&A process. CMS offers a phone number, email inbox, and a Service Center website: https://qpp.cms.gov/resources/help-and-support.
During measure development we made modifications to the measure’s attribution (adopting the multiple attribution method) based on TEP feedback on the measure.
Following measure implementation, no changes other than standard coding updates were implemented.
For the MIPS HWR (clinician group) measure we currently do not have access to measure scores that span a sufficient period of time to compare performance, following the start of public reporting.
However, we do know from the hospital-level measure that over time, observed (unadjusted) national readmission rates have declined across the entire cohort and across each specialty cohort. We examined national patient-level unadjusted outcomes for the hospital-level, claims-only HWR measure and found that mean observed (unadjusted) patient-level readmission rates declined across the entire cohort, from 15.3 percent in the 2016/2017 performance period to 14.6 percent in the 2021/2022 performance period, and also across all specialty cohorts (Figure 11 in the attachment). This suggests that public reporting of the HWR measure has the potential to result in better outcomes for patients and focus attention on improving processes that are tied to improved outcomes.
There have been no unexpected findings during implementation.
Comments
Staff Preliminary Assessment
CBE #3495 Staff Assessment
Importance
Strengths:
- The developer cites evidence from the primary literature in which hospital interventions that improve readmission rates, such as medication reconciliation ensuring patient readiness at discharge, improving communication between providers involved in the transition of care, and others. The developer notes that several hospital strategies can be used by groups of clinicians. Using data from Jan 1, 2021 through Dec 31, 2021, the developer reported a distribution of scores with an overall rate of 15.32% and a decile range of 13.62 – 17.41%. For a measure in which fewer readmissions is better, this distribution shows less then optimal performance for unplanned readmission, including slight variation in performance amongst providers
The developer cites interviews with patients and caregivers, who noted that readmissions resulted in confusion, frustration, and suffering.
Limitations:
- None identified.
Rationale:
- The developer posits that this measure identifies clinician groups performance on readmission rates, which promotes quality improvement and better inform consumers about care quality. The developer cites evidence from the primary literature in which hospital interventions that improve readmission rates, such as medication reconciliation ensuring patient readiness at discharge, improving communication between providers involved in the transition of care, and others. The developer notes that several hospital strategies can be used by groups of clinicians.
Using data from Jan 1, 2021 through Dec 31, 2021, the developer reported a distribution of scores with an overall rate of 15.32% and a decile range of 13.62 – 17.41%. For a measure in which fewer readmissions is better, this distribution shows less then optimal performance for unplanned readmission, including slight variation in performance amongst providers
The developer cites interviews with patients and caregivers, who noted that readmissions resulted in confusion, frustration, and suffering.
Feasibility Acceptance
Strengths:
- This an claims-based measure. As such, the developer states there is no added burden of reporting as all required data elements are routinely generated. The measure does not include any proprietary information.
Limitations:
- None identified.
Rationale:
- The developer did not conduct a feasibility assessment of an assessment of missing data for this measure, as it used a 100% sample of paid Medicare claims. The developer further states that since this is an claims-based measure, there is no added burden of reporting as all required data elements are routinely generated. The measure does not include any proprietary information.
Scientific Acceptability
Strengths:
- The measure is clear and well defined.
Entity level reliability (signal-to-noise) is calculated on data from 1/21-12/21 for the risk adjusted measure for five cohorts: cardio-respiratory, cardiovascular, medicine, neurology, and surgical. Minimum and maximum reliability values only are reported for each cohort in two ways: using all providers and using only providers with at least 200 patients. Minimum reliability values are above 0.6 for all clinician groups with at least 200 patients except for the surgical cohort which has a mean reliability of 0.84 and a minimum of 0.49.
Limitations:
- The mean reliability calculated using all providers is below 0.6 for the cardiovascular, medicine, and surgical cohorts, so the reliability is below 0.6 for more than 70% of all entities for these cohorts. The cardio-respiratory and neurology cohorts have a mean reliability slightly above 0.6, so it is unlikely that more than 70% of the entities have a reliability above 0.6 for those cohorts.
For providers with at least 200 patients, the surgical cohort has a minimum reliability of 0.49 and it is not possible to estimate the percentage of entities that may have a reliability below 0.6 for that cohort as only a minimum, mean and maximum reliability are reported.
Rationale:
- The measure is well defined. Reliability was assessed at the entity level. Reliability statistics are above the established thresholds for all entities with more then 200 patients per year, with the exception of the surgical cohort. Reliability rating is "Met" for enitites with at least 200 patients. It should be specifically stated that the measure be only reported for entities with at least 200 patients.
Strengths:
- The developer hypothesized a weak-to-moderate, negative relationship indicating that improved performance on this measure would result in improved Star rating scores. The developer reported weak and negative correlations, as expected. The developer also convened a technical expert panel (TEP) for face validity testing. 12/17 (70%) of TEP members agreed that the MIPS HWR measure scores are valid, useful, and can provide information for improving the quality of care.
The developer also conducted statistical risk adjustment for each cohort, including patient-level risk factors that were relevant to the cohort based on the developer’s conceptual model. The developer also explored social risk factors, such as dual eligibility and the Area Deprivation Index. The developer did not include these in the final models due to the minimal impact these social risk factors have on the measure scores. The developer reported c-statistics ranging from 0.63-0.68, which indicate good model discrimination.
Limitations:
- None identified.
Rationale:
- The developer conducted accountable entity-level validity testing by correlating the measure score to the CMS Hospital Overall Star Ratings and Hospital Star Ratings readmission group scores.
The developer hypothesized a weak-to-moderate, negative relationship indicating that improved performance on this measure would result in improved Star rating scores. The developer reported weak and negative correlations, as expected.
The developer also convened a technical expert panel (TEP) for face validity testing. 12/17 (70%) of TEP members agreed that the MIPS HWR measure scores are valid, useful, and can provide information for improving the quality of care.
The developer also conducted statistical risk adjustment for each cohort, including patient-level risk factors that were relevant to the cohort based on the developer’s conceptual model. The developer also explored social risk factors, such as dual eligibility and the Area Deprivation Index. The developer did not include these in the final models due to the minimal impact these social risk factors have on the measure scores. The developer reported c-statistics ranging from 0.63-0.68, which indicate good model discrimination.
Equity
Strengths:
- The developer identified differences in unadjusted patient-level outcomes for patients with dual eligibility and Area Deprivation Index.
Limitations:
- The developer identified differences in unadjusted patient-level outcomes in patients with dual eligibility and Area Deprivation Index, but it is unclear if these are statistically significant. The developer mentions that these social risk factors were not included in the final risk model, due to the very weak association between measure scores and the proportion of patients with social risk among clinician groups.
Rationale:
- The developer identified differences in unadjusted patient-level outcomes in patients with dual eligibility and Area Deprivation Index, but it is unclear if these are statistically significant. The developer mentions that these social risk factors were not included in the final risk model, due to the due to the very weak association between measure scores and the proportion of patients with social risk among clinican gorups. Since the social risk factors are not included in the model, the developer may consider doing significance testing on the differences of scores for cohorts with and without the social risk factor.
Use and Usability
Strengths:
- The measure is currently used in the MIPS program. The developer also describes various mechanisms in which feedback on the measure was obtained, including a developer-convened TEP, a public comment period, consultation from experts, and through CMS’ QPP Q&A process. The developer noted that feedback received from its TEP informed a modification to the attribution approach by adopting a multiple attribution approach.
The developer reports no unexpected findings.
Limitations:
- The developer also states that it does not have access to data that would span a time period for sufficiently comparing performance. The developer provides some results of the hospital-level readmission measure, positing that improvement with this measure means that reductions in readmissions are attainable.
Rationale:
- The measure is currently used in the MIPS program
The developer cites evidence of interventions that can improve readmissions, as noted previously.
The developer describes various mechanisms in which feedback on the measure was obtained, including a developer-convened TEP, a public comment period, consultation from experts, and through CMS’ QPP Q&A process. The developer noted that feedback received from its TEP informed a modification to the attribution approach by adopting a multiple attribution approach.
The developer states that it does not have access to data that would span a time period for sufficiently comparing performance. The developer provides some results of the hospital-level readmission measure, posting that improvement with this measure means that reductions in readmissions are attainable.
The developer reports no unexpected findings.
Committee Independent Review
Do not support
Importance
agree with staff
Feasibility Acceptance
agree with staff
Scientific Acceptability
agree with staff
The measure makes no distinction as to where the patient was discharged. A patient who is discharged to a Skilled Nursing Facility could experience an injury or poor care in the SNF and have to be admitted to the hospital for treatment of the resulting problems. However, under this measure, that admission will not be attributed to the SNF. Instead, it will be attributed to as many as three different clinicians who had nothing to do with the care delivered in the SNF or the problem that led to the admission.
Equity
Nothing in the risk adjustment methodology compensates for the problems described above; for example, there is no adjustment based on the reasons for the readmission in order to give higher weight to readmissions that are more likely to be related to the index admission or more likely to be preventable.
Moreover, even though it is obvious that patients with worse access to health care services in the community are more likely to be admitted to a hospital for both problems that are related to the original admission and problems that are not related, there are no factors in the risk adjustment to assess this. For example, patients who live in isolated rural areas and areas with shortages of healthcare services will have greater difficulty receiving follow-up care after discharge and greater difficulty receiving both primary care and specialty care for their other conditions, but there is nothing in the risk adjustment methodology that addresses this. The analysis of social risk factors described in the Measure Information Form fails to adequately examine these issues and inappropriately concludes that no adjustment is needed. As a result, the measure could discourage clinicians from treating such patients, which could even further reduce patients’ access to care and increase disparities in outcomes for disadvantaged patients.
Use and Usability
Patients and their caregivers might find this measure more meaningful if it considered the corporatization of healthcare with associated increased out-of-pocket expenses and the shift from solo to group practices; telemedicine, the impact of the COVID-19 pandemic, and included data from Medicare Advantage Plans.
Summary
The measure makes no distinction as to where the patient was discharged. A patient who is discharged to a Skilled Nursing Facility could experience an injury or poor care in the SNF and have to be admitted to the hospital for treatment of the resulting problems.
There is nothing in the risk adjustment methodology that addresses this. The analysis of social risk factors described in the Measure Information Form fails to adequately examine these issues and inappropriately concludes that no adjustment is needed. As a result, the measure could discourage clinicians from treating such patients, which could even further reduce patients’ access to care and increase disparities in outcomes for disadvantaged patients.
Patients and their caregivers might find this measure more meaningful if it considered the corporatization of healthcare with associated increased out-of-pocket expenses and the shift from solo to group practices; telemedicine, the impact of the COVID-19 pandemic, and included data from Medicare Advantage Plans.
Do not support
Importance
agree with staff assessment
Feasibility Acceptance
agree with staff assessment
Scientific Acceptability
concerned about attribution methodology which links the physicians responsible for the greatest costs (e.g., surgeon) with a hospital readmission which may be beyond their control. makes more sense as a hospital measure rather than physician.
problems with the attribution methodology would impact both reliability and validity of the measure.
Equity
agree with staff assessment; wound consider additional social risk factors to determine impact on measurement.
Use and Usability
agree with staff assessment; need data showing how measure has impacted performance, and whether results in clinically significant outcomes.
Summary
Unclear whether this measure makes sense, based on the physician's (esp the surgeon) ability to influence readmissions.
Some clarifications needed.
Importance
The developer cites evidence from the primary literature in which hospital interventions that improve readmission rates, such as medication reconciliation ensuring patient readiness at discharge, improving communication between providers involved in the transition of care, and others. The developer cites interviews with patients and caregivers, who noted that readmissions resulted in issues.
Feasibility Acceptance
Uses 100% Medicare claims and developer did not conduct feasibility.
Scientific Acceptability
For providers with at least 200 patients, the surgical cohort has a minimum reliability of 0.49 and it is not possible to estimate the percentage of entities that may have a reliability below 0.6 for that cohort as only a minimum, mean and maximum reliability are reported. The developer hypothesized a weak-to-moderate, negative relationship indicating that improved performance on this measure would result in improved Star rating scores. The c-statistic ranges from 0.63-0.68 which shows discrimination.
See above
Equity
The developer mentions that social risk factors was not included in the final risk model. Should this be relooked at.
Use and Usability
The measure was used in the MIPS program.
Summary
Some clarifications needed.
no additional comments
Importance
agree with staff assessment
Feasibility Acceptance
agree with staff assessment
Scientific Acceptability
agree with staff assessment
agree with staff assessment
Equity
agree with staff assessment
Use and Usability
agree with staff assessment
Summary
n/a
No additional comments
Importance
The developer has provided a decent rationale for the measure's importance, supported by empirical evidence and literature. It seems to be meaningful and beneficial for improving patient care by reducing unplanned hospital readmissions, by addressing quality improvement.
Feasibility Acceptance
The developer did not conduct a formal feasibility assessment or assess missing data for this measure, as it uses a 100% sample of paid Medicare claims. Since the measure is claims-based, all necessary data elements are routinely generated during care delivery, imposing no additional reporting burden on facilities.
Scientific Acceptability
agree with staff assessment
agree with staff assessment
Equity
Equity was not sufficiently addressed, and additional steps may be needed to fully incorporate and analyze the impact of social risk factors on measure performance.
Use and Usability
Though the measure was used in the MIPS program, more data should be showed to evaluate the measure’s impact on performance
Summary
No additional comments
Support
Importance
Agree w staff assessment
Feasibility Acceptance
Agree w staff assessment
Scientific Acceptability
Agree w staff assessment
Agree w staff assessment
Equity
Agree w staff assessment
Use and Usability
Agree w staff assessment
Summary
Full support of measure
Support
Importance
Agree with staff assessment.
Feasibility Acceptance
Agree with staff assessment.
Scientific Acceptability
Please clarify whether there are explicit volume or reliability thresholds being used for the measure. There is a large proportion of reported entities not meeting the minimum 0.6 threshold.
Agree with staff assessment.
Equity
Agree with staff assessment.
Use and Usability
I would like to understand when data will become available and the developer's plans to review performance improvement once data becomes available.
Summary
Overall support the measure and its intent. There are areas where more information is needed, specifically: addressing reliability issues, stratified equity analysis, and plans regarding assessing improvement.
Public Comments
Public Comment Shared During May 29 Listening Session
I think I'm glad to hear that this is in place, and that they'll be watching closer some of the unplanned readmission rates, and to try to reduce some of the burdens. How do you say? Because it's cost and effective? Sometimes clinicians may be maybe docked kind of because or because of something like an all-cause unplanned readmission. And so, I think I'm hoping that this new maintenance measure does look at some of that for the attribution. So, what happens is if a person has surgery, and they end up in the in the hospital 7 days later. Who might be considered the individual that might be sort of responsible. Doctors and clinicians and systems can get docked for the readmission, so to ensure that the readmission, what they're in there for is sort of aligned with whatever happened, right? Like something totally separate might happen. But anyway, thank you.
Public Comment Shared During May 29 Listening Session
Hi, I agree with Janice on this statement here [about attribution]. I believe the same. It's very important. Janice has supported me and guided me through this type of process here. Because of Janice, I'm part of this meeting and we're working with you guys here. So her input is very highly appreciated. So this is my input for the public comments is what I mean.
Endorsement Should Be Removed From This Measure
CBE ID: 3495
Hospital-Wide 30-Day, All-Cause, Unplanned Readmission Rate (HWR)
for the Merit-Based Incentive Payment System (MIPS) Eligible Clinician Groups
Endorsement should be removed from this measure. It is not a valid measure of the quality or efficiency of care provided by physicians due to problems with the definition of the numerator, the attribution methodology, and the risk adjustment methodology. Because of both the lack of validity and the low reliability of the measure, continued public reporting of the results could mislead patients about the quality of care their physicians provide, and continued use of the measure to modify physician payments could worse disparities in access and outcomes for patients.
Problems with the Numerator
The stated goal of this measure is to reduce the frequency with which patients are readmitted to the hospital after discharge due to “poor quality of care, inadequate coordination of care, or lack of effective discharge planning and transitional care.” However, the numerator of the measure includes events that have nothing to do with the original hospitalization, while excluding events that represent undesirable outcomes and avoidable costs, so it cannot be used to assess whether the goal is being achieved.
Inclusion of Unrelated Admissions. The measure defines a “readmission” as any unplanned admission to any hospital that occurs within 30 days after the patient was discharged. However, patients are admitted to hospitals for many reasons, and a patient who is discharged from the hospital after treatment for one problem could have an injury or unrelated acute illness or chronic condition requiring hospital treatment within 30 days after the initial discharge. It is inappropriate to treat all such hospital admissions as “readmissions” and to imply that all or most such admissions reflect poor quality of care by the physicians who treated the patient during the initial admission. For example, a patient who is discharged from the hospital following a knee replacement could be injured in an accident after leaving the hospital and require hospital treatment for those injuries, or they could experience an exacerbation of asthma or COPD several weeks after discharge and require hospital treatment for that exacerbation, but neither the hospital nor the surgeons who performed the knee surgery could have reasonably been expected to prevent either of those types of admissions. Including large numbers of unrelated and non-preventable admissions in the numerator means that it will be harder to identify which hospitals have higher rates of readmissions that are related to the original admission.
Exclusion of Related Complications That Do Not Result in Admission. Conversely, the measure only counts inpatient admissions to a hospital, not observation stays, emergency department visits, or visits to urgent care centers for treatment of complications resulting from the treatment during the original hospital stay. A patient could have a serious complication following hospital discharge, but if the complication can be successfully treated in an emergency department without admitting the patient to the hospital, or if the treatment is classified as an observation stay rather than an inpatient admission, the complication will not be counted by the measure. Hospitals that are experiencing shortages of inpatient beds may have to treat patients in the ED, and they will have lower “readmission” rates as a result. Moreover, because observation stays are not included, the measure definition creates an incentive to treat patients with complications in observation stays rather than admitting them as inpatients, since this will reduce the calculated readmission rate.
Failure to Include Deaths. The numerator does not include deaths resulting from the initial hospital stay, it only includes hospital admissions. Not only is death a more serious complication than a hospital admission, a hospital with a high rate of deaths after discharge could appear to have better performance on this measure. Moreover, excluding deaths from the measure creates a perverse incentive to avoid admitting a patient to the hospital for treatment during the 30 day post-discharge period, even if failure to admit the patient could result in their death.
The Measure Information Form provides no analysis regarding any of these issues. Because this measure is a “re-specified” version of the readmission measure that CMS uses for hospitals, the developers appear to simply assume that the definitions are automatically appropriate. However, research has demonstrated that the hospital-level readmission measure is problematic because of these same kinds of concerns (see, for example, Gupta A, Fonarow G. “The Hospital Readmissions Reduction Program – Learning from Failure of a Healthcare Policy,” European Journal of Heart Failure 20:1169-1174), and as a result, there have been growing calls to stop using the hospital-level measure to evaluate hospital performance (e.g., Figueroa JF and Wadhera RK. “A Decade of Observing the Hospital Readmission Reductions Program – Time to Retire an Ineffective Policy,” JAMA Network Open 5(11): e2242593).
Problems with the Attribution Methodology
The problems with the definition of the numerator are exacerbated because this measure attributes the “readmissions” to clinicians who had no reasonable opportunity to prevent the admissions. The measure attributes every post-discharge admission to up to three different clinicians – (1) the clinician who provided the majority of the inpatient care; (2) the clinician(s) involved with the patient’s discharge; and (3) the clinician who provided the largest number of primary care services prior to the hospital admission. There are multiple problems with this attribution approach:
Attributing Every Readmission to Every Clinician. The measure developers correctly recognize that multiple clinicians are typically involved with the care of a patient during and following discharge, and that each could play some role in reducing avoidable readmissions. For example:
However, the surgeon does not manage the patient’s chronic conditions, so the surgeon has no role to play in preventing exacerbations of those conditions. The patient’s primary care physician did not perform the surgery and had no role to play in preventing surgical site infections. Yet the attribution methodology in this measure assigns readmissions due to surgical site infections to primary care physicians as well as surgeons, and it assigns admissions for chronic disease exacerbations unrelated to the surgery to the surgeons as well as the primary care physicians. A hospitalist will have limited ability to control what happens to a patient after they leave the hospital, particularly if the patient lives a long distance away, yet the measure attributes any hospital admission that occurs anywhere, including in a distant community, to the hospitalist who discharged the patient from the original hospital stay as much as a month earlier.
Failure to Attribute Readmissions to Post-Acute Care Facilities. The measure makes no distinction as to where the patient was discharged. A patient who is discharged to a Skilled Nursing Facility could experience an injury or poor care in the SNF and have to be admitted to the hospital for treatment of the resulting problems. However, under this measure, that admission will not be attributed to the SNF. Instead, it will be attributed to as many as three different clinicians who had nothing to do with the care delivered in the SNF or the problem that led to the admission.
Attributing Admissions to Primary Care Clinicians No Longer Caring for the Patient. The measure attributes every admission that occurs following a hospital stay to the “clinician who provides [sic] the greatest number of claims for primary care services during the 12 months prior to the hospital admission date.” No effort is made to determine whether that clinician was still involved with the patient’s care either just prior to the admission or after discharge. If a patient had just moved to a different city before the admission, or if they were discharged to a skilled nursing facility located in a different city or to a family member living in a different city, it would be more difficult or impossible for a primary care clinician who was caring for the patient prior to the original hospital admission to continue playing a significant role in their care during the month following discharge.
None of these issues is discussed in the Measure Information Form. It does not appear that any effort was made to examine which clinicians are being attributed different types of readmissions for different types of patients to determine whether the attributions are valid indicators of the clinicians’ role in the patient’s care and their ability to prevent those readmissions.
These serious flaws in the attribution methodology mean that many clinicians could be inappropriately labeled as delivering poor quality care or penalized financially for events that had no relationship to the care they delivered and that they had no reasonable way of preventing.
Problems with the Risk Adjustment Methodology
Nothing in the risk adjustment methodology compensates for the problems described above; for example, there is no adjustment based on the reasons for the readmission in order to give higher weight to readmissions that are more likely to be related to the index admission or more likely to be preventable.
Moreover, even though it is obvious that patients with worse access to health care services in the community are more likely to be admitted to a hospital for both problems that are related to the original admission and problems that are not related, there are no factors in the risk adjustment to assess this. For example, patients who live in isolated rural areas and areas with shortages of healthcare services will have greater difficulty receiving follow-up care after discharge and greater difficulty receiving both primary care and specialty care for their other conditions, but there is nothing in the risk adjustment methodology that addresses this. The analysis of social risk factors described in the Measure Information Form fails to adequately examine these issues and inappropriately concludes that no adjustment is needed. As a result, the measure could discourage clinicians from treating such patients, which could even further reduce patients’ access to care and increase disparities in outcomes for disadvantaged patients.
Failure to Assess Classification Errors Resulting from the Measure
Unlike the hospital readmission measure, this measure is not intended to merely report whether the readmission rate calculated for a medical group is above or below average for other groups. It is being used to assign groups to performance deciles and to use the decile assignment as part of the Merit-Based Incentive Payment System to determine the group’s quality score, and that quality score will in turn be used to determine whether the clinicians in that group will have their Medicare payments increased or reduced.
If this measure inaccurately assigns a group to a low decile, it could result in a payment reduction to the physicians in that group, as well as lead patients to believe that the group provides poor quality care. Consequently, it is essential to estimate the misclassification probabilities associated with the measure, and also to determine the proportion of clinicians who are assigned to different deciles in successive years.
The Measure Information Form gives no indication that any analysis of classification error was conducted. It merely reports generic signal-to-noise reliability statistics. The Measure Information Form references an article by John Adams and others as the basis for the reliability statistics it reports, but it ignores the discussion of misclassification in that same article and it fails to use the methodology described in that article to estimate the classification error rates for this measure. Based on the reliability statistics reported in the Measure Information Form, it seems likely that the rates of misclassification would be very high for this measure.
Endorsement Should Be Removed for Failure to Meet Minimum Levels of Scientific Acceptability, Usability, and Equity
This measure fails to meet the criteria for endorsement. It is not a valid or reliable measure of the quality or efficiency of care delivered by clinicians. It does not provide useful information to clinicians that would enable them to improve the quality of care to their patients, and the results cannot be used by patients to judge the relative quality of care provided by different clinicians. It creates perverse incentives for hospitals and clinicians to avoid patients with more complex health problems and it can worsen disparities in access to care and outcomes for disadvantaged patients.
The National Association of…
The National Association of ACOs (NAACOS) requests that the developer clarify why this submission does not also provide evidence, performance gap, and reliability and validity testing for ACOs since this measure also applies to ACOs in the MIPS program. We believe that this submission should reflect all uses of the measure for continued endorsement.
3495 Hospital-Wide 30-Day, All-Cause, Unplanned Readmission (HWR
The American Medical Association (AMA) continues to be concerned with the evidence cited to support attributing 30-day, all-cause unplanned readmissions to clinician groups; specifically we believe that attribution must be determined based on evidence that the accountable unit is able to meaningfully influence the outcome. The measure attributes the "readmissions" to clinicians who had no reasonable opportunity to prevent the admissions. The measure attributes every post-discharge admission to up to 3 different clinicians-- 1. the clinician who provided the majority of the inpatient care; 2. the clinician(s) involved with the patient's discharge; and 3. the clinician who provided the largest number of primary care services prior to the hospital admission, which assumes that each could play a role in reducing avoidable admissions. It is also flawed because the measures makes no distinction as to where the patient was discharged and fails to attribute readmissions to post-acute care facilities. The measure also attributes admission to primary care clinicians who may no longer be caring for the patient and makes no effrot to determine whether that clinician is still involved with the patient's care either just prior to the admission or after discharge. CMS must begin to demonstrate these relationships with the accountable unit and we do not believe that the link has been adequately demonstrated.
While the AMA agrees that there is evidence to demonstrate that improved care coordination and programs focused on discharge planning can lead to reductions in hospital readmissions, the majority of the cited evidence references hospitals and the two studies by Adams and Sanchez cited as evidence for clinician groups leveraged hospital discharge processes, which a practice is less likely to be able to easily influence. As a result, we do not believe that sufficient evidence was provided to support attributing this outcome to clinician groups.
We also question whether this measure produces scores that can be considered sufficiently reliable since the minimum reliability was 0.66 for the medicine cohort and 0.49 for the surgical cohort even with the increased case minimum of 200 patients. We believe that measures must achieve a minimum reliability of at least 0.7 or higher in order to be achieve endorsement.
In addition, we ask the committee to carefully consider whether the empirical validity testing is sufficient as it compared clinician group performance to hospital performance and whether the lack of risk adjustment for the social risk factors (dual eligibility and area deprivation index) when there were weak but significant correlations is acceptable.