When the measure returns for maintenance in 5 years, the developer will have: 1) Explored the feasibility of two risk adjustment models: one for dementia and Alzheimer’s patients and one for patients with severe mental illness; 2) Explored whether community resources should be considered for risk adjustment.
The IPF Readmission measure is a facility-level measure that estimates an unplanned, 30-day, risk-standardized readmission rate for adult Medicare fee-for-service (FFS) patients discharged from an inpatient psychiatric facility with a principal discharge diagnosis of a psychiatric disorder or dementia/Alzheimer's disease. The performance period used to identify cases in the denominator is 24 months. Data from 12 months prior to the start of the performance period through the performance period are used to identify risk factors.
Measure Specs
General Information
The Thirty-Day All-Cause Unplanned Readmission Following Psychiatric Hospitalization in an Inpatient Psychiatric Facility (IPF Readmission) measure is a claims-based measure that estimates the unplanned, 30-day, risk-standardized readmission rate (RSRR) among Medicare fee-for-service (FFS) patients ages 18 and older discharged from reporting IPFs with a principal discharge diagnosis of a psychiatric disorder, dementia, or Alzheimer’s disease. A lower measure score indicates better performance and better quality of care. The IPF Readmission measure contains the following factors in the IPF Readmission risk-adjustment model for the RSRR: sex, age, and clinical risk-factors. IPFs receive scores of better than, worse than, or no different from the national rate based on the comparison of their RSRR with the national readmission rate.
In its January 2015 report, the Measure Applications Partnership identified readmissions as a key gap for the Inpatient Psychiatric Facility Quality Reporting (IPFQR) Program (National Quality Forum 2015). To address this key gap, the Centers for Medicare & Medicaid Services (CMS) adopted the IPF Readmission measure into the IPFQR Program starting with fiscal year (FY) 2019 payment determination, as stated in the FY 2017 IPF Prospective Payment Systems Final Rule published in August 2016 (Centers for Medicare & Medicaid Services 2016). The final rule noted that “an all-cause readmission rate was selected because it promotes a holistic approach to the treatment of patients with psychiatric disorders, who often have comorbid medical conditions. From the patient and caregiver perspective, these readmissions indicate a deterioration in the patient’s condition.”
Other care settings that have a readmission performance measure have seen benefits reducing facility readmission rates (Khera et al., 2020). The Hospital-Wide All-Cause Unplanned Readmission measure in the Hospital Inpatient Quality Reporting Program, and the Plan All-Cause Readmissions measure in the Medicaid Core Sets are two examples of measures related to the IPF Readmission measure. Moreover, because readmission is an outcome measure that is influenced by multiple care processes and structures, as well as the entire healthcare team, it promotes a systems approach to improving care. A readmission measure can promote shared accountability and collaboration with patients, families, and providers in other settings of care.
Reference
Khera R, Wang Y, Bernheim SM, Lin Z, Krumholz HM. Post-discharge acute care and outcomes following readmission reduction initiatives: national retrospective cohort study of Medicare beneficiaries in the United States. BMJ. 2020 Jan 15;368:l6831. doi: 10.1136/bmj.l6831. PMID: 31941686; PMCID: PMC7190056.
National Quality Forum. MAP 2015 Final Recommendations to HHS and CMS. January 2015. Retrieved from https://www.qualityforum.org/Setting_Priorities/Partnership/MAP_Final_R…;
Centers for Medicare and Medicaid Services. Medicare Program; Hospital Inpatient Prospective Payment Systems for Acute Care Hospitals and the Long-Term Care Hospital Prospective Payment System and Policy Changes and FY 2017 Rates; Quality-Reporting Requirements for Specific Providers; Graduate Medical Education; Hospital Notification Procedures Applicable to Beneficiaries Receiving Observation Services; Technical Changes Relating to Costs to Organizations and Medicare Cost Reports; Finalization of Interim Final Rules with Comment Period on LTCH PPS Payments for Severe Wounds, Modifications of Limitations on Redesignation by the Medicare Geographic Classification Review Board, and Extensions of Payments to MDHs and Low-Volume Hospitals, 81 F.R. 56761. August 2016. Retrieved from https://www.federalregister.gov/d/2016-18476.
The following Medicare files are used for the calculation of the IPF Readmission measure:
- Medicare beneficiary and coverage files. This data provides information on patient demographic, enrollment, and vital status information to identify the measure population and certain risk factors.
- Medicare Part A. Medicare Part A covers institutional claims data for hospital facilities, such as IPFs, acute care, and critical access hospitals, hospice care, and skilled nursing facilities (Source: https://www.ssa.gov/medicare/plan/medicare-parts).
- Medicare Part B. Medicare Part B covers services from providers such as physicians, physician assistants, nurse practitioners, clinical social workers, and other providers to identify certain risk factors. Claims for services such as laboratory tests, medical supplies, or other ambulatory services were not used; this ensures that diagnoses result from an encounter with a provider trained to establish diagnoses and not a claim for a diagnostic test. (Source: https://www.ssa.gov/medicare/plan/medicare-parts).].
Index admissions and readmissions are identified in the Medicare Part A data. Comorbid conditions for risk-adjustment are identified in the Medicare Part A and Part B data in the 12 months prior to and including the index admission. Demographic and Medicare FFS enrollment information are identified in the Medicare beneficiary and coverage files.
Numerator
The numerator for the IPF readmission measure is defined as any admission to an IPF or acute care hospital that occurs on or between days 3 and 30 post-discharge, except those considered planned by the CMS Planned Readmission Algorithm, Version 3.0.
The IPF Readmission measure is a risk-adjusted outcome measure and therefore does not have a traditional numerator. A readmission is defined as any admission, for any reason, to an IPF or a short-stay acute care hospital (including Critical Access Hospitals) that occurs within 30 days after the discharge date from an eligible index admission to an IPF, except those considered planned. The measure uses the CMS 30-day Hospital-Wide Readmission Measure Planned Readmission Algorithm to identify planned readmissions. The algorithm follows two principles to identify planned readmissions:
- Select procedures and diagnoses, such as transplant surgery, maintenance chemotherapy/radiotherapy, and rehabilitation care are always considered planned. For a full list of planned procedures and diagnoses, refer to the “PR1” and “PR2” tabs of the IPF Readmission codebook.
- Some procedures, such as colorectal resection or aortic resection, are considered planned or unplanned depending on the accompanying principal discharge diagnosis. For a full list of such procedures, refer to the “PR3” tab of the IPF Readmission codebook. Specifically, a procedure is considered planned if it does not coincide with a principal discharge diagnosis of an acute illness or complication. For a full list of such principal discharge diagnoses, refer to the “PR4” tab of the IPF Readmission codebook.
Denominator
The denominator for this measure includes Medicare FFS beneficiaries aged 18 years and older who are admitted to and discharged alive from an IPF with a principal diagnosis of a psychiatric disorder.
The IPF Readmission measure denominator population consists of eligible index admissions to IPFs. Index admissions are defined as admissions to IPFs for patients with the following characteristics:
- Age 18 or older at admission.
- Discharged alive.
- Enrolled in Medicare Fee-for-Service (FFS) Parts A and B during the 12 months before, during the month of, and at least one month after the index admission.
- Discharged with a psychiatric principal diagnosis included in the “PsychCCS” tab of the IPF Readmission codebook. The list of diagnoses uses the Agency for Healthcare Research and Quality (AHRQ) Clinical Classification Software (CCS) ICD groupings. Information on sorting ICD codes into clinically coherent groups is available on the AHRQ CCS web page at https://www.hcup-us.ahrq.gov/toolssoftware/ccsr/ccsr_archive.jsp#ccsr.
- Admitted for fewer than 180 days.
A readmission to an IPF within 30 days will also be eligible as an index admission if it meets all other eligibility criteria listed above. Patients may have more than one index admission within the measurement period.
Exclusions
The measure population excludes admissions for patients with the following characteristics:
- Discharged against medical advice (AMA) because the IPF may have limited opportunity to complete treatment and prepare for discharge.
- Unreliable demographic and vital status data, defined as:
- Age greater than 115 years.
- Missing sex.
- Discharge status of “dead” but with subsequent admissions.
- Death date prior to admission date.
- Death date within the admission and discharge dates but the discharge status was not “dead.”
- Readmissions on the day of discharge or day following discharge because those readmissions are likely transfers to another inpatient facility. The IPF that discharges the patient to home or to a non-acute care setting is accountable for subsequent readmissions.
- Readmissions two days following discharge because readmissions to the same IPF within two days of discharge are combined within the same claim as the index admission and do not appear as readmissions due to the interrupted stay billing policy. Therefore, complete data on readmissions within two days of discharge are not available.
Information required to identify and calculate exclusions from the denominator can be found in Inpatient Psychiatric Facility Quality Reporting Program: Claims-Based Measure Specifications manual.
Measure Calculation
The IPF Readmission measure is calculated by the following steps:
- Identify all IPF admissions in the performance period.
- Apply inclusion and exclusion criteria to identify index admissions.
- Identify readmissions to IPFs or short stay acute care hospitals within 30 days of discharge from each index admission.
- Apply the Planned Readmission Algorithm to identify planned readmissions and remove them from the outcome.
- Identify risk factors in the 12 months prior to index admission and during the index admission.
- Run hierarchical logistic regression to compute risk-standardized readmission rate (RSRR) for each IPF.
Hierarchical logistic regression is used to model the log-odds of readmission. The two-level specification allows reliable estimates for small-volume hospitals while accepting a certain amount of shrinkage toward the mean. The model includes risk factors as fixed effects and a hospital-specific intercept as random effect. The estimate of hospital-specific intercept reflects the quality of care received at an IPF after adjusting for case mix.
A standardized risk ratio (SRR), which is the “predicted” number of readmissions over the “expected” number of readmissions, is calculated for each IPF. The “predicted” number of readmissions is the number of readmissions, given the IPF’s performance and its observed case mix, which is calculated by summing the estimated probabilities of readmission for the index admissions contributing to the IPF, based on the IPF-specific intercept and all other risk factors. The “expected” number of readmissions is the number of readmissions given the national performance and its observed case mix, which is calculated by summing the estimated probabilities of readmission for the index admissions contributing to the IPF, based on the average intercept and all other risk factors. The confidence interval of the SRR is calculated by bootstrapping. An SRR greater than 1 indicates worse quality of care compared to the national average. An SRR less than 1 indicates better quality of care. The RSRR is calculated by multiplying the SRR with the overall national readmission rate for better interpretation.
Lower rates indicate better performance for the IPF Readmission measure.
This measure is not stratified.
The minimum sample size for the IPF Readmission measure is 25. IPFs must have a minimum of 25 cases to receive reliable performance scores.
Supplemental Attachment
Measure Record
Point of Contact
Not applicable
Perry Lazar
Baltimore, MD
United States
Erin Buchanan
Mathematica inc.
Washington, DC
United States
Importance
Evidence
Hospital readmissions for patients with a serious psychiatric conditions occur more frequently than for patients without serious psychiatric conditions (Buck et al. 2022, Jiang et al. 2023, Mark et al. 2013, & Winterstein et al. 2020). Readmission rates within 30-days of discharge for psychiatric diagnoses range widely in the literature and are impacted by study design, methodology, included diagnoses and patient population (Muhammad et al. 2023 & Zhou et al. 2023). In a study of 633,114 Medicaid patients readmitted within 30-days post-discharge for non-behavioral health conditions, Becker et al. (2017), found that 21.7 percent of patients had a serious psychiatric condition compared to 17.9 percent of patients without a co-existing serious psychiatric condition.
Literature supports linkages between multiple conditions and IPF readmissions. Owusu et al. (2022), in its scoping review of 75 studies, reported that patients with a shorter length of stay for their initial psychiatric hospitalization were associated with a higher risk of readmission. Owusu et al. (2022) noted that clinical diagnoses of major depressive disorder, bipolar disorder, schizoaffective disorder, psychotic disorder, seasonal affective disorder, and substance use disorders can be predictors of early readmissions. Owusu et al. (2022) also cited non-adherence to medications and suicidal ideation as factors associated with a higher risk of readmission. According to Owusu et al. (2022), IPF readmissions can be associated with increased risk of suicide attempts. Winterstein et al. (2020) found that patients diagnosed with schizophrenia spectrum and psychotic disorders, mood disorders, and substance use disorders were more likely to be readmitted to an IPF within 30 days. Muhammad et al. (2023) found that readmission rates within 30 days were higher for patients with a primary diagnosis of schizophrenia followed by personality disorders, bipolar disorder, and depressive disorder.
Many patient-level factors have been shown to influence the risk of readmission. Zhou et al. (2023) reviewed 16 studies for social-demographic factors associated with readmission and found that younger patients and male patients were more likely to be readmitted. Male patients were cited as being at higher risk for rehospitalization by an additional systematic review and metanalysis conducted by Muhammad et al. (2023). Other factors associated with readmission risk include hospitalization prior to index admission, lack of follow up after discharge, not providing the primary care provider with a discharge summary, lack of community resources, and lack of family support (Zhou et al. 2023). Meurs et al. (2021) found that patients with a greater number of hospital admissions during the prior six months and patients who completed an outpatient clinic visit prior to readmission had lower odds of a potentially preventable readmission.
Multiple studies have assessed interventions focused on the link between patient centeredness and care transitions and readmissions (Tyler et al. 2019, Kim et al. 2021, Pugh et al. 2021, Dickson et al. 2020, Rammohan et al. 2023, Shields et al. 2023, Jesus et al. 2024). Tyler et al. (2019) found that interventions that include patient education, positive therapeutic relationships, and increased care continuity can reduce readmissions. Educational interventions that focus on teaching patients various self-management techniques are associated with increased knowledge about their diagnoses, reduction in symptoms, increased treatment adherence, and improvement in readmission rates (Tyler et al. 2019). Transitional discharge models that strive to increase the continuity of care between the hospital from which the patient is being discharged and the community resources available after discharge were found to have an association suggesting a reduction in readmission (Tyler et al. 2019). The most promising interventions impacting readmission from this study were those that enhance mental health care transitions by providing education and creating professional and physical connections between the hospital and community (Tyler et al. 2019).
Patients and providers often view the causes of readmissions differently, revealing important gaps in perspective. Smeraglio et al. (2019) conducted a study of patient and provider perspectives about readmission in a general medical/surgical population. During the study period 13 percent (469/3601) of discharged patients were readmitted. The authors conducted interviews with 39 percent (182/469) of the readmitted patients using an 11-question tool to ascertain their perceived contributors to the readmission. The attending physician at the time of discharge for the patient’s index admission reviewed the medical record and provided feedback on the factors that may have contributed to the patient’s readmission. Nurse case managers reviewed the medical records to obtain information about the discharge process and follow up after discharge. Of the patients interviewed 58 percent thought a modifiable system issue such as lack of readiness or being discharged too early (21%), needing an earlier follow up with a doctor (7%), needing more help at home (7%), or a medication issue (5%) contributed to their readmission. A majority of the physicians indicated there was no modifiable cause (71%) for readmission and cited patient adherence issues (13%) as the number one modifiable cause. The case managers indicated there was no modifiable cause in 41 percent of the cases and identified multiple modifiable issues such as inadequate education prior to discharge (17%), lack of follow up after discharge (14%), insufficient assessment of post discharge level of care needs (14%), discharged too early (12%), and lack of medication and therapy understanding or adherence (10%). Physicians agreed with patients that modifiable system issues contributed to readmission in only 2 percent of the cases, whereas case managers agreed with patients about modifiable system issues contributing 45 percent of the time. The authors noted that the gap between patient and physician perspective of discharge is a factor that potentially contributes to preventable readmissions. While this study was of a general medical/surgical patient population it suggests that patients and providers may have different expectations and identify different issues associated with hospital readmission.
Dickson et al. (2022) identified various domains that can influence care transition practices, which include the environment of the hospital system, patient and population characteristics, geographic characteristics, community resource availability, and limited access to care. The literature suggests that lack of mental health resources, high costs of care, further distance from providers, and lack of providers accepting Medicare are primary drivers of increased 30-day admission rates (Dickson et al. 2022). Using data to identify high risk patients, shared access to patient health records, patient and family engagement in treatment planning, a clearly defined transition process, and staff dedicated to navigating care transitions have all been associated with improved outcomes (Dickson et al. 2022). In particular, discharge planning that supports scheduling of an outpatient follow up visit and successful follow up by the patient has been associated with lowered readmission rates (Dickson et al. 2022). While literature on provider characteristics that affect care transitions is lacking, available literature demonstrates that provider knowledge, understanding, and engagement in effective transition practices influences the quality-of-care transition (Dickson et al. 2022). The findings on the association of patient characteristics and readmission are consistent with that previously discussed (Dickson et al. 2022).
Patients with severe psychiatric conditions are at higher risk for readmission. There are many risk factors associated with readmission for psychiatric patients. Factors such as age, sex, and psychiatric diagnoses are examples of risk factors that providers and health systems can look to for opportunities to identify interventions to support patients and minimize the risk of avoidable readmissions. Providers and facilities should be aware of trends and interventions that are more strongly associated with reducing readmission rates, such as improving care transitions, improving patient-centered care experiences during index admissions at IPFs, and improving patient education.
References
Becker MA, Boaz TL, Andel R, et al. Risk of Early Rehospitalization for Non-Behavioral Health Conditions Among Adult Medicaid Beneficiaries with Severe Mental Illness or Substance Use Disorders. J Behav Health Serv Res. 2017 Jan;44(1):113-121. https://doi.org/10.1007/s11414-016-9516-9
Buck JA, Lowenstein L. Psychiatric Facility Readmissions of Medicare Inpatients. Psychiatr Serv. 2022 Mar 1;73(3):361. https://doi.org/10.1176/appi.ps.202100105
Dickson KS, Sklar M, Chen SZ, et al. Characterization of multilevel influences of mental health care transitions: a comparative case study analysis. BMC Health Serv Res. 2022 Apr 2;22(1):437. https://doi.org10.1186/s12913-022-07748-2
Jesus TS, Stern BZ, Lee D, et al. Systematic review of contemporary interventions for improving discharge support and transitions of care from the patient experience perspective. PLoS One. 2024 May 21;19(5):e0299176. https://doi.org/10.1371/journal.pone.0299176
Jiang HJ, Hensche MK. Characteristics of 30-Day All-Cause Hospital Readmissions, 2016–2020. HCUP Statistical Brief #304. Rockville, MD: Agency for Healthcare Research and Quality; September 2023. https://hcup-us.ahrq.gov/reports/statbriefs/sb304-readmissions-2016-202…
Kim B, Weatherly C, Wolk CB, et al. Measurement of unnecessary psychiatric readmissions in the context of care transition interventions: a scoping review. BMJ Open. 2021 Feb 8;11(2):e045364. https://doi.org/10.1136/bmjopen-2020-045364
Mark TL, Tomic KS, Kowlessar N, et al. Hospital readmission among medicaid patients with an index hospitalization for mental and/or substance use disorder. J Behav Health Serv Res. 2013 Apr;40(2):207-21. https://doi.org/10.1007/s11414-013-9323-5. Erratum in: J Behav Health Serv Res. 2015 Apr;42(2):273. https://doi.org/10.1007/s11414-013-9335-1. Mark, Tami [corrected to Mark, Tami L]. PMID: 23430287
Meurs EAIM, Siegert CEH, Uitvlugt E, et al. Clinical characteristics and risk factors of preventable hospital readmissions within 30 days. Sci Rep. 2021 Oct 11;11(1):20172. https://doi.org/10.1038/s41598-021-99250-8
Muhammad N, Talpur S, Sangroula N, et al. Independent Predictors of 30-Day Readmission to Acute Psychiatric Wards in Patients With Mental Disorders: A Systematic Review and Meta-Analysis. Cureus. 2023 Jul 26;15(7):e42490. https://doi.org/10.7759/cureus.42490
Owusu E, Oluwasina F, Nkire N, et al. Readmission of Patients to Acute Psychiatric Hospitals: Influential Factors and Interventions to Reduce Psychiatric Readmission Rates. Healthcare (Basel). 2022 Sep 19;10(9):1808. https://doi.org/10.3390/healthcare10091808
Pugh J, Penney LS, Noël PH, Neller S, Mader M, Finley EP, Lanham HJ, Leykum L. Evidence based processes to prevent readmissions: more is better, a ten-site observational study. BMC Health Serv Res. 2021 Mar 1;21(1):189. https://doi.org/10.1186/s12913-021-06193-x.
Rammohan R, Joy M, Magam SG, et al. The Path to Sustainable Healthcare: Implementing Care Transition Teams to Mitigate Hospital Readmissions and Improve Patient Outcomes. Cureus. 2023 May 15;15(5):e39022. https://doi.org/10.7759/cureus.39022
Shields MC, Hollander MAG, Busch AB, et al. Patient-centered inpatient psychiatry is associated with outcomes, ownership, and national quality measures. Health Aff Sch. 2023 Jun 20;1(1):qxad017. https://doi.org/10.1093/haschl/qxad017
Smeraglio A, Heidenreich PA, Krishnan G, et al. Patient vs provider perspectives of 30-day hospital readmissions. BMJ Open Quality 2019;8:e000264. doi:10.1136/
Tyler N, Wright N, Waring J. Interventions to improve discharge from acute adult mental health inpatient care to the community: systematic review and narrative synthesis. BMC Health Serv Res. 2019 Nov 25;19(1):883. https://doi.org/10.1186/s12913-019-4658-0
Winterstein AG, Bussing R, Goodin A, et al. Development and Validation of an Administrative Claims-based Measure for All-cause 30-day Risk-standardized Readmissions After Discharge From Inpatient Psychiatric Facilities. Med Care. 2020 Mar;58(3):225-233. https://doi.org/10.1097/MLR.0000000000001275
Zhou H, Ngune I, Albrecht MA, et al. Risk factors associated with 30-day unplanned hospital readmission for patients with mental illness. Int J Ment Health Nurs. 2023 Feb;32(1):30-53. https://doi.org/10.1111/inm.13042
Measure Impact
We convened a panel of individuals with lived experience as an admitted patient in an IPF and/or as a caregiver for someone who was a patient (n = 8) to explore the meaningfulness of this measure. Panel members were asked if they strongly agreed, agreed, disagreed, or strongly disagreed that, “The IPF Readmission measure is meaningful and provides information that is important and useful to me.” Respondents were split on the meaningfulness of this measure to them, although seven agreed unplanned readmissions reflect quality of care from the IPF . Three of the eight participants agreed it was meaningful, two disagreed, and three strongly disagreed. Participants who disagreed or strongly disagreed noted that while unplanned readmission may be an indicator of poor quality of care, there are a broad range of issues beyond the control of a specific facility that may contribute to an unplanned readmission, including challenges in accessing outpatient services, unstable housing, or systemic barriers such as insurance policies. Since this is a Medicare measure, the feedback related to commercial health insurance is not relevant. One participant who agreed with the statement noted that when combined with other IPF measures on Care Compare, this measure can be an indicator of the quality of care at an IPF. Participants were also asked if they strongly agreed, agreed, disagreed, or strongly disagreed that “The IPF Readmission measure provides information that is useful to me in making care decisions.” Two of the eight participants agreed that it did, four disagreed, and two strongly disagreed. Two participants noted value in the measure under specific conditions. For instance, individuals who use psychiatric advance directives or who have prior experience with particular facilities might find the readmission data relevant for future planning.
Performance Gap
We compared the national performance rates for the IPF Readmission measure from fiscal year (FY) 2019 (that is, the performance period of 7/1/2015 through 6/30/2017) to FY 2025 (that is, the performance period of 7/1/2021 through 6/30/2023). This data is publicly reported from CMS in the Provider Data Catalog (Source: https://data.cms.gov/provider-data/archived-data/hospitals) and is also available to facilities on Care Compare (Source: https://www.medicare.gov/care-compare/?providerType=Hospital). The national performance rate for the IPF Readmission measure maintained an average of 20.1 percent for FY 2019 through FY 2023, however, the national performance rate has declined slightly (19.6 percent in FY 2024, and 19.4 percent in FY 2025), which indicates better measure performance nationwide. Please see Table 1a: Number of IPFs with performance worse, same and better compared to the national average for FY2019 through FY2025.
We additionally assessed the FY 2025 claims data covering the period of July 1, 2021, through June 30, 2023, for gaps in performance by IPF. We used this same data in our testing activities and describe our findings throughout this form. This data included a total of 1,547 IPFs and 279,579 discharges. Out of the 1,547 IPFs, 1,410 met the case minimum of 25 eligible discharges, whose distribution of RSRRs are shown in Table 1. The row titled N of Persons/Encounters/Episodes reflects the number of eligible discharges for IPFs that meet the minimum number of eligible discharges. Although many IPFs have RSRRs below the overall national rate of 19.4 percent, there are still many with scores above this rate, suggesting that there is still room for improvement.
| Overall | Minimum | Decile_1 | Decile_2 | Decile_3 | Decile_4 | Decile_5 | Decile_6 | Decile_7 | Decile_8 | Decile_9 | Decile_10 | Maximum | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Mean Performance Score | 19.3% | 13.2% | 15.2% | 16.7% | 17.5% | 18.2% | 18.7% | 19.4% | 20.0% | 20.8% | 21.9% | 24.5% | 31.3% |
| N of Entities | 1410 | 1 | 141 | 141 | 141 | 141 | 141 | 141 | 141 | 141 | 141 | 141 | 1 |
| N of Persons / Encounters / Episodes | 278203 | 242 | 28012 | 25662 | 26312 | 25219 | 25812 | 27229 | 28154 | 28235 | 28205 | 35363 | 334 |
Care Gaps
Closing Care Gaps
This question was optional on the measure submission form; therefore, we did not include these testing results.
Feasibility
Feasibility
The IPF Readmission measure is claims-based, therefore IPFs generate the required data during routine patient care delivery processes and submit this information within their Medicare claims. IPF Readmission scores are generated using claims data, which are stored in structured table formats electronically within the CMS claims databases. Because claims data are used for processing measure scores, the risk of including missing or inaccurate data elements is minimized.
The IPF Readmission measure is a low burden to IPFs because it uses Medicare claims and does not require IPFs to do any additional work to calculate or report the measure. Claims-based measures are feasible to implement comprehensively and efficiently for large populations of patients. Data from claims-based measures are readily available, making the IPF Readmission measure feasible to implement and report in the IPFQR Program. Our testing proved that it is feasible to calculate the IPF Readmission measure, as specified.
CMS’s cell size suppression policy states that no cell (e.g. admissions, discharges, patients, enrollment, services, etc.) containing a value of 1 to 10 can be directly reported, and a value of zero does not violate the minimum cell size policy (Centers for Medicare & Medicaid Services, 2020). Therefore, patient confidentiality is not a concern for the IPF Readmission measure since measure scores are reported in aggregate at the IPF-level and only for facilities meeting the minimum case threshold of 25 eligible discharges.
Source: CMS Cell Suppression Policy | Guidance Portal https://resdac.org/articles/cms-cell-size-suppression-policy
There have been no changes to the measure as a result of the feasibility assessment.
Proprietary Information
Scientific Acceptability
Testing Data
The IPF Readmission measure was tested using Medicare beneficiary and coverage files, Medicare FFS Part A and B records. Both administrative and claims data were used for testing.
For measure calculation, the following Medicare files are required for the IPF Readmission measure:
- Medicare beneficiary and coverage files. Provides information on patient demographic, enrollment, and vital status to identify the measure population and certain risk factors.
- Medicare FFS Part A records. Contains final actions claims submitted by acute care and critical access hospitals, IPFs, home health agencies, and skilled nursing facilities to identify the measure populations, readmissions, and certain risk factors.
- Medicare FFS Part B records. Contains final action claims submitted by physicians, physician assistants, clinical social workers, nurse practitioners, and other providers to identify certain risk factors. For the IPF Readmission measure, claims for services such as laboratory tests, medical supplies, or other ambulatory services were not used. This ensures that diagnoses result from an encounter with a provider trained to establish diagnoses and not from a claim for a diagnostic test.
Index admissions and readmission are identified in the Medicare Part A claims data. Comorbid conditions for risk adjustment are identified in the Medicare Part and Part B data in the 12 months prior to and including index admission. Demographic and FFS enrollment information is identified in the Medicare beneficiary and coverage files. Adult IPF admissions with admission and discharges between July 1, 2021, through June 30, 2023, were used to calculate the measure performance rate.
Testing for this measure consists of data from June 1, 2021 through July 31, 2023.
Performance gap analyses included 1,410 IPFs with 25 or more discharges during the measurement period (July 1, 2021, through July 30, 2023) (n = 278,203 discharges total).
Scientific acceptability analyses included 1,410 IPFs (from the universe of 1,547 IPFs) with 25 or more discharges during the measurement period (July 1, 2021, through July 30, 2023) (n = 278,203 discharges total).
Risk adjustment analyses used the 1,547 IPFs with at least one eligible discharge during the measurement period (July 1, 2021, through July 30, 2023), which included 279,579 discharges total. We tested two patient-level variables (sex and age), 13 principal diagnosis categories, 35 clinical comorbidities, three patients’ claims history data (history of discharge AMA, and suicide history and aggression history).
This measure was tested at the facility level (IPF).
Risk adjustment testing used Medicare claims data for patients discharged from any of the 1,547 IPFs participating in the IPFQR Program during FY 2025 (that is, during the measurement period of July 1, 2021, through June 30, 2023). Of those 1,547 IPFs, 590 (38.14 percent) were free-standing facilities while 957 (61.86 percent) were from psychiatric units within a larger healthcare facility.
Of the 1,547 IPFs, 137 (8.86 percent) had fewer than the 25 eligible discharge threshold, leaving 1,410 (91.14 percent) of facilities with sufficient volume for reporting purposes. Of those 1,410 facilities, 72 were considered large, with over 500 eligible discharges. Scientific acceptability analysis consisted of the 1,410 facilities which met the 25 eligible discharge threshold.
See Table 3. Measured Entities: Inpatient Psychiatric Facilities (IPF's) by Type (Free-Standing vs. Unit) and Table 4. Measured Entities: Inpatient Psychiatric Facilities (IPF's) by volume in the Evidence and Scientific Acceptability Attachment.
To calculate Characteristics of Units of the Eligible Population, we tested the IPF Readmission measure using data from 179,831 patients across 279,579 index admissions across 1,547 IPFs. Among the eligible discharges, about 59.65 percent were for patients 18-64 years old while the remaining 40.35 percent were for patients 65 years or older. Approximately half of the discharges were male (50.94 percent). The majority of principal discharge diagnoses (~79 percent) were among four CCS categories: depressive disorder (22.39 percent), schizoaffective disorder (21.04 percent), psychosis (18.15 percent), or bipolar disorder (17.89 percent). See Table 5. Patient characteristics of eligible discharges used in testing for the IPF Readmission measure.
Reliability
We estimated measure reliability via the intra-class correlation coefficient (ICC), a reliability coefficient that reflects both correlation and agreement between measurements. We used a test-retest approach that examines the agreement between randomly selected subsamples of each IPF’s population during the same time period. We used a bootstrapping approach to generate independent samples of patients within the same IPF. The randomly sampled sets of admissions from a given hospital are assumed to reflect an independent set of re-measurement of readmission rates for the hospital. Adequate reliability is assumed if the risk-standardized measure rates calculated from the random datasets for the same IPF are similar. Higher ICC values indicate stronger agreement between measure scores in the samples and better measure reliability.
For the bootstrapping approach, we sampled 1,000 samples from the original measure cohort with replacement (stratified sampling by IPF), maintaining the sample size of a two-year measure within each IPF. For each of the bootstrap samples, we calculated the risk-standardized readmission rate (RSRR), thereby producing 1,000 RSRR’s for each hospital. Using the 1,000 bootstrap sample RSRR’s, we then calculated reliability for each individual IPF using the ICC formula. See the Reliability testing attachment for Full descriptions of reliability testing methods, including formulas.
As shown in Table 2, facility-level ICC reliability estimates obtained from the 1,000 bootstrapped samples ranged from 0.2632 to 0.948. The mean and median reliability estimates are 0.6629 and 0.6840302 respectively. Among the 1,410 facilities meeting the volume threshold of at least 25 discharges, 69.65% had ICC reliability of 0.60 or higher.
Table 2 provides the distribution of the ICC reliability estimates which are grouped by decile of hospital volume, or number of eligible discharges, for the 1,410 facilities with at least 25 discharges. The “reliability” row of this table contains the mean ICC estimate among hospitals in the first volume decile, second volume decile, and so on. Similarly, the “mean performance score” row of this table contains the mean risk standardized readmission rate (RSRR) among hospitals in the first volume decile, and so on. The row “N of Persons/Encounters/Episodes” of this table contains the total eligible discharges among hospitals in each decile group. The minimum and maximum columns of this table contain the mean reliability, mean RSRR, etc. of the hospital(s) with the minimum and maximum volume, respectively.
The ICC captures the effect of the IPF on the beneficiaries’ outcomes (RSRR) and could be interpreted as the correlation in the outcome between individuals randomly selected from the same IPF. There are no standard values for acceptable reliability using ICC. A low ICC could not only reflect the low degree of agreement but could also relate to the small number of subjects. Following Portney and Watkins, we rely on the following interpretation: ICC values less than 0.5 are indicative of poor reliability, values between 0.5 and 0.75 indicate moderate reliability, values between 0.75 and 0.9 indicate good reliability, and values greater than 0.90 indicate excellent reliability. The mean ICC of 0.66 obtained from the bootstrapping approach is considered moderate. The median ICC was 0.68, and 69.7% of hospitals had ICC reliability of 0.60 or higher.
References
Portney, L.G. and Watkins, M.P. (2009) Foundations of Clinical Research: Applications to Practice. 3rd Edition, Pearson Education, Inc., New Jersey.
| | Overall | Minimum | Decile_1 | Decile_2 | Decile_3 | Decile_4 | Decile_5 | Decile_6 | Decile_7 | Decile_8 | Decile_9 | Decile_10 | Maximum |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Reliability | 0.66 | 0.26 | 0.35 | 0.48 | 0.56 | 0.62 | 0.66 | 0.70 | 0.75 | 0.79 | 0.83 | 0.88 | 0.95 |
| Mean Performance Score | 19.3% | 18.1% | 18.8% | 19.1% | 19.0% | 19.2% | 19.3% | 19.1% | 19.2% | 19.7% | 19.8% | 19.7% | 17.7% |
| N of Entities | 1410 | 4 | 140 | 141 | 141 | 143 | 140 | 142 | 140 | 141 | 141 | 141 | 1 |
| N of Persons / Encounters / Episodes | 278203 | 100 | 5471 | 9093 | 12810 | 16448 | 19462 | 23547 | 28836 | 36327 | 47590 | 78619 | 1277 |
Validity
To assess validity, we examined the correlation with related measures and assessed for known-group validity.
Correlation with related measures. We examined validity of the IPF Readmission measure by analyzing the Spearman correlation between results from the IPF Readmission measure and conceptually-related Inpatient Psychiatric Facility Quality Reporting (IPFQR) measures: 1) 30-Day Risk-Standardized All-Cause Emergency Department Visit Following an Inpatient Psychiatric Facility (IPF-30 Day ED Visit Measure; CBE #4190), 2) 7-Day and 30-Day Follow-Up After Psychiatric Hospitalization (FAPH; CBE #0576), and 3) the Medication Continuation Following Inpatient Psychiatric Discharge measure (CBE #3205). We calculated the Spearman rank correlations of the IPF Readmission measure with three measures. We expected the IPF Readmission scores to be negatively correlated with the FAPH and Medication Continuation scores because readmissions may indicate a lack of care coordination, and higher IPF Readmission scores indicate lower quality. Our analysis of the peer-reviewed literature similarly suggests that patients with low and intermediate adherence to medication have higher readmission rates compared to patients with high adherence (Rosen et al. 2017). We expected positive correlation between the IPF Readmission and IPF ED Visit measures because patients who visit the ED within 30 days after discharge are more likely to be readmitted. Previous studies showed that approximately one in five patients presented to the ED within 30 days of an inpatient hospitalization and over half of these patients were readmitted (Gentil et al., 2021).
Hypothesis-based validity. A measure demonstrates validity if the measure scores could be used to discriminate between subgroups of patients with expected differences in the outcome based on peer-reviewed literature. We investigated validity by evaluating differences in the mean IPF Readmission rates among groups of patients based on the evidence from peer-reviewed publications on psychiatric readmissions. Consistent with the literature, we hypothesized readmission rates to be higher among males, patients with substance abuse disorder, patients with schizophrenia, and patients with shorter length of stay (LOS) at the IPF (Rienke et al., 2016; Mark et al., 2013; Carr et al., 2008).
To test for differences in IPF Readmission measure rates by patient subgroups, we compared the mean observed, predicted, and expected readmission rates for each subgroup of beneficiaries. Observed readmission rate is the percentage of IPF readmissions during the measurement period that were followed by an unplanned readmission to an IPF within 30 days. Predicted rate of readmissions is an estimated number of readmissions based on the IPF’s performance and its observed case mix. The expected rate of readmissions is based on the national observed readmission rate and the IPF’s observed case mix.
We used t-tests to compare mean group differences. With large sample sizes, small differences that are statistically significant may not always be practically or clinically meaningful. Therefore, we also computed Cohen's d effect size (the difference in mean scores divided by the pooled standard deviation). Following Cohen’s 1988 definitions, we defined effect size values for dichotomous variables as small (0.2), medium (0.5), or large (0.8) (Cohen 1988). For the ordered-categorical variables, we tested the difference in the mean IPF readmission rates between the 1st and 4th quartiles.
In our hypothesis-driven validity testing, we used unadjusted performance rates (i.e., observed measure rates), as risk-standardization reduces the variability in the distribution of measure scores and could obscure differences between subgroups of patients.
References
Appleby, L., Luchins, D. J., Desai, P. N., Gibbons, R. D., Janicak, P. G. & Marks, R. (1996) Length of inpatient stay and recidivism among patients with schizophrenia. Psychiatric Services: A Journal of the American Psychiatric Association, 47 (9), 987–990.
Burke RE, Donzé J, Schnipper JL. Contribution of psychiatric illness and substance abuse to 30-day readmission risk. J Hosp Med. 2013 Aug;8(8):450-5. doi: 10.1002/jhm.2044. Epub 2013 Apr 16. PMID: 23589474.
Carr VJ, Lewin TJ, Sly KA, Conrad AM, Tirupati S, Cohen M, et al. Adverse incidents in acute psychiatric inpatient units: rates, correlates and pressures. AustNz J Psychiat. 2008;19:267–82.
Carson, N. J., Vesper, A., Chen, C. N., & Cook, B. L. (2014). Quality of follow-up after hospitalization for mental illness among patients from racial-ethnic minority groups. Psychiatric Services, 65(7), 888–896. https://doi.org/10.1176/appi.ps.201300139
Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers.
Fontanella, C. A., Hiance-Steelesmith, D. L., Bridge, J. A., Lester, N., Sweeney, H. A., Hurst, M., & Campo, J. V. (2016). Factors Associated With Timely Follow-Up Care After Psychiatric Hospitalization for Youths With Mood Disorders. Psychiatric Services, 67(3), 324–331. https://doi.org/10.1176/appi.ps.201500104
Fuller, D. A., Sinclair, E., & Snook, J. (2016). Released, Relapsed, Rehospitalized. Length of Stay and Readmission Rates in State Hospitals A Comparative State Survey Executive Summary. Retrieved from https://www.treatmentadvocacycenter.org/storage/documents/released-rela…
Gentil L, Grenier G, Fleury MJ. Factors Related to 30-day Readmission following Hospitalization for Any Medical Reason among Patients with Mental Disorders: Facteurs liés à la réhospitalisation à 30 jours suivant une hospitalisation pour une raison médicale chez des patients souffrant de troubles mentaux. Can J Psychiatry. 2021 Jan;66(1):43-55. doi: 10.1177/0706743720963905. Epub 2020 Oct 16. PMID: 33063531; PMCID: PMC7890589.
Heeren, O., Dixon, L., Gavirneni, S., & Regenold, W. T. (2002). The Association Between Decreasing Length of Stay and Readmission Rate on a Psychogeriatric Unit. Psychiatric Services, 53(1), 76–79. https://doi.org/10.1176/appi.ps.53.1.76
Hung, Y. Y., Chan, H. Y., & Pan, Y. J. (2017). Risk factors for readmission in schizophrenia patients following involuntary admission. PLoS ONE, 12(10), 1–11. https://doi.org/10.1371/journal.pone.0186768
Joynt, K. E., Orav, E. J., & Jha, A. K. (2011). Thirty-day readmission rates for medicare beneficiaries by race and site of care. JAMA - Journal of the American Medical Association, 305(7), 675–681. https://doi.org/10.1001/jama.2011.123
Kalseth, J., Lassemo, E., Wahlbeck, K., Haaramo, P., & Magnussen, J. (2016). Psychiatric readmissions and their association with environmental and health system characteristics: A systematic review of the literature. BMC Psychiatry, 16(1), 1–9. https://doi.org/10.1186/s12888-016-1099-8
Lorant, V., Kampfl, D., Seghers, A., Deliege, D., Closon, M.-C., & Ansseau, M. (2003). Socio-economic differences in psychiatric in-patient care. Acta Psychiatrica Scandinavica, 107(3), 170–177. https://doi.org/10.1034/j.1600-0447.2003.00071.x
Madi N, Zhao H, Li JF. Hospital readmissions for patients with mental illness in Canada. Healthc Q. 2007;10(2):30-2. doi: 10.12927/hcq.2007.18818. PMID: 17491564.
Malla, A., Norman, R., Bechard-Evans, L., Schmitz, N., Manchanda, R., & Cassidy, C. (2008). Factors influencing relapse during a 2-year follow-up of first-episode psychosis in a specialized early intervention service. Psychological Medicine, 38(11), 1585–1593. https://doi.org/10.1017/S0033291707002656
Marcus S, Chuang CC, Ng-Mak D, Olfson M. (2017). Outpatient Follow-Up Care and Risk of Hospital Readmission in Schizophrenia and Bipolar Disorder. Psychiatric Services, 68(12), 1239–1246. https://doi.org/10.1176/appi.ps.201600498
Mark T, Tomic KS, Kowlessar N, Chu BC, Vandivort-Warren R, Smith S. Hospital readmission among medicaid patients with an index hospitalization for mental and/or substance use disorder. J Behav Health Ser R. 2013;40:207-21
Rieke, K., McGeary, C., Schmid, K. K., & Watanabe-Galloway, S. (2016). Risk Factors for Inpatient Psychiatric Readmission: Are There Gender Differences? Community Mental Health Journal, 52(6), 675–682. https://doi.org/10.1007/s10597-015-9921-1
Rømer Thomsen, K., Thylstrup, B., Pedersen, M. M., Pedersen, M. U., Simonsen, E., & Hesse, M. (2018). Drug-related predictors of readmission for schizophrenia among patients admitted to treatment for drug use disorders. Schizophrenia Research, 195, 495–500. https://doi.org/10.1016/j.schres.2017.09.026
Rosen, O., Fridman, R., Rosen, B., Shane, R., & Pevnick, J. (2017). Medication adherence as a predictor of 30-day hospital readmissions. Patient Preference and Adherence, Volume 11, 801–810. https://doi.org/10.2147/PPA.S125672
Turner JT, Wan TT. Recidivism and mental illness: the role of communities. Community Ment Health J. 1993;29:3–14. doi: 10.1007/BF00760626.
Vigod SN, Kurdyak PA, Dennis CL, Leszcz T, Taylor VH, Blumberger DM, et al. Transitional interventions to reduce early psychiatric readmissions in adults: systematic review. British Journal of Psychiatry. 2013 Mar;202(3):187-94.
Weiss A, Barret M, Heslin K, and Stocks C. Trends in Emergency Department Visits Involving Mental and Substance Use Disorders, 2006–2013. Rockville (MD): Agency for Healthcare Research and Quality (US); 2006
Correlation with established measures. Table 6c. Performance measure score correlation for IPF's meeting volume thresholds for each measure shows the correlation of IPF Readmission scores with the conceptually related IPFQR quality measures. Consistent with our expectations, after restricting the sample to facilities meeting the minimum sample size requirement for both measures (N≥25 for the IPF Readmission measure and N≥40 for the FAPH measure), the scores for the IPF Readmission measure were negatively correlated with the scores for the 7-Day FAPH (ρ = -0.275; p≤0.001) and 30-Day FAPH (ρ = -0.328; p≤0.001) measures.
Similarly, after excluding facilities that did not meet the minimum reporting sample size requirement for the IPF Readmission and Medication Continuation measures N≥25 for the IPF readmission and N≥75 for Medication Continuation the correlation between the measures was negative and statistically significant (ρ = -0.271, p<0.001).
Consistent with our hypothesis, we observed positive correlation between the IPF Readmission and IPF ED Visit measure scores (ρ = 0.348; statistically significant at p<0.001) when we restricted the sample to include facilities meeting the minimum sample size requirement for both measures (N≥25).
According to Cohen's guidelines for correlation (0.10 for small, 0.30 for medium, and 0.50 for large) effect size (strength of the association between the two measures; Cohen, 1969), the size of the observed Spearman correlation coefficients between the IPF Readmission, the 7-day FAPH and Medication Continuation measures corresponds represent a small effect size, and the correlation between the IPF Readmission and IPF ED Visit and 30-day FAPH measures can be interpreted as moderate. The direction of correlations between the IPF Readmission and all three criterion measures were consistent with our hypotheses.
Validity. All differences in the IPF readmission rates by subgroups were in the direction consistent with the literature and our hypotheses (Table 6a. Differences in the mean IPF Readmission rates by beneficiary subgroups). Differences in the IPF Readmission rates by patient subgroups ranged from 0.8 percentage points (pp) to 5.3 pp for the observed and 0.88 pp to 4.7 pp for the predicted and expected rates. For all three types of rates, the smallest differences were observed between beneficiaries with and without Alcohol/SUD disorder.
Consistent with the literature, we observed differences in the IPF Readmission rates by sex (lower rates of readmission for women), presence of schizophrenia diagnosis on admission (lower rates of readmission for beneficiaries without schizophrenia diagnosis), presence of alcohol and substance use disorder on admission (lower rates of readmission for beneficiaries without alcohol and substance use disorder diagnosis), and length of stay (higher rates of readmission for beneficiaries with shorter LOS). Observed differences in the IPF Readmission rates for beneficiaries with and without a substance use disorder (SUD) diagnosis on admission were small.
We also computed Cohen's d standardized effect size (the difference in mean scores divided by the pooled standard deviation across groups) for the differences in the mean IPF Readmission rates by beneficiaries’ subgroups (Table 6b. Effect sizes for differences in group means by beneficiary characteristics). We categorized effect size values for dichotomous variables as small (0.2), medium (0.5), or large (0.8).
We observed small to medium effect sizes for the differences in predicted and expected rates by patient subgroups. The effects ranged from 0.085 to 0.513 and 0.088 to 0.562 for predicted and expected rates, respectively.
For the observed rates, effects were smaller, ranging from 0.021 to 0.135. Smaller effects for the observed rates were due to more variability in the observed readmission rates, compared to the variability in the predicted and expected rates (Table 6a. Differences in the mean IPF Readmission rates by beneficiary subgroups). This is largely due to the shrinkage effect in hierarchical logistic regression which reduces the influence of unstable and noisy estimates for low-volume facilities (Clark et al. 2010; Quality Indicator Empirical Methods 2019). Readmission rates within larger IPFs will tend not to move much with smoothing, even if their rate differs from the reference population rate.
References
Agency for Healthcare Research and Quality. (2019) Quality Indicator Empirical Methods.
Retrieved from http://www.qualityindicators.ahrq.gov.
Clark DE, Hannan EL, Raudenbush SW. Using a hierarchical model to estimate risk-adjusted mortality for hospitals not included in the reference sample. Health Serv Res. 2010 Apr;45(2):577-87. doi: 10.1111/j.1475-6773.2009.01074.x. Epub 2010 Jan 8. PMID: 20070388; PMCID: PMC2838162.
Cohen, J. (1969) Statistical Power Analysis for the Behavioral Sciences. Academic Press, 101-105.
Fontanella, C. A., Hiance-Steelesmith, D. L., Bridge, J. A., Lester, N., Sweeney, H. A., Hurst, M., & Campo, J. V. (2016). Factors Associated With Timely Follow-Up Care After Psychiatric Hospitalization for Youths With Mood Disorders. Psychiatric Services, 67(3), 324–331. https://doi.org/10.1176/appi.ps.201500104
Figueroa, R., Harman, J. & Engberg, J. (2004) Use of claims data to examine the impact of length of inpatient psychiatric stay on readmission rate. Psychiatric Services: A Journal of the American Psychiatric Association, 55 (5), 560–565.
As shown in Table 6c. Performance measure score correlation for IPF's meeting volume thresholds for each measure, we found multiple instances supporting validity for the IPF Readmission measure. There is a small to moderate, yet meaningful, inverse relationship between the facilities’ rates on the IPF Readmission measure and the 7-day rate for the Follow-Up After Psychiatric Hospitalization (FAPH) measure (ρ = -0.275; p≤0.001), the 30-day rate for the FAPH measure (ρ = -0.328; p≤0.001), and the Medication Continuation Following Inpatient Psychiatric Hospitalization in an IPF (MedCont) measure (ρ = -0.271, p<0.0001). An inverse relationship between the measure rates indicates an increase in follow up visit rates within 7- and 30-days post discharge and adherence to the antipsychotic medications can lead to the reduction in the unplanned readmissions to an IPF within 30-days post discharge from an IPF.
Positive correlation between the IPF Readmission and IPF ED Visit measure scores (ρ = 0.348; statistically significant at p<0.001) suggests that patients who are more likely to visit the ED within 30-days post discharge from an IPF are also more likely to be readmitted to IPFs within 30-days. Moderate correlation between the IPF Readmission and IPF ED Visit measures may suggest inadequate discharge planning and/or outpatient management for patients with psychiatric conditions. This could also involve issues with medication adherence, access to timely follow-up care, or effective symptom management strategies post discharge. It is possible that patients with more severe psychiatric conditions may have higher rates of both ED visits and IPF readmissions. However, even in these cases, there may be opportunities for IPFs to improve care and reduce the frequency of readmissions.
As shown in Table 6b. Effect sizes for differences in group means by beneficiary characteristics, we observed small to medium effect sizes for the differences in predicted and expected rates by patient subgroups, and small effect sizes for the differences in the observed rates. As described above, there is less variation in the distribution of the predicted and expected rates due to the shrinkage of the corresponding rates to the national mean in hierarchical logistic regression. This, in turn, results in larger effect sizes (Cohen’s d) for the differences in rates. Therefore, for practical reasons, it is more useful to focus on the interpretation of the effect sizes for the observed (unadjusted) rates. To put the differences in the measure performance rates into context by beneficiaries’ subgroups:
- There is a difference of 4.0 percentage points (pp) between the mean readmission rate for the male and female beneficiaries, with male having high average readmission rates than females.
- There is a difference of 4.3 pp between the mean IPF readmission rate for beneficiaries with and without a schizophrenia diagnosis. Beneficiaries with a schizophrenia diagnosis have higher average readmission rates than those without a schizophrenia diagnosis.
- There is a difference of 0.8 pp between the mean IPF readmission rate for beneficiaries with and without Alcohol and Substance Use Disorder (SUD) diagnosis. Beneficiaries with Alcohol and SUD diagnosis have higher average readmission rates than those without.
- There is a difference of 3.7 pp between the mean IPF readmission rates for beneficiaries with long LOS (4th quartile) versus short LOS (1st quartile). On average, beneficiaries with shortest length of stay (1st quartile on the LOS variable) tend to have higher average readmission rates than those with the longest hospital stay (4th quartile on the LOS variable).
Risk Adjustment
The IPF Readmission measure is risk-adjusted because some patients have higher risks of experiencing readmission, regardless of the IPF’s quality of care (e.g., due to patients' preexisting conditions).
The conceptual model for risk adjustment considers patients’ sex, age, and clinical risk factors. The key criteria for the risk factors include:
- Observability and validity using administrative data across hospitals
- Clinical relevance to the outcome (readmission to an IPF)
- Adequate prevalence to obtain reasonably precise estimates of risk
- Systematic variation by hospital
The approach to risk adjustment is to consider as many risk factors as theoretically justified and computationally practical, and to apply a feature selection technique to select an optimal set of risk factors among the candidate variables before fitting a final risk adjustment model (see Figure 2. Conceptual model for patient risk factors that affect readmission following hospitalization for more details). The same set of selected risk factors is used in production each year, and coefficients are recalculated for each feature annually.
Table 7. Analysis of candidate risk factor frequencies using CCS categories of the evidence attachment for the distribution of the frequencies and readmission rates of all candidate risk variables. This table also shows whether the risk factor was used in the model.
The candidate risk factors for each measure included sex, age category, and both psychiatric and non-psychiatric problems that may necessitate readmission. The definition of the measure and construction of the risk-adjustment model are consistent with established standards for outcome measurement defined in the CBE guidance for outcome measures, the CMS Measures Management System guidance, and the American Heart Association scientific statement on statistical modeling of outcome measures. In ascertaining risk factors, we paid particular attention to both sensitivity and specificity by including diagnoses from outpatient billing records, which captured a variety of non-psychiatric comorbidities not recorded in the index admission claims. To ensure that the diagnoses assigned to outpatient encounters truly captured the manifestation of a disease as opposed to diagnostic work-up, we restricted outpatient claims to those with evaluation and management procedure codes and required a minimum of two claims with diagnoses within the same condition category (CC) grouping.
We then derived a parsimonious risk adjustment model by using logistic regression, which was repeated in 1,000 bootstrap samples from the entire population via random selection with replacement, and evaluated the percentage of samples in which the risk factor was statistically significant to the model. Those risk factors which were statistically significant with a p-value < 0.15 in at least 70% of the 1,000 bootstrap samples were selected for the final model.(See Table 8. Analysis of candidate risk factor frequencies using CCS categories). The candidate risk-factors included 74 variables excluding individual categories for categorical variables. The candidate risk-factors included 2 demographic variables (sex and age), 13 principal diagnosis categories, 35 comorbidities (26 were selected), 3 patients’ claims history data (history of discharge AMA, suicide history and aggression history), and 21 additional variables. The final IPF readmission measure includes 44 risk factors.
See Table 7. Analysis of candidate risk factor frequencies using CCS categories for the resulting list of risk factors selected for inclusion, coefficient estimates, and their 95% confidence intervals.
We assessed model performance using Hosmer-Lemeshow goodness-of-fit tests, calibration curves, and the concordance-statistic (C-statistic). Hosmer-Lemeshow tests divide patients into deciles (i.e., equal number of patients) based on the expected risk for 30-day readmission, from lowest to highest risk (See Table 9b. Results of the Hosmer-Lemeshow test for the risk-adjustment model). The range of expected risks of readmission within each decile is determined by the patients in that decile. The difference between the observed and expected readmissions for each decile is summarized by the Pearson chi-square statistic. The statistics are then summed over the ten deciles and are compared to the chi-square distribution. In addition, we assessed calibration using the calibration graph plotting observed versus predicted IPF readmission rates (Figure 3. Risk-decile calibration plot). In decile assessment, we should see similar numbers of observations in each decile group and increasing observed rates when we move from low to high deciles. We assessed model discrimination using the C-statistic, which reflects how accurately the model is able to distinguish between an index admission that does or does not have a readmission. A C-statistic of 0.5 represents random prediction and a C-statistic of 1.0 represents perfect prediction.
C-statistic for the risk-adjustment indicates moderate discrimination (0.658) comparable to other CBE-endorsed readmission measures developed for other settings (Readmission Measures Methodology, 2020).
The risk-decile calibration plot with observed outcomes versus expected probabilities of readmission was computed to localize possible deviations across risk strata. In the risk-decile calibration plot, the diagonal line is the line of perfect calibration. In a well-calibrated model, all markers representing deciles should be close to the diagonal line. In this graph, the markers appear close to the diagonal line, which indicates a close agreement between the observed and expected probabilities of the IPF readmission.
Risk adjustment model performance parameters showed excellent calibration with no indication of over-fitting. The mean observed IPF readmission rate range from 31.3 percent observed 30-day readmission rate in the highest decile to 13.2 percent in the lowest decile, an absolute difference of 18.1 percent, suggests good discrimination (See Table 9a. Distribution of the observed and risk-adjusted score. The ratio of observed to predicted IPF readmission rates is close to 1.0 for each decile, suggesting adequate calibration of the model. The C-statistic of 0.673 suggests moderate predictive discrimination, expressed as the model’s ability to distinguish between index admissions that are and are not followed by a readmission. Statistical findings of excellent calibration are confirmed when comparing observed to predicted probabilities by risk deciles. The results are in-line with the other NQF-endorsed readmission measures developed for other settings, such as Hospital 30-Day Heart Failure Readmission measure (0.601); Hospital 30-Day Pneumonia Readmission Measure (0.630); Hospital 30-Day Acute Myocardial Infarction Readmission Measure (0.630); Hospital-Wide Readmission Measure (0.64 to 0.71) (Source: Readmission Measures Methodology (2020). Available at: https://qualitynet.cms.gov/inpatient/measures/readmission/methodology. Accessed on, March 18, 2025).
References
Hosmer DW, Lemeshow S. Confidence interval estimates of an index of quality performance model based on logistic regression. Statistics in Med. 1995;14(19):2161-72.
We can interpret the risk factors included in the final risk adjustment model as the set of demographic and clinical risk-variables that provide an optimal balance between model complexity and performance. These final risk adjustment models showed adequate discriminative ability (C-statistics of 0.658) and calibration (decile plots showing approximately equal observed to expected risks across most deciles). These results suggest that the risk adjustment model can adequately predict patients’ risk of readmission using the patient or discharge characteristics (i.e., risk factors) included in the risk adjustment models. The model discrimination, as measured by the C-statistic, is in-line with the other CBE-endorsed readmission measures developed for other settings. See Table 9c. Final Model Risk Adjustment C-statistics.
Use & Usability
Use
The Inpatient Psychiatric Facility Quality Reporting (IPFQR) program was developed as mandated by section 1886(s)(4) of the Social Security Act, as added and amended by Sections 3401(f) and 10322(a) of the Affordable Care Act (Pub.L. 111-148).
The IPFQR pay-for-reporting program is intended to equip consumers with quality of care information to make more informed decisions about healthcare options. It is also intended to encourage hospitals and clinicians to improve the quality of inpatient care provided to beneficiaries by, first, ensuring that providers are aware of and reporting on best practices for their respective facilities and type of care.
The Inpatient Psychiatric Facility Quality Reporting (IPFQR) program is a nationwide program with
1,474 facilities participating in the program during Fiscal Year 2025. All eligible facilities that are paid under the Medicare Inpatient Psychiatric Facilities Prospective Payment System (IPF PPS) are required to participate in the IPFQR program.
The IPFQR program is a facility-level reporting program for inpatient psychiatric facilities.
Usability
There is no additional data submission for inpatient psychiatric facilities (IPFs) to report on this measure since the data used to calculate performance is based upon IPF claims already submitted to CMS. Multiple factors are associated with increased risk of readmission for patients with psychiatric conditions. These factors typically fall into categories of modifiable and non-modifiable risk factors. For IPFs to have an impact on readmission rates for this population they need to identify and address the modifiable factors associated with readmission. Interventions focused on patient centeredness and care transitions are typically the most successful and include teaching patients about their condition, self-management techniques and medications; developing positive therapeutic relationships with patients; increasing the continuity of care from the inpatient setting to the outpatient setting; establishing relationships between the IPF and community resources available to the patient after discharge; and scheduling an initial follow up visit with an outpatient clinic within seven days of discharge.
The first step toward reducing readmission rates is determining the facility’s current readmission rate. Efforts to reduce the readmission rate begin with acknowledging whether a facility’s readmission rate is higher than that of similar high performing IPFs. National level, state level and individual IPF performance rates for this measure are available on the CMS Care Compare Provider Data Catalog in the Inpatient Psychiatric Facility Quality Measure Data downloadable data sets. The IPF needs to understand that while bringing a readmission rate consistently down to zero is not realistic due to factors outside of the control or influence of the IPF, there is usually room for improvement. The degree of improvement that can be achieved may vary and be impacted by the baseline readmission rate, factors contributing to readmission, facility specific internal workflows and resources, availability of community resources and patient risk factors. The IPF will need to do an assessment of current patient care, discharge preparation, and discharge workflow and processes. Comparing these findings to evidence-based best practices can help the IPF identify where gaps may exist. Based on this information the facility can identify data driven measurable goals for reducing readmission rates.
A commitment needs to be made by the IPF to improve readmission rates. The commitment needs to be at all levels of the facility from the highest level of leadership to the caregivers closest to the patient’s bedside and include clinical and non-clinical individuals. The commitment to improvement requires an intention to make changes that will result in improvement, attention to the details that may impact each individual patient’s care, and investment in the resources and time to implement the changes that will lead to improvement in care and ultimately reduce readmission rates. To successfully implement meaningful sustainable changes the people implementing the changes need to see and feel the commitment of the entire patient care team and system supporting the team. This is the part of the process that if not properly supported and maintained most often leads to either inability to improve or inability to sustain improvement. Establishing a multi-disciplinary improvement team and identifying a clinical champion for the improvement efforts are essential. This helps to establish roles and chains of responsibility and accountability to help drive the improvement.
Providing education to providers and patient care givers is a crucial step in establishing the understanding that a readmission issue exists and that it is something that can be improved upon. Once it has been established with the care team that a readmission problem exists providing the care team with education about the evidence-based, best practices and interventions that can lead to reducing readmission is important. This helps establish a mutual understanding and is key to getting everyone on board with supporting readmission improvement. Despite best efforts there may be individuals who do not believe that readmission rates are too high, may believe that it is a patient problem or someone else’s problem not their problem, and that they cannot do anything about it. This is the point where clinician champions and leadership can support the improvement team. While they may not be able to get everyone on board, they can often establish a commitment from people to not get in the way of the improvement efforts.
Embracing and putting into place a focused group of interventions and a change strategy will help keep the improvement efforts on track and on target. The specific interventions that an IPF focuses on will vary depending on the findings from the assessment of current care and discharge processes and the resources available to address the gaps.
The IPF can encourage providers to start discharge planning at admission and revise it regularly through the stay. The discharge plans should be tailored to patient needs with attention to their mental health needs and status and social support needs. Frequent assessment of what the patient needs to be successful after being discharged and attention to those needs will help patients develop stronger self-management skills and support systems.
Not every patient care provider is an effective teacher, so helping build teaching skills is important. Successful education requires teaching at a level that is understandable and engaging to patients. It requires identifying what motivates the patient and tailoring education to the factors that motivate the patient. Confirming patient understanding is important to determining whether they can follow through with what they are taught after they are discharged. Simple techniques such as having the patient repeat back what they learn in their own words or demonstrating techniques they have learned to the teacher. This allows the teacher to assess the patient’s level of understanding and provide follow-up as needed. When patients understand more about their condition, triggers and limitations along with self-management techniques and appropriate medication adherence they are better able to prevent and manage crisis situations. Engaging family members and individuals that help support patients at home can help with crisis prevention and self-management. Print materials need to be designed with the patient in mind and made simple and easy to understand using plain language. Complex difficult to read and understand print materials are often discarded.
Real and perceived barriers between patients and the IPF and care team can result in patient avoidance and resistance to follow through with appropriate care and follow up after discharge. These barriers can easily develop without the IPF and patients being aware. Keeping the patient perspective, goals and motivations in mind are essential toward developing these relations. Developing positive therapeutic relations between the IPF, inpatient care team, and physicians with the patient can help break down real and perceived barriers to care and help establish the IPF and care team as trusted partners. IPFs should make sure that they include patients and their support persons in all decision making. By sharing, understanding and agreeing with treatment and treatment goals, patients will be better engaged with their care and support persons will be better able to support patients.
Developing and implementing a discharge or transition of care process that focuses on the continuity of care requires clear communication and seamless transfer of information from the IPF discharging the patient to the outpatient provider of care for the patient after discharge. Factors and issues that limit and result in a fragmented process need to be addressed. Use of integrated medical records or at a minimum a hand off process and materials that include information about what lead to the inpatient encounter, interventions and care provided to the patient during their IPF stay, patient level of understanding of self-management and medications, information and resources provided to the patient, and scheduling of a follow up outpatient visit within seven days of discharge can help support a smooth discharge and transition of care. IPFs providing follow up phone calls with patients within 48 to 72 hours after discharge can help identify potential patient needs not identified prior to or at discharge.
While the IPF may be limited in its ability to establish relationships with community resources if none exist, establishing relationships with those that do exist can help the IPF inform and connect the patient with those resources that are available to support patients in their community and home environments. Community resources and support help patients with self-management and development of relationships with peers who can support one another and develop a sense of self-worth and community. In communities where outpatient resources are limited, IPFs can help raise awareness of the importance of establishing community resources to support mental health.
Success also requires frequent measurement of meaningful metrics. The IPF’s rates of readmission for patients discharged from their facility are the focus of this measure and decreasing those rates is the goal. It is important to monitor other aspects of an IPF’s patients care and outcomes. Measurement of patient and staff satisfaction with the new processes can help assess the care and discharge process environment. Make sure that success with one part of the process does not inadvertently result in adverse impacts on other parts of the patient’s care and discharge process.
All IPFs will not be at the same starting point, some will have interventions in place whereas others may have none in place; they will not have the same level of resources available, some may have robust resources, and some may have limited resources. Each IPF will need to identify the gaps in care they have, determine the interventions that will provide the most benefit and work with the resources they have available.
Mathematica received four questions from implementers via ServiceNow on the IPF Readmission measure from January to December 2024. The questions pertained to clarifications of the denominator and readmission eligibility. The measure team provided applicable responses to those questions. Questions and feedback from implementers did not indicate a need to update the measure specifications. The measure team did not identify any questions about the measure or potential updates in 2024 that would necessitate discussion with a technical expert panel (TEP) or expert work group (EWG).
Mathematica has not revised the measure specifications or the quality reporting program implementation of the IPF Readmission measure.
As indicated in Section 2.4 Performance Gap, we compared the national performance rates for the IPF Readmission measure from FY 2019 (that is, the performance period of 7/1/2015 through 6/30/2017) to FY 2025 (that is, the performance period of 7/1/2021 through 6/30/2023). This data is publicly reported from CMS in the Provider Data Catalog (Source: https://data.cms.gov/provider-data/archived-data/hospitals). The national performance rate for the IPF Readmission measure maintained an average of 20.1 percent for FY 2019 through FY 2023, however, the national performance rate has declined slightly (19.6 percent in FY 2024, and 19.4 percent in FY 2025), which indicates better measure performance nationwide. Please see Table 1a: Number of IPFs with performance worse, same and better compared to the national average for FY2019 through FY2025 in Attachment-2-Evidence-Fall 2025-CBE 2860.
Since its implementation in the IPFQR Program, the IPF Readmission measure has received relatively little feedback from implementers via the CMS Q&A feedback tool, ServiceNow. Additionally, as indicated in Section 2.4 Performance Gap, the national measure rate has improved over the last few years, which indicates that readmissions to IPFs within 30 days post-discharge have decreased nationwide.
Comments
Staff Preliminary Assessment
CBE #2860 Staff Preliminary Assessment
Importance
Strengths
- A logic model is provided, depicting the relationships between the inputs (e.g., discharge workflows, clinician and patient education, community resource development), activities (e.g., implementing best practices, revising discharge workflows, connecting patients to community resources), and desired outcomes (e.g., increased patient engagement, reduced 30-day readmissions, improved patient satisfaction).
The measure is supported by a literature review, including systematic reviews and empirical studies, showing that interventions such as patient education, transitional discharge models, and care coordination reduce psychiatric readmissions. However, limitations exist to the relevance of the evidence (see Limitations).
The problem this measure addresses presents a significant burden for patients and cost impact, with approximately 21.7% of Medicaid patients with serious psychiatric conditions readmitted within 30 days of discharge compared to 17.9% without such conditions. Psychiatric readmissions indicate deterioration in a patient’s condition, disruption to continuity of care, and are associated with increased risk of suicide attempts and poor recovery. These readmissions also drive higher health care utilization and costs for Medicare and other payers, while persistent variation in facility-level rates (ranging from 13.2% to 31.3%) underscores gaps in care and opportunities for improvement.
The developer convened a patient/caregiver panel (n=8) to assess meaningfulness. Seven participants agreed that unplanned readmissions reflect quality of care, supporting the measure’s relevance to patient experience with some limitations.
The developer provides a distribution of measure scores by decile using data from January 1, 2022 – December 31, 2023. Reported RSRRs range from 16.3% in the lowest decile to 20.31% in the highest decile, with a mean of 18.06%, indicating a performance gap.
Limitations
- While the evidence review includes empirical studies, only a small subset are Medicare-specific, and the submission provides no discussion of intervention effectiveness for dementia/Alzheimer’s patients, whose care pathways differ significantly from those for psychiatric disorders. This gap limits confidence that the logic model’s activities (e.g., patient education, community resource linkage) will work equally well across both conditions. The submission could be strengthened by adding evidence specific to these populations.
While most patients of a developer-convened panel acknowledged that readmissions indicate quality, only three of eight agreed the measure is meaningful, and only two of eight found it useful for care decisions. Several noted that readmissions are influenced by factors outside facility control (e.g., housing, outpatient access), which may limit perceived usefulness. The developer acknowledges external factors in its logic model but does not provide detailed strategies for mitigating these influences or explain how they affect interpretation of measure results.
Rationale
- This maintenance measure is rated as “Not Met But Addressable” for importance due to limited evidence relevance and insufficient patient input/meaningfulness. While the evidence review includes empirical studies, only a small subset are Medicare-specific, and there is no discussion of intervention effectiveness for dementia/Alzheimer’s patients, whose care pathways differ significantly from psychiatric disorders.
Additionally, patient feedback was mixed—only three of eight panelists found the measure meaningful, and concerns were raised about factors outside facility control. Enhancements, including a more condition-specific literature review, clearer linkage of interventions to evidence, refinement of the logic model for Medicare IPFs, and more robust patient input, could elevate its importance.
Closing Care Gaps
The developer did not address this optional domain.
Feasibility Assessment
Strengths
- All required data elements are routinely generated during care delivery, and required elements are available in claims, which is an electronic data source, in structured fields. The measure is easily implemented with no data collection or confidentiality issues, calculated by CMS using hospital-submitted claims data, and, according to the developer, imposes no burden on facilities. Lastly, this measure is not proprietary and has no proprietary components.
Limitations
- None identified.
Rationale
- This measure meets all criteria for 'Met' due to the data elements being derived in structured fields from electronic sources during the normal process of care. The measure has minimal impact on provider burden and there are no concerns with patient confidentiality. The measures is also not proprietary.
Scientific Acceptability
Strengths
- The developer performed the required reliability testing for this maintenance measure, namely, they conducted accountable entity-level (“measure score”) reliability testing at the level for which the measure is specified. Data sources used for reliability analysis are adequately described and include administrative and claims data from Medicare beneficiary and coverage files, Medicare FFS Part A and B records for over 275K patients in 1,410 entities during the two-year period of July 1, 2021 to June 30, 2023.
The developer conducted signal-to-noise reliability testing at the accountable entity-level. More than 70% of the accountable entities meet the expected threshold of 0.6.
Limitations
- None identified.
Rationale
- This maintenance measure is rated as ‘Met’ for reliability because the developer performed the required reliability testing for this measure and results demonstrate sufficient reliability at accountable entity-level for a two-year period of performance.
Strengths
- The developer performed the required validity testing for this maintenance measure, namely, they provided conducted accountable entity-level (“measure score”) validity testing at the level for which the measure is specified. Data sources used for validity analysis are adequately described and include Medicare claims (Parts A and B) and Medicare beneficiary data during the period July 1, 2021 through July 30, 2023. The initial sample consisted of 1,410 (n = 278,203 discharges total) with 25 or more discharges during the measurement period. Resulting sample sizes for the correlation analyses ranged from 767 to 1,309 IPFs.
The developer conducted empirical validity testing at the accountable entity-level using two methods: Spearman’s rank-order correlation of the measure score with conceptually-related measures; and known groups analysis based on findings from the literature. For correlation analyses, the developer hypothesized the IPF 30-day readmission measure would be: 1) positively associated with the IPF 30-Day ED Visit measure (CBE 4190), based on prior research showing increased risk of readmission in this group; 2) negatively associated with the 7- and 30-Day Follow-Up After Psychiatric Hospitalization measure (FAPH; CBE 0576), reasoning that that quality care coordination reduces readmission risk; 3) negatively associated with the Medication Continuation Following Inpatient Psychiatric Discharge measure (CBE 3205), reasoning that medication continuation indicates quality care coordination, supported by prior research findings.
The results showed moderate strength correlations in the expected directions with IPF ED Visit (observed/unadjusted; rho = 0.35), FAPH (rho = -0.27), and Medication Continuation (rho = -0.29), each significant at the p < 0.001 level.
For known groups analysis, the developer hypothesized that males, persons with substance use disorder (SUD), schizophrenia, or a mood disorder, and persons who had a short length of stay (< 6 days vs. > 17 days) would experience higher rates of IPF readmission, based on prior research. The developer calculated the observed, expected, and predicted rates of IPF readmission for each subgroup, as well as Cohen’s d for the mean group differences between subgroups for observed, expected, and predicted rates. The developer reported results largely in line with hypotheses predicting the directions of effects (Tables 6a and 6b in the supplemental attachment), with the largest effects for gender and schizophrenia diagnosis, and the smallest effects for SUD diagnosis.
A well-developed logic model suggests adequate “ruling in” of mechanisms that can explain the measure focus. Acceptable reliability also supports an inference of validity.
The developer conducted statistical risk adjustment, based on a conceptual model, selecting risk factors that are present at the start of care and have a significant correlation with the outcome. The developer reported a c-statistic of 0.658-0.673, indicating moderate model discrimination.
Limitations
- While the developer provided hypotheses for the direction of the expected effects for correlation and known groups analyses, they did not provide expected relative magnitudes of the effects based in either a mechanistic explanation (in the case of the correlation analyses) or findings from prior research (especially for known groups analysis). For example, in the known groups analysis, no interpretation is offered for why the effect of gender appears to be substantially stronger than the effects of some psychiatric disorders, or whether this result was expected based on prior research.
The developer mentioned calculating t-tests for mean group differences, but results of these tests were not presented. In the absence of significance testing, it is unclear which, if any, mean group differences were significant. In addition, some group differences were small enough that they may not be meaningful. Known groups analysis results for mood disorder diagnosis were mixed between Tables 6a and 6b.
More discussion in these areas would bolster the validity claim.
The developer did not provide evidence demonstrating variation in the prevalence of risk factors across accountable entities. In the measure logic model, the developer noted that external factors such as lack of community resources and lack of family support are associated with higher readmission risk, but did not evaluate these factors during the risk adjustment assessment.
Rationale
- This maintenance measure is rated as ‘Not Met But Addressable’ for validity because the validity testing results partially support an inference of validity for the measure, suggesting that the measure somewhat accurately reflects performance on quality and can distinguish good from poor performance to a limited extent.
The risk adjustment methods used are appropriate and demonstrate that the risk factors contribute to unique variation in the outcome. The model performance is acceptable.
Use and Usability
Strengths
- The measure is currently implemented in an accountability program, the CMS IPFQR Program.
The developer provided a summary of how accountable entities can use the measure results to improve performance. Specifically, IPFs can use measure results to identify gaps and take actionable steps such as revising discharge workflows, improving care transitions, and educating staff and patients—actions reflected in the logic model and supported by evidence. The logic model also includes facilitators like structured discharge planning, patient education, and community resource linkage for reducing readmissions.
The developer notes that questions from measure implementers are received via ServiceNow, but does not provide any additional information on a feedback collection process. The submission could be strengthened with more detail about this process. The developer does state that questions received were clarifications of the denominator and readmission eligibility, which did not indicate a need to update the measure specification.
Limitations
- The logic model assumes communities have adequate outpatient resources. Yet, the developer notes that if community resources are limited, IPFs should establish relationships with those that do exist. However, the submission could be strengthened with additional evidence to support this claim.
National performance rates show only modest improvement (20.1% to 19.4% over six years), and the submission does not provide a rationale for limited progress.
The developer calls attention to the fact that it has received little feedback to date from implementers. However, it is unclear whether there are any unexpected findings specifically.
Rationale
- This maintenance measure is rated ‘Not Met, but Addressable’ for use and usability because while the measure is actively used in the CMS IPFQR Program, the national performance rates show only modest improvement (20.1% to 19.4% over six years) without a sufficient rationale for limited progress.
Committee Independent Review
Continued Endoresement
Importance
The measure targets a vulnerable population at high risk for early clinical deterioration, suicide attempts, and poor continuity of care. Psychiatric readmissions within 30 days are clinically meaningful and reflect failures in stabilization and care transitions.
Closing Care Gaps
Based on the submitted information, the IPF Readmission measure closes multiple, well-documented gaps in inpatient psychiatric care, particularly around discharge readiness, care transitions, continuity of treatment, and patient safety during the highest-risk post-discharge period.
Feasibility Assessment
The IPF Readmission measure is feasible to implement and maintain. It relies exclusively on routinely collected Medicare claims data, imposes no additional reporting burden on facilities, presents no new confidentiality concerns, and has demonstrated stable nationwide implementation through the IPFQR Program. Therefore, the measure meets feasibility requirements.
Scientific Acceptability
Since the outcome is objectively defined and consistently captured using claims data, reliability is supported through minimum case thresholds, hierarchical modeling, and a multi-year performance period. Low-volume facilities are appropriately excluded from public comparison, and categorical reporting mitigates over-interpretation. Collectively, these design features provide sufficient signal to support facility-level performance assessment.
Battelle staff are working with the commenter to clarify rating details.
Use and Usability
This measure is actively implemented in a national CMS quality reporting program, and produces interpretable and actionable facility-level results, supports quality improvement, and enhances transparency for consumers. The measure is appropriately designed for its intended users and purposes and meets use and usability criteria.
Summary
The IPF Readmission measure demonstrates strong feasibility, acceptable scientific reliability, and clear use and usability. It is operationally stable, imposes no reporting burden on facilities, produces reliable facility-level results, and is actively used in a national CMS quality reporting and payment program. The measure supports actionable quality improvement and enhances transparency for inpatient psychiatric care. Continued endorsement is recommended.
Measure review
Importance
Limited evidence provided, additional patient input would be helpful as mixed reviews currently
Closing Care Gaps
Did not address
Feasibility Assessment
would use structured data, no additional provider documentation seems to be required
Scientific Acceptability
reliability testing completed
results from validity testing seem to only partially support
Use and Usability
current performance improvement rates are not significant and no additional information provided to provide context on why/limitations
Summary
Additional information on limited performance improvement would be helpful context
Support
Importance
There is limited research. I hope there is more research focused on patients with dementia/Alzheimer’s in the near future.
Closing Care Gaps
This section is optional
Feasibility Assessment
The data are readily available.
Scientific Acceptability
The mean ICC reliability was 0.66, ranging from 0.263 to 0.948. Might the minimum case count be slightly higher? Would this improve the ICC? If so, how many IPF would get dropped?
All differences in the IPF readmission rates by subgroups were in the direction consistent with the literature and our hypotheses. Correlation results were mixed.
Use and Usability
It would be helpful to hear comments/thoughts about why the measure results have not improved. I think most people understand how to interpret readmissions measures.
Summary
-
N/A
Importance
Given the resources necessary for support and to mitigate risk for readmission are different for patients with primary psychiatric diagnosis versus Alzeheimers/Dementia would recommend separating these populations.
Closing Care Gaps
N/A
Feasibility Assessment
N/A
Scientific Acceptability
N/A
N/A
Use and Usability
Recommend distinguishing results for the two diagnostic populations.
Summary
N/A
final thoughts
Importance
I agree with the measure developer that the population measured faces high readmission rates and I am not as concerned that the evidence they cite is not Medicare-focused because the trends are similar in the Medicare and non-Medicare population.
I also appreciate that they include the patient and caregiver perspective as that is not always the case. But I do worry that based on the responses this is not of high importance to the population. I wonder if part of this is based in an understanding that readmissions to IPF are potentially not viewed the same as to acute care hospitals.
There is a clear performance gap across IPFs, similar in magnitude to what is often seen in acute care hospitals for other conditions. However, the rates have been relatively steady over time, although did come down in more recent years of data.
I also do not understand the response from the measure developer regarding the decision to combine the Dementia/Alzheimer’s patients with severe mental illness. They argue this is due to how they are grouped in the CCS. I realize this may make it simpler from a measure development perspective but for the public and for providers I am not sure why this approach makes more sense than separating the cohorts.
Closing Care Gaps
N/A
Feasibility Assessment
No concerns with feasibility given that it is a claims based measure
Scientific Acceptability
Reliability methods used were appropriate and well accepted. The results were right at the threshold of acceptance according to PQM standards for accountable entity testing. Reliability scores are closely tied to sample size, which is not a surprise, but may mean estimates are noisier for those closer to the minimum sample size requirements.
The authors examination of correlation with related measures is an acceptable way to assess measure validity. The hypothesized relationship between the measures made sense and the results supported their assertions.
The authors also examined differences in readmissions by subgroups to assess whether the hypothesized directions between the variables and the outcomes were correct. However, the differences between the raw and P/E rates were minimal and the analysis was not as informative as I would have liked. The same conclusions could have been reached by comparing the rates with and without the risk factors in Table 7 or the odds rations in Table 8.
The risk adjustment model process and criteria were acceptable and sound. But once again, the higher odds ratios for CCS 653 make me wonder if this is a different patient population and should not be combined with the rest of the cohort.
Model calibration and discrimination testing methods were generally appropriate. The c-statistic score was within an acceptable range, however the calibration plot shows important limitations at the tails as did the Hosmer-Lemeshow test. I would like to see model performance stratified by Dementia/Alzheimer's and mental illness to better understand if caring for different proportions of these patients may explain some of these results.
Use and Usability
The developer makes a clear argument for how IPFs can reduce readmissions, but based on the longitudinal data I wonder if that is actually the case. I would be more inclined to agree if 2024 data, when available, continued to demonstrate this recent trend.
Coming back to the Dementia/Alzheimer's topic, combining the cohorts into a single measure also makes it more challenging for patients to understand how facilities are performing, stratified reporting would help solve this issue.
As the MA population continues to expand I also worry this measure is not as usable as it is limited to only FFS.
Summary
The general concerns I have with the measure are related to the lack of more substantial improvement in performance over time and the decision to combine the Dementia/Alzheimer’s and serious mental illness cohorts. However, in general the measure is sound and the developers did a good job with their measure testing and evidence. I recognize that from CMS' perspective this is an important measure for IPFQR and that does support the importance of the measure somewhat.
Public Comments
No Public Comments
No public comments received.