After the endorsement meeting, the developer of CBE #4555 submitted an appeal due to the following rationales:
- Procedural error in the endorsement process.
- Misapplication of measure evaluation criteria, specifically risk adjustment.
The appellant posited the:
- Discussion focused on out-of-scope topics (broader population beyond Medicare FFS and ACO-REACH). Insufficient time for comprehensive discussion. Dominance of a few voices, limiting inclusive discussion.
- Measure meets risk adjustment criteria. Variables in the risk model influence the outcome and are present at the start of care. Model excludes unjustified factors associated with care inequities. Measure adjusts for dual eligibility, accounting for social risk factors. ACO REACH Model provides resources for coordinated care, supporting the measure's intent.
The Appeals Committee voted to uphold the appeal based on both rationales above. Therefore, the measures’ s endorsement decision was overturned to Endorsed with Conditions. When the measure returns for maintenance (5 years), the measure developer should have:
- Explored differences in scores for rural and urban settings and/or explore whether adjusting on these variables is needed.
- Explored the impact of variation in attribution based on utilization (in the denominator).
This is an ACO-level measure of days at home or in community settings (that is, not in acute care such as inpatient hospital or emergent care settings or post-acute skilled nursing) among adult Medicare Fee-for-Service (FFS) beneficiaries with complex, chronic conditions who are attributed to ACOs participating in the ACO REACH model. The measure includes risk adjustment for differences in patient mix across ACOs, with an additional adjustment based on patients’ risk of death. A policy-based nursing home adjustment that accounts for patients’ risk of transitioning to a long-term nursing home is also applied to incentivize community-based care. The performance period is one calendar year.
Measure Specs
- General Information(active tab)
- Numerator
- Denominator
- Exclusions
- Measure Calculation
- Supplemental Attachment
- Measure Record
- Point of Contact
General Information
The goal of this outcome measure is to reduce unnecessary or excessive hospitalizations for Medicare patients with complex chronic conditions, thus increasing their “days at home.” Through reporting this outcome measure, ACOs will be able to compare their performance in relation to others and use the results to identify the root causes of excess acute care utilization, including insufficient or low-quality outpatient care. Improvements in this outcome mean that patients will spend fewer days in acute care settings, and more days at home or in the community, which often reflects patient preferences as well as cost savings to patients and providers.
The measure uses CMS beneficiary enrollment and claims data for a given year accessed via the CMS Integrated Data Repository (IDR) and Chronic Conditions Data Warehouse Virtual Research Data Center (VRDC):
- Medicare inpatient claims
- Medicare outpatient claims
- Medicare SNF claims
- Medicare beneficiary enrollment data
The ACO-beneficiary alignment/attribution file is determined prospectively based on the requirements of the program; for testing, we used the 2018 Shared Savings Program ACO attribution file used by that program for its other claims-based measures and the 2022 ACO REACH alignment file furnished by CMS.
Numerator
The outcome measured for each eligible beneficiary is days spent “at home,” adjusted for clinical and social risk factors, risk of death, and risk of transitioning to a long-term nursing home. Days at home are defined as those days when a beneficiary is alive and not in care. A “day in care” is defined as any eligible patient day on which a patient receives care in one (or more) of the following specified care settings: inpatient acute and post-acute facilities (short term acute care hospitals, critical access hospitals (CAHs), inpatient rehabilitation facilities (IRFs), inpatient psychiatric facilities (IPFs), long-term care hospitals (LTCHs), and SNFs); emergency departments (ED); and observation stays. There are two exceptions:
1. A patient is always considered “at home” if they are enrolled in hospice, even if they receive care in settings normally counted as “days in care” (that is, if a patient enrolled in hospice is receiving care in an inpatient setting that will not count as a day in care)
2. Hospital admissions for childbirth, miscarriage, or termination are not counted as “days in care”
A “day at home” is defined as any eligible day that is not considered a “day in care” based on the above definition. “Eligible days” are all days in the measurement year that the beneficiary is alive.
“Days at home” are defined as those days when a beneficiary is alive and not in an acute or post-acute care setting.
A “day in care” is defined as any eligible patient day on which a patient receives care in one (or more) of the following specified care settings: inpatient acute and post-acute facilities (short-term acute care hospitals, critical access hospitals (CAHs), inpatient rehabilitation facilities (IRFs), inpatient psychiatric facilities (IPFs), long-term care hospitals (LTCHs), and SNFs); emergency departments (ED); and observation stays. There are two exceptions:
1. A patient is always considered “at home” if they are enrolled in hospice, even if they receive care in settings normally counted as “days in care” (that is, if a patient enrolled in hospice is receiving care in an inpatient setting that will not count as a day in care)
- Rationale: to promote effective and appropriate care for terminally ill patients
2. Hospital admissions for childbirth, miscarriage, or pregnancy termination are not counted as “days in care”
- Rationale: obstetric admissions may not indicate care quality; counting these admissions may create an inappropriate incentive to keep pregnant patients out of the hospital.
A “day at home” is defined as any eligible day that is not considered a “day in care” based on the above definition. “Eligible days” are all days in the measurement year that the beneficiary is alive.
Care in settings not listed above (including outpatient visits and procedures, hospice, residential psychiatric and substance abuse facilities, assisted living facilities and group homes, and home health and telehealth services) are not considered “days in care” in this measure; rather, they are treated as “days at home.”
Finally, days spent in a long-term or residential nursing home (except for SNF care) are not counted as “days in care” by this definition. However, as discussed in the “Measure Scoring” section, this measure includes an adjustment that accounts for patients’ risk of transitioning to a long-term nursing home, to encourage home- and community-based care in alignment with CMS’s policy goals.
Denominator
Eligible beneficiaries aligned with participating provider entities determined by the Innovation Center’s ACO REACH Model, including adults (age 18 or older); alive as of the first day of the performance year; continuously enrolled in Medicare FFS Parts A and B during the full performance year (up to date of death among patients who died) and one full year prior; and have an average Hierarchical Condition Category (HCC) composite risk score >= 2.0 in the year prior to the performance year.
Eligible beneficiaries are characterized by a combination of older age, chronic health conditions, more frequent healthcare utilization, serious illness, disability or frailty, and/or financial hardship that places them at greater risk of negative health outcomes and makes them more likely to require complex medical care. For this measure:
- Adult (age 18 or older);
- Alive as of the first day of the performance year;
- Continuously enrolled in Medicare FFS parts A and B during the full performance year (up to date of death among patients who died) and one full year prior; and
- Patients with complex, chronic conditions, which are defined as patients with an average Hierarchical Condition Category (HCC) composite risk score >= 2.0* in the year prior to the performance year.
The measure, as implemented, includes eligible beneficiaries who are aligned to an ACO participating in the ACO REACH model. Of note, this alignment is entirely prospective; the measure does not use any retrospective attribution of patients or outcomes to ACOs. These criteria are consistent with the cohort of the ACO REACH model and with its objective of emphasizing care of patients meeting a broad definition of serious illness or complex chronic disease.
*CMS originally developed the HCC risk score for the purpose of payment adjustment in Medicare Advantage plans, but now calculates the score monthly for all Medicare patients.1 The HCC score is calculated as a sum of values (“coefficients”) assigned to the following types of “risk:”
- Patient demographic characteristics:
- Age;
- Sex;
- Dual enrollment in Medicaid (an indicator of financial hardship);
- Disability-based Medicare eligibility;
- Long-term institutional (LTI) status, defined as living in a nursing home for 90 days or longer;
- Chronic conditions, such as heart failure or dementia
Patients with more risk factors or more serious conditions will have higher HCC risk scores than those with fewer or less serious risk factors. A risk score of 1.0 indicates a patient is expected to have average medical care costs; a risk scores greater than 1.0 indicates the patient is sicker and therefore at greater risk to need more expensive medical care. Higher costs of care are generally associated with more complex care delivery and more time in institutional health care settings.2
References
1. CMS. Medicare Managed Care Manual – Chapter 7, Risk Adjustment. 2014.
2. McCaffrey N, AgarM, Harlum J, Karnon J, Currow D, Eckermann S. Is home-based palliative care cost-effective? An economic evaluation of the Palliative Care Extended Packages at Home (PEACH) pilot. BMJ Support Palliat Care. 2013;3(4):431-435.
Exclusions
None. There are currently no denominator exclusions or exceptions for the measure. All patients meeting the denominator inclusion criteria are included.
None. There are currently no denominator exclusions or exceptions for the measure. All patients meeting the denominator inclusion criteria are included.
Measure Calculation
At a high level, the measure result (adjusted days at home) is calculated based on three risk-adjusted statistical models. We use the first model to calculate “excess days in care” for each patient, which represents the risk-adjusted days in acute care settings or SNFs among days alive in the year. Two additional risk-adjusted models are used to calculate the risk of mortality and risk of transition to nursing home for each patient. Finally, “excess days in care” are updated based on risk of death and risk of transition to nursing home care and then averaged across each ACO to produce the final measure scores. The details of each of these steps are described below.
First, “excess days in care” for each patient are modeled using a hierarchical negative binomial regression with an offset for days alive. “Excess days in care” is defined as predicted minus expected days in care, where “predicted” includes clinical and social risk adjustment, survival offset, and an ACO-specific effect, and “expected” includes only clinical and social risk adjustment and survival offset. More “excess days in care” indicates a patient spent more time in care than expected due to their ACO’s performance.
Second, mortality is modeled using a hierarchical logistic regression model with adjustment for the patient case-mix, to calculate a standardized mortality ratio (SMR) at the patient level. SMR is defined as predicted divided by expected risk of death, where “predicted” includes clinical risk adjustment and an ACO-specific effect, and “expected” includes only clinical risk adjustment. A high SMR indicates a patient at greater-than-expected risk of death due to their ACO’s performance.
Third, a patient’s risk of transitioning to a residential nursing home is modeled using a hierarchical logistic regression model with adjustment for patient case-mix and Medicaid dual-eligibility status, to calculate a standardized “nursing home ratio” (NHR) which is scaled to have the same mean and standard deviation as the SMR. NHR is defined as predicted divided by expected risk of transitioning to a nursing home during the performance year, where “predicted” includes clinical and social risk adjustment and an ACO-specific effect, and “expected” includes only clinical and social risk adjustment. A higher NHR indicates a patient at greater-than-expected risk of transitioning to a nursing home due to their ACO’s performance.
For the mortality adjustment for each patient, “excess days in care” is multiplied by SMR (if excess days >= 0) or divided by SMR (if excess days < 0), such that a greater SMR results in an absolute increase of “excess days in care” (that is, ACOs are rewarded for lower mortality than expected and penalized for greater mortality than expected given their case mix). Similarly, for the nursing home adjustment for each patient, “excess days in care” is multiplied by [0.5*NHR] (if excess days < 0) or divided by [0.5*NHR] (if excess days >= 0) so that ACOs are rewarded for lower rates of transition to the nursing home than expected given their case mix.
The SMR and NHR adjustments are combined additively to give a “mortality- and nursing home transition risk-adjusted excess days in care,” which is subtracted from the patient-level national average of days alive, resulting in a risk-, mortality-, and nursing home transition-adjusted measure of “days at home” for that patient.
Finally, the adjusted days at home are averaged over all patients of each ACO to summarize the ACO’s measure performance as the “ACO-level adjusted days at home.”
The measure, as specified, is not a stratified measure; ACOs overall scores are based on analysis of all patients pooled together. However, in implementation, the ACO REACH Model does include stratification of results, with both confidential reporting of stratified results and an equity-based payment adjustment (as described in Section 5.1). We include a similar stratification analysis here.
To promote improvements in disparities in care for patients with social risk factors, REACH ACO measure scores are stratified by three social risk factors: (1) dual-eligible status (DE); (2) low socioeconomic status (SES) as defined by the Area Deprivation Index (ADI); and (3) race/ethnicity other than white (i.e., non-white). As of the 2022 model performance year (Calendar Year 2022), CMS provides the stratified results to ACOs quarterly, in Quarterly Quality Reports (QQRs), and annually, in Annual Quality Reports (AQRs). The stratified results are provided to ACOs confidentially.
The three social risk factors used in stratified reporting are defined as:
- Medicare-Medicaid dual-eligible status: Full-benefit dual-eligible status for at least 1 month during the performance period.
- Living in a low-SES neighborhood: Defined as a neighborhood with an ADI percentile value of 81 or higher. We continue to use the 2022 version of ADI data due to differences between 2010 and 2020 Census boundaries and the limited prevalence of the 2020 boundaries among addresses within claims data. For beneficiaries with addresses that have no ADI match, we impute a county-level average ADI. More information about the ADI is available here.
- Non-white: Race/ethnicity other than white based on RTI_RACE_CD variable from the IDR.
The Days at Home measure specifications include Dual-Eligibility as a risk factor. However, for the confidential stratified reporting of the Days in Care component of Days at Home involving dual-eligible beneficiaries only, dual-eligible status is removed as a risk factor from the statistical model (as both stratifying and risk adjusting for the same variable is not statistically appropriate).
The stratified results are calculated through the following steps:
- The finder file, which is the first file created and used for building analytic files for each quality measure, creates the health equity indicator variables that are used for stratified reporting.
- Once the finder file is created, the health equity indicator variables are used to calculate the Days at Home measure for the ACOs included in the ACO REACH model as well as the benchmark population, which are non-ACO REACH ACOs.
- Summary statistics for each of the stratified populations are provided to ACOs in the QQRs. Values are not reported if the denominator volume (acute events) is less than 20.
Final risk factor coefficients on are tab “Final RF Coefficients” of the attached Code Set File.
There is no minimum sample size.
Supplemental Attachment
Measure Record
Point of Contact
Not applicable.
Chardae Buchanan
Baltimore, MD
United States
Clarissa Meyers
Yale CORE
New Haven, CT
United States
Importance
Evidence
The Days at Home measure seeks to support the preference of most patients with complex, chronic conditions to remain at home.1 The measure disincentivizes excessive care in acute inpatient or outpatient settings such as emergency departments (EDs), or post-acute settings such as SNFs, since prolonged or frequent utilization of these care settings often represents sub-optimal ambulatory care and worse quality of life for the patient. The Days at Home measure intends to encourage ACO Realizing Equity, Access, and Community Health (REACH) participants to improve quality of care, while reducing healthcare utilization among Medicare patients with complex, chronic conditions. Several recent studies have demonstrated an association between days spent at home with positive clinical outcomes, such as reduced disability1,2,3 and cost savings to patients and providers.4,5 One study found an inpatient and outpatient cost savings of $99 for each additional day at home during the first three months after stroke.4 Additionally, the measure intends to encourage better coordination of care between various providers and patients, which is associated with improved health outcomes such as healthy days at home. Evidence from the literature has shown that delivery of appropriate and timely primary and end-of-life care services 6-9 and improved care coordination and care transitions 10-12 can increase the number of days patients spend at home. For example, one study examined the association of frailty with intensity of end-of-life care (EOLC) for older adults with and without frailty who undergo emergency general surgery (EGS) but die within one year. Their findings suggest that many older EGS decedents, and particularly those who are frail, received worse quality of EOLC, spending a significant amount of time hospitalized and away from home, and few days in hospice. Their findings indicate an urgent need for in-hospital and post discharge interventions directed toward this vulnerable population to reduce hospital use and align treatments with patient preferences at the end of life.15
Several studies demonstrate that time spent at home differs substantially among older patients, which suggests that there is potential for improving the quality of care and resulting days at home for the elderly population.13,14 While the majority of patients spend all or most days at home, one study noted that patients aged 65 and older with multiple chronic conditions spend fewer days at home, with patients having three or more chronic conditions spending an average of 12.3 fewer days at home (mean 336.6 days, SD 3.0) in a one-year period than do all patients ages 65 and older (mean 348.9 days, SD 1.7).13
References
- Lee H, Shi SM, Kim DH. Home Time as a Patient-Centered Outcome in Administrative Claims Data. Journal of the American Geriatrics Society. 2019;67(2):347-351.
- Yu AYX, Fang J, Kapral MK. One-Year Home-Time and Mortality After Thrombolysis Compared With Nontreated Patients in a Propensity-Matched Analysis. Stroke. 2019:Strokeaha119026922.
- Ringden O, Remberger M, Holmberg K, et al. Many days at home during neutropenia after allogeneic hematopoietic stem cell transplantation correlates with low incidence of acute graft-versus-host disease. Biology of blood and marrow transplantation : journal of the American Society for Blood and Marrow Transplantation. 2013;19(2):314-320.
- Dewilde S, Annemans L, Peeters A, et al. The relationship between Home-time, quality of life and costs after ischemic stroke: the impact of the need for mobility aids, home and car modifications on Home-time. Disabil Rehabil. 2020;42(3):419-425.
- McCaffrey N, Agar M, Harlum J, Karnon J, Currow D, Eckermann S. Is home-based palliative care cost-effective? An economic evaluation of the Palliative Care Extended Packages at Home (PEACH) pilot. BMJ Support Palliat Care. 2013;3(4):431-435.
- Totten AM, White-Chu EF, Wasson N, et al. AHRQ Comparative Effectiveness Reviews. Home-Based Primary Care Interventions. Rockville (MD): Agency for Healthcare Research and Quality (US); 2016.
- Pham B, Krahn M. End-of-Life Care Interventions: An Economic Analysis. Ontario health technology assessment series. 2014;14(18):1-70.
- Wang SE, Liu IA, Lee JS, et al. End-of-Life Care in Patients Exposed to Home-Based Palliative Care vs Hospice Only. Journal of the American Geriatrics Society. 2019;67(6):1226-1233.
- Andersen SK, Croxford R, Earle CC, Singh S, Cheung MC. Days at Home in the Last 6 Months of Life: A Patient-Determined Quality Indicator for Cancer Care. Journal of oncology practice. 2019;15(4):e308-e315.
- Harrison JD, Auerbach AD, Quinn K, Kynoch E, Mourad M. Assessing the impact of nurse post-discharge telephone calls on 30-day hospital readmission rates. J Gen Intern Med. 2014;29(11):1519-1525.
- Hoyer EH, Brotman DJ, Apfel A, et al. Improving Outcomes After Hospitalization: A Prospective Observational Multicenter Evaluation of Care Coordination Strategies for Reducing 30-Day Readmissions to Maryland Hospitals. J Gen Intern Med. 2018;33(5):621-627.
- Wilson DM, Birch S. Moving from place to place in the last year of life: A qualitative study identifying care setting transition issues and solutions in Ontario. Health & social care in the community. 2018;26(2):232-239.
- Burke LG, Orav EJ, Zheng J, Jha AK. Healthy Days at home: A novel population-based outcome measure. Healthcare (Amsterdam, Netherlands). 2019:100378.
- YNHHSC/CORE. Condition-Specific Excess Days in Acute Care Measures Updates and Specifications Report. 2019.
- Sokas C, Lee KC, Sturgeon D, et al. Preoperative Frailty Status and Intensity of End-of-Life Care Among Older Adults After Emergency Surgery. Journal of pain and symptom management. 2021; 62 (1): 66-74.e3
Measure Impact
Most patients and families prefer spending time at home and in the community (“days at home”) rather than in the hospital,1-6 and more days at home are associated with both positive clinical outcomes and lower costs for patients and providers.2-4,6-9 Poor care coordination can lead to unnecessary and preventable hospital visits for patients.10-11 Given that patients with complex, chronic conditions often receive care from several clinicians and sites of care, this patient population may particularly benefit from improved care coordination.6
The Days at Home measure is intended to assess the extent to which provider entities such as Accountable Care Organizations (ACOs) help their patients living with complex, chronic conditions spend more time at home or in community settings, rather than acute or post-acute settings (such as hospitals or skilled nursing facilities [SNFs]). By capturing the days in which a patient is at home or in a community setting, the measure is intended to encourage healthcare use that supports:
- Maintenance of home- and community-based care;
- Prevention of health decline;
- Decreases in acute care use;
- Use of primary and preventive care; and
- Decreases in overuse of long-term institutional care, as feasible.
This measure will directly benefit Medicare patients by reducing unnecessary hospitalizations and incentivizing days at home or in the community, which often reflect patient preferences and quality of life. There are no competing measures for Medicare beneficiaries. The measure may help to incentivize care coordination between healthcare providers. The measure developers have reduced the risk of the measure having unintended negative consequences on patients through additional adjustment for risk of mortality and risk of transitioning to a nursing home; these adjustments would make it more difficult for providers to score well by reducing days in care in ways that lead to other poor outcomes. For example, measuring only days in acute care may create a perverse incentive for ACOs to more aggressively transfer patients to nursing homes (where they can be more easily supervised and receive constant unskilled nursing care) even though this is typically not what patients desire. Similarly, measuring only days in acute care may perversely incentivize providers to delay medically appropriate acute care resulting in worse outcomes overall, the most extreme of which is death. In short, while fewer days at home is generally an undesired outcome it has to be balanced against competing bad outcomes; in the absence of balancing quality measures, these adjustments still allow the Days at Home measure to account for this to some degree.
References
- Barnato AE, Herndon MB, Anthony DL, et al. Are regional variations in end-of-life care intensity explained by patient preferences?: A Study of the US Medicare Population. Medical care. 2007;45(5):386.
- Fonarow GC, Liang L, Thomas L, et al. Assessment of Home-Time After Acute Ischemic Stroke in Medicare Beneficiaries. Stroke. 2016;47(3):836-842.
- Higginson IJ, Sen-Gupta GJ. Place of care in advanced cancer: a qualitative systematic literature review of patient preferences. J Palliat Med. 2000;3(3):287-300.
- Lee H, Shi SM, Kim DH. Home Time as a Patient-Centered Outcome in Administrative Claims Data. Journal of the American Geriatrics Society. 2019;67(2):347-351.
- Leff B, Burton L, Guido S, Greenough WB, Steinwachs D, Burton JR. Home hospital program: a pilot study. Journal of the American Geriatrics Society. 1999;47(6):697-702.
- McDermid I, Barber M, Dennis M, et al. Home-Time Is a Feasible and Valid Stroke Outcome Measure in National Datasets. Stroke. 2019;50(5):1282-1285.
- Yu AYX, Fang J, Kapral MK. One-Year Home-Time and Mortality After Thrombolysis Compared With Nontreated Patients in a Propensity-Matched Analysis. Stroke. 2019:Strokeaha119026922.
- Stienen MN, Smoll NR, Fung C, et al. Home-Time as a Surrogate Marker for Functional Outcome After Aneurysmal Subarachnoid Hemorrhage. Stroke. 2018;49(12):3081-3084.
- Mishra NK, Shuaib A, Lyden P, et al. Home time is extended in patients with ischemic stroke who receive thrombolytic therapy: a validation study of home time as an outcome measure. Stroke. 2011;42(4):1046-1050.
- 20. Brooks EM, Winship JM, Kuzel AJ. A "Behind-the-Scenes" Look at Interprofessional Care Coordination: How Person-Centered Care in Safety-Net Health System Complex Care Clinics Produce Better Outcomes. Int J Integr Care. 2020;20(2):5.
- Valentijn PP, Schepman SM, Opheij W, Bruijnzeels MA. Understanding integrated care: a comprehensive conceptual framework based on the integrative functions of primary care. Int J Integr Care. 2013;13:e010.
There are no competing measures for Medicare beneficiaries, so the Days at Home measure for patients with complex, chronic conditions fills an important gap in measurement. The Days at Home measure fills this measurement gap with the intention to incentivize providers to improve care coordination, reduce unnecessary, excessive and/or preventable emergency, hospital, and skilled nursing facility care, and keep patients in their home and community. The measure further supports CMS’s Meaningful Measures Initiative to promote effective prevention and treatment of chronic disease in the areas of preventative care and management of chronic conditions. As noted by Burke et al (2019), while many quality measures capture outcomes (such as mortality or readmissions) related to specific episodes of care (for example a condition-specific hospital admission or surgical procedure), there are relatively few population-based outcome measures that focus on improving outcomes broadly; a measure like Days at Home that reflects how well healthcare organizations like ACOs keep their patients healthy and out of institutions altogether would help to fill this gap.1
Reference
1. Burke LG, Orav EJ, Zheng J, Jha AK. Healthy Days at home: A novel population-based outcome measure. Healthcare (Amsterdam, Netherlands). 2019:100378.
The Days at Home measure is a risk-adjusted population-based outcome measure that uses the outcome of days at home to assess care quality. A spectrum of institutional acute and post-acute care use, including emergency department (ED) visits, observation stays, or admissions to hospitals or skilled nursing facilities (SNFs) are reported as “days not at home,” or “days in care.” Remaining days during the measurement period that a patient is alive are considered “days at home,” including days in which a patient utilizes outpatient procedures or services (other than ED visits and observation stays), institutional or home-based hospice, and residential treatment facilities.
The measure disincentivizes care in acute inpatient or outpatient settings such as emergency departments (EDs), or post-acute such as skilled nursing facilities (SNFs), since prolonged or frequent utilization of these care settings often represents sub-optimal care and worse quality of life for the patient, which holds significant meaning for the target patient population.
Evidence demonstrates that generally, patients prefer to remain at home and avoid unnecessary hospitalizations or transitions to long-term care facilities1-6. Lee, Shi, & Kim (2019) found that days at home are associated with other patient-centered outcomes, such as self-reported health status, social activity, and depression.4
One study evaluated short- and long-term measures of health care utilization (i.e., days in the emergency department (ED), inpatient (IP) care, and rehabilitation in a post-acute care (PAC) facility) to understand how home time (i.e., days alive and not in an acute or PAC setting) corresponds to quality of life (QoL). The results showed that the impact of receiving care in one setting is not equivalent to that in another setting. In the short term (6 months), PAC utilization emerged as the most salient predictor of decreased QoL, whereas no setting predominated in the long term (18 months). This study advances the science of home time measures by exploring how home time measures are associated with quality of life (QoL), which health care settings to include, and which time frames of utilization are important. This study indicates that patients with more DAH report higher quality of life and greater satisfaction with their care, reinforcing DAH as a meaningful measure for the patients.
In addition to condition- or procedure-specific articles, Lee et al. (2019) validated the days at home outcome against other patient-centered outcome measures among Medicare patients.4 For example, the incidence of poor self-rated health ranged from 2% to 21%, and days at home ranged from 365 days for those with the lowest self-rating of poor health (as in, patients with the best health) to less than 337 days for those with the highest self-rating of poor health (as in, patients with the poorest health). They found similar relationships between days at home and other patient-centered outcomes; participants with the highest ratings of mobility impairment, depression, limited social activity, and difficulty in self-care had the lowest ratings of days at home. As a result, Lee et al. (2019) determined days at home to be a valid, patient-centered outcome.
Moreover, the American population is living longer, and elderly patients have an increasing number of chronic conditions and comorbidities over time8; a measure focusing on improving primary care services for this high-risk cohort of patients is increasingly important and relevant.
As part of the development process, we convened a Technical Expert Panel (TEP) that included several patient and advocate representatives (who were themselves patients or caregivers of patients with complex chronic illness). The TEP as a whole, including these patient partners, broadly valued the concept of a Days at Home measure as one that logically reflects an ACO’s ability to act in accord with patient preferences, with an emphasis on more comprehensive and patient-centered “whole-person care” over siloed approaches. The TEP and these patient representatives further noted the importance of such care to patients with complex chronic illness in particular, for whom navigating a fragmented care landscape is highly burdensome in its own right. Panelists noted that even hospitalizations that are planned or medically necessary are disruptive and can adversely affect patients’ quality of life, and supported the concept of a Days at Home measure that would capture outcomes more holistically.
References
1. Barnato AE, Herndon MB, Anthony DL, et al. Are regional variations in end-of-life care intensity explained by patient preferences?: A Study of the US Medicare Population. Med Care. 2007;45(5):386-393.
2. Fonarow GC, Liang L, Thomas L, et al. Assessment of Home-Time After Acute Ischemic Stroke in Medicare Beneficiaries. Stroke. 2016;47(3):836-842.
3. Higginson IJ, Sen-Gupta GJ. Place of care in advanced cancer: a qualitative systematic literature review of patient preferences. J Palliat Med. 2000;3(3):287-300.
4. Lee H, Shi SM, Kim DH. Home Time as a Patient-Centered Outcome in Administrative Claims Data. Journal of the American Geriatrics Society. 2019;67(2):347-351.
5. Leff B, Burton L, Guido S, Greenough WB, Steinwachs D, Burton JR. Home hospital program: a pilot study. Journal of the American Geriatrics Society. 1999;47(6):697-702.
6. Sayer C. “Time Spent at Home” — A Patient-Defined Outcome. NEJM Catalyst. 2016.
7. Dennis A, Stechuachak K, Van Houtven C, et al. Informing a home time measure reflective of quality of life: A data driven investigation of time frames and settings of health care utilization. Health Services Research. 2023; 58(6): 1233-1244.
8. Totten AM, White-Chu EF, Wasson N, et al. Home-Based Primary Care Interventions. Rockville MD2016.
Performance Gap
Using Medicare FFS administrative claims and enrollment information from CY 2021 in the pre-measure year and 2022 (January 1, 2021 – December 31, 2022) in the measure year, downloaded from CMS’s Integrated Data Repository and Virtual Research Data Center.
The mean ACO-level adjusted days at home was 323.04, ranging from 300.31 to 334.01 days.
| Overall | Minimum | Decile_1 | Decile_2 | Decile_3 | Decile_4 | Decile_5 | Decile_6 | Decile_7 | Decile_8 | Decile_9 | Decile_10 | Maximum | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Mean Performance Score | 323.04 | 300.31 | 315.19 | 320.24 | 321.66 | 322.46 | 323.06 | 323.76 | 324.41 | 325.21 | 325.79 | 327.84 | 334.01 |
| N of Entities | 99 | 1 | 9 | 10 | 10 | 10 | 10 | 10 | 10 | 10 | 10 | 10 | 1 |
| N of Persons / Encounters / Episodes | 169,324 | 1,220 | 21,647 | 22,900 | 13,128 | 21,641 | 11,620 | 12,713 | 15,143 | 17,766 | 13,377 | 19,389 | 361 |
Equity
Equity
Reporting and reducing disparities are a key area of focus for quality measures and payment models. As described in Section 6.1.4 Program Details, this measure is used in the ACO REACH model, which has a primary goal of advancing health equity, by requiring participating ACOs to have a robust plan for meeting the needs of underserved communities and make measurable changes to address health disparities. The model supports this through a payment approach to better support care delivery and coordination for patients in underserved communities, including quality measures such as Days at Home.1 The REACH model adopts a similar approach to SDOH adjustment as other CMS programs such as the Hospital Readmission Reduction Program (HRRP) by generally adjusting for social risk factors at the level of payment in the program, rather than at the quality measure level. This promotes fairness in calculating payments, so as not to penalize measured entities with a high proportion of patients with social risk, but still allows for transparency in terms of outcomes for patients with social risk factors. Specifically, the ACO REACH model, for 2024, adjusts payments based on dual-eligibility status and ADI, resulting in higher payments to ACOs serving higher proportions of underserved beneficiaries. The 2024 model will adjust ACO benchmarks by $30 per-beneficiary, per-month (PBPM) for beneficiaries with equity scores in the top decile, $20 PBPM for beneficiaries in the second decile, $10 PBPM for the third decile, and $0 PBPM for the next four deciles. For any aligned beneficiary in the bottom 50%, an ACO's benchmark will be reduced by $6 PBPM.2 The model also promotes reducing disparities through confidential stratified reporting of quality measure data to model participants.
Use of stratified quality measures—that is, calculating and reporting quality measure results separately for persons with and without social risk factors—can illuminate gaps in quality of care within and across entities. The Days at Home measure, as a broad population-based outcome measure of primary care, care coordination, and post-acute case management among patients with complex chronic illness, provides a unique window to measure and address disparities. While most other CMS outcome measures are fairly narrow (for example, looking for death or readmission within 30 days after discharge for an acute care visit), Days at Home counts how much time a patient spends alive and out of acute care facilities over an entire year, accounting for person-time at risk and the duration of acute/inpatient visits in addition to case mix variation. Stratification of the Days at Home measure can illuminate differences in outcomes between different populations that other measures may not.
To address this, CORE assessed potential disparities in the Days at Home measure, comparing days in care at the patient and ACO levels by select SDOH factors. We used the CY 2022 ACO REACH model dataset.
Based on available data and to be consistent with the approach to stratified reporting adopted by the ACO REACH model, we identified three SDOH factors for analysis: Medicare-Medicaid dual-eligible (DE) status, University of Wisconsin Area Deprivation Index (ADI), and race.3
- DE status is an indicator of Medicare patients who are also enrolled in Medicaid, which is a commonly used indicator for conditions of financial hardship and related barriers to high-quality outcomes compared to non-DE patients (the reference group).4 DE status for all Medicare patients is included in the master beneficiary summary file (MBSF)
- ADI is an area-level indicator of deprivation that quantifies factors such as income, education, employment, and housing quality using variables from US Census data. ADI is scored at the census block-group level from 1 (indicating low deprivation and thus lowest risk) to 100 (indicating high deprivation and thus highest risk). Individuals experiencing deprivation are more likely to face barriers such as transportation or adequate housing that makes maintenance of health at home more difficult and may require special approaches by ACOs to receive the highest quality care. For this analysis we defined beneficiaries from areas with highest quintile ADI (at or above 80th percentile) as “high ADI” (the high-risk group) and compared to those from areas with ADI below the top quintile (“low ADI,” the reference group). ADI is linked to each patient record via ZIP code.
- Race is another commonly studied SDOH factor and can serve as a proxy indicator for discrimination and exposure to structural racism. We dichotomized race as “non-white” (the high-risk group) and white (the reference group), consolidating all non-white options to ensure sufficient sample size. Race is derived from the MBSF, which includes values imputed for those with missing data.
Methods
Because the full Days at Home measure is a combination of three separate risk adjustment models, it is not possible to stratify the full score while maintaining statistical rigor; instead, we report results here based on the Days in Care model only. As shown in Table 15 of the supplemental document, the Days in Care model provides most of the quality signal, with the nursing home and mortality adjustments mostly affecting those with more extreme outlying performance. The effect of including mortality and nursing home transitions yields a range of -5.02 to +2.22 “extra” days at home compared to using Days in Care only, but for the middle 80% of ACOs the difference is between -0.37 and +0.54, and the interquartile range (encompassing half of all ACOs) it is between just -0.07 and +0.19 days at home per person year. By comparison the interquartile range for the measure score itself is much greater, from 321.7 to 325.3, which suggests that for a large majority of ACOs the days in care model alone is a reasonable proxy for performance on the whole measure. Also, notably, dual-eligible status has been removed from the risk model for analyses stratifying by dual-eligible status (though retained in analyses stratified by ADI and race).
To accomplish this, we used the CMS Disparity Method (previously developed for the CMS Hospital Equity Index and subsequently expanding to use in other contexts) to estimate “within” differences at the ACO level.5 In brief, the “within” disparity method seeks to answer the question, “How does quality within this ACO differ between its patients with vs. without a given factor?” For example, an ACO with equivalent outcomes among both its dual and non-dual eligible patients would have a within-score of 0 days, while one with worse outcomes among dual-eligible patients would have a negative within-score.
First, we reported the patient-level distribution of each SDOH factor, the mean unadjusted Days in Care per person, and the mean unadjusted Days in Care rate (days per person-year) among each stratum.
Second, we report the ACO-level distribution of patient volume and number of patients with each SDOH factor, as well as the distribution of SDOH proportion across the ACOs.
Third, we use the CMS “Within” disparity method to calculate the relative rate (RR) of days in care (per person-year) by strata, and the adjusted rate difference (RD) for each SDOH factor. Results are reported for ACOs with at least 12 patients in each given stratum to minimize impact of low-volume outliers and ensure a more reliable result. Specifically, we use a two-stage approach to 1) calculate a patient-level risk score estimate using the 50+ clinical risk factors, then 2) calculate a provider-level score estimated for each ACO using the patient-level SDOH stratum as the only risk factor (with the stage 1 patient-level risk score as an offset). This yields a fixed-effect intercept and coefficient for the SDOH factor for each ACO (instead of the random intercept/random slope of the hierarchical model approach). The coefficient of the SDOH factor is then used to estimate the rate difference, with a difference of 0 indicating no disparity and a positive difference indicating more days in care (worse outcomes) for patients with the risk factor compared to the reference group, and a negative difference indicating fewer days in care (better).
References
1. ACO REACH. Cms.gov.https://www.cms.gov/priorities/innovation/innovation-models/aco-reach
2. ACO REACH Model RFA. CMS.gov. https://www.cms.gov/priorities/innovation/media/document/aco-reach-rfa
3. Neighborhood Atlas – home. Wisc.edu.https://www.neighborhoodatlas.medicine.wisc.edu/
4. Segal, M., Rollins, E., Hodges, K., & Roozeboom, M. (2014). Medicare-medicaid eligible beneficiaries and potentially avoidable hospitalizations. Medicare & Medicaid Research Review, 4(1), E1–E13. https://doi.org/10.5600/mmrr.004.01.b01
5. 2022 MUC list & pre-rulemaking resources now available. Cms.gov. https://mmshub.cms.gov/2022/2022-12/2022-muc-list-pre-rulemaking-resources-now-available
Results
Table 16 in the supplemental document reports the number of patients in each stratum, along with the mean observed days in care (per person) and days in care rate (per person-year) in the 2022 REACH dataset (N=169,324 patients). Dual-eligible patients comprise 23.2% of the population, non-white patients 16.2%, and high ADI patients 11.1%. On average, dual-eligible patients have more days in care per person than non-dual (20.9 to 9.7), non-white have more than white (13.6 to 12.1), and high ADI have more than low ADI (14.3 to 12.1). Before adjustment, on a rate basis, dual-eligible patients spend 13.6 days in care per person-year in excess of non-dual eligible (28.3 days vs. 14.7 days), non-white patients spend 2.1 days in care per person-year in excess of white patients (19.7 days vs. 17.6 days), and high ADI patients spend 3.2 days per person-year in excess of low ADI patients (20.7 days vs. 17.5). Please note, the scale of the Days in Care measure is such that a higher number [more days in care] indicates a worse outcome [fewer days at home].
Table 17 in the supplemental document reports the distribution of patient volume and SDOH proportion at the ACO level (n=99 REACH ACOs). The mean ACO has 1710.3 patients, ranging from 60 to 16,390 (median 971, interquartile range 478-2099). The average ACO has 397.4 dual-eligible patients (median 186, IQR 69-418, range 13-4,657), 276.6 non-White patients (median 116, IQR 67-287, range 8-2,660), and 190.0 high ADI patients (median 30, IQR 5-173, range 0-4,464). The average ACO has a dual eligible proportion of 27.8% (median 21.6%, IQR 10.1%-38.1%, range 2.7%-96.3%), a non-white proportion of 20.4% (median 14.1%, IQR 7.4%-26.0%, range 2.0%-94.0%), and a high ADI proportion of 9.9% (median 5.1%, IQR 0.6%-15.0%, range 0.0%-55.8%).
Table 18 in the supplemental document reports the distribution of mean adjusted days in care rate (per person-year) by stratum, as well as the adjusted rate difference between those with vs. without each SDOH factor, among ACOs with at least 12 cases in each strata. At the average ACO, dual-eligible patients spend 6.92 more days in care per person year than non-dual-eligible (median +6.85, IQR +5.53 to +8.14, range +2.38-+13.47). At the average ACO, non-white patients spend 0.74 fewer days in care (that is, -0.74 more days) per person year than white patients (median -0.78, IQR -1.30 to -0.054, range -4.77 to +1.21). At the average ACO, high ADI patients spend 0.84 more days in care per person than low-ADI patients (median +0.76, IQR +0.60 to +1.21, range -0.55 to +2.24).
Interpretation
In unadjusted analyses, it is evident that dual-eligible patients spend typically spend more days in care than non-dual eligible patients (28.3 days per person-year vs. 14.7, difference +13.6). The difference between non-white and white (19.7 days per person-year vs. 17.6 respectively, +2.1) and high and low ADI (20.7 days per person year vs. 17.5 respectively, +3.2) is modest by comparison, but still shows that on average non-white and high ADI patients tend to have more days in care.
At the ACO level, there is a skew in the distribution of volume and SDOH share, with a smaller group of ACOs having a disproportionately large share of patients overall and within strata – the median ACO has 971 patients, while the median by SDOH factor is 186 dual-eligible patients, 116 non-white patients, and just 30 high ADI patients. This skew is also observed in SDOH proportion, in which a majority of ACOs have a sparser SDOH patient population than average (the average SDOH proportions are 27.8% dual-eligible, 20.4% non-white, and 9.9% high ADI, while the median SDOH proportions are 21.6%, 14.5%, and 5.0% respectively). This suggests that some providers care for a disproportionately large population of patients with SDOH factors and so may have a much higher burden to provide high-quality equitable care, while a larger number of providers have relatively few patients with SDOH factors and may find providing equitable care less burdensome.
After adjustment, there remains a substantial disparity in care based on dual-eligible status. At all 98 ACOs for which a score could be calculated (those with at least 12 dual-eligible and 12 non-dual-eligible patients), the dual-eligible patients had a higher rate of days in care (mean +6.92, median +6.85). However, there is substantial variation among ACOs, with some nearly closing the gap (minimum +2.38, 10th percentile +4.58, Q1 +5.53) while others have a much greater disparity (Q3 +8.14, 90th percentile +9.50, max +13.47). This suggests there is meaningful opportunity for many ACOs to improve the quality of care delivered to their dual-eligible patients.
As noted previously, the Days in Care component model (as well as the Nursing Home Transition model) does include dual-eligible status as a risk factor in the final measure specifications, due mainly to systematic differences in the accessibility and ability to pay for different services among dual-eligible patients compared to non-dual eligible patients that lie outside ACO’s control, and to avoid disincentivizing treatment of dual-eligible patients (as dual-eligible status remains a significant predictor of outcomes even after adjusting for clinical factors). In this sense dual-eligible status is not merely an SDOH factor or an indicator for some underlying difference in risk, it reflects a structural difference in patients’ ability to access and pay for services, particularly home- and community-based options that are important to safely remaining at home. While mitigating disparities due to underlying SDOH factors that correlate with dual-eligible status is within the control of a large organization such as an ACO, the structural differences in payment and access for dual-eligible patients vs. non-dual-eligible are not
Note, because dual-eligible status is included in the Days in Care risk model, disparities in the final measure score related to dual-eligible status will be attenuated compared to those observed in the stratified Days in Care model analysis alone (for which dual-eligible status was removed as a risk factor).
Adjusted rate differences for both race and ADI were modest. The RD of negative-0.74 for non-white race suggests that at most ACOs non-white patients tend to have fewer days in care than white patients, although there again is variation with some ACOs having almost no gap while others have a larger disparity in either direction. Interestingly, this reverses the observation in unadjusted results in which non-white patients had more days in care. The causes of this range in outcomes are unclear – for example, it could be that some ACOs are undertreating non-white patients for a given risk profile, but it is also possible that some ACOs are overtreating white patients (resulting in unnecessary extra days in care). When interpreting stratified results such as rate differences, it is important to interpret findings in the context of overall performance. Additionally, while not as extreme as the difference for dual-eligible status, the range of 5.99 days per person year difference (-4.77 to +1.21) is still meaningful in this context, where even a difference of one or two days per person year can correspond to a substantial number of total days across an ACO’s entire population.
The ADI analysis is somewhat limited by sample size – only 64 ACOs had at least the 12 high ADI and 12 low ADI patients needed to receive a score. At the average ACO, high ADI patients tend to have slightly more days in care, with a range in disparity from -0.55 to +2.24. A limitation, particularly in interpretation of results stratified by ADI, is due to ACO REACH being a voluntary model that ACOs choose to participate in. This may lead to some self-selection bias if ACOs are less likely to participate if they serve a higher ADI footprint and limits the generalizability of these findings beyond the ACO REACH model. However, the findings remain valid within the ACO REACH population.
In conclusion, there is substantial evidence of disparities in days at home for dual-eligible patients, while disparities related to race or ADI appear less extreme but still exist to some extent. As noted before, dual-eligible status corresponds to structural difference in patients’ ability to access and pay for certain services, so it may be difficult for ACOs to entirely mitigate these disparities – however, the extent of the disparity varies substantially by ACO, with some almost entirely eliminated, suggesting it can be addressed to at least some extent. While dual-eligible status is included in the measure risk model for this reason, the stratified Days at Home provides a powerful tool for ACOs and CMS to monitor this disparity, while also ensuring that continued disparity is not masked by its inclusion as a risk factor. By comparison, the data don’t currently show a clear systemic disparity based on race or ADI, but there is still a gap in performance between ACOs that demonstrates opportunity to achieve more equitable outcomes. Furthermore, continued future monitoring to ensure new gaps don’t open up and that ACOs are making progress in closing existing gaps (particularly as model participants join and depart, or if the measure is adopted for use in another model or setting) will help ensure more equitable outcomes for all.
Feasibility
Feasibility
The Days at Home measure uses routinely submitted claims data to identify the measure cohort, risk-adjustment variables, and outcome. Administrative claims are frequently used as data sources for quality measures to assess the quality of care delivered by ACOs. A major benefit of using claims data is that there are no additional costs for providers and the administrative burden is low since there is no need to submit additional data to support measure implementation.
Given the initial feasibility assessment, and implementation of the measure, no changes have been made to the measure specifications in response to the feasibility assessment. The measure still uses routinely submitted claims data to identify critical components of the measure specifications (cohort, risk-adjustment variables, and outcome), and there is no cost to providers and low administrative burden on clinician-groups, and their providers, and participating ACOs.
Proprietary Information
Scientific Acceptability
Testing Data
For initial development and validation of the measure in 2021, we used Medicare FFS administrative claims and enrollment information from CYs 2017 through 2018 (January 1, 2017 – December 31, 2018), downloaded from the CMS’s Integrated Data Repository (IDR) and Virtual Research Data Center (VRDC). The measure was initially tested on patients attributed to ACOs participating in the Medicare Shared Savings Program (SSP) for initial validation of the measure, using CY 2018 as a proxy for the performance year and CY 2017 as the pre-performance year (for risk adjustment lookback). This data used for initial development and validation of the measure 2021 was also used to test social risk factors from the following data sources:
• We derived Agency for Healthcare Research & Quality (AHRQ) Socioeconomic Status (SES) Index from the 2013-2017 American Community Survey (ACS)
• We derived Percentage of Residents Living Alone and Percentage of Residents Never Married from the 2014-2018 ACS
• We obtained Density of Primary Care Providers, Density of Specialists, and Density of Hospital Beds from the 2018 Health Resources & Services Administration Area Health Resources File (AHRF)
• We obtained Density of Nursing Home Beds from Nursing Home Compare from November 2020
• We identified Rural Residence (2013 data) from the United States Department of Agriculture Economic Research Service.
For updated testing and maintenance of the measure during 2023-2024, we used Medicare FFS administrative claims and enrollment information from CYs 2021 through 2022 (January 1, 2021 – December 31, 2022), downloaded from CMS’s IDR and CCW/VRDC. We tested the measure using CMMI/ACO REACH beneficiary alignment file in the CMMI/ACO REACH 2022 program, using CY 2022 as the performance year and CY 2021 as the pre-performance year (for risk adjustment lookback).
For the measure cohort, we used the monthly HCC risk scores in CY 2021 and the status of enrollment in Medicare Parts A and B for CYs 2021 and 2022 from the CMS Medicare Enrollment Database (EDB). For risk adjustment:
- We ascertained clinical comorbidities using Medicare FFS institutional inpatient and outpatient claims, as well as non-institutional carrier claims, from CY 2021.
- Frailty indicators were identified using 2021 durable medical equipment (DME) claims.
- The current reason for Medicare entitlement in 2021 came from the Medicare EDB.
- We derived Medicare/Medicaid dual-eligibility status from the 2022 Medicare Master Beneficiary Summary File (MBSF).
In addition, for model performance testing we derived two separate datasets (the “development sample” and the “validity sample”) by splitting the 2022 ACO REACH dataset cohort in half, randomly sampling patients into one of two groups.
The updated testing results for testing and maintenance of the measure during 2023-2024 reflect final specifications that include data from the above-mentioned Medicare sources (Medicare FFS administrative claims and enrollment information from CY 2021-2022, using data from participants aligned in the CMMI/ACO REACH 2022 program. Both 2018 and 2022 data sets were used for analysis as described in the relevant sections; we have indicated which data source all analyses are derived from.
As described above, the initial measure development and validation used data from CY 2017-2018 to test social risk factors’ inclusion in the model. Ultimately, only dual-eligible status was retained as a risk factor in the final measure (Days in Care and Nursing Home Transition component models only). This testing was not duplicated using the 2022 ACO REACH dataset.
Additional testing (most prominently the empiric validity assessment and health equity assessment) was completed more recently using only the 2022 ACO REACH dataset.
Descriptive statistics for the 2022 development and validity samples were not substantially different from the full population and so are not additionally reported here; both samples are directly comparable to the full cohort with the only meaningful difference being the number of patients is halved in each.
Measured entities included all ACO REACH innovation model ACOs active in 2022 (N=99) from a nationally representative geographic footprint.
The ACO Reach model consists of three “components” with slightly different requirements; each participating ACO belongs to one of these tracks. Of the 99 total participating ACOs 8 are in the High Needs component (“high needs ACOs”), 13 in the new entrant component, and the remaining 78 are in the standard component.
High needs ACOs serve beneficiaries with complex needs and are expected to use a model of coordinated care designed to serve these individuals similar to that employed by the Programs of All-Inclusive Care for the Elderly (PACE). To be aligned with a High Needs ACO, in addition to the eligibility criteria (enrolled in Medicare Parts A and B, not enrolled in Medicare Advantage or other Medicare health plan; have Medicare as the primary payer; be a resident of the United States; and reside in a county included in the ACO’s service area), an individual must also have conditions that impair their mobility or neurological function, significant chronic or other serious illness, other chronic or serious illness with multiple unplanned hospitalizations in the past year, or signs of frailty.
Of note, the measure is only implemented for the High Needs component and is not used in performance scoring for Standard and New Entrant ACOs. For testing purposes, we have pooled ACOs from all components to ensure a sufficient population of patients and providers for reliable analysis.
The measure cohort was not based on a sample and included all eligible patients aligned to participating ACOs.
Characteristics of the population in the initial 2018 development dataset (used for testing social drivers/determinants of health (SDOH) risk factors, reported here) are shown in Table 2 of the supplemental document. The 2018 cohort was predominantly White (84.90%), and predominantly aged 65 years or older (84.04%). 22.78% of the cohort was dually eligible for Medicaid, indicating disability and/or low income. Patients also had evidence of serious illness and frailty (with 12.97 % using SNF care, and 40.54% having an HCC risk score ≥ 3.0 in 2021). Per the measure inclusion criteria, 100% of the cohort has an HCC score of ≥ 2.0.
Characteristics of the 2022 ACO REACH dataset cohort are reported in Table 3 of the supplemental document. The cohort was predominantly White (83.83%), and predominantly aged 65 years or older (90.08%). 22.5% of the cohort was dually eligible for Medicaid, indicating disability and/or low income. Patients also had evidence of serious illness and frailty (with 55.35% using DME care, 12.06 % using SNF care, and 38.43% having an HCC risk score ≥ 3.0 in 2021). Per the measure inclusion criteria, 100% of the cohort has an HCC score of ≥ 2.0. While including a much smaller volume of patients, the characteristics of this cohort were extremely similar to those of the 2018 SSP ACO cohort.
Reliability
Measure score reliability testing of the Days at Home measure was performed using split-half methodology. We randomly divided the cohort in half at the patient level, calculated the ACO-level measure scores separately for each half, and compared the results between each half. A high level of agreement between the results for each “test” demonstrates greater reliability (that is, less sensitivity to chance variations in the underlying data) for the measure. We specifically compared the ACO-level intraclass correlation coefficient (ICC) for the final Days at Home outcome metric.1
Because the measure score is calculated from three different models, it was not possible to apply signal-to-noise reliability and therefore we used the split-half approach. The split-half approach results in a single statistic that does not have a distribution.
Reference
1. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychological bulletin. 1979;86(2):420-8
After splitting the cohort randomly in half and calculating the measure independently in each half, the intraclass correlation coefficient for each ACO’s score is 0.7893, indicating very good agreement in score between the two samples. This indicates that the measure is highly consistent and not greatly sensitive to chance variations in the underlying data. The reliability results for this measure meet Battelle's threshold for split-half reliability (>0.6).
The intraclass correlation coefficient for each ACO’s score is 0.7893, indicating very good agreement in ACO-level score between the two samples. This indicates that the measure is highly consistent and not greatly sensitive to chance variations in the underlying data. The reliability results for this measure surpass Battelle’s minimum threshold for split-half reliability (>0.6).
Validity
Face Validity: We engaged a large national technical expert panel (TEP) comprising a diverse group of clinicians, quality measurement and performance improvement experts, and patient advocates (including patients and family caregivers) to provide input on the measure concept and the developing specifications throughout the measure development process. Following presentation of final measure specifications and testing results, CORE asked members of the TEP to respond to a survey assessing the face validity of the Days at Home measure specifications. Each member was asked if they “strongly agree”, “agree,” “somewhat agree,” “somewhat disagree,” “disagree,” or “strongly disagree” with the statement, “The Days at Home measure, as specified, can be used to distinguish between better or worse performance at ACOs or provider groups,” and to provide a rationale for their response.
Construct Validity: To empirically evaluate the measure’s construct validity, we correlated performance on the Days at Home measure among 99 REACH ACOs in CY 2022 to their performance on select other quality measures used by ACO REACH in the same period. We identified the candidate measures as those that might capture quality constructs related to the Days at Home measure including care coordination, avoidance of acute care, and prevention of health decline. The Pearson correlation was calculated for each relating measure. The measures are:
- All-cause unplanned admissions for patients w/ multiple chronic conditions;
- Risk-standardized all condition readmission;
- Timely Follow-up after Acute Exacerbations of Chronic Conditions;
- Consumer Assessment of Healthcare Providers and Systems (CAHPS): Getting Timely Appointments, Care, and Information Summary Survey Measure (SSM);
- CAHPS: Care Coordination SSM;
- CAHPS: Shared Decision-Making SSM;
- CAHPS: Patient Rating of Provider SSM;
- CAHPS: Courteous and Helpful Office Staff SSM; and
- CAHPS: Health Promotion and Education SSM
As claims-based risk-adjusted outcomes measures of similar constructs, we expected strong inverse correlation with the all-cause unplanned admissions and all-condition readmission measures. Higher scores on these measures are a signal that an ACO is not successfully preventing acute exacerbations, which could result from inadequate preventive services and post-acute follow-up care aimed at maintaining health. While the cohort and outcome of these measures differ, there is strong conceptual alignment with the Days at Home measure that we expect to result in strong empirical correlation in ACO performance.
We also expect weak positive correlation with the Timely Follow-up process measure, for which a higher score indicates a higher (better) rate of qualifying follow-up visits following an acute exacerbation of one of six chronic conditions. The empiric association with Days at Home may be attenuated because a) the timely follow-up measure is not risk-adjusted and so may not fully reflect differences in clinical need between ACOs and b) only captures the process of follow-up after acute events but does not measure any other processes or factors that affect patients’ days at home.
The CAHPS measures are a family of patient reported outcome (PRO)-based performance measures that measure patient experiences with healthcare services, which can be used for assessing satisfaction with the quality of care and healthcare services delivery from the patients’ perspective. We anticipated weak positive correlations with the Timely Appointments, Care and Information measure, the Care Coordination measure, the Courteous and Helpful Staff measure, and the Patient Rating. We would expect that patient satisfaction with the coordination timeliness of their care, the engagement of their care staff would reflect ACOs that more proactively and effectively engage these patients, and we would expect this to correspond with improved clinical outcomes including more days at home for patients with complex chronic illness. The association may be limited because a) patient satisfaction with a service or process does not guarantee the quality and will not track perfectly with clinical outcomes, and b) the cohort for these measures is much broader than just patients with complex chronic conditions (that is, they will only partially reflect the experience of those patients in the Days at Home cohort).
We also assessed two other CAHPS measures (Shared Decision-making, and Health Promotion & Education) but did not expect a substantial correlation with Days at Home. In addition to the limitations of the other HCAHPS measures listed above (particularly that they include a broader cohort than just chronically ill patients in the Days at Home target population), patient satisfaction with these concepts is likely further removed from the Days at Home outcome. For example, some patients may be very satisfied with shared decision-making that results in days in care, while others may be dissatisfied with inadequate shared decision-making even if they have more days at home as a result. Similarly, patients may be satisfied with education they feel helps them better manage their condition during their time at home, but which does not reflect the level of engagement and coordination that is really needed to prevent acute events.
Face Validity: Following completion of the measure specifications, CORE surveyed the measure development TEP to assess the face validity of the measure. Of the 21 TEP members, 19 responded to the survey, of whom all 19 (100%) agreed with the given statement. Specifically, 2 members indicated “strongly agree,” 15 indicated “agree,” and 2 indicated “somewhat agree.” No TEP members disagreed that the specified Days at Home measure can be used to distinguish performance among provider groups such as ACOs.
Several TEP members agreed that the measure reflects an important outcome and would incentivize care coordination and home-based care, noting this is currently a gap in measurement. They noted that to perform well on the measure, providers will have to implement practices and interventions across multiple aspects or systems of care to prevent acute care use and support days at home, which will improve the patient experience as well as reduce overall spending.
Several TEP members also expressed support for the measure methodology (particularly the definition to count “days at home”) and risk adjustment approach, suggesting that the measure fairly captures the outcome of interest.
Construct Validity: The Pearson correlation and p-value of ACO performance between Days at Home and each relating measure are as follows, and reported in Table 4 of the supplemental document:
- All-cause unplanned admissions for patients w/ multiple chronic conditions was -0.51 (<0.0001);
- Risk-standardized all condition readmission was -0.39 (<0.0001);
- Timely Follow-up after Acute Exacerbations of Chronic Conditions was 0.15 (0.16);
- CAHPS: Getting Timely Appointments, Care, and Information SSM was 0.36 (0.0003);
- CAHPS: Care Coordination SSM was 0.28 (0.005);
- CAHPS: Shared Decision-Making SSM was 0.07 (0.50);
- CAHPS: Patient Rating of Provider SSM was 0.20 (0.05);
- CAHPS: Courteous and Helpful Office Staff SSM was 0.28 (0.007), and
- CAHPS: Health Promotion and Education SSM was -0.14 (0.17).
The feedback from the TEP demonstrated that the Days at Home measure concept and specifications are sensible to a diverse range of stakeholders with interest in chronic condition management and treatment, and that the measure can fairly represent quality of different ACOs.
In general, the empiric validity testing results were consistent with our hypothesized relationships. Both risk-adjusted outcome measures for which we expected a strong inverse correlation indeed demonstrated a substantial inverse correlation. In addition, three of the four patient satisfaction measures for which we hypothesized a weak positive correlation demonstrated a statistically significant positive correlation, while the two for which we expected no correlation demonstrated no significant correlation. Finally, the timely follow-up process measure for which we expected weak positive correlation demonstrated subtle positive (though not statistically significant) correlation.
These results demonstrate that outcomes of the Days at Home measure indeed track with other related measures of quality while still maintaining independence and capturing a dimension of quality not reflected elsewhere.
Risk Adjustment
The goal of risk adjustment is to account for differences across provider entities in patient demographic and clinical characteristics that might be related to the outcome but are unrelated to quality of care. Patients’ ability to remain at home and avoid acute care may be influenced by degree of illness, financial capacity to receive quality care in home settings, access to healthcare facilities and services, as well as the quality of care they receive. We therefore considered patient demographic factors, comorbidities, and social risk factors to develop our candidate risk variables to better isolate the impact of care quality and decisions in modelling the outcome.
To select risk variables during initial measure development in 2018, we leveraged the final list of candidate risk variables that a fully developed similar measure: the All-Cause Unplanned Admission Rate for Patients with Multiple Chronic Conditions (MCC), referred to here as “the UAMCC measure.” The UAMCC measure is elsewhere used in the Medicare SSP and the CMS Merit-based Incentive Payment System (MIPS). The UAMCC measure assesses the number of acute, unplanned hospital admissions per 100 person-years for patients aged 65 or older with MCCs attributed to participating provider entities. The UAMCC measure patient population is clinically similar to the Days at Home measure cohort of patients with complex, chronic conditions and has a comparable diversity of severity of illness, socioeconomic status, and geographical access to care, despite differences in age eligibility for the cohorts (that is, age 18 or older for Days at Home but 65 or older for the UAMCC measure).
The UAMCC measure development team used a rigorous methodology to select and group candidate risk variables that were strongly predictive of patients’ likelihood of having unplanned hospital visits, such as patient frailty. Therefore, original measure specification adapted the UAMCC measure’s candidate risk variables, including 37 clinical comorbidity variables (defined using hierarchical Condition Categories [CCs] and/or standalone ICD-10-CM codes), 9 cohort-qualifying chronic conditions (defined by CMS’s Chronic Conditions Warehouse [CCW]), 6 variables related to frailty or disability (for example, walking aids, durable medical equipment, and reason for current Medicare entitlement), and age as our list of candidate risk variables for further consideration.
To select variables for consideration in the Days at Home measure in original measure specification, first, we conducted descriptive analyses of the candidate risk variables to verify the conceptual validity of using them for the Days at Home measure. We also examined characteristics of patients in the Days at Home cohort who are eligible for the UAMCC cohort to those who are not eligible for the UAMCC measure cohort, and then compared the frequency of each candidate risk variable between patients in the days at home cohort who are eligible versus not eligible for the UAMCC measure cohort.
Next, we conducted empiric analyses with the candidate risk variables to verify their appropriateness as risk variables for preliminary Days at Home analyses. We calculated the frequencies and bivariate associations with the days in care (as in, days not at home), mortality, and nursing home transition outcomes in the Days at Home cohort. We also calculated the Pearson correlation coefficients between all risk factors. Finally, clinical experts reviewed the risk variables for clinical sensibility.
Finally, we selected final variables based on prevalence, bivariate associations between each risk factor and the measure outcomes, and clinical sensibility.
During measure reevaluation September 2023 – September 2024, we transitioned the measure from CMS’s CCW-27 algorithms, which were used during original measure specification as described above, to the updated CCW-30 algorithm. There was much overlap in CCW-27 and -30 variables, but some factors had more substantial variations including multiple codes or clinical concepts added.
As noted by multiple TEP members and clinical experts in 2018, a patient’s ability to remain safely at home is influenced by the patients’ environment and available resources, known as social determinants/drivers of health (SDOH) factors. While provider entities may have some ability to address or mitigate SDOH factors in care of their patients, the TEP members and experts expressed concern that these factors may be largely out of a provider entity’s control given the cohort and outcome of this measure, especially given concerns about a patient’s ability to obtain adequate in-home support. Adjusting for SDOH factors may have the unintended consequence of establishing inequitable standards for outcomes and quality, while not adjusting for SDOH factors may adversely discourage providers from caring for patients at greater risk. While some stakeholders have advocated that this measure account for SDOH factors, a recent Office of the Assistant Secretary for Planning and Evaluation (ASPE) report has recommended that generally outcomes measures should not adjust for SDOH factors. The decision of which, if any, SDOH factors the measure should include requires considering the incentives and potential consequences for reducing inequities.
We conducted empiric analyses with the 53 candidate risk variables to verify their appropriateness as risk variables for the Days at Home measure. We present the number of patients with each risk factor and the prevalence as a percentage of all patients eligible for the Days at Home measure (n=1,154,779). For each risk factor, we present the odds ratio of death for patients with that diagnosis during 2018, along with the p-value. (The odds ratio is the relative odds of death for patients with that risk factor compared to those without; an odds ratio > 1 indicates patients with that risk factor have greater risk of death). Similarly, we present the odds ratio for transition to a nursing home for a patient with that diagnosis in 2022, along with p-value. We also present the mean days in care for patients with each risk factor, along with the standard deviation and a p-value (obtained using a t-test) to examine if patients with that risk factor are likely to spend more days in care than those without.
We also calculated the Pearson correlation coefficient between each pair of risk variables to assess if any are highly correlated with each other (results not tabulated here). Only one pair of variables – age and current reason for Medicare entitlement – were highly correlated (r = -0.73).
We found that nearly all risk variables were significantly associated with days in care, mortality, and nursing home transitions in bivariate analyses. We included all candidate clinical risk variables, with two exceptions (leaving 50 variables total). We excluded pancreatic disease (pancreatitis) due to very low prevalence in the cohort (<1%), and current reason for Medicare entitlement which was highly correlated with age (correlation coefficient -0.73).
To select risk variables for the Days at Home measure, we originally started with the set of risk factors used in a fully-developed related measure, the Risk-Standardized Unplanned Acute Admission Rate for Patients with Multiple Chronic Conditions (UAMCC). The measure is also currently implemented in the “High Needs Population ACOs” component of the CMS Innovation Center’s ACO REACH Model and elsewhere.
The UAMCC patient population is clinically similar to the Days at Home measure cohort of patients with complex, chronic conditions and has a comparable diversity of severity of illness, socioeconomic status, and geographical access to care. There is a difference in eligibility based on age for the measure cohorts: patients who are age 18 or older are eligible for Days at Home, but patients are required to be 65 or older for the UAMCC measure. The UAMCC measure assesses the number of admissions within the performance year and thus the risk variables selected and grouped by the UAMCC measure (such as patient frailty) were strongly predictive of patients’ likelihood of having unplanned hospital visits.
The Days at Home measure uses the candidate clinical risk variables of the UAMCC measure (defined by Condition Categories (CCs)) for each of the three component models. These consist of 37 clinical comorbidity variables, nine chronic conditions, six variables related to frailty or disability (for example, walking aids, durable medical equipment, and reason for current Medicare entitlement), and age. Two variables (pancreatic disease due to very low prevalence and reason for Medicare entitlement due to high collinearity with age) were originally excluded, and a third (cardiomyopathy due to overlap with heart failure and non-ischemic heart disease) was excluded following testing with the 2022 data (which also supported continued exclusion of pancreatic disease and reason for Medicare entitlement) leaving 50 clinical risk variables. These same clinical risk variables are used in each of the three component risk models of the Days at Home measure.
In addition, beneficiaries’ Medicare-Medicaid dual-eligibility status is included as a risk variable in the Days in Care and Nursing Home Transition component risk models. Dual-eligible patients have structurally different means to pay for various services and may have fewer resources and social support to remain safely at home compared to Medicare-only beneficiaries. Adjusting for dual-eligible status acknowledges that ACOs may have limited ability to address this risk factor and reduces any incentive for ACOs to select against dual-eligible patients. Dual-eligible status is not included in the Mortality risk model.
Table 5 in the supplemental document presents the number of patients with each risk factor and the prevalence as a percentage of all patients eligible for the Days at Home measure, in both the original development dataset (CY2018 SSP ACOs) and the updated ACO REACH testing dataset (CY2022).
In original measure development (tested using the 2018 dataset), upon finalizing the clinical risk model we considered several other potential social drivers/determinants of health (SDOH) as potential risk factors in addition to dual-eligible status – namely AHRQ SES index, urbanicity, specialist and PCP density, hospital and nursing home bed density, and density of individuals never married or living alone. We ultimately did not include other SDOH risk factors (SRFs) for several reasons: they are area-level indicators that may not represent the actual circumstance of each patient, the empirical support for inclusion is weaker, and they require abstraction of data from external sources which could present a problem in measure implementation. In addition, some of these risk factors (particularly physician density and density of hospital or nursing home beds) can be directly addressed and mitigated by DCEs, in which case statistical adjustment would be inappropriate. This testing was not repeated using the 2022 data as the chief concerns with these other variables remain unchanged. Table 6 in the supplemental document lists each of these factors along with the adjusted risk ratio for reference, from the original 2018 development testing.
In 2022, we conducted additional analyses stratifying the measure by three SDOH factors: dual-eligible status (already discussed), local deprivation, and race. Using the Area Deprivation Index (ADI), we dichotomized deprivation as high (above 80th percentile, more deprived) or low (less deprived). We dichotomized race as White or non-White. Table 7 in the supplemental document presents the prevalence of these characteristics in the 2022 ACO REACH cohort (n=169,324) used for this testing.1
Reference
- Neighborhood Atlas - home. (n.d.). Wisc.edu. Retrieved September 24, 2024, from https://www.neighborhoodatlas.medicine.wisc.edu/
Table 8 in the supplemental document lists the Days in Care rate ratio and mortality and nursing home odds ratios (with 95% confidence intervals) in the final model used in original development testing with the 2018 SSP ACO dataset. Table 9 in the supplemental document details the number of patients with each risk factor and the prevalence as a percentage of all patients eligible for the Days at Home measure, using CY 2022 data. The odds ratio (OR) and its 95% confidence interval (CI) lower and upper bounds are reported for the mortality and nursing home transition logistic regression models, while the rate ratio (RR) and its 95% CI bounds are reported for the days in care negative binomial model. Note in Table 9, due to updates in the underlying source codes available, some of the risk factors were modified slightly between 2018 and 2022 analyses.
Table 10 in the supplemental document reports the relationship between ACO-level Adjusted Days at Home and SDOH factors, listing the prevalence of individuals with each SDOH factor among patients aligned to ACOs by quartile of ACO overall performance score. Dual-eligible and non-White patients tend to make up a slightly higher proportion of patients at first- and fourth-quartile ACOs, while high ADI patients are disproportionately at first-quartile ACOs. Note that in all analyses stratifying the measure by dual-eligible status, dual-eligibility is removed as a risk factor from the statistical risk model, though it is retained in stratifications by race or ADI.
To test discrimination of the models, we computed the c-statistic (area under the ROC curve) for the Mortality and Nursing Home Transition logistic regression models. We evaluated the goodness-of-fit of the Days in Care negative binomial count model using the deviance R-squared and the Spearman rank correlation coefficient.
We assessed the calibration of the Days at Home component models using a split-half methodology. We split the cohort randomly in half, fit each model to the first half of data (the development sample), and then used those coefficients in the second half (the validation sample) to confirm the models are generalizable and well-calibrated. We also plotted the observed and expected proportion of days alive spent in care among deciles of expected days in care.
We also assessed the calibration of the model for select patient subgroups (dual-eligible vs. non-dual-eligible, white vs. non-white, and high- vs. low-ADI) by comparing the observed days in care to deciles of predicted (model estimated among the full population).
We conducted empiric analyses with the 53 candidate risk variables to verify their appropriateness as risk variables for the Days at Home measure. We present an unadjusted bivariate analysis for each of the candidate risk factors with each of the three parts of the outcome – days in care, mortality, and transitioning to a nursing home. We present the number of patients with each risk factor and the prevalence as a percentage of all patients eligible for the Days at Home measure (n=169,324). For each risk factor, we present the odds ratio of death for patients with that diagnosis during 2022, along with the 95% confidence interval. (The odds ratio is the relative odds of death for patients with that risk factor compared to those without; an odds ratio > 1 indicates patients with that risk factor have greater risk of death). Similarly, we present the odds ratio for transition to a nursing home for a patient with that diagnosis in 2022, along with 95% confidence interval. We also describe the mean days in care for patients with each risk factor, along with the standard deviation and a p-value (obtained using a t-test) to examine if patients with that risk factor are likely to spend more days in care than those without.
During the measure development we also calculated the Pearson correlation coefficient between each pair of risk variables to assess if any are highly correlated with each other (results not tabulated here). Only one pair of variables – age and current reason for Medicare entitlement – were highly correlated (r = -0.73).
We found that nearly all risk variables were significantly associated with days in care, mortality, and nursing home transitions in bivariate analyses. We included all candidate clinical risk variables, with three exceptions (leaving 50 variables total). We excluded pancreatic disease (pancreatitis) due to very low prevalence in the cohort (<1%), cardiomyopathy due to its conceptual and empiric overlap with heart failure & non-ischemic heart disease, and current reason for Medicare entitlement which was highly correlated with age (correlation coefficient -0.73).
Using the 2022 ACO REACH test dataset, we evaluated the discrimination and goodness-of-fit of the Days at Home component models to assess their ability to differentiate outcomes.
- We found a C-statistic of 0.736 for the Mortality model, which indicates good classification of observed deaths using model predictions. We found a C-statistic of 0.753 for the Nursing Home Transition model, which indicates similarly good model performance. Potential values of the C-statistic range from 0, indicating perfectly poor classification to 0.5, meaning the model predictions are no better than chance, to 1.0, indicating perfect classification; “perfect classification” here implies that patients’ outcomes can be predicted by the specified risk factors alone and that no other factors, including performance of healthcare providers, play a role in patients’ outcomes.
- We evaluated the goodness-of-fit of the Days in Care count model using the deviance R-squared, which we computed to be to be 0.0183. It is important to note that the deviance R-squared is distinct from the “standard” R-squared definition that is often reported for linear regression models and should not be interpreted in the same way; deviance R-squared is a separate measure of fit applicable to count models estimated using maximum-likelihood methods (equal to one minus the ratio of the log-likelihoods of the final model and the null model).1 Deviance R-squared values are typically lower than standard R-squared values; the deviance R-squared of 0.0183 for the Days in Care model is comparable to that observed in other similar count models used in CMS measures (specifically values of 0.060, 0.028, and 0.038 for the count models of the 30-day Excess Days in Acute Care measures for Acute Myocardial Infarction, Heart Failure, and Pneumonia respectively).2
Using the split-half development and validation sample results, we computed discrimination and calibration metrics for each of the component models (Mortality, Nursing Home Transition, Days in Care). Overall, the models fitted in the development sample show very similar performance and results when applied to data in the validation sample.
- Of particular note, the overfitting indices (γ0, γ1) in the validation sample of the mortality model (-0.044, 0.974) and nursing home transition model (-0.036, 0.978) are very close to the development sample (by definition: 0, 1), indicating that those models are generalizable and have high predictive ability.
- Furthermore, the C-statistics based on the validation sample (0.734 for Mortality, 0.752 for Nursing Home Transitions) are approximately equal to those based on the development sample (0.738 and 0.754), indicating that the model discrimination is maintained when applied to new data.
- Similarly, the deviance R-squared of the Days in Care model is similar between the development and validation sample results (both 0.0184 respectively), which indicates similar goodness-of-fit in both samples.
The calibration deciles plots of the Days in Care model for both the development and validation samples show close alignment between the observed and expected days in care (as proportion of days alive), with the only deviation being in the highest decile (in which the predicted values exceed the observed by about 20% in both plots). This demonstrates the generalizability of the risk model as its calibration is maintained even in data on which it was not fit.
Finally, calibration plots based on dual-eligible status, race, and ADI demonstrate that the model is well-calibrated among these populations of interest, and that there are not systemic differences between these populations that the model fails to capture. Similarly to the overall calibration plots among the development and validation samples, the observed days in care per survival day is very close to the predicted value in every decile except the highest, in which the predicted values exceed the observed by some margin. Among dual-eligible patients, both observed and predicted values are greater than those among non-dual-eligible, which is consistent with other observations that dual-eligible patients are at greater risk for days in care; however, the alignment between observed and predicted within each subgroup indicates that this difference in risk is well captured by the statistical model. There is a much more subtle difference in calibration plot between white and non-white patients; non-white patients have slightly more observed and predicted days in care by decile but good calibration within each subgroup. Only for ADI does calibration begin to deviate, with a tendency for the model to under-predict days in care among high ADI patients compared to observed; this could be related to potential disparities related to ADI as discussed further in Section 5.
References
1. Cameron AC, Windmeijer F. R-Squared Measures for Count Data Regression Models with Applications to Health-care Utilization. Journal of Business & Economic Statistics. 1996;14(2):209-220.
2. YNHHSC/CORE. Condition-Specific Excess Days in Acute Care Measures Annual Updates and Specifications Report. 2021.
Use & Usability
Use
ACO Level
Usability
Freed et al. (2022) evaluated the viability of using DAH as a quality measure for providers participating in Alternative Payment Models (APMs) such as ACO that focused on the care of seriously ill beneficiaries1. The study found that ACOs with high DAH scores spent more on Durable Medical Equipment (DME) as a percentage of total spending than ACOs with low DAH scores. In other words, ACOs with high DAH scores prioritize or use more Durable Medical Equipment relative to their overall expenditures than those with lower DAH scores. This could indicate that high DAH score ACOs might have a greater reliance on or investment in such equipment as part of their care strategies. Similarly, Kaufman et al. (2021) tried to characterize spending on seriously ill beneficiaries in ACOs with Medicare Shared Savings program (MSSP) contracts and the association of spending with ACO shared savings2. The results suggest that increased investment in serious illness spending may lead to a reduction in per capita expenditures overall for ACOs. Coupled with findings of Freed et al. (2022), these studies suggest that ACOs investments in supporting beneficiaries in their home or community setting by offering services such as home-based care (e.g., DME), preventive services, and post-acute follow-up may be associated with reduced utilization and related costs of acute services for both providers and beneficiaries.
References
1. Freed S, Kaufman BG, Harold C, Saunders RS. Using a home time measure to differentiate ACO performance for seriously ill populations. Journal of the American Geriatrics Society. 2022;70(9):2666-2676.
2. Kaufman BG, Anderson D, Bleser WK, et al. Association of ACO shared savings success and serious illness spending. J Healthc Manag. 2021; 66(3): 227-240.
The Days at Home measure was fully implemented beginning 2023, with only one measurement year completed. Without at least two full years of performance data, it is not possible to evaluate trends in performance. To date, no concerns have been raised from measured entities has been reported to the measure developers. ACOs participating in the REACH model have several opportunities to ask questions and provide feedback (such as through webinars, through a Q&A support tool, and the ACO REACH Helpdesk).
During measure development, we convened a Technical Expert Panel (TEP) including among other members several ACO clinicians and quality improvement leaders. These individuals provided valuable feedback throughout the development process; for example, they agreed that the measure is patient-centered and easy to understand and promotes holistic patient care and can improve care coordination; on the measure cohort, they agreed with no diagnosis-related exclusion criteria, and supported risk adjustment for various clinical and social risk factors. TEP members also offered feedback to inform decisions to carefully consider the various clinical settings, such as Skilled Nursing Facilities, when defining the measure outcome of “days in care”. Along with that of the other TEP members, we carefully considered feedback provided by panelists who were clinicians and quality leaders in ACO settings. This feedback particularly informed our decisions to consider planned versus unplanned admissions and unintended consequences, leading to specifically counting all acute care use as whole days not at home, or “days in care”.
The Days at Home measure was fully implemented beginning in 2023, with only one measurement year completed. Without at least two full years of performance data, it is not possible to evaluate trends in performance.
We have no unexpected findings to report at this time.
Comments
Staff Preliminary Assessment
CBE #4555 Staff Assessment
Importance
Strengths:
- This new measure is intended to incentivize accountable care organizations (ACOs) to reduce excessive care in acute and post-acute are settings, including skilled nursing facilities (SNFs), by increasing the number of days at home.
- The developer cites evidence that increasing days at home has been associated with improved clinical outcomes and cost savings for both patients and providers.
- The developer posits that this measure supports maintenance of home-based care, prevention of health decline, reduction in acute care use, and decreases in long-term institutional care, ultimately benefiting Medicare patients by reducing unnecessary hospitalizations. The developer states that this measure will encourage improved care coordination between providers and lists evidence-based actions ACOs can take to increase days at home. These include delivery of appropriate and timely primary and end-of-life care services and improved care coordination and care transitions.
- A logic model diagram is provided, depicting that ACOs establish relationships with patients with complex, chronic conditions and then offers services (e.g., care coordination, home-based care, preventative services, post-acute care follow-up). The diagram shows that this will lead to reduced acute care needs, thus leading to better clinical outcomes and reduced costs.
- The developer posits that this measure addresses a significant gap in Medicare measurement by incentivizing ACOs to enhance care coordination and reduce unnecessary acute care use. Additionally, the developer states that the measure is unique with no competing measures.
- Medicare fee-for-service (FFS) data from 2021 and 2022 show some variability in days at home, with decile ranges from 315.19 to 327.84 days.
- The developer provides evidence indicating that more days at home correlate with better self-reported health status, social activity, and lower depression, making this measure highly relevant and meaningful to patients.
- The measure has been validated against other patient-centered outcomes, demonstrating its effectiveness in reflecting patient preferences and enhancing quality of life, particularly for the elderly with chronic conditions.
- Lastly, the developer notes that its technical expert panel (TEP) included patient and advocate representatives who supported the measure, emphasizing its role in providing patient-centered, whole-person care and reducing the burden on patients with complex chronic illnesses.
Limitations:
- As a new measure, performance gap results are not required. However, given the performance gap results provided, it is unclear what quality impact would be with respect to a difference of roughly 12 days (between decile 1 and 10), including whether such differences can determine a worse or better performing provider. The developer should consider providing additional evidence for maintenance reviews to support whether 12 days is a meaningful gap in care.
Rationale:
- This new measure aims to incentivize accountable care organizations (ACOs) to reduce excessive care in acute and post-acute settings, such as skilled nursing facilities (SNFs), by increasing the number of days patients spend at home, which has been linked to improved clinical outcomes and cost savings. It encourages ACOs to enhance care coordination and deliver timely primary and end-of-life care services, which are expected to reduce acute care needs, thus improving clinical outcomes and reducing costs. For maintenance review, the developer should consider providing additional evidence to support whether 12 days is a meaningful gap in care. What impact on health care costs and/or patient outcomes occurs with differences across deciles.
Feasibility Acceptance
Strengths:
- The measure utilizes routinely submitted claims data to define the measure cohort, risk-adjustment variables, and outcomes, leveraging administrative claims, minimizing financial and administrative burdens on providers.
- The developer notes that no changes have been made to the measure given the feasibility assessment and implementation of the measure. The measure is not a proprietary measure and no proprietary components.
Limitations:
- None.
Rationale:
- The measure uses routinely submitted claims data to define its cohort, risk-adjustment variables, and outcomes, maintaining its feasibility and non-proprietary nature without additional financial or administrative burdens on providers.
Scientific Acceptability
Strengths:
- Data Sources and Dates: Data used for testing were sourced from Medicare Fee-for-Service administrative claims and enrollment information during the period of CY2021-CY2022. The reliability has been calculated using 2 years of data, and so the reliability results are only applicable if the measure is also calculated using two years of data.
- Accountable Entity-Level Reliability: The developer conducted split half reliability testing at the accountable entity-level. A single ICC value of 0.789 which is above the threshold for ICC values.
Limitations:
- None.
Rationale:
- The results demonstrate sufficient reliability at the accountable entity level.
Strengths:
- The developer provided face validity evidence demonstrating consensus among the TEP (N=19) with the statement ""The Days at Home measure, as specified, can be used to distinguish between better or worse performance at ACOs or provider groups”. The developer also provided an association study that demonstrated correlations with selected ACO measures: All-cause unplanned admissions for patients w/ multiple chronic conditions; Risk-standardized all condition readmission; Timely Follow-up after Acute Exacerbations of Chronic Conditions; Consumer Assessment of Healthcare Providers and Systems (CAHPS): Getting Timely Appointments, Care, and Information Summary Survey Measure (SSM); CAHPS: Care Coordination SSM; CAHPS: Shared Decision-Making SSM; CAHPS: Patient Rating of Provider SSM; CAHPS: Courteous and Helpful Office Staff SSM; and CAHPS: Health Promotion and Education SSM
- The developer conducted statistical risk adjustment, based on a conceptual model, selecting risk factors that have a significant correlation to the outcome. The developer also explored social risk factors, such as dual eligibility and included these in the final model. The developer reported c-statistics of 0.736 and 0.753, indicating good model discrimination.
Limitations:
- In general, a new measure submission would provide person- or encounter-level validity evidence. The data are largely from administrative or well studied data sources, so the omission may be less problematic. For accountable entity validity, causal claims based on association (correlation) studies alone are prone to bias (i.e., confounding due to a common cause cannot be ruled-out). Additional support from mechanism studies that confirm the existence of a suitable (plausible) mechanism capable of accounting for the observed correlation would strengthen the causal claim. Otherwise, statements about the relative magnitude of the observed correlations and whether those magnitudes are greater or lesser than what one might anticipate are difficult to evaluate. Going forward, additional studies that either rule-out potential confounding or describe features of potential mechanisms will strengthen causal claims. The lack of variation in the Importance Table does not support a causal association between the entity and the measure focus.
Rationale:
- A new measure should at least include an argument for the validity of the data elements used in the measure specification. The data are largely from administrative or well studied data sources, so the omission may be less problematic. Additionally, going forward, additional studies that either rule-out potential confounding (in addition to risk-adjustment) or describe features of potential mechanisms will strengthen causal claims.
- The risk adjustment methods used are appropriate and demonstrate variation in the prevalence of risk factors across measured entities, contribute to unique variation in the outcome, and show the impact of risk adjustment for providers at high or low extremes of risk. The model performance is acceptable.
Equity
Strengths:
- The developer’s analysis of the measure within the ACO REACH model focused on Medicare-Medicaid dual-eligible (DE) status, ADI, and race (non-white and white) to identify and address potential disparities.
- The Days in Care model was used for reporting due to the challenges of maintaining statistical rigor when combining three separate risk adjustment models for the Days at Home measure.
- To analyze disparities within ACOs, the CMS Disparity Method was used, focusing on "within" differences in quality between patients with and without specific social determinants (SDOH) of health factors.
- This method calculates the relative rate (RR) and adjusted rate difference (RD) of days in care per person-year for each SDOH factor, ensuring robust results by only including ACOs with sufficient patient volumes in each stratum.
- The analysis process involves first reporting unadjusted days in care at both the patient and ACO levels, stratified by SDOH factors. Subsequently, a two-stage approach is used to calculate a risk-adjusted provider-level score for each ACO, which helps in identifying disparities by comparing days in care for patients with a given SDOH factor against those without, aiming to highlight areas needing targeted improvements in care delivery.
- After adjustments, dual-eligible patients at the average ACO spend significantly more days in care per person-year compared to non-dual eligible patients. Interestingly, non-white patients spend slightly fewer days in care per person-year than white patients, while high ADI patients spend more days in care than those from low ADI areas, highlighting disparities in care needs and outcomes among these groups.
Limitations:
- None.
Rationale:
- The developer's analysis within the ACO REACH model utilized the Days in Care model and the CMS Disparity Method to address potential disparities based on Medicare-Medicaid dual-eligible status, ADI, and race. The findings revealed disparities, particularly for dual-eligible patients who experienced more days in care compared to their non-dual eligible counterparts, while non-white patients had slightly fewer days in care than white patients, and high ADI patients had more days in care than those from low ADI areas.
Use and Usability
Strengths:
- This new measures is currently in us within the CMS Innovation Center's ACO REACH Model.
- The developer refers to two studies, indicating that ACOs' investments in home-based care, preventive services, and post-acute follow-up can reduce the need and costs associated with more intensive acute services.
- ACOs have multiple channels for feedback and inquiries, including webinars and a dedicated helpdesk. The developer states that no concerns have been raised from measured entities to date.
- During the measure’s development, the developer notes that feedback from a TEP comprising ACO clinicians and quality leaders shaped the measure to be patient-centered and inclusive of various clinical settings and risk factors, emphasizing holistic care and improved care coordination. Since the measure was recently implemented in 2023 and only one year of data available, the developer notes that it is currently not possible to assess performance trends.
- The developer reports no unexpected findings have been identified.
Limitations:
- None.
Rationale:
- This new measure is currently utilized within the CMS Innovation Center's ACO REACH Model. Studies referenced by the developer suggest that ACOs' investments in home-based care, preventive services, and post-acute follow-up can significantly reduce the need and costs associated with more intensive acute services. Although the measure was only implemented in 2023 and lacks sufficient data for trend analysis, no unexpected findings or concerns have been reported, and ACOs benefit from multiple feedback channels including webinars and a dedicated helpdesk.
Committee Independent Review
CBE 4555 Should Not Be Endorsed
Importance
As defined, this measure has very limited value because the denominator is limited to a subset of the Medicare beneficiaries who are “attributed” to the Accountable Care Organizations participating in the REACH ACO demonstration program. It is impossible to use the measure as it is defined in order to determine whether Medicare beneficiaries who are part of REACH ACOs spend more or fewer days at home compared to beneficiaries in other ACOs or to beneficiaries who are not part of any ACO. The measure provides no information at all about differences in days at home for REACH ACO patients that have risk scores below 2.0.
Feasibility Acceptance
Since the measure is calculated entirely from claims data, it would be feasible to calculate the measure for all Medicare beneficiaries and make it far more meaningful for quality improvement than the narrow version that is proposed.
Scientific Acceptability
The developer did not provide an adequate assessment of reliability for the measure. Since the explicit purpose of the measure is to compare ACOs to each other, it is important to carefully assess the probability of misclassification for individual ACOs and the reliability of the measure over time, and that was not done. This measure should not be endorsed without a better assessment of its reliability.
Although the measure is represented as counting the average number of days that Medicare beneficiaries spend at home or in community settings, the actual number of days at home is adjusted by an estimate of the beneficiary’s risk of death and by a “policy-based” estimate of the beneficiary’s “risk of transitioning to a long-term nursing home.” The developer indicated that these adjustments had very little impact on most ACOs, so it is not clear that there is any value in including them. No information was presented regarding whether these adjustments had a large impact on individual ACOs and whether the reasons for those impacts were investigated to ensure they made sense.
It is notable that the measure’s mortality adjustment is ostensibly designed to avoid creating an incentive for ACO providers to withhold appropriate care, yet the developer has refused to adjust for mortality in its readmission measures even though the same potential incentive would exist there.
The deviance R-squared for this model was determined to be only 0.0183; since the statistic can range between 0 and 1, this indicates a very poor model fit. The developer says it is “not trying to maximize the predictive ability of the model,” but it provides no analysis indicating the reasons for the poor fit and whether the poor fit is due to problems with the model or unwarranted variation in care.
Equity
Only those beneficiaries with a Hierarchical Condition Category (HCC) score >= 2.0 in the previous year are included, so no assessment is made of how much time is spent at home by the large number of other attributed beneficiaries. Since HCC scores depend on the completeness and accuracy of diagnostic coding as well as actual differences in patient health status, patients with limited access to healthcare may have lower HCC scores, and the measure would fail to identify disparities in care between those patients and other patients.
Use and Usability
The calibration plots show underprediction in most of the higher deciles and very large overprediction in the highest decile. The developer says this is “a function of the underlying data rather than the model’s predictive ability,” which implies that somehow the model is right and the data are wrong. The developer says it “suspects that the model overpredicts at the higher deciles due to the complex comorbidities of the extreme cases with high numbers of days in care,” but apparently no effort has been made to improve the model to address this. As a result of these problems, it is not clear why anyone should rely on this measure to indicate either the presence or absence of performance problems at individual ACOs.
None of the analyses provided by the developer support using this measure outside of REACH ACOs, yet it appears that CMS is planning to calculate the same measure for non-REACH ACO participants in order to establish performance benchmarks for the REACH ACO participants. (The CMS PY 2025 Quality Measurement Methodology Report states: “When calculating Quality Measure scores for REACH ACOs, CMS will also calculate scores for non-ACO REACH provider groups. CMS will use scores from both REACH ACOs and non-ACO REACH provider groups to create the QPB distributions for evaluating performance.”)
Summary
This measure should not be endorsed. It is desirable to assess the time Medicare beneficiaries spend at home, but this measure does so only for a small number of beneficiaries “attributed” to organizations participating in the ACO REACH program. This makes it impossible to determine whether beneficiaries receiving care from REACH ACOs spend more or fewer days at home than (1) those not attributed to an ACO, (2) beneficiaries who are receiving care from other types of ACOs, or (3) beneficiaries enrolled in Medicare Advantage plans. Moreover, it only assesses days at home for the subset of beneficiaries with an HCC risk score of 2.0 or greater, even though many beneficiaries with lower risk scores have preventable admissions. Rather than simply counting the number of days at home, the measure uses a complex formula based on the estimated risk of mortality and the estimated risk of “transitioning to a long-term nursing home,” which makes the measure impossible to understand and also makes it difficult to understand the reasons for any differences between ACOs. The “policy-based” adjustment for nursing home transition risk is inappropriate because it is not intended to more accurately measure outcomes but to “incentivize community-based care.” The risk adjustment model appears to perform poorly and the reliability of the measure was not assessed adequately. Use of this measure could be misleading to patients and unfairly reward or penalize individual ACOs.
Agree with staff assessment
Importance
Agree with staff assessment
Feasibility Acceptance
Agree with staff assessment
Scientific Acceptability
Agree with staff assessment
Agree with staff assessment
Equity
This measure includes comparisons by SDOH which is very important. Agree with staff assessment
Use and Usability
Agree with staff assessment
Summary
I think that future measures should apply the same approach to traditional Medicare and Medicare Advantage programs so that patients across their coverage options.
Refer to comments above
Importance
The measure only uses patients from ACO REACH demonstration. Should expand to larger more representative population
Limited quality impact as narrow population
Feasibility Acceptance
Claims data
Scientific Acceptability
Data is from administrative data and other well-studied data sources.
Data is from administrative data and other well-studied data sources. Need to see additional studies to address any confounding variables.
Equity
The data showed disparities especially for dual eligible
Use and Usability
Measure was implemented in 2023 and there is not enough data for trend analysis therefore we do not know about unexpected consequences or concerns based on what has been reported.
Summary
Refer to comments above
Support
Importance
Agree with staff. Appreciate the new focus on patient quality of life (time outside of healthcare settings) not necessarily events like readmissions and complications.
Feasibility Acceptance
Agree with staff.
Scientific Acceptability
Agree with staff.
Agree with staff. As a new measure, will need refinement as we gain experience with this measure, assess impactability.
Equity
Agree with staff.
Use and Usability
Agree with staff.
Summary
Good patient-centered measure.
More information needed
Importance
As has been noted in other comments, the measure is limited to comparison within REACH ACOs only. It would be good to have estimates of the measure for larger populations and entities to understand how the variability within the REACH ACO sample compares to variability in a larger sample. This would also help in assessing whether the 12 day 10-90th decile variation is important.
Feasibility Acceptance
Claims based measure.
Scientific Acceptability
The additional information from multiple split sample tests is helpful and reasures about the reliability.
I would like some discussion of the decision to weight the adjustment for SMR at 1 and the Nursing home transition ratio at 0.5. These seem like arbitrary adjustment.
Equity
DE and social determinants included in risk model.
Use and Usability
New measure. Would expect more information on how used and how useful from providers on maintenance renewal.
Summary
It's an interesting measure but to evaluate it, need more information on how much variance is important, and basis for weighting mortality risk at 1 and nursing home transition at 0.5
Do Not Support
Importance
Limited population to be measured. Unclear how differences in days at home measures a providers performance.
Feasibility Acceptance
Agree with prelim assessment
Scientific Acceptability
Agree with prelim assessment
Potential for bias at the entitly level
Equity
Agree with prelim assessment
Use and Usability
Limited population/reach, lack of trend data
Summary
measures should be applied to all Medicare and Medicare Advantage populations versus a subset. additional data needed on variance and how that applies to provider performance.
Feel this measure is very…
Importance
agree with staff assessment
Feasibility Acceptance
agree with staff assessment
Scientific Acceptability
- ADI is not meant to be used as a dichotomous variable so the analysis of how relevant that variable is was not adequate
- ADI clearly has an impact on model performance in the analytics provided by the measure developer.
- Risk adjustment still doesn’t look like it performs very well, even if using the deviance r square. C-statistic is too low for a publicly used measure of performance or payment.
- White vs. non-white for race is not sufficient to pick up unique differences in minority groups.
- Validity testing is inadequate.
Equity
- White vs. non-white for race is not sufficient to pick up unique differences in minority groups.
Use and Usability
As stated it should be used for a very limited and focused purpose.
Summary
Feel this measure is very close to being where it needs to be for its stated purpose, but needs a some performance improvement (or better performance metrics for validity).
no additional comment
Importance
no additional comment
Feasibility Acceptance
no additional comment
Scientific Acceptability
no additional comment
no additional comment
Equity
no additional comment
Use and Usability
no additional comment
Summary
no additional comment
Public Comments
This Measure Should Not Be Endorsed
This measure should not be endorsed. Although it is desirable to assess how much time Medicare beneficiaries spend at home rather than in the hospital or other institutions, this measure does so only for a small number of Medicare beneficiaries who are “attributed” to the Accountable Care Organizations (ACOs) that participate in the ACO REACH demonstration program. As a result, it is impossible to determine whether beneficiaries attributed to these ACOs spend more or fewer days at home than those who are not attributed to the ACOs, whether beneficiaries attributed to REACH ACOs spend more or less time at home than beneficiaries who are part of other ACOs, or whether beneficiaries attributed to REACH ACOs spend more or less time at home than beneficiaries enrolled in Medicare Advantage plans. Moreover, rather than simply counting the number of days at home, the measure adjusts the count using a complex formula based on the estimated risk of mortality and the estimated risk of “transitioning to a long-term nursing home,” which makes it impossible to understand and impossible to determine the reasons for differences between ACOs. The “policy-based” adjustment for nursing home transition risk is inappropriate because it is not intended to more accurately measure outcomes but to “incentivize community-based care.” The risk adjustment model appears to perform poorly and the reliability of the measure was not assessed adequately. Use of this measure could be misleading to patients and unfairly reward or penalize individual ACOs.
Problems with the Denominator
Problems with the Numerator
Inadequate Risk Adjustment Methodology
The developer assessed the goodness-of-fit for the Days in Care risk adjustment model using the deviance R-squared statistic. The deviance R-squared for this model was determined to be only 0.0183; since the statistic can range between 0 and 1, this indicates a very poor model fit. The developer tries to justify the low value by saying that it is “comparable” to that observed in the models it developed for Excess Days in Acute Care, but the value for this model is much lower than those calculated in the other models. The developer did not report any of the many other measures of model fit that are available, nor was any effort made to identify the sources of unexplained variation. The calibration plots in Figures 4 and 5 show underprediction in most of the higher deciles and very large overprediction in the highest decile, but there is no indication that any efforts were made to develop better risk adjustment models.
Failure to Adequately Assess the Reliability of the Measure
The developer did not report signal-to-noise (inter-unit) reliability for the measure, claiming that “it is not possible” to apply signal-to-noise reliability because the measure score is calculated from three different models. Signal-to-noise reliability is a function of the variance between entities compared to the total variance, so it can be calculated regardless of how the predicted values are determined. In fact, in The Reliability of Provider Profiling: A Tutorial, John Adams explains how to do this in a chapter on “The Reliability of Composite Scores.”
Instead, the developer calculated an intraclass correlation coefficient (ICC) based on a single split sample. A single split sample is not an adequate way of estimating an ICC (Nieser KJ and Harris AHS, “Split-sample reliability estimation in health care quality measurement: Once is not enough.” Health Services Research 2024;50:e14310) and an overall ICC does not assess the reliability of the measure for individual ACOs.
The most appropriate way to assess the reliability of the measure is to calculate the misclassification probability for individual ACOs, and that was not done. This measure should not be endorsed without a better assessment of its reliability.
Response to Public Comment submitted by Harold Miller
Thank you for taking the time to comment on the Days at Home measure. We (Yale/CORE) provide a detailed response to the comment below.
The purpose of the Days at Home (DAH) measure is to assess care provided to FFS beneficiaries, with each ACO’s performance evaluated compared to its peers. The ACO REACH model by design only includes Medicare Fee-For-Service (FFS) beneficiaries and the measure was developed for that context, to assess the relative quality of care provided by these specific organizations participating in this model. Other use cases may use different comparison populations. Assessment among FFS beneficiaries is a standard practice across many CMS quality programs. It is not intended to generalize performance among non-FFS populations for which other measures can or have been developed. While these are valid research questions, some of them are outside the scope of this measure as a performance assessment tool in its intended use and setting. Furthermore, the measure intends to assess the patient-centered ‘days at home’ for those patients with complex, chronic conditions, or patients of higher risk, at greater risk for use of and costs associated with acute care use. The measure counts ‘days in care’ and makes adjustment for greater risk of mortality and risk of transitioning patients to a nursing home to guardrail against inappropriate nursing home use or care that would lead to fewer days at home and poorer outcomes. The Days at Home Measure is in use in the ACO REACH Model and has undergone testing supporting the scientific acceptability of the measure and has met all of the criteria for measure endorsement. We respond to specific concerns within the public comment, below.
Denominator
Comment: Exclusion of Medicare Beneficiaries Not Attributed to REACH ACOs. The denominator of this measure is limited to Medicare beneficiaries who are “attributed” to the Accountable Care Organizations participating in the REACH ACO demonstration program. As a result, it is impossible to use the measure to determine whether Medicare beneficiaries who are part of REACH ACOs spend more or fewer days at home compared to beneficiaries in other ACOs or to beneficiaries who are not part of any ACO.
Response: This comment accurately describes the design of the ACO REACH Model and the use of this Days at Home measure in the model. The ACO REACH Model uses performance-based payment adjustment for its aligned participants and their beneficiaries, and the measure was designed for use among this population, which is limited to the Medicare FFS population. As specified, it is not applicable to other payers, such as Medicare Advantage, due to the capitated payment structure. This comment addresses implementation and use of the measure; as the measure is currently used and specified, it is not intended to compare beneficiaries’ days at home between Model Participants and those outside of the ACO REACH Model; instead, it is intended to allow comparison of performance among REACH model participants. The decision to expand use of this Days at Home measure for beyond the ACO REACH Model lies with Centers for Medicare and Medicaid and the Innovation Center.
Comment: Exclusion of Beneficiaries With Low Risk Scores. Only those beneficiaries with a Hierarchical Condition Category (HCC) score >= 2.0 in the previous year are included, so no assessment is made of how much time is spent at home by the large number of other attributed beneficiaries. Since HCC scores depend on the completeness and accuracy of diagnostic coding as well as actual differences in patient health status, some ACOs may have higher or lower performance on the measure solely because of differences in how their beneficiaries are coded.
Response: The goal of the ACO REACH Model is to lower the cost of care for Medicare beneficiaries while maintaining or improving the quality of care provided. The ACO REACH Model strategy provides achievable performance criteria that aim to incentivize changes in care delivery that reduce unnecessary utilization while improving quality of care. The Days at Home measure is one of five quality measures used in this model.
The ACO REACH Model uses the Days at Home measure only for High Needs ACOs that serve FFS Medicare beneficiaries with complex needs, as these patients are at the greatest risk for spending subsequent days in care. This provides insight as to why the inclusion criteria for Days at Home defines a Hierarchical Condition Category (HCC) score >= 2.0 in the previous year. Furthermore, HCC scores are used in many quality measures, and presumably, incompleteness and inaccuracy in diagnostic of the HCC scores would have similar impact in differentiating between higher and lower performers across many measures, inclusive of this Days at Home measure. The CORE measure development team takes this point into consideration.
Comment: Exclusion of Beneficiaries on Medicare Advantage. Beneficiaries with Medicare Advantage plans are excluded from the measure, making it impossible to compare the performance of REACH ACOs to Medicare Advantage plans.
Response: As stated above, the measure is specified to only include Medicare Fee-for-Service (FFS) patients to align with the ACO REACH Model. The quality measures in the model are used for payment adjustment, with REACH ACOs compared to the performance of other REACH ACOs. Due to Medicare Advantage’s capitated payments, beneficiaries of Medicare Advantage could not be included in this model.
Numerator
Comment: The Numerator is Not “Days at Home.” Although the measure is represented as counting the average number of days that Medicare beneficiaries spend at home or in community settings, the actual number of days at home is adjusted by an estimate of the beneficiary’s risk of death and by a “policy-based” estimate of the beneficiary’s “risk of transitioning to a long-term nursing home.” The developer indicated that these adjustments had very little impact on most ACOs, so it is not clear that there is any value in including them. No information was presented regarding whether these adjustments had a large impact on individual ACOs and whether the reasons for those impacts were investigated to ensure they made sense.
Response: The intent of adjusting for an ACO’s risk of mortality and risk of transitioning to a nursing home was to minimize risk of unintended consequences of the measure as the measure enters use on a pay-for-performance basis and as ACOs respond to the resulting change in incentives. An ACO whose providers withhold appropriate acute care (that would count as a ‘day in care’) do so at the cost of their patient’s health, potentially leading to poorer health, and the most severe outcome of death. Experts CORE engaged with during measure development felt a similar adjustment should be made for risk of transitioning to a nursing home, so providers at ACOs would not, at the cost of the maintaining or improving their Days at Home score, prematurely transition their patients to a nursing home without first exploring home- and community- based services to keep them at home longer. So, while a patient’s nursing home is treated as their “home” once admitted, it was important for the measure to have guardrails for nursing home use.
As our testing demonstrates, the mortality and nursing home risk adjustments have the greatest impact on a few higher and lower performers.
As shown in Table 15 of the submission form, the Days in Care model provides most of the quality signal, with the nursing home and mortality adjustments mostly affecting those with more extreme outlying performance. The effect of including mortality and nursing home transitions yields a range of -5.02 to +2.22 “extra” days at home compared to using Days in Care only, but for the middle 80% of ACOs the difference is between -0.37 and +0.54, and the interquartile range is between -0.07 and +0.19 days at home per person-year. By comparison, the interquartile range for the measure score itself is much greater, from 321.7 to 325.3, which demonstrates that for a large majority of ACOs, the days in care component alone is a reasonable proxy for performance on the whole measure.
Finally, the testing data used for this new measure was not based on entities directly incentivized to perform well. As of 2022, the measure was only used for monitoring and confidential reporting among the handful of High Needs ACOs and was not used at all among the other REACH ACOs included in the test dataset. The impetus for these adjustments is in anticipation of how ACOs could respond to the incentives of measure implementation that create other adverse outcomes for their patients. Monitoring for such unintended consequences is a key part of maintenance reevaluation for measures already in use, but as this is a new measure there is no available data reflecting how ACOs’ responses to the new incentives would affect their patients’ outcomes. However, we heard strong stakeholder concern about these possibilities in particular as being vulnerable to unintended consequences and implemented these adjustments as a mechanism to guard against them.
Risk Adjustment
Comment: The developer assessed the goodness-of-fit for the Days in Care risk adjustment model using the deviance R-squared statistic. The deviance R-squared for this model was determined to be only 0.0183; since the statistic can range between 0 and 1, this indicates a very poor model fit.
The calibration plots in Figures 4 and 5 show underprediction in most of the higher deciles and very large overprediction in the highest decile, but there is no indication that any efforts were made to develop better risk adjustment models.
Response: The deviance R-squared is similar to what was shown in other similar measures or studies, specifically being on the order of that of the three endorsed condition-specific Excess Days in Acute Care measures (AMI, heart failure, and pneumonia). While those measures use a very similar modeling approach, the Days in Care model has some key differences that likely contribute to a lower deviance R-square – most notably, modelling outcomes for a broader population over a full calendar year rather than within only the 30 days after specific inpatient discharges. Because the intent is to account for patient risk factors, we are not trying to maximize the predictive ability of the model. The outcomes will be affected by the care received by the patients and ACO plays a significant role in the care of patients. We expect some variation in the outcomes will be explained by the care delivered, which is what we intend to assess. We do not include variables that may be predictive of the outcomes but reflect the care provided by an ACO.
The model is generally well-calibrated except for the highest decile of expected days in care, where the model overpredicts. Of note, the calibration curve is nearly the same in both the development and validation samples, which suggests that the miscalibration in the highest decile is a function of the underlying data rather than the model’s predictive ability. We suspect the model overpredicts at the higher deciles due to the complex comorbidities of the extreme cases with high numbers of days in care. The CORE team plans to further explore potential explanations for this slight miscalibration at the higher deciles.
Reliability
Comment: The developer did not report signal-to-noise (inter-unit) reliability for the measure, claiming that “it is not possible” to apply signal-to-noise reliability because the measure score is calculated from three different models. Signal-to-noise reliability is a function of the variance between entities compared to the total variance, so it can be calculated regardless of how the predicted values are determined. In fact, in The Reliability of Provider Profiling: A Tutorial, John Adams explains how to do this in a chapter on “The Reliability of Composite Scores.”
Response: The Days at Home measure is based on a risk-adjusted count model and is not within the scope of the “Reliability of Composite Scores” chapter which focuses narrowly on composites of simple binary (pass/fail) measures. As noted in that chapter, challenges remain for calculating signal-to-noise reliability of more complicated measures. Additionally, the Days at Home measure uses hierarchical modeling to estimate the provider-level effects that account for clustering of patients to providers; simplified methods of reliability do not account for this hierarchical data structuring and can be inadequate as a result – specifically, this chapter notes that signal to noise estimation requires error variance for the entity-level scores, which cannot be produced for the full Days at Home score with all three models.
Measuring reliability is the proxy for assessing misclassification probability; the methods mentioned by the commenter, citing Nieser et al., have been used by CORE in reliability testing across other CORE-maintained claims-based measures. The results produced across those measures were found to be similar in magnitude to the ICC reported in this Days at Home measure. Based on the commenter’s feedback we conducted additional reliability testing using permutation split-sample reliability (SSR) method, using the Nieser et al. method recommended by the commenter. We randomly split the study cohort into two halves without replacement for 100 trials and calculated the ICC in the two halves each time.
The results show that the permutation SSR method results in a distribution where the mean across 100 trials (0.773) is similar to the single ICC submitted in the full measure submission form (0.789). Furthermore, even the minimum observed value (0.650) still surpasses the criterion of 0.6 that is established by PQM for split-half reliability testing. The full ICC distribution using the permutation SRR method is as follows:
Mean (S.D): 0.773 (0.044)
Minimum: 0.650
25th percentile: 0.745
Median: 0.781
75th percentile: 0.805
Maximum: 0.858