The measure calculates the percentage of long-stay residents, those living in the skilled nursing facility for 100 days or more, who are satisfied. This patient reported outcome measure is based on the CoreQ: Long-Stay Resident questionnaire that is a three-item questionnaire.
Measure Specs
General Information
Collecting satisfaction information from skilled nursing facility (SNF) patients is more important now than ever. We have seen a philosophical change in healthcare that now includes the patient and their preferences as an integral part of the system of care. The Institute of Medicine (IOM) endorses this change by putting the patient as central to the care system (IOM, 2001). For this philosophical change to person-centered care to succeed, we have to be able to measure patient satisfaction for these three reasons:
(1) Measuring satisfaction is necessary to understand patient preferences.
(2) Measuring and reporting satisfaction with care helps patients and their families choose and trust a health care facility.
(3) Satisfaction information can help facilities improve the quality of care they provide.
The implementation of person-centered care in SNFs has already begun, but there is still room for improvement. The Centers for Medicare and Medicaid Services (CMS) demonstrated interest in consumers’ perspective on quality of care by supporting the development of the Consumer Assessment of Healthcare Providers and Systems (CAHPS) survey for patients in nursing facilities (Sangl et al., 2007).
Further supporting person-centered care and resident satisfaction are ongoing organizational change initiatives. These include: the Advancing Excellence in America’s Nursing Homes campaign (2006), which lists person-centered care as one of its goals; Action Pact, Inc., which provides workshops and consultations with nursing facilities on how to be more person-centered through their physical environment and organizational structure; and Eden Alternative, which uses education, consultation, and outreach to further person-centered care in nursing facilities. All of these initiatives have identified the measurement of resident satisfaction as an essential part in making, evaluating, and sustaining effective clinical and organizational changes that ultimately result in a person- centered philosophy of care.
The importance of measuring resident satisfaction as part of quality improvement cannot be stressed enough. Quality improvement initiatives, such as total quality management (TQM) and continuous quality improvement (CQI), emphasize meeting or exceeding “customer” expectations. William Deming, one of the first proponents of quality improvement, noted that “one of the five hallmarks of a quality organization is knowing your customer’s needs and expectations and working to meet or exceed them” (Deming, 1986). Measuring resident satisfaction can help organizations identify deficiencies that other quality metrics may struggle to identify, such as communication between a patient and the provider.
As part of the U.S. Department of Commerce renowned Baldrige Criteria for organizational excellence, applicants are assessed on their ability to describe the links between their mission, key customers, and strategic position. Applicants are also required to show evidence of successful improvements resulting from their performance improvement system. An essential component of this process is the measurement of customer, or resident, satisfaction (Shook & Chenoweth, 2012). Bhattacharyya et al. (2022) note that “satisfaction is an integral part of nursing home (NH) quality of care.”
The CoreQ: Long-Stay Resident questionnaire can strategically help nursing facilities achieve organizational excellence and provide high quality care by being a tool that targets a unique and growing patient population. Over the past several decades, care in nursing facilities has changed substantially. Statistics show that more than half of all elders cared for in nursing homes are now discharged home (approximately 1.6 million residents; CMS, 2009). Moreover, when satisfaction information from current residents (i.e., long stay residents) is compared with those of elders discharged home, substantial differences exist (Castle, 2007). This indicates that long stay and short stay residents are different populations with different needs in the nursing facilities. Thus, the CoreQ: Long-Stay Resident questionnaire measure is needed to improve the care for long-stay SNF patients.
Furthermore, improving the care for long-stay nursing home patients is tenable. A review of the literature on satisfaction surveys in nursing facilities (Castle, 2007) concluded that substantial improvements in resident satisfaction could be made in many nursing facilities by improving care (i.e., changing either structural or process aspects of care). This was based on satisfaction scores ranging from 60 to 80% on average.
It is worth noting, few other generalizations could be made because existing instruments used to collect satisfaction information are not standardized. Thus, benchmarking scores and comparison scores (i.e., best in class) were difficult to establish. The CoreQ: Long-stay Resident measure has considerable relevance in establishing benchmarking scores and comparison scores.
This measure’s relevance is furthered by recent federal legislative actions. The Affordable Care Act of 2010 requires the Secretary of Health and Human Services (HHS) to implement a Quality Assurance & Performance Improvement Program (QAPI) within nursing facilities. This means all nursing facilities have increased accountability for continuous quality improvement efforts. In CMS’s “QAPI at a Glance” document there are references to customer-satisfaction surveys and organizations utilizing them to identify opportunities for improvement. Lastly, the new “Medicare and Medicaid Programs; Reform of Requirements for Long-Term Care Facilities” proposed rule includes language purporting the importance of satisfaction and measuring satisfaction. CMS states “CMS is committed to strengthening and modernizing the nation’s health care system to provide access to high quality care and improved health at lower cost. This includes improving the patient experience of care, both quality and satisfaction, improving the health of populations, and reducing the per capita cost of health care.” There are also other references in the proposed rule speaking to improving resident satisfaction and increasing person-centered care (Medicare and Medicaid Programs; Reform of Requirements for Long-Term Care Facilities, 2015). The CoreQ: Long-Stay Resident measure has considerable applicability to both of these initiatives.
References
Bhattacharyya KK, Molinari V, Hyer K. Self-Reported Satisfaction of Older Adult Residents in Nursing Homes: Development of a Conceptual Framework. Gerontologist. 2022 Sep 7;62(8):e442-e456. doi:10.1093/geront/gnab061. PMID: 33979428.
Castle, N.G. (2007). A literature review of satisfaction instruments used in long-term care settings. Journal of Aging and Social Policy, 19(2), 9-42.
CMS (2009). Skilled Nursing Facilities Non Swing Bed - Medicare National Summary. http://www.cms.hhs.gov/MedicareFeeforSvcPartsAB/Downloads/NationalSum20…
CMS, University of Minnesota, and Stratis Health. QAPI at a Glance: A step by step guide to implementing quality assurance and performance improvement (QAPI) in your nursing home. https://www.cms.gov/Medicare/Provider-Enrollment-and- Certification/QAPI/Downloads/QAPIAtaGlance.pdf.
Deming, W.E. (1986). Out of the crisis. Cambridge, MA. Massachusetts Institute of Technology, Center for Advanced Engineering Study.
Institute of Medicine (2001). Improving the Quality of Long Term Care, National Academy Press, Washington, D.C., 2001. Medicare and Medicaid Programs; Reform of Requirements for Long-Term Care Facilities; Department of Health and Human Services. 80 Fed. Reg. 136 (July 16, 2015) (to be codified at 42 CFR Parts 405, 431, 447, et al.).
MedPAC. (2015). Report to the Congress: Medicare Payment Policy. http://www.medpac.gov/documents/reports/mar2015_entirereport_revised.pd….
Sangl, J., Bernard, S., Buchanan, J., Keller, S., Mitchell, N., Castle, N.G., Cosenza, C., Brown, J., Sekscenski, E., and Larwood, D. (2007). The development of a CAHPS instrument for nursing home residents. Journal of Aging and Social Policy, 19(2), 63-82.
Shook, J., & Chenoweth, J. (2012, October). 100 Top Hospitals CEO Insights: Adoption Rates of Select Baldrige Award s and Processes. Truven Health Analytics. http://www.nist.gov/baldrige/upload/100-Top-Hosp-CEO-Insights-RB-final….
The collection instrument is the CoreQ: Long-Stay Resident Satisfaction Questionnaire. The resident information comes from facility health information systems used by all SNFs such as billing systems. The exclusions are from the facility health information systems that all facilities have in place such as MDS.
Numerator
The numerator is the sum of the individuals in the facility that have an average satisfaction score of =>3 for the three questions on the CoreQ: Long -Stay Resident questionnaire.
The numerator includes all of the long-stay residents that had an average response =>3 on the CoreQ: Long Stay Resident questionnaire that do not meet any of the exclusions (see exclusions).
The calculation of an individual patient’s average satisfaction score is done in the following manner:
- Respondents within the appropriate time window and who do not meet the exclusions are identified.
- A numeric score is associated with each response scale option on the CoreQ: Long-Stay Resident
questionnaire (that is, Poor=1, Average=2, Good=3, Very Good=4, and Excellent=5). -The following formula is utilized to calculate the individual’s average satisfaction score. [Numeric Score Question 1 + Numeric Score Question 2 + Numeric Score Question 3]/3. - The number of respondents whose average satisfaction score >=3 are summed together and function as the
numerator. For residents with one missing data point (from the 3 items included in the questionnaire) imputation is used (representing the average value from the other two available questions). Residents with more than one missing data point, are not counted in the measure (i.e., no imputation is used for these residents since their responses are excluded). Imputation details are described.
No risk-adjustment is used.
Denominator
The denominator includes all of the residents that have been in the SNF for 100 days or more regardless of payer status; who received the CoreQ: Long-Stay Resident questionnaire (e.g. people meeting exclusions do not receive the questionnaire), who responded to the questionnaire within the two month time window, who did not have the questionnaire completed by somebody other than the resident, and who did not have more than one item missing.
The target population includes all current individuals in the SNF on a given day who have been in the SNF for 100 days or more and respond to the CoreQ: Long-Stay Resident questionnaire and completed the survey within the two month time window.
Residents have up to two months to complete and return the survey. The length-of-stay is identified from nursing facility records (MDS item A1600 “Entry Date”).
Exclusions
Exclusions made at the time of sample selection are the following: (1) Residents who have poor cognition defined by the BIMS score; (2) residents receiving hospice; (3) residents with a legal court appointed guardian; and (4) residents who have lived in the SNF for less than 100 days.
Additionally, once the survey is administered, the following exclusions are applied: a) surveys received outside of the time window (two months after the administration date) b) surveys that have more than one questionnaire item missing c) surveys from residents who indicate that someone else answered the questions for the resident. (Note this does not include cases where the resident solely had help such as reading the questions or writing down their responses.)
Individuals are excluded based on information from the Minimum Data Set (MDS) 3.0 assessment.
- Residents who have poor cognition: Then the Brief Interview for Mental Status (BIMS), a well validated dementia assessment tool is used. BIMS ranges are 0-7 (lowest); 8-12; and 13-15 (highest).
- Residents with BIMS scores of equal or less than 7 are excluded. (MDS Section C0200-C0500 items are used) (Saliba, et al., 2012).
- Residents receiving or having received any hospice. This is recorded in the MDS as Hospice O0100K1 = 1 (“the patient was on hospice in the last 14 days while not a resident”), O0100K2 = 1 (“the patient was on hospice in the last 14 days while a resident”), A1800=07 (“entered from hospice”), or A2100=07 (“discharged to hospice”).
- Residents with court appointed legal guardian for all decisions will be identified from nursing facility health information system.
- Residents who have lived in the SNF for less than 100 days will be identified from the MDS. This is recorded in the MDS (Section A1600, Entry Date).
- Residents that respond after the two month response period.
- Residents whose responses were completed by someone other than the resident will be excluded. Identified from an additional question on the CoreQ: Long-Stay Resident questionnaire.
- Residents without usable data (defined as missing data for 2 or 3 of the survey questions).
Reference
Saliba D, Buchanan J, Edelen MO, Streim J, Ouslander J, Berlowitz D, ChodoshJ. J Am Med Dir Assoc. 2012 Sep;13(7):611-7. doi: 10.1016/j.jamda.2012.06.004. Epub 2012 Jul 15.
Measure Calculation
1.Identify the residents that have been residing in the SNF for 100 days or more. Length of stay so far is the MDS target date (TRGT_DT) -MDS admission date (A1900).
2. Take the residents that have been residing in the SNF for >=100 days and exclude the following:
a. Residents who have poor cognition defined as any residents with BIMS scores of 7 or lower. (MDS Section C0200-C0500 used) (Saliba, et al., 2012).
b. Patients receiving or having received any hospice. This is recorded in the MDS as Hospice O0100K1 = 1 (“the patient was on hospice in the last 14 days while not a resident”), O0100K2 = 1 (“the patient was on hospice in the last 14 days while a resident”), A1800=07 (“entered from hospice”), or A2100=07 (“discharged to hospice”). c. Residents with Court appointed legal guardian for all decisions will be identified from nursing facility health information system.
3. Administer the CoreQ: Long-stay Resident questionnaire to these individuals. The questionnaire should be administered to all residents in the SNF after exclusions in step 2 above. Communicate that residents have four weeks to respond to the survey. Note, we will include surveys received up to two months from administration but specify four weeks to help increase response rate and completion within a timely manner. This also allows providers to use follow-up strategy at 4 weeks to get responses by the 8 week cut off.
4.Create a tracking sheet with the following columns:
i. Data Administered
ii. Data Response Received
iii. Time to Receive Response ([Date Response Received – Date Administered])
5.Exclude any surveys received after 2 months from administration.
6.Exclude responses not completed by the intended recipient (e.g. questions were answered by a friend or
family members (Note: this does not include cases where the resident solely had help such as reading the
questions or writing down their responses).
7.Exclude responses that are missing data for 2 or 3 of the CoreQ questions.
8.All of the remaining surveys are totaled and become the denominator.
9.Combine the CoreQ: Long-Stay Resident questionnaire items to calculate a resident level score. Responses for each item should be given the following scores:
a. Poor = 1,
b. Average = 2,
c. Good = 3,
d. Very Good =4 and
e. Excellent = 5.
10.Impute missing data if only one of the three questions are missing data.
11.Calculate resident score from usable surveys.
a. Patient score= (Score for Item 1 + Score for Item 2 + Score for Item 3) / 3.
i. For example, a resident rates their satisfaction on the three CoreQ questions as excellent = 5, very good = 4,
and good = 3. The resident’s total score will be 5 + 4 + 3 for a total of 12. The resident total score (12) will then be divided by the number of questions (3), which equals 4.0. Thus the residents average satisfaction rating is 4.0. Since the resident’s score is >3.0, this resident will be counted in the numerator.
b. Flag those patients with a score equal to or greater than 3.0. These residents will be included in the numerator.
12. Calculate the CoreQ: Long-Stay Resident Measure which represents the percent of residents with average scores of 3.0 or above. CoreQ: Long-Stay Resident Measure= ([number of respondents with an average score of =3.0] / [total number of respondents])*100.
13.No risk-adjustment is used.
Reference
Saliba, D., Buchanan, J., Edelen, M.O., Streim, J., Ouslander, J., Berlowitz, D, & Chodosh J. (2012). MDS 3.0: brief interview for mental status. Journal of the American Medical Directors Association, 13(7): 611-617.
The measure is not stratified.
1. Administer the CoreQ: Long-Stay Resident questionnaire to SNF residents who have resided in the SNF for >=100 days and who do not fall into one of the following exclusions:
a. Identify that the SNF resident has resided in the facility for >= 100 days. Using MDS (Section A1600, Entry Date).
b. Remove individuals with the following exclusions from the sample:
i. Residents who have poor cognition; Residents with BIMS scores of 7 are lower are excluded. (MDS Section C0200-C0500 used) (Saliba, et al., 2012).
ii. Patients receiving or having received any hospice. This is recorded in the MDS as Hospice O0100K1 = 1 (“the patient was on hospice in the last 14 days while not a resident”), O0100K2 = 1 (“the patient was on hospice in the last 14 days while a resident”), A1800=07 (“entered from hospice”), or A2100=07 (“discharged to hospice”).
iii. Residents with Court appointed legal guardian for all decisions will be identified from nursing facility health information system.
2. Administer the CoreQ: Long-Stay Resident questionnaire to residents.
3. Instruct residents that they must respond to the survey within 2 months.
4. The response rate is calculated based on the number of usable surveys returned divided by the number of surveys administered.
a. Surveys with missing responses for two or more questions, surveys received outside of the time window (more than two months after administration date), and surveys who were completed by someone else other than the intended resident are excluded
b. A minimum response rate of 30% needs to be achieved for results to be reported for a SNF.
5. Regardless of response rate, SNFs must also achieve a minimum number of 20 usable questionnaires (e.g. denominator). If after 2 months, less than 20 usable questionnaires are received then a facility level satisfaction measure is not reported.
6. All the questionnaires that are received (other than those with more than one missing value; or those returned after 2 months; or those completed by another person other than the intended resident) must be used in the calculations.
Reference
Saliba, D., Buchanan, J., Edelen, M.O., Streim, J., Ouslander, J., Berlowitz, D, & Chodosh J. (2012). MDS 3.0: brief interview for mental status. Journal of the American Medical Directors Association, 13(7): 611-617.
A minimum sample size of 20 and overall response rate of 30% is needed for the measure.
Supplemental Attachment
Measure Record
Point of Contact
Not applicable
Valerie Brandon
Washington, DC
United States
Nicholas Castle
University of West Virginia
Morgantown, WV
United States
Importance
Evidence
The definition of quality in a nursing facility has shifted from a focus on structure and process criteria to clinical outcomes, resident satisfaction, and quality of life. This shift was first supported by nursing home reform legislation included in the Omnibus Budget Reconciliation Act of 1987 (OBRA, 1987). Furthering the movement, the Institute of Medicine (IOM) put the patient as central to the care system (Castle, 2007; IOM, 2001) – necessitating the collection of satisfaction information. As mentioned previously (see 1b.1), a focus on person-centered care and satisfaction is also evident in the Quality Assurance & Performance Improvement Program (QAPI) for nursing facilities and proposed Reform Requirements for Long-Term Care Facilities (Medicare and Medicaid Programs; Reform of Requirements for Long-Term Care Facilities, 2015). Also, the Institute for Healthcare Improvement (IHI) included “improving the patient experience” as part of the Triple Aim for improving quality.
Measuring and reporting satisfaction of nursing home care is important in many ways. First, residents are more likely to follow medical advice when they rate their care as satisfactory (Hall, Milburn, Roter, & Daltroy, 1998). Second, because resident satisfaction can influence the quality of care provided and the outcomes of treatment (Hudak and Wright 2000), satisfaction surveys can be used as measures of clinical and organizational accountability. Third, measuring and reporting resident satisfaction can help nursing facilities identify and improve aspects of quality. Furthermore, if publicly released, information on satisfaction with care can help elders and their families choose a nursing facility (Kwon and Bowblis, 2024).
Researchers have recently studied the association of satisfaction sores and the star rating system. The resident’s perspective is imperative in selecting a nursing home where they intend to spend their lives. Nursing homes with higher satisfaction scores tend to have higher star ratings thus proving that the nursing home provides higher quality care (Kwon and Bowblis, 2024). The data shows that for every 1%-point increase in resident satisfaction score, the probability of being a 4- or 5-star nursing home increases by 0.7% points across all star ratings (Kwon and Bowblis, 2024). Other researchers identified similar findings (e.g., Kusmaul et al., 2024). The addition of a publicly reported satisfaction measure in the star rating system or in Care Compare allows current and future consumers of nursing homes with the knowledge to make informed decisions about their long-term care.
Several research efforts have concluded consumer satisfaction is an important indicator of quality of care in nursing homes (Gesell, 2001; Bangerter et al., 2016; Shippee et al 2015; Kajonius and Kazemi, 2016). An online review cites studies showing a relationship between satisfaction and other quality measures, such as turnover (https://www.dhcs.ca.gov/services/medi-cal/Documents).
In addition, other studies have concluded nursing resident satisfaction data provides information about quality of care that is different from clinician perspectives and clinical indicators (Berlowitz, Du, Kazis, & Lewis, 1993; Riccio, 2000; Uman & Urman, 1997). This exemplifies the need for resident satisfaction data to achieve person-centered care. Only by hearing from the patient can we ensure the care provided is person-centered. Bhattacharyya et al. (2022) note that “satisfaction is an integral part of nursing home (NH) quality of care.”
References
Bangerter, L.R., Heid, A.R., Abbott, K, & Van Haitsma, K. (2016). Honoring the Everyday Preferences of Nursing Home Residents: Perceived Choice and Satisfaction with Care. The Gerontologist. (Advance online publication): 1-8.
Berlowitz, D. R., Du, W., Kazis, L., & Lewis, S. (1995). Health-related quality of life of nursing home residents: Difference in patient and provider perceptions. Journal of the American Geriatric Society, 43, 799-802.
Bhattacharyya KK, Molinari V, Hyer K. Self-Reported Satisfaction of Older Adult Residents in Nursing Homes: Development of a Conceptual Framework. Gerontologist. 2022 Sep 7;62(8):e442-e456. doi: 10.1093/geront/gnab061. PMID: 33979428.
Castle, N.G. (2007). A literature review of satisfaction instruments used in long-term care settings. Journal of Aging and Social Policy, 19(2), 9-42.
CMS, University of Minnesota, and Stratis Health. QAPI at a Glance: A step by step guide to implementing quality assurance and performance improvement (QAPI) in your nursing home. https://www.cms.gov/Medicare/Provider-Enrollment-and-Certification/QAPI…;
Gesell, S.B. (2001). A measure of satisfaction for the assisted-living industry. Journal for Healthcare Quality, 23(2), 16-25.
Hall J, Milburn M, Roter D, Daltroy L. (1998). Why are sicker patients less satisfied with their medical care? Tests of two explanatory models. Health Psychol.17(1):70–75.
Hudak, P. L. & J.G. Wright. (2000). The Characteristics of Patient Satisfaction Measures. Spine 25 (24): 3167-3177.
Institute of Medicine (2001). Improving the Quality of Long-Term Care, National Academy Press, Washington, D.C., 2001.
Kajonius, P. & Kazemi, A. (2016). Advancing the Big Five of user-oriented care and accounting for its variations. International Journal of Health Care Quality Assurance. 29(2): 162 – 176.
Kusmaul, N., Miller, R.J., Diehl, C., & Stockwell, I.A. (2024). A facility-level analysis of nursing home compare five star rating and Maryland’s family satisfaction with care survey. The Gerontologist, 64.
Kwon, J. H., & Bowblis, J. R. (2024). Association Between Nursing Home Five-Star Ratings and Consumer Satisfaction. Journal of the American Medical Directors Association, 25(12), 105322. https://doi.org/10.1016/j.jamda.2024.105322
Medicare and Medicaid Programs; Reform of Requirements for Long-Term Care Facilities; Department of Health and Human Services. 80 Fed. Reg. 136 (July 16, 2015) (to be codified at 42 CFR Parts 405, 431, 447, et al.).
Omnibus Budget Reconciliation Act (OBRA) of 1987. (1987, December 22). Public Law 100-203. Subtitle C: Nursing Home Reform.
Riccio, P.A. (2000). Quality Evaluaiton of home nursing care: Perceptions of patients, physicians, and nurses. Nursing Administration Quarterly 24(3): 43-52.
Shippee, T.P., Henning-Smith, C., Kane, R.L, & Lewis, T. (2015). Resident- and Facility-Level Predictors of Quality of Life in Long-Term Care. The Gerontologist. 55(4):643-655
Uman, C & Urman, H. (1997). Measuring consumer satisfaction in nursing home residents. Nutrition 13: 705-707.
Measure Impact
The consumer movement has fostered the notion that patient evaluations should be an integral component of health care. Patient satisfaction, which is one form of patient evaluation, became an essential outcome of health care widely advocated for use by researchers and policy makers. Managed care organizations, accreditation and certification agencies, and advocates of quality improvement initiatives, among others, now promote the use of satisfaction surveys. For example, satisfaction information is included in the Health Plan Employer Data Information Set (HEDIS), which is used as a report card for managed care organizations (NCQA, 2016).
Measuring and improving patient satisfaction is valuable to patients, because it is a way forward on improving the patient-provider relationship, which influences health care outcomes. A 2014 systematic review and meta-analysis of randomized controlled trials, in which the patient-provider relationship was systematically manipulated and tracked with health care outcomes, found a small but statistically significant positive effect of the patient-provider relationship on health care outcomes (Kelly et al., 2014). This finding aligns with other studies that show a link between patient satisfaction and the following health-related behaviors:
1. Keeping follow-up appointments (Hall, Milburn, Roter, & Daltroy, 1998);
2. Disenrollment from health plans (Allen & Rogers, 1997); and,
3. Litigation against providers (Penchansky & Macnee, 1994).
The positive effect of person-centered care and patient satisfaction is not precluded from skilled nursing facilities. A 2013 systematic review of studies on the effect of person-centered initiatives in nursing facilities, such as the Eden Alternative, found person-centered care associated with psychosocial benefits to residents and staff, notwithstanding variations and limitations in study designs (Brownie & Nancarrow, 2013).
As person-centered care (PCC) continues to evolve in long-term care, states are creating programs geared towards training and educating nursing home staff about PCC. The Promoting Excellence Alternatives in Kansas (Peak) 2.0 program incentivizes nursing homes in Kansas to implement PCC by empowering staff to ensure that residents have a choice about their care (Poey et.al, 2017). Resident satisfaction is a great way to determine the effectiveness of PCC but also, it’s a way to gain knowledge about nursing home quality. The program found the nursing homes that fully implemented PCC reports higher resident satisfaction. Programs that incorporate PCC benefits the residents but also have benefits for the nursing home by showing that it is a reputable location for older adults to age in place.
From the nursing facility and provider perspective, there are numerous ways to improve patient satisfaction. One study found conversations regarding end-of-life care options with family members improve overall satisfaction with care and increase use of advance directives (Reinhardt et al., 2014). Another found an association between improving symptom management of nursing home residents with dementia and higher satisfaction with care (Van Uden et al., 2013). Improvements in a nursing home food delivery system also were associated with higher overall satisfaction and improved resident health (Crogan et al., 2013). The association of specific factors (e.g., food, staff, programs) with satisfaction are discussed in a scoping review by Li et al. (2023). The advantage of the CoreQ: Long-Stay Resident questionnaire is it is broad enough to capture patient dissatisfaction on various provided services and signal to providers to drill down and discover ways of improving the patient experience at their facility.
Specific to the CoreQ: Long-Stay Resident questionnaire, the importance of the satisfaction areas assessed were examined with focus groups of residents and family members. The respondents were patients (N=40) in five nursing facilities in the Pittsburgh region. Table 1c.5 in Section 7. Supplemental Information shows the score of the importance for question included in the CoreQ: Long-Stay Resident questionnaire. The overall ranking used was 10=Most important and 1=Least important. The final four questions included in the measure had average scores ranging from 9.50 to 9.69; this clearly shows that the respondents value the items used in the CoreQ: Long-Stay Resident measure.
References
Allen HM, & Rogers WH. (1997). The Consumer Health Plan Value Survey: Round Two. Health Affairs. 1997;16(4):156–66.
Brownie, S. & Nancarrow, S. (2013). Effects of person-centered care on residents and staff in aged-care facilities: a systematic review. Clinical Interventions In Aging. 8:1-10.
Crogan, N.L., Dupler, A.E., Short, R., & Heaton, G. (2013). Food choice can improve nursing home resident meal service satisfaction and nutritional status. Journal of Gerontological Nursing. 39(5):38-45.
Hall J, Milburn M, Roter D, Daltroy L (1998). Why are sicker patients less satisfied with their medical care? Tests of two explanatory models. Health Psychol. 17(1):70–75.
Kelley J.M., Kraft-Todd G, Schapira L, Kossowsky J, & Riess H. (2014). The influence of the patient-clinician relationship on healthcare outcomes: a systematic review and metaanalysis of randomized controlled trials. PLoS One. 9(4): e94207.
Kellogg, C., Zhu, Y., Cardenas, V., Vazquez, K., Johari, K., Rahman, A., & Enguidanos, S. (2018). What consumers say about nursing homes in online reviews. The Gerontologist, 58(4), e273–e280.
Kwon, Jenny H. et al. (2024). Association Between Nursing Home Five-Star Ratings and Journal of the American Medical Directors Association, Volume 25, Issue 12.
Li, Y., Cai, X., Ye, Z., Glance, L.G., Harrington, C., & Mukamel, D.B. (2013). Satisfaction with Massachusetts nursing home care was generally high during 2005-09, with some variability across facilities. Health Affairs. 32(8):1416-25.
Li, X., Mpofu, E., Collins, S., Yin, C., & Shaw, T. (2023). Resident satisfaction indicators in long term care settings in the United States: A scoping review. Aging and Health Research, 3.
Lin, J., Hsiao, C.T., Glen, R., Pai, J.Y., & Zeng, S.H. (2014). Perceived service quality, perceived value, overall satisfaction and happiness of outlook for long-term care institution residents. Health Expectations. 17(3):311-20.
National Committee for Quality Assurance (NCQA) (2016). HEDIS Measures. http://www.ncqa.org/HEDISQualityMeasurement/HEDISMeasures.aspx. Accessed March 2016.
Penchansky and Macnee, (1994). Initiation of medical malpractice suits: a conceptualization and test. Medical Care. 32(8): pp. 813–831.
Plaku-Alakbarova, B., Punnett, L., & Gore, R. J.; Procare Research Team. (2018). Nursing home employee and resident satisfaction and resident care outcomes. Safety and Health at Work, 9(4), 408–415.
Poey, J. L., Hermer, L., Cornelison, L., Kaup, M. L., Drake, P., Stone, R. I., & Doll, G. (2017). Does Person-Centered Care Improve Residents' Satisfaction With Nursing Home Quality?. Journal of the American Medical Directors Association, 18(11), 974–979. https://doi.org/10.1016/j.jamda.2017.06.007.
Reinhardt, J.P., Chichin, E., Posner, L., & Kassabian, S. (2014). Vital conversations with family in the nursing home: preparation for end-stage dementia care. Journal Of Social Work In End-Of-Life & Palliative Care. 10(2):112-26.
Van Uden, N., Van den Block, L., van der Steen, J.T., Onwuteaka-Philipsen, B.D., Vandervoort, A., Vander Stichele, R., & Deliens, L. (2013). Quality of dying of nursing home residents with dementia as judged by relatives. International Psychogeriatrics. 25(10):1697-707.
Performance Gap
The data provided here comes from LTC Trend Tracker, a data tool managed by American Health Care Association (AHCA) to allow nursing homes to benchmark and trend their performance on various metrics, including CoreQ. On a voluntary basis, nursing homes can upload data themselves or delegate a customer satisfaction vendor to upload on their behalf.
The data provided below reflects data from 2024q2-2025q1 representing 727 providers and 9,792 patients. With an average satisfaction rate of 64.3%, there is still room for improvement.
| Overall | Minimum | Decile_1 | Decile_2 | Decile_3 | Decile_4 | Decile_5 | Decile_6 | Decile_7 | Decile_8 | Decile_9 | Decile_10 | Maximum | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Mean Performance Score | 64.30 | 0.00 | 0.00 | 11.40 | 43.50 | 56.20 | 68.30 | 78.20 | 87.10 | 97.70 | 100.00 | 100.00 | 100.00 |
| N of Entities | 727 | 111 | 72 | 73 | 73 | 73 | 72 | 73 | 73 | 73 | 73 | 72 | 187 |
| N of Persons / Encounters / Episodes | 9792 | 188 | 119 | 442 | 695 | 1187 | 1272 | 2020 | 2106 | 1025 | 402 | 524 | 1104 |
Performance Scores by Decile, 2024q2-2025q1
| Overall | Min | Decile 1 | Decile 2 | Decile 3 | Decile 4 | Decile 5 | Decile 6 | Decile 7 | Decile 8 | Decile 9 | Decile 10 | Max | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Mean Performance Score | 64.30% | 0.00% | 0.00% | 11.40% | 43.50% | 56.20% | 68.30% | 78.20% | 87.10% | 97.70% | 100.00% | 100.00% | 100.00% |
| N of Entities | 727 | 111 | 72 | 73 | 73 | 73 | 72 | 73 | 73 | 73 | 73 | 72 | 187 |
| N of Persons / Encounters / Episodes | 9792 | 188 | 119 | 442 | 695 | 1187 | 1272 | 2020 | 2106 | 1025 | 402 | 524 | 1104 |
Care Gaps
Closing Care Gaps
Multiple studies in the past twenty years have examined racial disparities in the care of nursing facility residents and have consistently found poorer care in facilities with high minority populations (Fennell et al., 2000; Mor et al., 2004; Smith et al., 2007). Work on racial disparities in nursing facilities’ quality of care between elderly white and black residents within nursing facility has shown clearly that nursing homes remain relatively segregated and that specifically nursing home care can be described as a tiered system in which blacks are concentrated in marginal-quality homes (Li, Ye, Glance & Temkin-Greener, 2014; Fennell, Feng, Clark & Mor, 2010; Li, Yin, Cai, Temkin-Greener, Mukamel, 2011; Chisholm, Weech-Maldonado, Laberge, Lin, & Hyer, 2013; Mor et al., 2004; Smith et al., 2007). Such homes tend to have serious deficiencies in staffing ratios, performance, and are more financially vulnerable (Smith et al, 2007; Chisholm et al., 2013). Based on a review of the nursing facility disparities literature, Konetzka and Werner concluded that disparities in care are likely related to this racial and socioeconomic segregation as opposed to within-provider discrimination (Konetzka and Werner 2009). This conclusion is supported, for example, by Grunier and colleagues who found that as the proportion of black residents in the nursing home increased the risk of hospitalization among all residents, regardless of race, also increased (Grunier et al., 2008). Thus, adjusting for racial status has the unintended effect of adjusting for poor quality providers not to differences due to racial status and not within-provider discrimination.
Therefore, lower satisfaction scores for both Caucasian and Blacks and other ethnicities are likely to increase as the proportion of black residents increases in a SNF, indicating that the best measure of racial disparities in satisfaction rates is one that measures scores at the facility level. That is, ethnic and social economic status differences are related to inter-facility differences not to intra-facility differences in care. Therefore, the literature suggests that racial status should not be risk adjusted otherwise one is adjusting for the poor quality of the SNFs rather than differences due to racial status.
The CoreQ information was collected from nursing homes and assisted living facilities during 2021. These facilities were primarily located in MA, PA, and NJ. These facilities were a sample of convenience, in that they had voluntarily participated in collecting CoreQ information during 2020 and continued doing so in 2021. In addition to the CoreQ items, in 2021 demographic questions including asking for the respondents’ race were also included.
In these facilities, discharged residents and family members were sent a CoreQ questionnaire, a letter describing the questionnaire, and postage paid return envelope. The questionnaires were anonymous, but they did include the facility name as part of the survey. For long-stay residents, social workers in the facilities collected CoreQ information. The questionnaires used were also anonymous. In all cases, the standard data collection exclusions for collecting CoreQ were used.
The proportion of black residents in nursing homes is not uniform, and some facilities have higher concentrations of black residents than others. This has led to investigations of both within-nursing home care examining if differences exist by race within the same facility, and between-nursing home care examining if differences exist across facilities. Both within- and between-nursing home differences were examined.
For all three samples (discharge, resident, and family), overall scores for black residents are lower than those for white residents. However, we know that black residents are disproportionately cared for in lower quality facilities (described above). This may influence the overall scores. Indeed, when we examine nursing homes with >20% black residents, the resulting scores for black vs. white residents are almost identical.
References
Chisholm L, Weech-Maldonado R, Laberge A, Lin FC, Hyer K. (2013). Nursing home quality and financial performance: does the racial composition of residents matter? Health Serv Res;48(6 Pt 1):2060–2080.
Fennell ML, Feng Z, Clark MA, Mor V. (2010). Elderly Hispanics more likely to reside in poor-quality nursing homes. Health Aff (Millwood);29(1):65–73.
Grabowski, D.C. (2004). The admission of Blacks to high-deficiency nursing homes. Medical Care 42(5): 456-464.
Gruneir, A., Miller, S. C., Feng, Z., Intrator, O., & Mor, V. (2008). Relationship between state Medicaid policies, nursing home racial composition, and the risk of hospitalization for black and white residents. Health Services Research, 43(3), 869-881.
Konetzka, R. T., & Werner, R. M. (2009). Review: Disparities in long-term care building equity into market-based reforms. Medical Care Research and Review, 66(5), 491-521.
Li Y, Yin J, Cai X, Temkin-Greener J, Mukamel DB. (2011). Association of race and sites of care with pressure ulcers in high-risk nursing home residents. JAMA;306(2):179–186.
Li Y, Ye Zhiqiu, Glance, Laurent & Temkin-Greener, Helena. (2014). Trends in family rating experience with care and racial disparities among Maryland nursing homes. Med Care, 52(7): 641-648.
Mor, V., Zinn, J., Angelelli, J., Teno, J. M., & Miller, S. C. (2004). Driven to tiers: socioeconomic and racial disparities in the quality of nursing home care. Milbank Quarterly, 82(2), 227-256.
Smith, D. B., Feng, Z., Fennell, M. L., Zinn, J. S., & Mor, V. (2007). Separate and unequal: racial segregation and disparities in quality across US nursing homes. Health Affairs, 26(5): 1448-1458.
Feasibility
Feasibility
The data elements are routinely generated through the satisfaction survey. All data elements are in defined fields in a combination of electronic sources. In an effort to keep administrative burden low to encourage collection of satisfaction data, which is important in the field, there are no efforts to develop a plan for electronic collection.
Facilities have no data entry burden. However, they do have data collection burden. In the work we have done with CMS for a different CoreQ survey (NH discharge survey) the cost burden for the facility was calculated to be $2.80 per respondent. This calculation was based on requiring more than 20 data elements, whereas, here only 4 are needed. The cost will likely be less than $2.80. Because CoreQ can be added to the beginning of existing surveys, the cost and burden of adoption can be minimal.
No barriers were encountered with the measure specifications. The measure calculation was sometimes confused with an average score. The CoreQ measure is not an average. This is explained in reports produced and in the technical manual.
All of the patient surveys are anonymous. In addition, scores are only calculated with 20 or more survey returns. Thus, patient confidentiality is protected. The only identifier on the survey is the facility name so that a score can be attributed to the facility.
This is a maintenance application. As detailed above we have continued to collect CoreQ data to examine any changes in scores and implementation issues. We also continued to text the exclusions and inclusions. No adjustment to the measure has occurred.
Proprietary Information
There are no fees, licensing, or other requirements for this measure. The Long-Stay Resident survey/questionnaire and the methodology are proprietary. The CoreQ name is trademarked my the American Health Care Association/National Center for Assisted Living.
Scientific Acceptability
Testing Data
- Data utilized for testing came from CoreQ: Long-Stay Resident questionnaire from CY2025. To validate the measure; we also utilized CASPER Quality Indicators and data from Centers for Medicare and Medicaid Services (CMS) Nursing Home Compare from CY2025 from a national sample of facilities.
- Additionally, Updated testing based on the Pilot CoreQ: Long-Stay Resident questionnaire was completed for CY2025 from a national sample of facilities.
- Performance gap data was collected from LTC Trend Tracker, a data tool managed by the American Health Care Association (AHCA) to allow nursing homes to benchmark and trend their performance on various metrics, including CoreQ. The data provided below reflects data from 2024q2-2025q1.
Reliability and Validity data are derived from the CoreQ: Long-Stay Resident questionnaire from CY2025.
Note: Data from CY2014 is included in the testing documents as a comparison to CY2025 data but was not used for testing in this submission
The dates of the data used for testing are:
- Centers for Medicare and Medicaid Services (CMS) Nursing Home Compare, CASPER Quality indicators, Five-Star, and Payroll-Based Journal data from CY2025 (January-December 2025).
- Pilot CoreQ: Long-Stay Resident questionnaire data from CY2025 (January-December 2025).
- CoreQ: Long-Stay Resident questionnaire data from CY2025 (January-December 2025).
- Performance gap data was collected from LTC Trend Tracker for 2024q2-2025q1.
Note: Data from CY2014 is included in the testing documents as a comparison to CY2025 data but was not used for testing in this submission.
We conducted two levels of testing in the development of the CoreQ: Long-Stay Resident measure. The first focused on testing (e.g., reliability, validity, exclusions) of the CoreQ: Long-Stay Resident questionnaire. The first source of data (pilot data) was utilized in developing and choosing the items to be included in the CoreQ: Long-Stay Resident questionnaire. This included using a questionnaire with 22 items. Below we call this the Pilot CoreQ: Long-Stay Resident questionnaire.
Once the CoreQ: Long-Stay Resident questionnaire was developed, a second source of data was used to test the validity of the CoreQ: Long-Stay Resident measure (i.e., facility and summary score validity).
For this E&M similar analyses were conducted using CY2025 data. This was to check the continued validity of the CoreQ: Long-Stay Resident measure.
Reliability Testing: CoreQ: Long-Stay Resident Questionnaire data from CY2025 (January 2025-December 2025) a national sample (n=387, 12,771 residents) at the data element, person/questionnaire, and measure (facility) level.
Validity Testing: CoreQ: Long-Stay Resident Questionnaire data from CY2025 (January 2025-December 2025) a national sample (n=387, 12,771 residents). To test the validity of the measure, CASPER Quality Indicators, CMS Nursing Home Compare, Five-Star, and Payroll-Based Journal data from CY2025 (January 2025- December 2025) was used.
The testing and analysis included three data sources, one of which had additional variables collected for a subset of respondents:
The Pilot CoreQ: Long-Stay Resident questionnaire was examined using responses from 1,714 residents from a national sample of nursing facilities.
a. In addition, resident-level sociodemographic (SDS) variables were examined using this same sample of 1,714 residents (#1 below) in nursing facilities across the US.
- Validity testing of the Pilot CoreQ: Long-Stay Resident questionnaire was examined using responses from 100 residents from the Pittsburgh area.
- CoreQ: Long-Stay Resident measure was examined using 223 facilities and included responses from 7,307 residents. These nursing facilities were located in multiple states across the US.
- Repeat data came from a national sample of facilities collected in CY2025 (January 2025- December 2025, Source 4). This included 387 facilities and included responses from 12,771 residents. These nursing facilities were located in multiple states across the US.
Some basic descriptive characteristics of these facilities (data sources) are provided below in table 5.1.3.
Table 5.1.3: Descriptive Statistics of Centers Included in the Analyses
Data Source | Average Number of Licensed Beds | Average Daily Census | Sample Size of Residents (N) |
Source 1 | 139 | 121 | 1,714 |
Source 2 | 202 | 188 | 100 |
Source 3 | 137 | 130 | 7,307 |
Source 4 | 141 | 132 | 12,771 |
Data Source: CoreQ: Long-Stay Resident Questionnaire (Source 1-3, CY2014; Source 4, CY2025)
Note: Data from CY2014 is included in the testing documents as a comparison to CY2025 data but was not used for testing in this submission.
Resident Level of Analysis
Data was used from the CoreQ: Long-Stay Resident questionnaire. The questionnaire was administered to all eligible long-stay residents (with the exclusions described in the Specifications section of this application). The testing and analyses included:
The Pilot CoreQ: Long-Stay Resident questionnaire was examined using responses from 1,714 residents from a national sample of nursing facilities. (Data Source #1 above Table 5.1.3)
a. In addition, resident-level sociodemographic (SDS) variables were examined using this same sample of 1,714 residents (Data Source #1 above Table 5.1.3) in nursing facilities across the US.
- Validity testing of the Pilot CoreQ: Long-Stay Resident questionnaire was examined using responses from 100 residents from the Pittsburgh area. (Data Source #2 above Table 5.1.3)
- CoreQ: Long-Stay Resident questionnaire measure was examined using 223 facilities and included responses from 7,307 residents. These nursing facilities were located in multiple states across the US. (Data Source #3 above Table 5.1.3)
The descriptive characteristics of the residents are given in the following table that includes information from all of the data used (the education level and race information comes only from the sample described above with 1,714 respondents, as this data was not collected for the other samples).
Table 5.1.4: Patient Demographics
| DEMOGRAPHICS |
| Percent (CY2014 samples 1,2,3 pooled) | Percent (CY2025 data) |
| How long were you a resident at this facility? | <6 Months | 12% | 9% |
6Months-1Yr | 18% | 26% | |
1-2Yrs | 25% | 31% | |
2-3Yrs | 17% | 21% | |
>3yrs | 28% | 23% | |
| Are you male or female? | Male | 35% | 26% |
Female | 65% | 74% | |
| What year were you born? | Average | 1931 | 1939 |
| What is the highest grade or level of school that you have completed? | Some HS | 24% | 21% |
HS or GED | 34% | 37% | |
Some College/ 2yr Degree | 20% | 24% | |
4yr College Degree | 17% | 24% | |
>4yr College Degree | 4% | 4% | |
| What is your race? | White | 86% | 85% |
Black | 6% | 12% | |
Asian | 2% | 3% | |
Native Hawaiian | <0% | <0% | |
American Indian | 1% | <0% |
Data Source: Data Source: CoreQ: Long-Stay Resident Questionnaire (Source 1-3, CY2014; Source 4, CY2025)
Note: Data from CY2014 is included in the testing documents as a comparison to CY2025 data but was not used for testing in this submission.
Reliability
We measured reliability at the: (1) data element level; (2) the person/questionnaire level; and, (3) at the measure (i.e., facility) level. More detail of each analysis follows.
1. Data Element Level
To determine if the CoreQ: Long-Stay Resident questionnaire items were repeatable, producing the same results a high proportion of the time when assessed in the same population in the same time period, we re-administered the questionnaire to residents 1 month after their completion of the first survey. The Pilot CoreQ: Long-Stay Resident questionnaire had responses from 100 residents; we re-administered the survey to 50 of these same residents. The re-administered sample was a sample of convenience as they represented residents from the Pittsburgh area (the location of the team testing the questionnaire). To measure the agreement, we calculated first the distribution of responses by question in the original round of surveys, and then again in the follow-up surveys (they should be distributed similarly); and second, calculated the correlations between the original and follow-up responses by question (they should be highly correlated).
2. Person/Questionnaire Level
Having tested whether the data elements matched between the pilot responses and the re-administered responses, we then examined whether the person-level results matched between the Pilot CoreQ: Long-Stay Resident questionnaire responses and their corresponding re-administered responses. In particular, we calculated the percent of time that there was agreement between whether or not the pilot response was poor, average, good, very good or excellent, and whether or not the re-administered response was poor, average, good, very good or excellent.
3. Measure (Facility) Level
Last, we measured stability of the facility-level measure when the facility’s score is calculated using multiple “draws” from the same population. This measures how stable the facility’s score would be if the underlying residents are from the same population but are subject to the kind of natural sample variation that occurs over time. We did this by bootstrap with 10,000 repetitions of the facility score calculation, and present the percent of facility resamples where the facility score is within 1 percentage point, 3 percentage points, 5 percentage points, and 10 percentage points of the original score calculated on the Pilot CoreQ: Long-Stay Resident questionnaire sample.
4. Accountable Entity-Level Reliability
Entities were ranked by mean performance score and grouped into deciles of approximately equal size. Reliability was estimated using a signal-to-noise framework in which reliability represents the proportion of total variance attributable to true between-entity differences rather than measurement error. For each decile, we summarized the mean performance score, number of entities, and total number of persons/encounters/episodes contributing to the estimates. Reliability values ≥0.70 were considered acceptable and values ≥0.80 were considered good.
Data Source: CoreQ: Long-Stay Resident Questionnaire for CY2025.
1.Data Element Level
Table 5.2.3a shows the four CoreQ: Long-Stay Resident questionnaire items, and the response per item for both the pilot survey of 100 residents and the re-administered survey of 50 residents. The responses in the pilot survey are not statistically significant from the re-administered survey. This shows that the data elements were highly repeatable and produced the same results a high proportion of the time when assessing the same population in the same time period. Similar findings are shown from the repeat 2025 data.
2. Person/Questionnaire Level
Table 5.2.3c shows the CoreQ: Long-Stay Resident questionnaire items, and the agreement in response per item for both the pilot survey of 100 residents compared with the re-administered survey of 50 residents. The person-level responses in the pilot survey are not statistically significant from the re-administered survey. This shows that a high percent of time there was agreement between whether or not the pilot response was poor, average, good, very good or excellent, and whether or not the re-administered response was poor, average, good, very good or excellent. Table 5.2.3d shows the agreement between the pilot and re-administered responses. In summary, 98% or more of the re-administered responses agreed with their corresponding pilot responses, in terms of whether or not they were rated in the categories of poor or average or good, very good or excellent. Similar findings are shown from the repeat 2025 data.
3. Measure (Facility) Level
After having performed 10,000-repitition bootstrap 7.7% of bootstrap repetition scores were within 1 percentage point of the score in the original sample, 35.9% were within 3 percentage points, 62.5% were within 5 percentage points, and 73.2% were within 10 percentage points.
* Tables are available in Additional Reliability Testing Results
Note: Data from CY2014 is included in the testing documents as a comparison to CY2025 data but was not used for testing in this submission.
In summary, the measure displays a high degree of element-level, questionnaire-level, and measure (facility)-level reliability. First, the CoreQ: Long-Stay Resident questionnaire data elements were highly repeatable, with pilot and re-administered responses agreeing between 97% and 99% of the time depending on the question. That is, this produced the same results a high proportion of the time when assessed in the same population in the same time period. Second, the questionnaire level scores were also highly repeatable, with pilot and re-administered responses agreeing 98.5% of the time (or more). Third, a facility drawing residents from the same underlying population will only vary modestly. The 10,000-repetition bootstrap results show that the CoreQ: Long-Stay Resident measure scores from the same facility are moderately stable given the minimum sample size of 20 we set for this measure; and the maximum sample size was 122.
The performance measure demonstrated good overall reliability (0.834). Reliability estimates increased across performance deciles, ranging from 0.63 among the lowest-performing entities to 0.93 among the highest-performing entities. Reliability was acceptable to excellent across nearly all deciles, indicating that the measure consistently distinguished true differences in entity performance. Higher-performing entities demonstrated particularly strong measurement stability, while reliability among lower-performing entities remained moderate to acceptable.
| | Overall | Minimum | Decile_1 | Decile_2 | Decile_3 | Decile_4 | Decile_5 | Decile_6 | Decile_7 | Decile_8 | Decile_9 | Decile_10 | Maximum |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Reliability | |||||||||||||
| Mean Performance Score | 64.30 | 0.00 | 0.00 | 11.40 | 43.50 | 56.20 | 68.30 | 78.20 | 87.10 | 97.70 | 100.00 | 100.00 | 100.00 |
| N of Entities | 727 | 111 | 72 | 73 | 73 | 73 | 72 | 73 | 73 | 73 | 73 | 72 | 187 |
| N of Persons / Encounters / Episodes | 9792 | 188 | 119 | 442 | 695 | 1187 | 1272 | 2020 | 2106 | 1025 | 402 | 524 | 1104 |
Accountable Entity Level Reliability Testing Results by Denominator, January - December 2025
Overall | Min | Decile 1 | Decile 2 | Decile 3 | Decile 4 | Decile 5 | Decile 6 | Decile 7 | Decile 8 | Decile 9 | Decile 10 | Max | |
| Reliability | 0.834 | 0.68 | 0.63 | 0.76 | 0.79 | 0.84 | 0.84 | 0.87 | 0.92 | 0.85 | 0.93 | 0.91 | 0.93 |
| Mean Performance Score | 83% | 32% | 39% | 45% | 55% | 63% | 69% | 76% | 83% | 87% | 91% | 99% | 100% |
| N of Entities | 670 | 11 | 67 | 67 | 67 | 67 | 67 | 67 | 67 | 67 | 67 | 67 | 24 |
| N of Persons / Encounters / Episodes | 19055 | 196 | 1982 | 1901 | 1966 | 1905 | 1927 | 1841 | 1830 | 1896 | 1814 | 1993 | 1039 |
Note: Table 2a used Cronbach's alpha to show computations per decile.
Accountable Entity Level Reliability Testing Results by Reliability Score, January - December 2025
Overall | Min | Decile 1 | Decile 2 | Decile 3 | Decile 4 | Decile 5 | Decile 6 | Decile 7 | Decile 8 | Decile 9 | Decile 10 | Max | |
| Reliability | 0.834 | 0.68 | 0.63 | 0.76 | 0.79 | 0.84 | 0.84 | 0.87 | 0.92 | 0.85 | 0.93 | 0.91 | 0.93 |
Note: Table 2b used Cronbach's alpha to show computations per decile.
Validity
In the development of the CoreQ: Long-Stay Resident questionnaire, three sources of data were used to perform three levels of validity testing.
The first source of data (data from a sample of convenience collected near the researchers developing the questionnaire in Pittsburgh) (See Table 5.1.3 above) was used in developing and choosing the format to be utilized in the CoreQ: Long-Stay Resident questionnaire (i.e., response scale).
The second source of data (See Table 5.1.3 above) was pilot data collected from a national sample of 1,714 residents. This data was used in choosing the items to be used in the CoreQ: Long-Stay Resident questionnaire (i.e., questionnaire items). This data was also used in examining resident-level sociodemographic (SDS) variables.
The third source of data (See Table 5.1.3 above) (collected from 223 facilities) was used examine the validity of the CoreQ: Long-Stay Resident measure (i.e., facility and summary score validity). These residents / nursing facilities were from multiple states across the U.S.
Thus, the following sections describe this validity testing:
1. Validity Testing of the questionnaire format used in the CoreQ: Long-Stay Resident questionnaire (using data source 1, from above);
2. Testing the items for the CoreQ: Long-Stay Resident questionnaire (using data source 2, from above);
3. Testing to determine if a sub-set of items could reliably be used to produce an overall indicator of satisfaction (Core Q: Long-Stay Resident measure) (using data source 3, from above);
4. Validity testing for the CoreQ: Long-Stay Resident measure (also using data source 1, from above).
Validity Testing for the Questionnaire Format used in the CoreQ: Long-Stay Resident Questionnaire
A. The face validity of the domains used in the CoreQ: Long-Stay Resident questionnaire was evaluated via a literature review. The literature review was conducted to examine important areas of satisfaction for LTC residents. Specifically, the research team examined 12 commonly used satisfaction surveys and reports to determine the most valued domains when looking at satisfaction. These surveys were identified by completing internet searches in PubMed and Google. Key terms that were searched included: resident satisfaction, long-term care satisfaction, and elderly satisfaction.
B. The face validity of the domains was also examined using a focus group of residents. The overall ranking used was 1=Most important and 22=Least important. That is residents were asked to rank the domains from most important to least important. The respondents were residents (N=40) in five nursing facilities in the Pittsburgh region.
C. The face validity of the Pilot CoreQ: Long-Stay Resident questionnaire response scale was also examined. The respondents were residents (N=40) in five nursing facilities in the Pittsburgh region. The percent of respondents that stated they “fully understood” how the response scale worked, could complete the scale, AND in cognitive testing understood the scale was used.
D. The Flesch-Kinkaid scale (Streiner & Norman, 1995) was used to determine if respondent correctly understood the questions being asked.
Reference
Streiner, D. L. & Norman, G.R. (1995). Health measurement scales: A practical guide to their development and use. 2nd ed. New York: Oxford.
1. Testing the Items for the CoreQ: Long-Stay Resident Questionnaire
The second series of validity testing was used to further identify items that should be included in the CoreQ: Long-Stay Resident questionnaire. This analysis was important, as all items in a satisfaction measure should have adequate psychometric properties (such as low basement or ceiling effects). For this testing, (1) A pilot group of 40 residents was first used in focus groups; (2) a Pilot version of the CoreQ: Long-Stay Resident questionnaire survey was administered consisting of 18 items (N= 1,714 residents). The testing consisted of:
A. Residents were asked to rate the 18 different satisfaction questions related to their experience in SNFs. This was conducted with a pilot group of 40 residents in focus groups.
B. The Pilot CoreQ: Long-Stay Resident questionnaire items performance with respect to the distribution of the response scale and with respect to missing responses. (using 1,714 residents described above)
C. The intent of the Pilot instrument was to have items that represented the most important areas of satisfaction (as identified above) in a parsimonious manner. Additional analyses such as exploratory factor analysis (EFA) were used to further refine the pilot instrument. This was an iterative process that included using Eigenvalues from the principal factors (unrotated) and correlation analysis of the individual items. (using 1,714 residents described above)
2. To determine if a Sub-Set of Items could Reliably be used to Produce an Overall Indicator of Satisfaction (The Core Q: Long-Stay Resident Measure).
The CoreQ: Long-Stay Resident measure under development was meant to represent overall satisfaction with as few items as possible. The testing given below describes how this was achieved.
A. To support the construct validity that the idea that the CoreQ items measured a single concept of “satisfaction” – we performed a correlation analysis using all items in the instrument.
B. In addition, using all items in the instruments a factor analysis was conducted. Using the global items Q1 (“How satisfied are you with the facility?”) the Cronbach’s Alpha of adding the “best” additional item was examined.
3. Validity Testing for the Core Q: Long-Stay Resident Measure.
A. To determine if the 3 items in the CoreQ: Long-Stay Resident questionnaire were a reliable indicator of satisfaction, the correlation between these three items (the “CoreQ: Long-Stay Resident Measure”) and ALL of the items on the Pilot CoreQ instrument was conducted.
B. We performed additional validity testing of the facility-level CoreQ: Long-Stay Resident measure by examining the correlations between the CoreQ: Long-Stay Resident measure scores and i) measures of regulatory compliance and other quality metrics from the Certification and Survey Provider Enhanced Reporting (CASPER) data, and ii) several other quality metrics from Nursing Home Compare. If the CoreQ Long Stay Family scores correlate negatively with the measures that decrease as they get better, and positively with the measures that increase as they get better, then this supports the validity of the CoreQ Long Stay Resident measure.
Repeat data came from a national sample of facilities collected in 2025. This included 387 facilities and included responses from 12,771 residents. These nursing facilities were located in multiple states across the US.
1. Validity Testing for the Questionnaire Format used in the CoreQ: Long-Stay Resident Questionnaire
A. The face validity of the domains used in the CoreQ: Long-Stay Resident questionnaire was evaluated via a literature review (described above).
The research team examined the surveys and reports to identify the different domains that were included. The research team scored the domains by simply counting if an instrument included the domain. Table 5.3.4a gives the domains that were found throughout the search, as well as a score. An example is the domain clinical care, this was used in 10 out of the 12 surveys identified in the literature. An interpretation of this finding would be that items addressing clinical care are extremely important in satisfaction surveys. These domains were used in developing the pilot CoreQ: Long-Stay Resident questionnaire items. A recent scoping review by Li et al. (2023) provides very similar results.
Reference
Li, X., Mpofu, E., Collins, S., Yin, C., & Shaw, T. (2023). Resident satisfaction indicators in long term care settings in the United States: A scoping review. Aging and Health Research, 3.
B. The face validity of the domains was also examined using residents (described above). The following abbreviated table shows the rank of importance for each group of domains. The overall ranking used was 1=Most important and 22=Least important. The ranking of the 3 areas used in the CoreQ: Long-Stay Resident questionnaire are shown. Note, the food domain was ranked third – but was excluded from the CORE Q based on additional analyses showing that it was highly correlated with the overall domain; thus, it added little to the measure.
C. The face validity of the pilot CoreQ: Long-Stay Resident questionnaire response scale was also examined (described above). Table 5.3.4c gives the percent of respondents that stated they “fully understood” how the response scale worked, could complete the scale, AND in cognitive testing understood the scale.
D. The CoreQ: Long-Stay Resident questionnaire was purposefully written using simple language. No a priori goal for reading level was set, however a Flesch-Kinkaid scale score of six, or lower, is achieved for all questions.
2. Testing the Items for the CoreQ: Long-Stay Resident Questionnaire
A. Each resident was asked to rate on a scale of 1 to 10 (with 10 as the best) how important they thought the question was for evaluating the experience with SNF care. The three questions included in the CoreQ were highly rated out of all the questions and in analysis of resident’s responses to 18 questions. That is, these three items were shown to provide unique information to distinguish satisfaction with SNFs. Specifically, “In recommending this facility to your friends and family, how would you rate it overall?” had an average score of 9.69; “Overall, how would you rate the staff?” had an average score of 9.56; and, “How would you rate the care you receive?” had an average score of 9.5. This shows a very pervasive influence of the satisfaction items with the experience of SNF care. (See Table 1c.5 in Section 7. Supplemental Information).
B. The pilot CoreQ: Long-Stay Resident questionnaire items are shown in Table 5.3.4d in the attached validity results. It also shows that the items performed well with respect to the distribution of the response scale and with respect to missing responses.
C. Using all items in the instruments (excluding the global item Q1 (“How would you rate the facility?”)) exploratory factor analysis (EFA) was used to evaluate the construct validity of the measure. The Eigenvalues from the principal factors (unrotated) are presented in the Table below. In this analysis, the first Eigenvalue is overwhelmingly greater than the second Eigenvalue, this supports the proposition that the CoreQ instrument is measuring a single global concept of customer satisfaction – rather than a number of sub-concepts of customer satisfaction. Sensitivity analyses using principal factors and rotating provide highly similar findings.
3. To determine if a Sub-Set of Items could be used to Produce an Overall Indicator of Satisfaction (The Core Q: Long-Stay Resident measure).
A. To support the construct validity that the idea that the CoreQ items measured a single concept of “satisfaction” – we performed a correlation analysis using all items in the instrument. The analysis identifies the pairs of CoreQ items with the highest correlations. The highest correlations are shown in the Table 5.3.4f (see attached validity testing results). Items with the highest correlation are potentially providing similar satisfaction information. Note, the table provides six sets of correlations, the analysis was conducted examining all possible correlations between items. Because items with the highest correlation were potentially gathering similar satisfaction information they could be eliminated from the instrument.
B. In addition, using all items in the instrument a factor analysis was conducted. Using the global items Q1 (“How satisfied are you with the facility?”) the Cronbach’s Alpha of adding the “best” additional item is shown in the table below. Cronbach’s alpha measures the internal consistency of the values entered into the factor analysis; a value of 0.7 or higher is generally considered acceptably high. The additional item(s) is considered best in the sense that it is most highly correlated with the existing item, and therefore provides little additional information about the same construct. So this analysis was also used to eliminate items. Note, the table again provides 7 sets of correlations, the analysis was conducted examining all possible correlations between items. See table 5.3.4g in attached validity testing results.
Thus, using the correlation information and factor analysis 3 items representing the CoreQ: Long-Stay Resident questionnaire were identified.
4. Validity Testing for the Core Q: Long-Stay Resident Measure.
The overall intent of the analyses described above was to identify if a sub-set of items could reliably be used to produce an overall indicator of satisfaction, the CoreQ: Long-Stay Resident questionnaire.
A. The items were all scored according to the rules identified elsewhere. The same scoring was used in creating the 3 item CoreQ: Long-Stay Resident questionnaire summary score and the satisfaction score using the Pilot CoreQ: Long-Stay Resident questionnaire. The correlation was identified as having a value of 0.89. That is, the correlation score between actual the “CoreQ: Long-Stay Resident Measure” and all of the 18 items used in the Pilot instrument indicates that the satisfaction information is approximately the same if we had included either the 3 items or the 18 item Pilot instrument.
B. We performed additional validity testing of the facility-level CoreQ: Long-Stay Resident measure by measuring the correlations between the CoreQ: Long-Stay Resident measure scores and i) measures of regulatory compliance and other quality metrics from the Certification and Survey Provider Enhanced Reporting (CASPER) data, and ii) several other quality metrics from Nursing Home Compare.
The summary score from the 3 CoreQ: Long-Stay Resident questionnaire items is calculated in the following way: Respondents answering poor are given a score of 1, average = 2, good =3, very good =4 and excellent =5. For the 3 questionnaire items the average score for the resident is calculated. The facility score represents the percent of residents with average scores of 3 or above. This score should be associated with quality. Therefore, for each facility in the sample the correlation with other quality indicators was examined.
(i) Relationship with CASPER Quality Indicators
Certification and Survey Provider Enhanced Reporting (CASPER) contains data collected as part of state/federal nursing home inspections. In short, nursing facilities that accept residents with Medicare and/or Medicaid payments are surveyed. This includes most (i.e., 97% [16,000 facilities]) nursing homes in the U.S. The survey process occurs approximately yearly, and includes the recording of many quality characteristics of the nursing home. These include restraint use; pressure ulcers; catheter use; antipsychotic use; antidepressant use; antianxiety use; and, use of hypnotics. These are commonly used quality indicators used for examining the quality of nursing homes.
In addition, when a nursing home is determined not to meet a certification minimum standard a deficiency citation is issued. These deficiency citations are also commonly used in the analyses of the quality of nursing homes. Approximately 180 deficiency citations exist and are grouped into 16 categories. These 16 categories group like areas together. They were developed by CMS and have considerable face validity; although, one limitation of using these categories is that they were not defined using empirical estimation (such as factor analysis).
(ii) Relationship with Nursing Home Compare (NHC) Quality Indicators, Five Star ratings, and staffing levels
Nursing Home Compare (NHC) is a nursing home report card. After several years of pilot testing, the Centers for Medicare and Medicaid Services (CMS) released this report card on the world-wide web in November of 2002. Briefly, Nursing Home Compare provides information for facility location, structural factors (such as ownership), and staffing characteristics (such as registered nurse [RN] staffing levels). Most significantly, standardized quality information is presented in what are called Quality Measures (QMs). These are calculated from MDS information.
At the time period of for this study (i.e., 2014) CMS reported on 19 measures – these are called the core Quality Measures. The Quality Measures address specific areas of resident care, 5 are for short-stay residents and 14 are for long-stay residents. Long-stay measures are for those residents staying at a facility 3 months or more and short-stay measures are for residents staying at a facility less than 3 months. The long-stay measures are most pertinent to the CoreQ: Long-Stay Resident questionnaire; therefore, these were used in the analyses.
Nursing Home Compare also uses a five-star rating for facilities. This is based on information from the health inspection, direct care staffing, and the MDS quality measures. A five star facility is the highest score and a 1 star facility the lowest score. With respect to staffing, two measures are used: 1) RN hours per resident day; and 2) total staffing hours (RN+ LPN+ nurse aide hours) per resident day.
Tables mentioned above are attached in 5.3.4a.
1. Validity Testing for the Questionnaire Format used in the CoreQ: Long-Stay Resident Questionnaire
A. The literature review shows that domains used in the Pilot CoreQ: Long-Stay Resident questionnaire items have a high degree of both face validity and content validity.
B. Residents overall rankings, show the general “domain” areas used indicates a high degree of both face validity and content validity.
C. The results show that 100% of residents are able to complete the response format used. This testing indicates a high degree of both face validity and content validity.
D. The Flesch-Kinkaid scale score achieved for all questions indicates that respondents have a high degree of understanding of the item.
2. Testing the Items for the CoreQ: Long-Stay Resident Questionnaire
A. The percent of missing responses for the items is very low. The distribution of the summary score is wide. This is important for quality improvement purposes, as nursing facilities can use benchmarks etc.
B. EFA shows that one factor explains the common variance of the items. A single factor can be interpreted as the only “concept” being measured by those variables. This means that the instrument measures the global concept of satisfaction and not multiple areas of satisfaction. This supports the validity of the CoreQ instrument as measuring a single concept of “customer satisfaction”. This testing indicates a high degree of criterion validity.
3. Testing to Determine if a Sub-Set of Items could Reliably be used to Produce an Overall Indicator of Satisfaction (The Core Q: Long-Stay Resident measure)
A. Using the correlation information of the Core Q: Long-Stay Resident questionnaire (18 items) and the 3 items representing the CoreQ: Long-Stay Resident questionnaire a high degree of correlation was identified. This testing indicates a high degree of criterion validity.
B. EFA shows that one factor explains the common variance of the items. A single factor can be interpreted as the only “concept” being measured by those variables. This means that the instrument measures the global concept of satisfaction and not multiple areas of satisfaction. This supports the validity of the CoreQ instrument as measuring a single concept of “customer satisfaction”. This testing indicates a high degree of criterion validity.
4. Validity Testing for the Core Q: Long-Stay Resident Measure
A. The correlation of the 3 item CoreQ: Long-Stay Resident measure summary score (identified elsewhere in this document) with the overall satisfaction score (scored using all data and the same scoring metric) gave a value of 0.89.
That is, the correlation score between actual the “CoreQ: Long-Stay Resident Measure” and all of the 18 items used in the Pilot instrument indicates that the satisfaction information is approximately the same if we had included either the 3 items or the 18 item Pilot questions.
This indicates that the CoreQ: Long-Stay Resident measure score adequately represents the overall satisfaction of the facility. This testing indicates a high degree of criterion validity.
(i) Relationship with CASPER Quality Indicators
The 8 CASPER Quality Indicators all had a reasonable level of positive correlation with the CoreQ: Long-Stay Resident measure in the direction as expected (higher satisfaction is associated with better quality. These correlations range from 0.27 to 0.40. The CoreQ: Long-Stay Resident measure is associated with these quality indicators. This testing indicates a reasonable degree of construct validity and convergent validity.
(i) Relationship with Nursing Home Compare (NHC) Quality Indicators, Five Star ratings, and staffing levels
The strongest associations were observed for agency staffing (0.40), staff stability (0.39), and staff turnover (r = 0.38), indicating that satisfaction is most closely related to workforce consistency and staffing composition. These relationships support the expected theoretical linkage between continuity of care and resident-reported experience, providing evidence that the satisfaction measure is sensitive to core workforce conditions.
Weaker but still significant associations were found for Five-Star rating (0.29) and CNA hours per resident day (0.27). This suggests that regulatory quality scores and staffing intensity are related to satisfaction, but less strongly than workforce stability measures. This pattern is consistent with construct theory distinguishing structural quality indicators (e.g., staffing levels, ratings) from satisfaction-based outcomes (resident satisfaction), supporting discriminant validity of the satisfaction construct.
However, the positive direction of associations for staff turnover and agency staffing should be interpreted cautiously, as these relationships are typically expected to be negative in established care quality frameworks. This may reflect coding direction, case-mix differences, or unmeasured confounding, which limits full confidence in directional interpretation.
We repeated some of this testing with 2025 data. This also included additional staffing data such as turnover, stability, and agency use. These factors were used as previous studies have shown strong associations with quality in nursing homes. We find a similar relationship with CoreQ. This updated testing indicates a high degree of construct validity and convergent validity.
In summary, moderate correlations were identified between the quality measures. As noted by Mor and associates (2003, p.41) “there is only a low level of correlation among the various measures of quality.” Castle and Ferguson (2010) also show the pattern of findings of quality indicators in nursing facilities is consistently moderate with respect to the correlations identified. The magnitude of correlations of the CoreQ with quality metrics are consistent with other findings in this setting. Thus, it is not surprising that “very high” levels of correlations were not identified. Nevertheless, moderate correlations were identified and with a larger 2025 sample and more reliable data the correlations were consistent.
Summary
Validity testing demonstrates moderate convergent validity with established staffing and quality indicators based on statistically significant associations across all variables (all p < 0.001). The pattern of correlations suggests that the measure is meaningfully aligned with known structural determinants of care quality while still capturing a distinct experiential construct.
References
Castle NG, Ferguson JC. What is nursing home quality and how is it measured? Gerontologist. 2010 Aug;50(4):426-42. doi: 10.1093/geront/gnq052. Epub 2010 Jul 14. PMID: 20631035; PMCID: PMC2915498.
Mor V, Berg K, Angelelli J, Gifford D, Morris J, Moore T. The quality of quality measurement in U.S. nursing homes. Gerontologist. 2003 Apr;43 Spec No 2:37-46. doi: 10.1093/geront/43.suppl_2.37. PMID: 12711723.
Risk Adjustment
Risk adjustment is noted in some recent publications examining long-term care satisfaction. For example, one recent report noted “As of the date of this report, risk adjustment for geography (Twin Cities Metro vs. Other) is recommended. The U of MN suggests evaluating risk adjustment for size once there is sufficient data from small facilities (<20 residents), to conduct meaningful analysis.” (Shippee, Woodhouse, & Skarpho, 2023). The rationale for this has some merits, as this same report notes this “allows for more fair comparisons between providers who may serve residents with different needs and have different resources.”
However, this does come with some complications. First, it is not clear what risk adjustment methodology should be used. A simple stratification here would seem appropriate. Second, the notion of fair comparisons comes from a provider perspective. An alternative view is that a consumer should know the quality (i.e., satisfaction level) of a facility based on a standard and uniform metric. So risk adjustment for CoreQ using facility metrics could be misleading.
A memo regarding the Home Health Care CAHPS (HHCAHPS) Survey patient-mix adjustments (MUC2024, Adjustment Factors) noted that in 2022 HHCAHPS field test were used to determine which patient characteristics (patient mix) affected patients’ assessment of the home health care they received. As with prior HHCAHPS field tests, the results of this showed differences in responses attributable to patient mix characteristics. Patient mix characteristics that were important included: Age, Education, Self-reported health status, and Mental/emotional status. Some have noted issues with HHCAHPS risk-adjustment (Lines eta l., 2020).
In long-term care hospitals (LTACs) Zuckerbraun and associates (2020) identified the need for the adjustment of a CHAPS like survey. Patient risk factors included: age, gender, education, ethnicity, race, marital staus, and overall health.
Kwon and Bowblis (2024) included the percent of residents with dementia, psychiatric illness and depression in their risk adjustment. For testing of CoreQ, a similar approach was not possible as the surveys are not linked directly to patient characteristics.
For CoreQ, an overall score is calculated for a facility (and as described elsewhere is not an average score of the items). Thus, the CoreQ score cannot be calculated using specific characteristics (such as age). So in our testing we used the average score for one question item (In recommending this facility to your friends and family, how would you rate it overall?). We acknowledge that one limitation of this is that the risk adjustment (if any) may be different for the other question items in CoreQ. Also, no interaction effects are used (e.g., male and <65). Factors such as SES and other factors not included may have an influence. And, lastly the theoretical reasons for the potential differences in scores is not well explained in the literature.
CoreQ: Long-Stay Resident
| Age | Mean Score |
| <65 | 3.77 |
| 65-75 | 3.82 |
| >75 | 3.84 |
| Sex | |
| Male | 3.69 |
| Female | 3.75 |
| Education | |
| Some High School | 3.81 |
| High School Graduate/GED | 3.82 |
| College Graduate or More | 3.78 |
Testing summary. Conducted in 2025 with 2,970 residents. The facilities used were geographically diverse, and representative of national long-term care characteristics for size, payor mix, and ownership). The score distribution is 1 (low) to 5 (high). The average scores do vary by the patient mix.
The average scores do vary by the patient mix. However, the influence is small and given the numerous unknowns in this area (described above) risk-adjustment is not recommended at this time.
To date, results from satisfaction surveys have mostly not used risk adjustment. The CoreQ measures overall satisfaction. Providing the scores across entities without any adjustment does provide a fair comparison. In addition, the data elements that could be used for any adjustment are not collected as part of the CoreQ (the surveys are anonymous).
References
Kwon, J. & Bowblis, J.R. (2024). Association between nursing home five-star ratings and consumer satisfaction. JAMDA, 25, 105322.
Lines, L.M., Anderson, W.L., Gordek, Ha., & Kenyoun, A.E. (2020). Risk adjustment in home health care CAHPS. AJMC, 25(2), 58-59.
MUC2024. Patient-Mix Adjustment Factors for Home Health Care CAHPS Survey Field Test Results.
Shippee T., Woodhouse, M., & Skarpho, T. (2023). Building Resident Quality of Life and Family Satisfaction Measures for the Minnesota Assisted Living Report Card. A Report to the Minnesota Department of Human Services.
Zuckerbraun, S.M., Deutsch, A., Eicheldinger C., et al. (2020). Risk adjustment, mode adjustment, and nonresponse bias analysis on quality measures from a long-term care hospital experience of care survey. Archives of Physical Medicine and Rehabilitation, 101, 841-51.
Use & Usability
Use
The AHCA/NCAL National Quality Awards Program is a progressive program that is based on the Baldrige Criteria for Performance Excellence. This nationally recognized approach to performance excellence focuses on system-based quality improvement to create.
The geographic area is the nation. The AHCA/NCAL National Quality Awards Program is used across the nation. Over 1,700 entities have received an award to date. This represents about 180,000 patients/residents.
The level of analysis is the facility-level. The care settings are skilled nursing and assisted living facilities.
The program allows skilled nursing and assisted living organizations to benchmark personal metrics to those of their peers and examine ongoing quality improvement efforts.
Skilled Nursing and Assisted living facilities across the United States utilize LTC Trend Tracker. About 15,266 Skilled Nursing Facilities and 9,280 Assisted Living Facilities use the program.
The level of analysis is the facility-level. The care settings are skilled nursing and assisted living facilities.
Rhode Island's Health Care Quality Reporting Program promotes quality in the state's healthcare system by developing a healthcare quality performance measures and reporting program to guide quality improvement initiatives.
Rhode Island: 73 SNFs and 19,948 residents.
The level of analysis is the facility-level. The care setting is skilled nursing facilities.
Measuring and improving patient satisfaction is valuable to patients, because it is a way forward on improving the patient-provider relationship, which influences health care outcomes. The target population is all residents in long-term care, that includes Medicare, Medicaid and private pay. Functional long-term care accountability programs often include a satisfaction component. These programs often prioritize the following components:
• Quality Assurance and Performance Improvement (QAPI) system in place,
• Person-centered care,
• Data-driven monitoring of performance metrics,
• Strong Leadership,
• And ethical compliance.
Together, all these components create a framework for quality improvement.
Usability
Improving performance relies on the testing of change and benchmarking. Frequently collecting data is a necessary step to enhance and maximize quality improvement. Data collected during tests provides critical insight that is needed to determine the best path forward. Benchmarking is a process used to measure the quality and performance of your organization. Benchmarking plays a significant role in identifying patterns, providing context, and then guiding decision-making processes.
The CoreQ Long-Stay Resident Satisfaction measure allows skilled nursing facilities to measure the impact of tests of change and benchmark their performance relative to other facilities. Specifically, facilities can increase the number of staff and/or improve staff training and measure the impact using CoreQ. Similarly, improvements in reduced adverse events, such as falls and hospitalizations, increase resident rating of care received and increase satisfaction. Finally, facilities can understand and address the needs and wants of residents, like certain activities or food, to increase their willingness to recommend the facility and CoreQ performance
The actions needed to improve performance are not difficult once a process or plan for improvement is developed (e.g. Quality Assurance/Performance Improvement (QAPI)). Measured entities can overcome difficulties by monitoring data and results. Monitoring data often ensures you preserve the advances of the quality improvement effort. Developing a feedback and monitoring system to sustain continuous improvement helps providers preserve the advances of the quality improvement effort.
The CoreQ measure for skilled nursing residents has elevated the resident and family voice as well as help guide consumer choices as another way for potential residents to review the quality of a care facility. Specifically, the CoreQ measure has been independently tested as a valid and reliable measure of customer satisfaction. The CoreQ is a short survey with three to four questions which reduces response burden on residents and allows organizations to benchmark their results with consistent questions and response scale. Satisfaction vendors and providers have particularly appreciated how easy it is to integrate the CoreQ questions to their satisfaction surveys. They believe the short length relative to other survey tools, like HCAHPS, helps increase and maintain high response rates.
AHCA/NCAL developed LTC Trend Tracker, a web-based tool that enables long term and post-acute care providers, including assisted living, to access key information that can help their organization succeed. The CoreQ report and upload feature within LTC Trend Tracker includes an API (application programming interface) for vendors performing the survey on behalf of SNFs to upload data, so that the aggregate CoreQ results will be available to providers. Given that LTC Trend Tracker is the leading method for AHCA/NCAL members to profile their quality and other data, the incorporation of CoreQ into LTC Trend Tracker means it will immediately become the de facto standard for customer satisfaction surveys for the long-term care industry. AHCA/NCAL continues to work with customer satisfaction vendors to promote CoreQ and receives requests for vendors to be added to the list of those incorporating CoreQ. Currently, there are over 40 vendors across the nation who can administer the CoreQ survey.
We are also working with states who require satisfaction measurement to incorporate CoreQ into their process. AHCA/NCAL has a presence in each state, and our state affiliates continue to promote the use of the CoreQ.
Feedback is continuously obtained through meetings with facility operators and vendors serving on AHCA/NCAL’s Customer Experience Committee and the CoreQ Vendors’ Workgroup. The purpose of the Customer Experience Committee is to champion the importance of meeting customer expectations now and in the future. This includes defining quality from the consumer’s perspective. Key areas of focus include collecting, analyzing, and using data to drive performance improvement, and the application of successful practices. The CoreQ Vendors’ Workgroup was created to help improve CoreQ usage and discuss ways to best support the CoreQ Vendors’ who administer the surveys.
AHCA/NCAL developed LTC Trend Tracker, a web-based tool that enables long term and post-acute care providers, including assisted living, to access key information that can help their organization succeed. The CoreQ report and upload feature within LTC Trend Tracker includes an API for vendors performing the survey on behalf of skilled nursing facility's to upload data, so that the aggregate CoreQ results will be available to providers. Given that LTC Trend Tracker is the leading method for AHCA/NCAL members to profile their quality and other data, the incorporation of CoreQ into LTC Trend Tracker means it will immediately become the de facto standard for customer satisfaction surveys for the skilled nursing industry. AHCA/NCAL continues to work with customer satisfaction vendors to promote CoreQ and receives requests for vendors to be added to the list of those incorporating CoreQ.
Among providers and vendors, we receive feedback during committee and workgroup meetings. For feedback on LTC Trend Tracker, we scope out the cost and feasibility of suggested enhancements. For example, we added a more graphical user interface option for the API, in addition to the original command line interface that was more technical, based on feedback from vendors.
For some of the feedback we receive, we use it as an opportunity to educate about best practices in survey collection and administration. For example, some vendors and providers inquire about administering CoreQ over the phone or other mixed modes of collection. In this instance, we caution vendors and providers about possible response or interviewer bias and recommend using written surveys as the primary method because it has been tested and shown to be reliable and valid.
The impact of COVID pandemic and the gradual recovery shows in the performance trend. Prior to the pandemic, the average satisfaction rate in LTC Trend Tracker was 81.4% (n=1,666). Over the next two years, the average rate dropped to a low of 54.8% (n=985) as the industry faced a massive staffing crisis, limited visitation, and had intense isolation procedures in place to limit potential outbreaks. Since, the average rate has been on the rise. The latest average rate is 84.4% (n=534). We suspect the increase in industry workforce as reported by the U.S. Bureau of Labor Statistics and the return to more normal operations related to visitors, activities, and patient interactions have helped increase satisfaction rates.
There were no negative consequences to individuals or populations identified during testing or evidence of unintended negative consequences to individuals or populations reported since the implementation of the CoreQ: Long-Stay Resident questionnaire or the measure that is calculated using this questionnaire. This is consistent with satisfaction surveys in general in nursing facilities. Many other satisfaction surveys are used in nursing facilities with no reported unintended consequences to patients or their families.
There are no potentially serious physical, psychological, social, legal, or other risks for patients. However, in some cases the satisfaction questionnaire can highlight poor care for some dissatisfied patients, and this may make them further dissatisfied.
Comments
Staff Preliminary Assessment
CBE #2615 Staff Assessment
Importance
Strengths:
- A clear logic model is provided, depicting the relationships between inputs (e.g., competency of staff, responsiveness of management), activities (e.g., discharge instructions, RN assessments), and desired outcomes (e.g., ratings of care and short stay discharge satisfaction). This model demonstrates how the measure's implementation will lead to the anticipated outcomes.
- The problem this measure addresses aligns with past initiatives to improve nursing home care, and more recent evidence demonstrates that residents’ satisfaction with care is associated with measures of nursing home quality.
- Data from quarter 2 of 2024- quarter 1 of 2025 of the Long Term Care (LTC) trend tracker show a performance gap, with decile ranges from 0.00% to 100%, with an average 64.3%, indicating variation in measure performance and less than optimal performance across the target population.
- Description of patient input supports the conclusion that the measured outcome is meaningful with at least moderate certainty. Patient input was obtained through literature review and focus group of residents and family members. However, the literature evidence provided was older and no timeframe for when patient input was obtained was provided.
Limitations:
- The literature review mainly includes studies that are more than 10 years old, including their review of nursing home quality initiatives. The submission could be strengthened by discussion of the quality, quantity, and consistency of the older evidence, and providing updated references, if possible.
- The literature provided for importance to patients is older and no timeframe for when patient input was obtained is provided. The submission could be strengthened by discussion of the quality, quantity, and consistency of the older evidence, and providing updated references, if possible.
Rationale:
- This maintenance measure meets all criteria for 'Met' for importance due to the significance of the problem it addresses its robust evidence base, a documented performance gap, and well-articulated logic model, making it essential for addressing patient satisfaction. There is at least moderate confidence that the business case is adequate, i.e., the anticipated impacts of the measure on patient outcomes and justify use of the measure.
Closing Care Gaps
Strengths:
- None identified.
Limitations:
- The developer referenced analyses performed using CoreQ survey data collected in 2021. The developer stated they found that average measures scores for Black residents were lower than for white residents, and that this difference may have been due to between-facility differences in care rather than within-facility differences in care. However, the developer did not describe the statistical method used or report statistical results to support their statements. The developer did not provide recommended actions entities can take to close care gaps, such as targeted quality improvement initiatives or policy changes, if relevant.
Rationale:
- The rating for Closing Care Gaps is 'Not Met' due to insufficient information provided. While the developer attempted to assess gaps in care across race subgroups, statistical methods and results for their analyses were not reported.
Feasibility Assessment
Strengths:
- All required data elements are routinely generated during care delivery, and required elements are available from digital or electronic sources.
The developer indicated there have been no changes to the measure specifications. The developer stated that no feasibility issues were found requiring adjustment of the final measure specifications. - The developer described the costs and burden associated with data collection and data entry, validation, and analysis. They discussed current barriers that could be encountered in implementing or reporting the measure, which include collection burden and cost burden. They noted that the cost burden is minimal and that CoreQ can be added to the beginning of existing surveys to reduce adoption burden.
- The developer described how all required data elements can be collected without risk to patient confidentiality because surveys are anonymous and scores are only calculated with 20 or more survey returns.
Any fees, licensing, or other requirements to use any aspect of the measure (e.g., value/code set, risk model, programming code, algorithm) are clearly described and justified.
Limitations:
- There is a small cost and collection burden, but these are minimal and mitigations are addressed by the developer.
Rationale:
- This maintenance measure meets all criteria for 'Met' for feasibility due to its well-documented feasibility assessment, clear and implementable data collection strategy, and transparent handling of patient confidentiality, burden, licensing, and fees. These factors collectively ensure that the measure can be implemented effectively and sustainably in a real-world healthcare setting.
Scientific Acceptability
Strengths:
- The developer describes signal-to-noise reliability estimation which is needed for this maintenance measure.
Limitations:
- The developer did not provide any details about the data used for accountable entity-level reliability testing.
- Based on the submission, the developer seems to have estimated signal-to-noise reliability for each decile of entities ranked by performance score rather than estimate signal-to-noise reliability for each entity. The percentage of entities which meet the expected threshold of 0.6 for signal-to-noise reliability cannot be determined from this submission.
Rationale:
- This maintenance measure is rated as ‘Not Met But Addressable’ for reliability because the description of the data used for accountable entity-level reliability was insufficient and the reliability metrics provided by the developer do not allow for determining whether established thresholds were met. However, the identified limitations are deemed addressable, as the developer may consider providing details about the data used for accountable entity-level reliability and applying the signal-to-noise reliability test method to each entity and then updating Tables 2a and 2b with the mean reliability of entities within each decile. By addressing these issues, there is potential to meet the reliability requirements.
Strengths:
- The developer performed the required validity testing for this maintenance measure, namely, they conducted accountable entity-level (“measure score”) validity testing at the level for which the measure is specified. The data used for the validity analysis was collected from a national sample of 387 facilities (n=12,771 residents) in January to December 2025, and included the CoreQ Long Stay Questionnaire, CMS Nursing Home Compare measures and the CMS Five-star rating, and Certification and Survey Provider Enhanced Reporting (CASPER) quality indicators. Note that testing results based on data collected before 2020 cannot be considered in the rating.
- The developer hypothesized moderate positive correlations between the CoreQ Short Stay measure and the five other measures for which 2025 results are reported in Table 5.4.3h in the "Additional Validity Testing" attachment, including "Staff Stability" (r=0.52, p<0.001), "Agency Staffing" (r=0.46, p<0.001), "CNA hours per resident day" (r=0.35, p<0.001); "Staff Turnover" (r=0.37, p<0.001); and Five-Star rating (r=0.38, p<0.001).
- A thorough, well-developed logic model supports an inference of validity for this measure.
Limitations:
- There are several limitations to note regarding the accountable entity level validity testing presented:
- The type of correlation analysis performed was not reported.
- Measures used as comparators in correlation analyses provided in the attachment were referenced but not described (e.g., how is "staff turnover" defined, and is a higher score better?), and the dataset each measure comes from is not clearly stated.
- The developer reported only the range of estimates for the correlation analyses of the measure with CASPER quality indicators, and the specific CASPER indicators included in the analysis were not named or described. In addition, hypotheses and supporting background regarding the expected direction and strength of each correlation observed are required to understand the meaning of any results.
- Literature was cited to support expectations for "moderate correlations" between measures, but references provided are from 2003 and 2010, and the specific measures the cited studies reported on are not provided.
Note: - Several tables referenced in the submission ( 5.3.4c, 5.3.4d, 5.3.4f, and 5.3.4g), were not included in the attachment; however, these findings appear to refer to data element testing (i.e., for the questionnaire) and are not required for this maintenance measure.
- The developer referenced two other measures (community discharge, rehospitalization) and provided correlation results in the text, but these results use data collected in 2014 and are not considered in the rating.
- The face validity testing described appears to refer to psychometric testing of the instrument, rather than accountable-entity level testing of the measure score itself. Person- or encounter-level testing of the measure is not required for a maintenance submission, and is not considered in the validity rating.
- The developer did not conduct risk or case-mix adjustment or stratification. The developer provided the rationale that a risk adjustment approach at the patient-level was not possible, as surveys are not linked directly to patient characteristics, and use of facility-level metrics could be misleading. The developer examined average survey scores for one question by appropriate case-mix factors and noted minimal variation in measure score by case-mix factor.
Rationale:
- This maintenance measure is rated as ‘Not Met But Addressable’ for validity because the validity testing results partially support an inference of validity for the measure, suggesting that the measure somewhat accurately reflects performance on quality and can distinguish good from poor performance to a limited extent.
The developer did not conduct risk adjustment or stratification, but provided a reasonable rationale for why and supported the rationale with literature and basic analysis.
Use and Usability
Strengths:
- The measure is currently used in Rhode Island’s Healthcare Quality Reporting Program, LTC Trend Tracker, and American Health Care Association and the National Center for Assisted Living (AHCA/NCAL) National Quality Award Program. Attributes of a suitable program for this measure are described, and these include a target population is all residents in long-term care, that includes Medicare, Medicaid and private pay.
- The developer provided a summary of how accountable entities can use the measure results to improve performance. Specifically, benchmarking and monitoring data to measure resident needs and wants, satisfaction, and staff competency.
- Feedback is gathered via committee and workgroup meetings. Feedback is scoped for feasibility and cost. Some feedback is used as an opportunity to educate vendors about best practices in survey collection and administration. Feedback has also been used to make updates to the interface for the API.
Pre-pandemic, the mean performance score was 81.4%, which dipped to 54.8% during the early pandemic years and post pandemic, has risen to 84.4%. The developer provided a clear rationale for these changes, noting that the pandemic drove significant reductions in staffing and implementation of strict isolation protocols. - The developer reported no unexpected findings. The developer noted that the satisfaction questionnaire can highlight poor care for some dissatisfied patients, and this may make those patients further dissatisfied.
Limitations:
- No specific timeframe for performance data was given. The most recent number of reporting facilities is less than half of the number in earlier reporting periods, which could potentially affect mean ratings through selection bias. The submission could be strengthened with a discussion of any potential effects.
Rationale:
- This maintenance measure is rated ‘Met’ for use and usability because it is actively used in at least one accountability application with a systematic feedback approach that allows for continuous updates based on stakeholder feedback. The measure also demonstrates a positive trend in performance results, affirming its ongoing usability. The developer reported no unexpected findings.
Committee Independent Review
Agreement with staff assessment
Importance
Concur with staff review about reliance on older literature. Missing dates/timeframes for testing with convenience sample make it hard to assess relevance
Closing Care Gaps
Concur with staff rating - submission describes a care gap but does not specify how it could be addressed. Noting that this domain is optional.
Feasibility Assessment
Concur with staff assessment
Scientific Acceptability
Concur with staff assessment of missing detail around testing approaches and results. Much of the literature is older and predates the COVID PHE, which disproportionately impacted the nursing home industry. Missing information on some testing activities - was pilot testing in 2014?
See above
Use and Usability
Generally concur with staff assessment but would have liked to see more detail about how results could drive quality improvement, apart from frequently repeated data collection. Measure does not yield information on the factors driving satisfaction that could be targeted for improvement activities. Would also be helpful to have examples of successful efforts to improve scores from discrete facilities, chains or states.
Summary
See additional comments and suggestions in individual domains.
Public Comments
No public comments
No public comments