The ICH CAHPS Survey is designed to measure the experiences of people receiving in-center hemodialysis care from Medicare-certified dialysis centers. The survey is designed to meet the following three broad goals:
- Produce comparable data from the patient’s perspective that will allow objective and meaningful comparisons between dialysis centers on domains that are important to consumers.
- Create incentives for dialysis centers to improve their quality of care.
- Enhance public accountability in health care by increasing the transparency of the quality of care provided in return for public investment.
Specifically, the survey measures patients’ experiences on topics that are important from the perspective of patients and help them make more informed choices when selecting a dialysis center as well as helping dialysis centers improve the quality of dialysis care for their patients.
The survey is administered semiannually to patients who have received in-center hemodialysis for at least 3 months from Medicare-certified dialysis centers. Data collection for each survey period is 12 weeks. The survey is available in mail-only, phone-only, and mail with phone follow-up.
Survey results publicly reported include two ratings and two measures:
• Global rating of dialysis center staff
• Global rating of dialysis center
• Quality of Dialysis Center Care and Operations measure (QDCCO, calculated from 13 survey questions)
• Providing Information to Patients measure (PIP, calculated from 9 survey questions)
The ICH CAHPS Quality of Dialysis Center Care and Operations (QDCCO) measure captures whether dialysis patients feel that their that dialysis center staff communicated well, kept patients as comfortable and pain-free as possible, behaved in a professional manner, and kept the center clean.
The questions included in this measure are:
3. In the last 3 months, how often did the dialysis center staff listen carefully to you?
4. In the last 3 months, how often did the dialysis center staff explain things in a way that was easy for you to understand?
5. In the last 3 months, how often did the dialysis center staff show respect for what you had to say?
6. In the last 3 months, how often did the dialysis center staff spend enough time with you?
7. In the last 3 months, how often did dialysis center staff make you as comfortable as possible during dialysis?
8. In the last 3 months, did you feel comfortable asking the dialysis center staff everything you wanted about dialysis care?
11. In the last 3 months, how often did dialysis center staff check you as closely as you wanted while you were on the dialysis machine?
13. In the last 3 months, how often was the dialysis center staff able to manage problems during your dialysis?
14. In the last 3 months, how often did dialysis center staff behave in a professional manner?
15. In the last 3 months, how often did dialysis center staff explain blood test results in a way that was easy to understand?
21. In the last 3 months, when you arrived on time, how often did you get put on the dialysis machine within 15 minutes of your appointment or shift time?
22. In the last 3 months, how often was the dialysis center as clean as it could be?
31. In the last 12 months, how often were you satisfied with the way they handled these problems?
Measure Specs
- General Information(active tab)
- Numerator
- Denominator
- Exclusions
- Measure Calculation
- Supplemental Attachment
- Point of Contact
General Information
One of the goals of the CMS National Quality Strategy is to foster engagement and to bring the voices of patients to the forefront. As part of fostering engagement, it is critical to hear the voice of individuals by obtaining feedback from them on in-center hemodialysis (ICH) facility performance and incorporating it as part of CMS’s comprehensive approach to quality. Patient-centeredness is a central goal of dialysis care and can be directly measured through surveys of dialysis patients. CMS created the In-Center Hemodialysis CAHPS® Survey, a component of the End-Stage Renal Disease (ESRD) Quality Incentive Program (QIP), to ensure that an assessment of the patient-centeredness of care would be included to monitor dialysis facility performance, promote quality improvement, and inform consumer decision making in the selection of a dialysis facility via public reporting of results. The ICH CAHPS Survey is a standardized survey instrument and data collection methodology for measuring ICH patients’ perspectives on their care in Medicare-certified dialysis centers. The survey is administered semiannually to patients who have received dialysis for at least 3 months from Medicare-certified dialysis centers.
The ICH CAHPS Quality of Dialysis Center Care and Operations (QDCCO) multi-item measure captures whether patients feel that their dialysis center staff communicated well, kept patients as comfortable and pain-free as possible, behaved in a professional manner, and kept the center clean.
This measure reflects patient experiences with their dialysis facility across a variety of domains that were identified as important to patients and stakeholders based on focus groups and cognitive interviews during the survey revision development phase. This measure will be publicly reported on Medicare.gov Care Compare tool to help ICH facilities with quality improvement and help ESRD patients find high quality dialysis facilities.
In addition to survey responses from patients, we receive data from a CMS database. ICH facilities are required to enter dialysis information for all patients into the ESRD Quality Reporting System (EQRS). EQRS data are then used to create the samples for each ICH CAHPS Survey period. In addition, information such as sex and age are pulled from the EQRS data and merged with survey response data files for analysis purposes. We work with the EQRS team on a continuous basis to remain up to speed on the data, changes to the data or data format, and to mitigate any issues with the data.
Numerator
CMS calculates ICH CAHPS Survey measure scores using top-box scoring. The top-box score refers to the percentage of respondents that give the most positive response(s): 12 items in the ICH CAHPS QDCCO measure use a “Never/Sometimes/Usually/Always” response scale where the top-box numerator is the number of respondents who answer “Always.” 1 item uses a “Yes/No” response scale where top-box numerator is the number of respondents who answer “Yes.”
CMS calculates ICH CAHPS Survey multi-item measure scores using top-box scoring. The top-box score refers to the percentage of respondents that give the most positive response(s) (answers: Always or Yes).
12 items in the ICH CAHPS QDCCO multi-item measure use a “Never/Sometimes/Usually/Always” response scale where the top-box numerator is the number of respondents who answer “Always.” 1 item uses a “Yes/No” response scale where top-box numerator is the number of respondents who answer “Yes.”
The QDCCO measure is composed of responses to the following survey items:
- In the last 3 months, how often did the dialysis center staff listen carefully to you?
- In the last 3 months, how often did the dialysis center staff explain things in a way that was easy for you to understand?
- In the last 3 months, how often did the dialysis center staff show respect for what you had to say?
- In the last 3 months, how often did the dialysis center staff spend enough time with you?
- In the last 3 months, how often did dialysis center staff make you as comfortable as possible during dialysis?
- In the last 3 months, did you feel comfortable asking the dialysis center staff everything you wanted about dialysis care?
- In the last 3 months, how often did dialysis center staff check you as closely as you wanted while you were on the dialysis machine?
- In the last 3 months, how often was the dialysis center staff able to manage problems during your dialysis?
- In the last 3 months, how often did dialysis center staff behave in a professional manner?
- In the last 3 months, how often did dialysis center staff explain blood test results in a way that was easy to understand?
- In the last 3 months, when you arrived on time, how often did you get put on the dialysis machine within 15 minutes of your appointment or shift time?
- In the last 3 months, how often was the dialysis center as clean as it could be?
- In the last 12 months, how often were you satisfied with the way they handled these problems?
Denominator
ICH CAHPS Survey respondents are adult eligible patients who received dialysis care from an in-center hemodialysis facility during the sampling window. A survey is defined as completed when at least 50 percent of the questions applicable to all patients are answered.
The denominator for the ICH CAHPS QDCCO measure is the number of respondents with completed surveys who answer at least one item within the multi-item measure over the 2 semiannual periods being publicly reported.
ICH CAHPS Survey respondents are adult eligible patients who received dialysis care from an in-center hemodialysis facility during the sampling window. A survey is defined as completed when at least 50 percent of the questions applicable to all patients are answered.
The denominator for the ICH CAHPS QDCCO multi-item measure is the number of respondents with completed surveys who answer at least one item within the multi-item measure over the 2 semiannual periods being publicly reported.
Exclusions
Cases are excluded from the multi-item measure denominator if:
- Patients are under 18 years of age
- Patients’ eligibility is unclear in mail survey
- Patients are not currently receiving dialysis
- Patients are deceased or receiving hospice
- Patients have not received ICH dialysis for at least 3 months
- Patients receive dialysis at a nursing home where they reside or at home
- Patients reside in jail or prison
- Patients are mentally or physically incapable
- Patients are no longer receiving care at sampled facility
- Patients have a language barrier
- Survey completed by a proxy
Denominator exclusions are based on final disposition codes from survey vendors, and approved during data cleaning activities. The cases with the following disposition codes assigned are excluded:
130 – Completed mail survey, survey eligibility unknown
140 – Ineligible: not currently receiving dialysis
150 - Deceased
160 – Ineligible: does not meet eligibility criteria (under 18, hospice, nursing home, less than 3 months, doesn’t receive dialysis at a center)
170 – Language barrier
180 – Mentally or physically incapacitated
190 – Ineligible: No longer receiving care at sampled facility
199 – Ineligible: Completed by proxy
Measure Calculation
The survey is administered semiannually to patients who have received in-center hemodialysis for at least 3 months from Medicare-certified dialysis centers. Data collection for each survey period is 12 weeks.
Survey results publicly reported include two ratings measures and two multi-item measures:
• Global rating of dialysis center staff
• Global rating of dialysis center
• Quality of Dialysis Center Care and Operations multi-item measure (QDCCO, calculated from 13 survey questions)
• Providing Information to Patients multi-item measure (PIP, calculated from 9 survey questions)
CMS calculates ICH CAHPS Survey measure scores using top-box scoring for completed surveys (a survey is defined as completed when at least 50% of the core questions applicable to all patients are answered). The top-box score refers to the percentage of respondents who give the most positive response(s).
ICH CAHPS Survey respondents are eligible adult patients who have received care from an ICH dialysis center for at least 3 months. The numerator for the multi-item measures is the number of most positive responses (9/10 for ratings and “Yes” or “Always” for multi-item measures). The denominator for the ICH CAHPS multi-item measures is the total number of respondents with completed surveys who answered at least one item within the multi-item measure.
Cases are excluded from the measure denominator if:
- Patients are under 18 years of age
- Patients’ eligibility is unclear in mail survey
- Patients are not currently receiving dialysis
- Patients are deceased or receiving hospice
- Patients have not received ICH dialysis for at least 3 months
- Patients receive dialysis at a nursing home where they reside or at home
- Patients reside in jail or prison
- Patients are mentally or physically incapable
- Patients are no longer receiving care at sampled facility
- Patients have a language barrier
- Survey completed by a proxy
More information on the risk adjustment and related calculations can be found in the two attachments submitted for 1.18a. One of the attachments is in Section 7 (supplemental).
The measure is not stratified.
The ICH CAHPS Survey is conducted semiannually each spring and fall. CMS-trained survey vendors can offer survey administration via mail-only, telephone-only, or mixed mode (mail with phone follow-up) to their client ICH facilities. The survey is available in English, Spanish, traditional and simplified Chinese, Samoan, and Vietnamese; the telephone interview is only available in English and Spanish. Vendors must administer the survey in English, but can choose whether to offer other languages.
We do not yet know response rates for the revised survey since it has not been implemented in the national implementation. Our overall response rate for the current ICH CAHPS Survey (not the revised survey) is around 30%; the mixed mode has the highest response rates of the 3 offered modes. Response rates are calculated for the instrument as a whole:
Response Rate = Total Number of completed surveys/(Total Number of Surveys Fielded - Total Number of Ineligible Surveys)
There is no minimum response rate requirement on ICH CAHPS. We are continuously working with survey vendors and ICH facilities to help improve response rates, by offering things such a flyers/posters and waiting room FAQs to place in facilities, training telephone interviewers on avoiding refusals, and using the CMS logo on mailing materials. We also have a dialysis patient page on our project website with survey FAQs and we reference this in the materials that are mailed to sample patients.
Every facility’s sample size differs, depending on the number of survey-eligible patients in the EQRS data, and the size of the facility. There is no minimum sample size per ICH facility. If a facility did not serve 30 survey-eligible patients in the preceding year, they are not required to participate in that year’s surveys. In order for ICH CAHPS scores (for the 2 ratings and 2 multi-item measures) to be publicly reported, a facility must have 30+ completed surveys across the two survey periods that are being reported during that Care Compare refresh on medicare.gov. For example, for the 2024 October Care Compare refresh period, a facility needed at least 30 completed surveys between the 2023 Spring and 2023 Fall Surveys, in order for the scores for the global ratings and QDCCO and PIP multi-item measures to be reported.
Supplemental Attachment
Point of Contact
Not applicable.
Elizabeth Goldstein
Woodlawn, MD
United States
Tracy Kline
RTI International
Research Triangle Park, NC
United States
Importance
Evidence
The Consumer Assessment of Healthcare Providers and Systems (CAHPS) set of patient experience surveys are well-established measures of healthcare quality. Each semiannual survey period, between 350,000 and 415,000 ICH patients are selected to receive the ICH CAHPS Survey. Public reporting of these survey results creates incentives for dialysis centers to improve their quality of care, directly impacting the patients who receive it. Because of this, it is important to ensure that the survey aligns with what patients believe constitutes high-quality care. Specifically, the survey measures patients’ experiences on topics that are important from the perspective of patients and where the patient is the best source of information. The results help dialysis patients make more informed choices when selecting a dialysis center as well as helping dialysis centers improve the quality of dialysis care for their patients.
In addition to a number of psychometric analyses, RTI conducted literature reviews, several rounds of focus groups and cognitive interviews with dialysis patients, conversations with ESRD stakeholders and Technical Expert Panels (TEP), and discussions with the CAHPS Consortium prior to finalizing a revised survey to test in a 2022 field test/mode experiment. Following the field test, results were discussed with the same groups of experts and all supported the revised survey.
During the focus groups, dialysis patients were asked about characteristics that were important to them in regards to high quality dialysis care. They were then asked about specific survey items identified for removal in the psychometric analysis. For each survey item, participants were asked how important questions were in rating and evaluating the care that they receive at their dialysis center. For the QDCCO measure, there was a mix of opinions on each question identified to be removed, but the consensus of the focus group participants was that if questions had to be removed from the measure, then these were the best to remove.
The 2020 and 2023 TEPs regarding survey revisions included 10 members each, consisting of dialysis patients, ESRD network representatives, a survey expert, dialysis patient advocates, and large dialysis organization representatives.
Data from the 2022 ICH CAHPS Survey field test/mode experiment do not allow for direct assessment of the relationship between survey measures and structures or processes. However, given that the modified instrument-derived multi-item measures (QDCCO and PIP) are the same or similar to current ICH CAHPS measures, CMS anticipates that these revised measures will exhibit similar relationships to those of the existing ICH CAHPS Survey measures. For example, an exercise comparing the current QDCCO multi-item measure (prior to item removal) to the revised QDCCO multi-item measure results in a near-perfect correlation (r=0.991) and illustrating the strong relationship between the current and revised state of the QDCCO multi-item measure.
The survey measures patients’ experiences on topics that patients have identified as important to measure the quality of care for facilities and for which the patient is the best source of this information. These survey measures help dialysis patients make more informed choices when selecting a dialysis center as well as helping dialysis centers improve the quality of dialysis care for their patients. The QDCCO multi-item measure asks questions regarding the dialysis center’s quality of care and operations. Focus groups/cognitive interviews with dialysis patients, conversations with ESRD stakeholders and Technical Expert Panels, and discussions with the CAHPS Consortium were conducted prior to the 2022 field test/mode experiment that tested the revised survey and after the field test/mode experiment results were finalized. All supported the revised survey, including modifying the existing QDCCO multi-item measure by dropping 4 questions that were deemed not critical to ask, leaving the QDCCO measure with 13 instead of 17 questions.
Summary of literature findings associated with this measure:
1. Al Nuairi A, Bermamet H, Abdulla H, Simsekler MCE, Anwar S, Lentine KL. Identifying Patient Satisfaction Determinants in Hemodialysis Settings: A Systematic Review. Risk Manag Healthc Policy. 2022 Sep 30;15:1843-1857. doi: 10.2147/RMHP.S372094. PMID: 36203651; PMCID: PMC9531609.
- Quality Improvement: Understanding these determinants can help healthcare providers and administrators develop targeted interventions to enhance patient satisfaction (training programs to improve staff communication skills or facility upgrades to enhance comfort can be implemented).
- Personalized Care: Recognizing the impact of patient demographics and health status on satisfaction can lead to more personalized care approaches, addressing specific needs and preferences.
- Policy Development: The findings can inform policy decisions aimed at improving service delivery in HD centers, ensuring that patient satisfaction is a central component of quality care metrics.
- By enhancing relational skills, nurses can foster trust and emotional support, leading to improved patient satisfaction, adherence to treatment plans, and overall health outcomes.
- Empathy is not just a moral attribute but a crucial skill that fosters trust and effective nurse–patient relationships.
- Recognizing individual patient needs and adjusting communication strategies accordingly can enhance patient satisfaction and treatment adherence.
- Especially for long-term patients, ongoing efforts to strengthen the therapeutic bond can lead to better health outcomes and patient well-being.
- The study underscores that establishing a therapeutic relationship is fundamental in hemodialysis. Such relationships are pivotal in achieving goals, enhancing patient well-being, and improving the overall quality of care.
2. Mancin, S.; Palomares, S.M.; Sguanci, M.; Palmisano, A.; Gazineo, D.; Parozzi, M.; Ricco, M.; Savini, S.; Ferrara, G.; Anastasi, G.; et al. Relational skills of nephrology and dialysis nurses in clinical care settings: A scoping review and stakeholder consultation. Nurse Educ. Pract. 2024, 82, 104229.
3. Hreńczuk M. Therapeutic relationship nurse–patient in hemodialysis therapy. Nurs Forum. 2021;56:579‐586. 10.1111/nuf.12590
Measure Impact
A successful survey should be relevant to the target audience and produce meaningful results that can be used to inform decisions. For the ICH CAHPS Survey, this includes a survey that accurately measures characteristics of quality dialysis care from the patient’s perspective and produces results that ICH centers can use to improve their care.
Each survey period, between 80 and 100k sample patients respond to the current ICH CAHPS Survey, indicating that the survey is meaningful and that they feel that providing their information is valuable. The survey focuses on topics that ESRD patients have reported as being important in defining high-quality dialysis care; these patients are the only source of this information. During survey revisions, we met with 2 focus groups of 9 people who all noted that they felt the ICH CAHPS Survey is important and were happy to know that it was being reduced in length. CMS also receives a number of letters from dialysis patients noting that they believe the survey is valuable and that they appreciate being able to convey their thoughts and have their voices heard.
During the 2024 Pre-Rulemaking Measure Review (PRMR) process, the Patients for Patient Safety US organization commented the following: We certainly support this, because the experiences of patients is a valuable data source, and we're happy to see it included in instruments like CAHPS as well as other ways in which we use PROMs and PREMs. I will say that our organization Patients for Patient Safety US does see the CAHPS surveys as an incredibly important tool, not only for feeding back patient experience, but improving health equity, by bringing in people who are less likely to use these surveys by asking the right kinds of questions and including meaningful questions in them and I think this falls into that category.
Activities conducted by RTI during the survey revision process, such as focus groups, interviews, and TEP meetings, helped to ensure that the survey is relevant to the target audience, and that survey results are meaningful to both patients and the ICH facilities that provide care.
During initial TEP discussions, TEP members were presented with questions that could possibly be removed from the QDCCO measure, based on psychometric analyses. Based on those discussions, focus groups and cognitive interviews were conducted with patients, asking about the removal of each question narrowed down by the TEP. It was determined that 3 questions could safely be removed from the QDCCO measure. When presented to the CAHPS Consortium, they agreed with the removal of the 3 identified questions, and also asked for a 4th question to be removed. Following additional analyses, all 4 identified questions – by TEP, patients, and the CAHPS Consortium – were removed from the measure.
Performance Gap
We have provided the distribution of scores in Table 1 as requested. The information in the table is based on top-box calculated scores. The data used to calculate the deciles comes from the 2023 Fall and 2023 Spring National Implementation data. Due to the sampling design for the mode experiment, to have the least impact on public reporting, facility-level scores could not be calculated so national implementation was used as proxy. The facility-level performance scores for each survey period are created using top-box, and then combined to create 2023 performance scores. In total, 6700 facilities and 178,480 survey responses are represented in this data.
| Overall | Minimum | Decile_1 | Decile_2 | Decile_3 | Decile_4 | Decile_5 | Decile_6 | Decile_7 | Decile_8 | Decile_9 | Decile_10 | Maximum | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Mean Performance Score | 65.64 | 5.48 | 51.813 | 56.785 | 60.198 | 63.049 | 65.704 | 68.154 | 71.075 | 74.702 | 79.959 | 100 | 100 |
| N of Entities | 6,700 | 1 | 670 | 670 | 670 | 670 | 670 | 670 | 670 | 670 | 670 | 670 | 3 |
| N of Persons / Encounters / Episodes | 178,480 | 1 | 16,168 | 18,923 | 20,241 | 20,726 | 20,491 | 19,472 | 18,590 | 16,703 | 15,811 | 11,355 | 3 |
Equity
Equity
This domain is optional for Spring 2025.
Feasibility
Feasibility
As these patient-experience data are collected from patients based on their ICH dialysis care experiences, the structured data are not available in electronic sources outside of this data collection. Web-based data collection was tested in the 2022 ICH CAHPS Survey field test/mode experiment, but due to a lack of email addresses available in the sampling data, we had very few responses. We are working now to determine whether we can obtain additional email addresses to retest web data collection in a future mode experiment.
Proposed revisions to the current ICH CAHPS Survey instrument include shortening it and was developed through focus groups, cognitive interviews, and other instrument-development activities conducted for this revised instrument. The changes to the current ICH CAHPS survey were implemented to mitigate the challenges and barriers to responding to the current full survey. The shortened survey reduces patient burden and is expected to increase response rates.
The ICH CAHPS Survey is administered by independent survey vendors that are approved by CMS, and CMS’s implementation contractor provides oversight on a regular basis to ensure that the vendors are following established protocols. Additionally, the survey vendors are required to conduct regular review and monitoring of their own operational systems, whether the survey is administered by mail or telephone. Data is assessed semiannually for accuracy and missing data.
ICH CAHPS Survey results for the updated survey instrument will be fully publicly reported in October 2027 (2026 Spring + 2026 Fall data). However, because the April 2027 refresh would include a survey period that used the current survey (2025 Fall) and a survey period that used the revised survey (2026 Spring), we would plan to reanalyze the 2025 Fall data based on the revised survey measures and case-mix, then combine the reanalyzed data with the 2026 Spring data for public reporting in April 2027; therefore, we are not missing a refresh for ICH CAHPS data.
The following revisions were made to shorten the ICH CAHPS Survey:
The Nephrologist Communication and Caring (NCC) Measure was removed, which included the following questions:
- In the last 3 months, how often did your kidney doctors listen carefully to you?
- In the last 3 months, how often did your kidney doctors explain things in a way that was easy for you to understand?
- In the last 3 months, how often did your kidney doctors show respect for what you had to say?
- In the last 3 months, how often did your kidney doctors spend enough time with you?
- In the last 3 months, how often did you feel your kidney doctors really cared about you as a person?
- Using any number from 0 to 10, where 0 is the worst kidney doctors possible and 10 is the best kidney doctors possible, what number would you use to rate the kidney doctors you have now?
- Do your kidney doctors seem informed and up-to-date about the health care you receive from other doctors?
The following questions were removed from the QDCCO Measure:
- In the last 3 months, did dialysis center staff keep information about you and your health as private as possible from other patients?
- In the last 3 months, how often did dialysis center staff insert your needles with as little pain as possible?
- In the last 3 months, did dialysis center staff talk to you about what you should eat and drink?
- In the last 3 months, how often did you feel your kidney doctors really cared about you as a person?
The following questions were also removed from the survey; these items were not included in a measure:
- In the last 3 months, has anyone on the dialysis center staff asked you about how your kidney disease affects other parts of your life?
- Medicare and your State have special agencies that check the quality of care at this dialysis center. In the last 12 months, did you make a complaint to any of these agencies?
- Are you being treated for high blood pressure?
- Are you being treated for diabetes or high blood sugar?
- Are you being treated for heart disease or heart problems?
- Are you deaf or do you have serious difficulty hearing?
- Are you blind or do you have serious difficulty seeing, even when wearing glasses?
- Because of a physical, mental, or emotional condition, do you have serious difficulty concentrating, remembering, or making decisions?
- Do you have serious difficulty walking or climbing stairs?
- Do you have difficulty dressing or bathing?
- Because of a physical, mental, or emotional condition, do you have difficulty doing errands alone, such as visiting a doctor’s office or shopping?
- Who helped you complete this survey?
ICH facilities that served 30 or more survey-eligible patients in the preceding year are required by the ESRD PPS Rule to participate in the current calendar year’s two surveys. They are required to contract with one of the CMS-approved ICH CAHPS Survey vendors, who administers the survey on their behalf. Neither CMS nor RTI are involved in payment discussions between the vendors and facilities, and we have no cost information in terms of the facilities. The proposed revisions will not cause a significant burden to vendors or CMS’s implementation contractor, other than the modification of existing computer programs. Regarding burden to sample dialysis patients, the revised survey will take approximately 12 minutes of their time up to two times each year.
The sample is drawn from EQRS data, meaning that facilities are not involved and do not know who was selected to participate in the survey (unless a sample patient specifically tells them). Sample patients are asked to not ask for help from facility staff, and facilities are told not to assist their patients in completing the survey if asked and not to ask any patient whether they were sampled for the survey.
ICH CAHPS Survey data submitted to the ICH CAHPS Data Center (CMS’s implementation contractor) are de-identified. Case-level data are assigned a unique case ID, randomly generated so that there is no identifying information embedded in it. ICH CAHPS Survey results are aggregated at the facility level and publicly reported at the facility-, state- and national-level.
Vendors are allowed to provide unofficial aggregated results to their facility clients; however, there are rules on sharing such information when the number of responses is too low.
After the 2022 field test/mode experiment, and after the results were discussed with stakeholders (Technical Expert Panel members representing different ESRD organizations and large dialysis organizations, as well as the CAHPS Consortium), CMS approved the deletion of multiple items from the current nationally fielded ICH CAHPS Survey. The revised instrument is being submitted via this Instrument submission. For the 4 instrument-derived measures (2 ratings and 2 multi-item measures), 3 are the same as the current ICH CAHPS Survey and one is slightly modified.
Proprietary Information
Scientific Acceptability
Testing Data
The revised ICH CAHPS Survey was tested in fall 2022 during a field test/mode experiment. The total sample for the field test/mode experiment, based on EQRS data, was 24,354. Data were collected in English and Spanish, by mail, telephone, mail with telephone follow-up, and web with mail follow-up. Analyses are based on the field test/mode experiment data collected between October 2022 and January 2023, including 4605 respondents representing 3211 facilities. Field test/mode experiment data were used for the internal consistency analyses (Reliability – Section 5.2), factor analysis (Validity – Section 5.3), correlations among ratings and multi-item measures (Validity – Section 5.3), and risk analysis (IDM – Risk Adjustment – Section 5.4) analyses. For the signal-to-noise (IDM – Reliability – Section 5.2) and performance gap (IDM – Performance Gap – Section 2.4) analyses, national implementation data was used, which was collected during the 2023 Spring and 2023 Fall Surveys.
The national ICH CAHPS Survey is offered in six languages: English, Spanish, Chinese (simplified and traditional), Samoan, and Vietnamese. Additional translations will be made as needed.
There are no fees or licensing for use of the ICH CAHPS® Survey, training or oversight activities, or for accessing publicly reported ICH CAHPS® Survey measure scores or star ratings on the CMS Medicare.gov website.
None
For the field test/mode experiment that was conducted from October 2022 through January 2023, we took a sample of 24,354 ICH patients, using a stratified sampling method so as to have the least impact on public reporting. There were 6,736 facilities represented in the field test/mode experiment sample, located throughout the country. Facilities ranged in size from very small to very large. To have the least impact in public reporting, facilities that always meet the public reporting threshold (always have 30+ completed surveys for the two reporting periods) and facilities never meet the public reporting threshold (do not come close to the 30+ completes) were oversampled, and facilities that are normally right on the threshold line were undersampled. We randomly assigned sampled patients to one of the four data collection modes using the inverse of the estimated response rates to achieve the desired sample size in each mode. The final sample closely mirrored the national population of dialysis centers. Analyses were conducted using data from the 4605 respondents, who represented 3211 facilities.
For the 2023 Spring/Fall national implementation data used for facility-level analyses, all facilities registered on the ICH CAHPS website, by each dialysis facility that has determined they are required to participate in the surveys, were included in the sample if they had sample in the EQRS data. For the 2023 Spring/Fall Survey national implementation data, the following sample strategy was used:
Facilities with up to 240 patients: A census of all survey-eligible patients will be conducted for facilities with fewer than 240 survey-eligible patients at each semiannual sampling wave. Thus, patients at these ICH facilities may be sampled twice in a 12-month period. Facilities with 240 or more patients.
For dialysis centers with 240 or more survey-eligible ICH patients: A simple random sample will be selected for each sampling period, with the goal of obtaining 200 completed surveys per year while attempting to minimize the overlap of patients between subsequent semiannual waves of sampling.
The 2023 Spring Survey included 6675 facilities across the US and its territories, and the 2023 Fall Survey included 6674 facilities. Facility size differed from small to large; facilities were required to participate if they served 30+ survey-eligible patients in the prior calendar year.
For the field test/mode experiment, our sampling design randomly assigned sample patients to one of the four data collection modes, with a target of completing approximately 1,570 interviews for each mode. This design allowed us to measure mode effects related to nonresponse and measurement differences (e.g., because of social desirability) and also allowed for case-mix analyses. 24,354 ICH patients were sampled for the field test/mode experiment, using a stratified sampling method so as to have the least impact on public reporting. To have the least impact in public reporting, facilities that always meet the public reporting threshold (always have 30+ completed surveys for the two reporting periods) and facilities never meet the public reporting threshold (do not come close to the 30+ completes) were oversampled, and facilities that are normally right on the threshold line were undersampled. We randomly assigned sampled patients to one of the four data collection modes using the inverse of the estimated response rates to achieve the desired sample size in each mode. The final sample closely mirrored the national population of dialysis centers. Analyses were conducted using data from the 4605 respondents, who represented 3211 facilities.
For the 2023 Spring/Fall Survey national implementation data, the following sample strategy was used:
Facilities with up to 240 patients: A census of all survey-eligible patients will be conducted for facilities with fewer than 240 survey-eligible patients at each semiannual sampling wave. Thus, patients at these ICH facilities may be sampled twice in a 12-month period. Facilities with 240 or more patients.
For dialysis centers with 240 or more survey-eligible ICH patients: A simple random sample will be selected for each sampling period, with the goal of obtaining 200 completed surveys per year while attempting to minimize the overlap of patients between subsequent semiannual waves of sampling.
For the 2023 Spring, 378,513 patients were sampled. 91,914 respondents, representing 6609 facilities across the nation and its territories, were included in the analyses. For the 2023 Fall, 383,116 patients were sampled, 86,566 respondents representing 6595 facilities, were included in the analyses.
Eligibility criteria for the field test/mode experiment sample mirrored the criteria used for national implementation. Sample patients must:
- Be at least 18 years of age or older,
- Have received hemodialysis at an in-center facility for 3 months or longer,
- Be alive as of the last day of the sampling window, and
- Not be institutionalized or receiving hospice.
Attachment 5.1.4 with patient characteristics can be found in Supplemental 7.1 zip file.
Reliability
Psychometric testing at the instrument level includes a reliability assessment using the Cronbach’s alpha estimate, from the internal consistency analysis of measurement error. The reliability analysis was conducted at the item and patient level, using SAS’s PROC CORR procedure with ALPHA notation specified, which employs listwise deletion of missing data. Available response data from the revised survey field test/mode experiment was included as input for the internal consistency analysis.
Output from the PROC CORR was evaluated against psychometric thresholds of acceptable internal consistency. In general, Cronbach’s alpha estimates of above 0.9 were evaluated as good internal consistency and 0.7 was considered a minimum threshold for acceptability in this analysis (Nunnally & Bernstein, 1994).
Citation:
Nunnally J, Bernstein L. Psychometric theory. New York: McGraw-Hill Higher, INC; 1994.
The attachment provides the internal consistency results for the Quality of Dialysis Center Care and Operations (QDCCO) and Providing Information to Patients (PIP) item sets. The item-total correlations are provided in the second column from the right, and the alpha estimate for the set (if a particular item is removed) is in the far right. In summary, the 13 QDCCO items have a standardized Cronbach’s alpha estimate of 0.930 which exceeds the threshold for good internal consistency. In addition, the standardized item-total correlations range from 0.54 to 0.78 and none of the alpha estimates improve substantially with any item removed.
The 9 PIP items have a standardized Cronbach’s alpha estimate of 0.743 which is just above the minimal acceptability threshold. The standardized item-total correlation range for the PIP item set has a minimum of 0.29 and a maximum of 0.55, and again, none of the alpha estimates improve substantially with any item removed.
The internal consistency estimates for the QDCCO item set (α = 0.930), and PIP item set (α = 0.743) are above the minimal acceptability threshold (Nunnally & Bernstein, 1994). In addition,
most of the item-total correlations across multi-item measures are above 0.5, the remaining primarily above 0.3, with one value at 0.29. Therefore, the items assigned to each multi-items measure (QDCCO and PIP) function well together and we conclude that the items perform consistent measurement of the representative constructs.
The signal-to-noise analysis, termed inter-unit in this documentation, was conducted at the facility and derived measure level using the CAHPS 5.0 macro. For the overall estimate, 2023 Spring and Fall survey data was subset to facilities that had at least 30 completed cases between the two survey periods, to match the Public Reporting standards. 2023 Spring and Fall National Implementation survey data was used as a proxy due to the design of the 2022 field test/mode experiment, which was sampled in a way to ensure that the impact on Public Reporting data would be minimal and therefore not suitable for this analysis.
The CAHPS 5.0 macro calculates the global F-test to determine differences among entities on measures or items. F-statistics are transformed into reliability such that if the F-statistic is around 1, the reliability will be close to zero and indicate differences are due to random variation. The larger the real differences among facilities, the larger the F-statistic, and the higher the reliability estimate (approaching 1.0) and reliabilities above 0.7 are considered acceptable.
CITATION: Keller S, O’Malley AJ, Hays RD, Matthew RA, Zaslavsky AM, Hepner KA, Cleary PD. Methods Used to Streamline the CAHPS® Hospital Survey. Health Services Research, 2005, 40, 2057-2077. PMIDp: 16316438.
The QDCCO multi-item measure’s overall inter-unit reliability, using the 2023 Spring and Fall National Implementation survey data, is 0.76 (F=4.17, p<0.001). The information in Table 2 provides overall information as well as reliability by decile. Deciles for this table are based on psychometrics done within the CAHPS Macro; and the “mean performance score” row in this table is showing variation in the mean value by sample size. Deciles were specified on facilities that have 30 or more completes to match the Public Reporting standards. The extreme specifications included facilities with the smallest possible (1 or 2 completes) and close to the maximum number (100 completes or more) were specified as such to provide enough facilities to conduct the analysis.
| | Overall | Minimum | Decile_1 | Decile_2 | Decile_3 | Decile_4 | Decile_5 | Decile_6 | Decile_7 | Decile_8 | Decile_9 | Decile_10 | Maximum |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Reliability | 0.76 | 0.98 | 0.73 | 0.76 | 0.73 | 0.75 | 0.74 | 0.75 | 0.76 | 0.80 | 0.82 | 0.72 | 0.91 |
| Mean Performance Score | 3.19 | 3.07 | 3.19 | 3.20 | 3.66 | 3.18 | 3.18 | 3.17 | 3.18 | 3.21 | 3.16 | 3.17 | 3.18 |
| N of Entities | 2319 | 74 | 441 | 243 | 328 | 239 | 215 | 225 | 206 | 153 | 162 | 107 | 22 |
| N of Persons / Encounters / Episodes | 100828 | 115 | 13652 | 8118 | 11766 | 9312 | 9009 | 10236 | 10301 | 8468 | 10385 | 9581 | 2564 |
The inter-unit reliability assessment for the ICH CAHPS QDCCO multi-item measure indicates that overall, and at each decile of interest, the measure maintains acceptable performance (e.g., greater than 0.7) in finding real variation among facilities relative to random variation.
Validity
Changes to the items within the QDCCO multi-item measure were assessed with face validity. Next, structural and convergent validity were used to determine whether the QDCCO and PIP item sets measure their respective construct as intended.
Confirmatory factor analysis (CFA) was used to determine structural validity of the QDCCO and PIP item sets through model fit statistics compared to acceptability thresholds. The thresholds for model acceptability are as follows:
RMSEA of 0.05 or less (Browne & Cudeck, 1993)
CFI of 0.90 at a minimum (Hu & Bentler, 1999)
TLI of 0.90 at a minimum historically, 0.95 indicates good fit
Convergent validity was also assessed through CFA by evaluating factor loadings for each item. CFA analyses were conducted using the Mplus analytic software (Version 8.11; 2023) using the Weighted Least Squares Mean and Variance adjusted (WLSMV) estimator to account for categorical data with pairwise deletion for missing data. In addition, correlations, conducted using SAS PROC CORR, evaluating the two multi-item measures from the item sets (QDCCO and PIP) with ratings of the Dialysis Center and Dialysis Staff, were provided for convergent validity.
Citations:
Browne, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp. 136–162). Thousand Oaks, CA: Sage.
Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55. http://dx.doi.org/10.1080/10705519909540118
Muthén, L. K., & Muthén, B. O. (1998-2023). Mplus (Version 8.11) [Computer software]. Los Angeles, CA: Muthén & Muthén
Glossary:
RMSEA - root mean square error of approximation
CFI - comparative fit index
TLI - Tucker-Lewis Index
All experts confirmed face validity and agreed that the remaining items still provided construct representation of the quality of a facility’s care and operations. In addition, psychometric results showed that the removal of 4 questions from the QDCCO did not negatively impact the measure’s validity.
Confirmatory factor analysis (CFA) model fit to assess structural validity produced acceptable values, consistent with a well-fitting model. These estimates are:
RMSEA=0.038 (95% CI = 0.036 0.040)
CFI=0.987
TLI=0.985
The attachment provides the estimates for each standardized factor loading on the representative items to assess convergent validity. Correlative information on the factors is also provided, and moderate. All factor loadings for the QDCCO and PIP item sets are above 0.4, with the majority above 0.5, and all are statistically significant (p < 0.001).
The final assessment of convergent validity through correlation information results in the relationship between each averaged measure, and the Dialysis Staff Rating (rQDCCO = 0. 0.757, rPIP = 0.434) and the Dialysis Center Rating (rQDCCO = 0.737, rPIP = 0.403), all of which are statistically significant (p<0.001).
The QDCCO and PIP item sets continue to effectively measure consistent constructs. Confirmatory factor analysis (CFA) produced appropriate model fit indices and statistically significant factor loadings. Correlational analyses showed moderate relationships to patient ratings of the overall facility and the staff. The findings confirm the valid measurement of the QDCCO and PIP constructs.
Validity was assessed at the measure and facility level using correlational methods and SAS’s PROC CORR procedure, which employs pairwise deletion of missing data. Available measure scores from each facility were derived from the National Implementation survey data, specified down to reflect the revised QDCCO item set (13 questions). The 2023 Spring and Fall National Implementation data was used as a proxy due to the design of the 2022 field test/mode experiment, which was sampled in a way to ensure minimal Public Reporting data impact. Output from the PROC CORR analysis was evaluated against standard correlational thresholds. In general, a correlation of 0.50 shows a moderate relationship and estimates higher are indicative of stronger relationships.
Discriminant validity was assessed at the facility level by examining correlations among the QDCCO items and the derived measure for both the QDCCO and PIP item sets. The expectation is that an item’s correlation with the QDCCO measure (without the item of interest) will be higher than the correlation with PIP measure.
Convergent validity was also assessed through correlations at the multi-item measure and facility level and examined the relationship between the QDCCO measure and rating of the Dialysis Center, and then again with the rating of Dialysis Center Staff.
CITATION
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillside, NJ: Lawrence Erlbaum Associates
Discriminant validity results, using the 2023 Spring and Fall National Implementation survey data, are presented in the attached table. As expected, the QDCCO items show stronger relationships with the QDCCO multi-item measure than the PIP multi-item measure. In addition, convergent validity results show that the relationship between the QDCCO measure, and the Dialysis Center Staff Rating (rQDCCO = 0.824) and the Dialysis Center Rating (rQDCCO = 0.789) are stronger than the established moderate threshold (0.50) and statistically significant (p<0.001).
The QDCCO multi-item measure consistently represents a patient’s perspective on the care they receive at their dialysis facility. Correlational analyses showed smaller relationships to alternative measures not intended to measure the quality of the facility and strong relationships to patient ratings of the overall center and the center staff. The findings confirm that the ICH CAHPS items result in the valid measurement, specifically of the revised QDCCO multi-item measure, without replicating other information within the survey.
Risk Adjustment
All case-mix/risk adjustment analyses were conducted using the data from the 2022 field test/mode experiment. The independent variables that were included in the mode and case-mix models are those hypothesized to affect response tendencies, and the results of the variables included in the two multi-item measures and two ratings. These independent variables are used as case-mix adjusters for ICH CAHPS scores for ICH facilities. These independent variables are not under the control of the facility, meaning a facility can’t change these descriptors of their patients (e.g., a facility can’t control the demographic make-up of their population with regards to age, sex, overall health, overall mental health, etc.). Adjusting for this helps to ensure that variations in case-mix do not affect the CAHPS scores calculated and publicly reported for ICH facilities because these factors impact response tendencies and are outside the control of the ICH facilities.
These independent variables came from four sources: (1) the ICH CAHPS Survey mode experiment implementation; (2) the ICH CAHPS Survey mode experiment as patients’ self-reported characteristics; (3) the Common Medicare Environment (CME) database; and (4) the EQRS database.
Using the case-mix model, the facility’s case-mix adjusted scores are adjusted for differences between a facility’s patient composition according to the ICH CAHPS case-mix characteristics and the overall national composition of ICH patients on these same characteristics. This adjustment allows consumers to compare different ICH facilities based on the same overall patient composition, thus allowing for a comparison that isn’t affected by the given facility’s patient composition.
We have attached a file showing the descriptive statistics, submitted under 7.1 other attachments.
We conducted the multivariate regression analysis assessing the candidate case-mix variables. We estimated 50 multivariate regression models for the mode and case-mix analysis, including one regression model for each question comprising the two multi-item measures and two ratings for the top-box outcomes. When estimating the multivariate regression models, we used the individual patient as the unit of analysis. For the top-box outcomes the dependent variables for all the multivariate regression models were converted to a binary score with the most positive response set to 1 and all other nonmissing responses set to 0.
We used ordinary least squares models to investigate the candidate case-mix variables which included:
• Mode of survey administration.
• Language survey was completed. Binary yes/no response categories for whether the respondent completed the survey in English or Spanish.
Patients’ Self-Reported Characteristics Independent Variables
• Overall health. Including five categories for self-reported overall physical health status.
• Overall mental health. Including five categories for self-reported mental or emotional health status.
• Education. Including six response categories for self-reported education.
• Did someone help you complete this survey. Including binary yes/no response categories for whether the respondent had someone else help them to complete the ICH CAHPS Survey.
• Low-income subsidy or dually enrolled. This variable is calculated from variables indicating if the respondent had a low-income subsidy or was dually enrolled in both Medicare and Medicaid at any time during the survey period.
• Age. This variable was calculated from the respondent’s date of birth. We created five categories for a categorical age variable.
• Sex. Binary male/female categories.
• Total years on dialysis. This variable includes six response categories for different time periods for the respondent since they originally started dialysis treatment. This variable is calculated from the EQRS variable for the date dialysis started for this patient.
• Was diabetes the primary cause of ESRD. This variable includes binary yes/no response categories for whether diabetes was the cause of ESRD for this patient. This variable is calculated from the EQRS variable for the cause of ESRD.
The facility indicator variable was not included in the models for this analysis. The number of respondents from each of the facilities ranged from only one to eight with the majority having only one respondent, thus the cluster effects were not important. Generally, the linear form of the multivariate regression models was:
Dependent variable = sum of (coefficients*mode indicators) + sum of (coefficients*patient characteristic indicators)
When there were categorical patient characteristics for an independent variable, such as age groups, one group was used as the reference category from the set of categories included in the regression model. That group is the reference to which the effects of the other categories for that variable are compared, which simplifies the interpretation of the regression coefficients.
RTI randomly assigned patients to mode.
Sequential Modeling
We estimated each of the regression models using the independent case-mix variables previously noted. All of the independent variables were statistically significant for at least one of the regression models.
The first set of multivariate regression models run on the 25 dependent variables included as independent variables all of the case-mix factors in their original form. We refer to this first set of models as the full model or Model 1. In the second set of multivariate regression models (Model 2), we recategorized the mental health question by rolling together the fair and poor categories to reduce the VIFs for this variable (a finding from the multicollinearity analysis). In the third and final set of regression models (Model 3), we dropped the low-income subsidy or dually enrolled variable due to the difficulty of obtaining this variable and the high level of missing values (13.18%) for this variable. Model 3 was our final model.
To determine the recommended regression model, we calculated the patient-level case-mix adjusted scores from the three models investigated. Then we calculated the two scores by averaging the patient-level case-mix adjusted scores for each survey item comprising a given multi-item measure. We used the results of the patient-level case-mix adjusted and global rating scores from the three models and the unadjusted scores to conduct an impact analysis. The goal of the impact analysis was to understand how the results from the three case-mix regression models changed compared to the unadjusted scores and to each other.
Using the three sets of adjusted patient-level scores and the unadjusted scores, we calculated the absolute value of the difference between each score considering all possible combinations (e.g., unadjusted vs. Model 1, unadjusted vs. Model 2, unadjusted vs. Model 3). The mean of these percentage point differences is found in Table 1of the attachment in 5.4.5a.
For the top-box scores, the absolute values of the percentage point differences range from 0.95 for QDCCO case-mix adjusted top-box score using Model 2 compared to the case-mix adjusted top-box score using Model 1 to 10.43 for the global rating of the dialysis center unadjusted top-box score compared to patient-mix adjusted top-box score using Model 2. In general, the percentage point differences between the case-mix adjusted scores for the three models is small with the differences between Model 2 and Model 3 being the smallest.
Using the three sets of adjusted patient-level scores and the unadjusted scores, we calculated the Spearman’s rank correlation coefficient. Spearman’s rank correlation quantifies the strength and direction of association between two ranked variables (i.e., how similar they are). A high correlation (close to 1) indicates that the two variables are strongly related and a low correlation (close to 0) indicates that the two variables have a weak or no relationship. Negative values (close to −1) indicate that the two variables are strongly inversely related.
In this analysis, correlations close to 1 indicate that the different sets of patient-mix models are adjusting the scores similarly. The Spearman’s Rank Correlations are found in Table 2of the attachment in 5.4.5a for the two global ratings and two multi-item measure outcome scores. These correlations demonstrate the strength of the relationship of the global ratings and multi-item measures between the adjusted patient-level scores using the three patient-mix models and the unadjusted scores. For each measure the lowest correlations existed when comparing to the unadjusted scores. For example, for the top-box global rating of the dialysis center, the correlation for the unadjusted scores and Model 1 was 0.81. The same correlation was found for the unadjusted and Model 2, and for unadjusted and Model 3.
For the two ratings and two multi-item measures the pairwise Spearman’s Rank Correlations for the adjusted scores using Model 2 compared to Model 3 were either 0.98 or 0.99 and the correlations using Model 1 to Model 2 ranged from 0.97 to 0.99. Given that all the correlations between the adjusted scores were so close to 1, we concluded that there are very few differences in the patient-level adjusted scores when using the different sets of patient-mix variables in the multiple regression models.
Final Case-Mix Regression Model
The three sets of multiple regression models tested performed very similarly for top-box. The Spearman’s Rank Correlations were all close to 1 demonstrating that the ranks of the adjusted scores were very similar. This suggests very small differences in the results of the adjusted scores for the different sets of case-mix models. Furthermore, the percentage point differences between the three models were less than 2 for all measures, hence we recommend the simplest model without the low-income subsidy or dually enrolled variable. This is Model 3 with the following case-mix variables:
overall health,
overall mental health (with fair and poor categories rolled together),
patient age,
patient sex,
patient education,
language survey was conducted (English or Spanish),
help completing the survey,
total years on dialysis, and
was diabetes primary cause of ESRD
Use & Usability
Use
Dialysis facility
Usability
Dialysis center scores will increase as they target these measures, therefore improving patients’ perspective of their dialysis care. Each dialysis center may have different training mechanisms or internal procedures which can help improve the overall quality of the services they deliver.
In an attempt to increase survey response rates (performance on the survey overall) to have enough data to be publicly reported, ICH facilities are encouraged to hang posters/flyers and waiting room FAQs in their facilities, so that patients understand that the ICH CAHPS survey is legitimate.
ICH facility scores will increase as facilities target this QDCCO measure, which asks for patients' perception of whether their dialysis center staff communicated well, kept patients as comfortable and pain-free as possible, behaved in a professional manner, and kept the center clean. Each ICH center may have different training mechanisms or internal procedures which can help improve the overall quality of the services they deliver.
The revised instrument has not been implemented beyond the field test stage; however, we did receive positive feedback during the Pre-Rulemaking Measure Review (PRMR) process. The PRMR hospital committee recommended the revised survey. The PRMR committee recognized the importance of patient experience of care data and supported efforts to reduce the length of the survey while maintaining scientific acceptability. The CAHPS Consortium noted that we should remove one additional question (which was done after more analyses) and then move forward with the revised survey; they felt that it would help survey burden and was a great start at reducing a very long survey. In addition, feedback was obtained from several large dialysis organizations (LDOs) during TEP meetings and email correspondence. They noted that this was a good start at reducing the survey but would like to see it reduced even more in the future. The consensus from the LDOs was that the revised survey didn’t remove any major questions that would raise alarm.
The proposed QDCCO multi-item measure (modified from the current QDCCO measure used in national implementation) will be one of four ICH CAHPS publicly reported.
Feedback from ESRD stakeholders and the CAHPS Consortium, as well as patients during focus groups and cognitive interviews, was positive for the changes made to the QDCCO measure (the removal of 4 questions). Refer to 2.6 (derived) for more information about the feedback from the TEP and CAHPS Consortium.
Although this revised survey has not been implemented yet, final decisions were based on feedback from many stakeholders, including representatives from our LDOs. During Technical Expert Panel (TEP) meetings and via email discussions, feedback was provided on which questions could safely be dropped from the current survey, to create a revised survey to reduce patient burden. The TEP supported efforts to shorten the survey to reduce burden on respondents. They noted that there were no major concerns with the questions removed and appreciated the effort to make this a little easier on the patients responding to the survey.
Although this revised survey has not been implemented yet, final decisions were based on feedback from many stakeholders, including representatives from our LDOs. During TEP meetings and via email discussions, feedback was provided on which questions could safely be dropped from the QDCCO measure, to create a revised survey to reduce patient burden.
The revised survey has not been implemented yet so we have no improvement data based on this revised measure as of yet.
The scores for the current QDCCO multi-item measure (not the revised QDCCO), for facilities with at least 20 responses from 2015 thru 2023, have increased over time. In 2015 the median score for the QDCCO measure was 61.3 and in 2023 the median score was 64.4. Similarly, the 20th decile in 2015 was 54.7 and in 2023 was 57.4. The 80th percentile in 2015 was 68.6 and in 2023 it was 71.4.
The revised survey has not been implemented beyond the field test/mode experiment phase. No unexpected findings were observed during the field test/mode experiment. Respondents appreciated the shorter survey.
There have been no unexpected findings of the QDCCO multi-item measure based on current data, which is based on the current ICH CAHPS Survey (not the revised survey). We have not implemented the revised survey yet but do not anticipate any unexpected findings after removing 4 questions form this measure.
Comments
Staff Preliminary Assessment
CBE #0258-4 Staff Preliminary Assessment
Importance
Strengths
- A logic model is provided, in which the composite scale ‘Quality of Dialysis Center Care and Operations (QDCCO)’ measure is positioned as an outcome, with inputs that include ‘staffing and resources’, and impacts such as ‘selecting and engaging patient in dialysis treatment best for them’ and ‘Patient Driven Dialysis Center.’ The developer cited a review article, reporting on the value of interventions, such as staff training, to enhance patient experience, and the importance of addressing different patient needs/preferences and building empathy and relational skills in nursing staff for improving overall quality of care.
- National data from the 2023 Fall and 2023 Spring surveys for the currently implemented measure show a performance gap, with decile ranges from 51.8% in decile 1 to 100% in decile 10 (mean 65.6%), indicating variation in measure performance across the target population.
- The revised measure removes four items.
- The developer’s description of patient input sought for revision of the instrument overall supports the general conclusion that this measured patient-reported experience is meaningful with at least moderate certainty. Patients were asked to provide input on the items they felt could be removed from the QDCCO measure, and three of the four items removed were drawn from their recommendations.
Limitations
- The logic model provided is a starting point for demonstrating how the measure's implementation will lead to the anticipated outcomes, and inputs and activities are limited to those required to report the measure. The logic model could be improved by specifying common clinical quality activities hemodialysis centers could use to improve on this measure focus. Several of the factors identified in the review article the developer summarized could be suitable as inputs or activities in the logic model, but were not included there.
- The submission does not directly address the scope of the population the measure covers, e.g., the estimated number of U.S. adults receiving in-center hemodialysis care, and could be strengthened by providing a few details about this population.
- The developer’s evidence review is focused primarily on their efforts to shorten the survey by identifying the content that is most meaningful to patients. The literature review provided that explores relationships between the topic addressed by this measure and material outcomes is minimal. The developer cited a second review article and one empirical study focused on the patient-nurse relationship in dialysis care, but did not discuss them.
- The submission could be strengthened by expanding the literature review and discussing the quality, quantity, and consistency of existing studies, or explicitly stating that this area is a gap in the available literature and explaining how they determined this.
Rationale
- This maintenance measure is rated as 'Not Met But Addressable' due to sparse review of evidence and a vague logic model. Enhancements, including a more complete discussion of available literature and more complete logic model, could elevate its importance.
Closing Care Gaps
The developer did not address this optional domain.
Feasibility Assessment
Strengths
- Required data elements are collected via patient survey. The developer tested web survey collection in the 2022 field test and cited hemodialysis centers’ lack of patients’ email addresses as barriers to online collection.
- The developer reported finding no feasibility issues resulting from the revised survey. The revisions to the survey are clearly described.
- As the survey is collected directly from patients, there is no burden on health care providers. The developer noted that the survey has been shortened, reducing burden for patients. The QDCCO measure specifically has also been shortened by four items.
- The developer explained that hemodialysis centers are not involved in sampling patients and that data submitted to the ICH CAHPS data center are deidentified. Vendors may share aggregate data with clients, but rules prohibit sharing when responses are “too low.”
- There are no fees, licensing, or other requirements to use any aspect of the measure (e.g., value/code set, risk model, programming code, algorithm).
Limitations
- The developer did not provide typical or average costs associated with contracting with a survey vendor.
Rationale
- This maintenance measure meets all criteria for 'Met' due to its well-documented feasibility assessment, clear and implementable data collection strategy, and transparent handling of patient confidentiality, burden, licensing, and fees. These factors collectively ensure that the measure can be implemented effectively and sustainably in a real-world healthcare setting.
Scientific Acceptability
Strengths
- The developer performed the required reliability testing for this maintenance measure, namely, they conducted accountable entity-level (“measure score”) reliability testing at the level for which the measure is specified. They also provided evidence of person/encounter-level (“data element”) reliability testing for all critical data elements. Data sources used for reliability analysis are adequately described. For the accountable entity-level testing, national implementation data collected during the 2023 Spring and Fall Surveys was used. The entities included in the analysis were characterized by accountable entities with at least 30 completes. There were 100,828 encounters across 2,319 qualifying entities. The data for the data element reliability analyses were from the field test/mode experiment data collected 11/22-1/23 and consisted of 4,605 encounters across 3,211 facilities.
- The developer conducted test-retest reliability testing at the accountable entity-level, however the decile table isn't quite correct. Decile sizes range from 107 to 441 entities. An overall IUR of 0.76 is reported and a rough estimation based on the performance gap table suggests that more than 70% of accountable entities are likely meet the expected threshold of 0.6 at the levels for which the measure is specified, but only for entities with at least 30 completes (about 35% of the entities in the importance table).
- The developer also conducted internal consistency analysis of the 13 Quality of Dialysis Center Care and Operations (QDCCO) items. The internal consistency estimates for the QDCCO item set (α = 0.930) is above the expected threshold of 0.7.
Limitations
- The reliability decile table isn't quite correct, each decile should include approximately the same number of entities. Decile sizes range from 107 to 441 entities. The performance gap tables have not been limited to a minimum of 30 completes and include nearly three times as many entities than the reliability tables but with only about 75% more of the total number of persons.
Rationale
- The developer performed the required reliability testing for this maintenance measure and the results suggest sufficient reliability at data element and accountable entity levels.
Strengths
- The developer provides an Importance Table and Logic Model, providing a plausible causal association between the entity response to the measure and the measure focus. Empirical support for ruling out confounders includes adequate reliability (although the method IUR is not recommended), adequate risk-adjustment, and a correlation with a related experience measures with construct overlap (Staff Communication and Professionalism Training, Facility Environment Enhancements, Therapeutic Nurse–Patient Relationships, Patient-Centered Operational Adjustments, Relational Skills Development). Specifically, the argument is that the individual items are more highly correlated with QDCCO (because of the construct overlap) than with PIP (where there is less construct overlap). Similar, the results suggest that the construct overlap (the common causal factors) across the ICH measures is significant, perhaps 80% or more (r=0.824 for staff rating; r=0.789 for center rating).
- Developer conducted statistical case-mix adjustment, based on a conceptual model, selecting risk factors that have a significant correlation with the outcome. The developer also explored social factors, including dual eligibility/low-income subsidy. The developer did not include these factors in the final models due to the minimal impact these social case-mix indicators have on the measure scores and to the model overall.
Limitations
- The main limitation is that the nature of the construct overlap among the related experience measures, and the nature of the responsible mechanisms that make up that construct, are not explicitly stated, apart from the logic model, or supported by empirical studies (apart from the reliability and risk adjustment). Residual risk for confounders not ruled out includes: Patient demographics and health status, Variation in survey response rates, and Patient expectations and interpretations. Residual risk for a responsible mechanism includes the potential counter-acting mechanisms (none are explicitly stated).
- The developer does not provide model testing results that indicate the impact on provider scores at the high or low extremes of case-mix is due to the presence of case-mix factors.
Rationale
- The measure developer provides some support for the causal claim that the entity response to the measure is causally related to the measure focus. The developer provides empirical support for ruling out confounders (always with some residual risk of unstated or unexamined confounders) and for ruling in responsible mechanisms (always with some residual risk that the explicit mechanisms are only partially responsible for the measure focus).
- Opportunities to strengthen the inference of validity, that do not impact the domain rating, include: a more explicit consideration of potential confounders and ruling out such confounders as causal explanations of the variation among entities, groups, and over time. Conditions might also reflect a more explicit consideration of responsible mechanisms and ruling in such mechanisms as causal explanations of the variation among entities, groups, and over time.
- The case-mix adjustment methods are appropriate and demonstrate that variation in the prevalence of case-mix indicators across facilities contribute to unique variation in the outcome.
- The impact of patient-level factors on provider scores at high or low extremes of case-mix is unclear in testing data.
Use and Usability
Strengths
- This measure is currently in use in the End Stage Renal Disease (ESRD) Quality Incentive Program.
- The developer referenced input from the TEP regarding recommended revisions to the survey and described positive feedback on the revised instrument offered by stakeholder groups such as the CAHPS Consortium and the Pre-Rulemaking Measure Review (PRMR) Hospital Committee.
- The developer also referenced support received from these groups and from patients for the revisions made to the QDCCO measure.
- The developer reported an overall increase in median performance score from 2015 to 2023 (61.3% to 64.4%). Performance scores for intervening years were not reported.
- The developer reported no unexpected or adverse findings related to the currently implemented measure, and indicated they do not expect any for the revised measure once the new survey is implemented.
Limitations
- The developer argued that accountable entities can use the measure results to improve performance; however, no material actions that entities can take to improve performance are suggested.
- Feedback from measured entities on the currently implemented measure is not described.
Rationale
- The rating for this maintenance measure is ‘Not Met.’ The measure is actively used in at least one accountability application and has shown improvement since 2015. However, the developer does not address actions that accountable entities can use to improve performance.
Public Comments
ASN Comments on ICH-CAHPS Related Measures Spring 2025 E&M
Please see attached for the American Society of Nephrology's comments on ICH-CAHPS related measures.
Dear American Society of…
Dear American Society of Nephrologists,
We thank you for your letter and the opportunity to respond to your comments about the ICH CAHPS Survey.
Low Response Rates: CMS shares your concerns about response rates and notes that this is not unique to the ICH CAHPS Survey. CMS’s contractor for the ICH CAHPS Survey continues to work with our approved third-party survey vendors and large dialysis organizations to determine ways to increase patient engagement and improve response rates. We have provided several tools for participating dialysis facilities to use in these efforts, such as flyers/posters and Waiting Room FAQs to post in their facilities. In addition, we have created an FAQ page for patients on our website and reference this in the letters sent to all sampled patients. In addition, we continue to encourage facilities to update patient contact information in EQRS, the source for sampling, as poor and outdated contact information continues to lend to lower response rates.
The feedback we have consistently received from participating facilities and patients is that shortening the survey is key to reducing survey burden of this population and our hope is that this will help improve response rates over time; the ICH CAHPS Survey is currently the longest CAHPS survey being used. The revised survey removes 23 questions. CMS has no plans to decrease the frequency of the survey at this time. After many analyses, it was determined that an annual survey would lessen the number of facilities that meet the criteria for their data to be publicly reported and that the survey data would be outdated by the time it was actually reported. Our goal is that by reducing the length of the survey, the burden on patients is decreased.
We recognize that differences exist between responders and non-responders to the survey as Nonresponse Bias Analyses have been conducted on all ICH Mode experiments and the results have suggested that case-mix analysis is sufficient to adjust for the differences in response rates. Most recently for the 2022 ICH Mode Experiment we tested the following variables that were known for both responders and nonresponder: age, sex, was diabetes the primary cause, years on dialysis, and if dually eligible for both Medicare or Medicaid or receiving a low-income subsidy. We calculated nonresponse weights using a response propensity logistic model. The response propensity logistic model found significant differences in the patient characteristics age and if dually eligible for both Medicare or Medicaid or receiving a low-income subsidy. Next we performed case-mix analysis on the data and obtained the residuals of the regression models that used the case-mix variables. We calculated the Pearson correlation coefficients between the nonresponse-adjusted weights and the residuals from the regression models using the case-mix adjusters for all survey items used to create the composites. The correlation coefficients were nonsignificant all survey items (p-value > 0.10) except one (p-value = 0.038). The lack of overall significant correlation lead us to conclude that, when using our case-mix adjustment model, nonresponse-adjusted weights are not needed to further adjust the case-mix adjusted scores.
We continue to monitor the number of facilities that meet the 30+ completed survey threshold for publicly reporting; at this time, CMS does not have plans to lower this threshold. A major concern about lowering the threshold is the amount of variability associated with the publicly reported scores; fewer completes per CCN will result in scores with higher variability (i.e., less precise scores). We will continue to re-evaluate this over time.
Need for home dialysis survey: CMS and its ICH CAHPS contractor recently met with the Alliance for Home Dialysis and have begun work to determine whether the ICH CAHPS Survey can be modified to also include the home dialysis population. We have just completed several rounds of cognitive interviews and focus groups with home dialysis patients, to learn the topics most important from their perspective and to determine the feasibility of a combined ICH/home survey. The results of these efforts will be discussed with a Technical Expert Panel in the coming months, and a decision will be made whether to test a combined survey via a field test/mode experiment in the future, should CMS funding be available.
Again, we thank you for the opportunity to respond to your concerns about the ICH CAHPS Survey.
The CMS and RTI ICH CAHPS Teams