When this measure comes back for maintenance (3 years), the developer should: Provide entity-level empirical testing of reliability and validity using child survey data to support the extrapolation argument.
The Consumer Assessment of Healthcare Providers and Systems (CAHPS) Clinician & Group Survey 3.1 (CG-CAHPS) is a standardized survey instrument that asks patients to report on their experiences with primary or specialty care received from providers and their staff in ambulatory care settings over the preceding 6 months. CG-CAHPS Survey Version 1.0 was endorsed by NQF in July 2007 (NQF #0005) and version 2.0 received maintenance endorsement in early 2015. Version 3.0 was released in July 2015 and was last endorsed in 2019. The 3.1 version of the survey updates the 3.0 version to prompt respondents to consider in-person, phone, and video visits when they answer the questions and to report which type(s) of visits they had. The survey is part of the CAHPS family of patient experience surveys and is available at https://www.ahrq.gov/cahps/surveys-guidance/cg/index.html
The Adult CG-CAHPS Survey 3.1 is administered to patients aged 18 and over who had at least one visit to a selected provider during the past 6 months. The survey has 32 questions including one overall rating of the provider and 12 questions used to create four (4) composite measures.
The Child CG-CAHPS Survey 3.1 is administered to the parents or guardians of pediatric patients under the age of 18. The survey has 40 questions including one overall rating of the provider and 11 questions used to create four (4) composite measures.
The composite measures are:
- Getting Timely Appointments, Care, and Information (Access)
- How Well Providers Communicate With Patients (Provider Communication)
- Helpful, Courteous, and Respectful Office Staff (Office Staff)
- Providers’ Use of Information to Coordinate Patient Care (Care Coordination)
The survey also has a single-item rating measure:
- Rating of Provider
A guidance document is available on the AHRQ CAHPS website (https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/surveys-guidance…) which explains how to administer the survey including how to choose the sample, maintain confidentiality, collect the data, track returned questionnaires, and calculate the response rate.
The Child CG-CAHPS Survey Provider’s Use of Information to Coordinate Patient Care (Care Coordination) composite measure assesses experiences with how often providers knew the child’s medical history and followed up to give results of tests.
Measure Specs
General Information
The CAHPS Clinician & Group (CG-CAHPS) Survey assesses aspects of health care delivery that are important to patients and for which patients are the best or only source of information (Cleary, 2016). Further, the CG-CAHPS survey focuses on patient-centered care (Cleary, Edgman-Levitan, 1997; Cleary, 2016), one of the six central aims identified by the Institute of Medicine for improving the health care system (IOM, 2001). A focus on the patient experience has the potential to enhance clinical outcomes, improve patient safety, and reduce unnecessary medical services. Moreover, assessing patient experience through surveys that include data on the demographic characteristics of respondents, such as race and ethnicity, can help identify the extent to which positive experiences are distributed equitably across patients (Haviland et al., 2003).
Use of this measure will benefit both patients, providers, and medical groups:
- Patients can use information from the measures to help make more informed choices about which practice or medical group to use.
- Medical groups and their providers can use data from the surveys for quality improvement initiatives and incentives.
- Researchers can use data files from the surveys to help answer important health services research questions.
Patient experience encompasses the range of interactions that patients have with the healthcare system. The terms patient satisfaction and patient experience are often used interchangeably, but they are not the same. CAHPS surveys ask patients to report on what they experienced in a healthcare encounter—for example, whether something happened or how often it happened. Patient experience of care surveys provide actionable, objective information for quality improvement. Patient satisfaction surveys, on the other hand, use ratings to measure whether a patient’s expectations about a health encounter were met.
The CG-CAHPS Survey is a standardized survey instrument for measuring patients’ perspectives on their care. The survey is generally administered annually to patients who have received care in the last 6 months.
References
Cleary, PD, Edgman-Levitan, S. (1997). Health care quality. Incorporating consumer perspectives. JAMA. 278(19), 1608-12.
Cleary, PD. (2016). Evolving concepts of patient-centered care and the assessment of patient care experiences; optimism and opposition. J Health Pol, Policy & Law, 41 (4), 675-696.
Haviland, M. et al, (2003). Do health care ratings differ by race or ethnicity? Joint Commission Journal on Quality and Safety. 29(3), 134-145.
Institute of Medicine. (2001). Crossing the Quality Chasm: A New Health System for the 21st Century. Accessible at https://nap.nationalacademies.org/catalog/10027/crossing-the-quality-ch….
The Child CG-CAHPS Survey Care Coordination composite measure captures how often the provider knew important information about the patient’s medical history and how often someone from the provider’s office followed up with the parent/guardian of the child to give test or imaging results. Having coordinated care helps ensure that patients receive the right care at the right time, helps to avoid duplication of tests, and helps practice sites align practices with reimbursement strategies tied to quality and efficiency (e.g., Accountable Care Organizations). Improving performance on this measure facilitates seamless communication among providers, timely follow-up on test results, and comprehensive medication management, which reduces errors, prevents unnecessary costs, and leads to better health outcomes and higher satisfaction.
The data were obtained from the Centers for Medicare & Medicaid Services (CMS) Center for Medicare and Medicaid Innovation (CMMI) Primary Care First (PCF) Patient Experience of Care Survey (PECS), which is based on the Adult CG-CAHPS Survey and adds Patient-Centered Medical Home Supplemental items (https://pcfpecs.org/Survey-and-Protocols) The survey is administered annually via mail with phone follow-up. The 2023 survey data included 2,490 practice sites and 238,204 respondents. For testing and performance scores, practice sites with less than 10 surveys were excluded. After the restriction, the number of sites is 2,486.
Numerator
AHRQ calculates this composite measure using a top box scoring method. The numerator for each question in the composite measure is the number of respondents who answered “always.” The final numerator for the composite measure is based on the number of unique respondents who answered “always” to the composite measure questions where the composite measure is an average of the question level proportions.
This measure includes the following questions:
- Q20: In the last 6 months, how often did this provider seem to know the important information about your child’s medical history?
- Q24: In the last 6 months, when this provider ordered a blood test, x-ray, or other test for your child, how often did someone from this provider’s office follow up to give you those results?
The response options for each item in this composite measure are:
- Never
- Sometimes
- Usually
- Always
No additional detail, refer to 1.14.
Denominator
For each question in the composite measure, the denominator is the number of respondents who answered the question. The final denominator for the composite measure is based on the number of unique survey respondents who completed at least one question within the composite measure. The composite measure is the based on the average of the question level proportions.
The target population for the survey is patients who have had at least one visit to the selected provider/practice in the target 6-month time frame. This time frame is also known as the look back period. The sampling frame is a person-level list and not a visit-level list.
The question “In the last 6 months, when this provider ordered a blood test, x-ray, or other test for your child, how often did someone from this provider’s office follow up to give you those results?” was only asked of respondents who indicated that the provider ordered a blood test, x-ray, or other test for their child (i.e., responded yes to the question “In the last 6 months, did this provider order a blood test, x-ray, or other test for your child?”).
Exclusions
Individuals are excluded from the denominator if:
- They are deceased.
- They did not receive care from the participating medical group or practice in the last 6 months.
The denominator is the total number of surveys fielded minus the total number of ineligible surveys. The total number of ineligible surveys includes sample cases deemed ineligible: does not meet the eligible population criteria (refer to Section 1.15b). No other cases are excluded from the denominator, but cases are excluded from the denominator of the measure if they did not answer any item within the measure.
Measure Calculation
Respondents report on their experiences accessing and using care over the past 6 months.
AHRQ calculates CG-CAHPS Survey this composite measure scores using a top box scoring method.
Composite Measures:
There are two basic steps to calculating a composite measure score for a practice site:
- Calculate the proportion of responses in the top box or most positive response category for each question in a composite measure.
- Calculate the mean or average top box scores across all questions in a composite measure to determine the composite measure’s top box score.
For the top box or “top proportion” score, the numerator is the number of respondents who answered that they “Always” received the desired care or service for a given measure. For example, if 400 out of 1,000 total respondents answered “Always” to a composite measure item, the top box score for that item would be 40 percent [i.e., (400 ÷ 1,000)*100 = 40%].
Lower proportion and middle proportion composite measure scores can also be calculated following the same methodology where the lower proportion is the proportion answering “Never” or “Sometimes” and the middle proportion is the proportion answering “Usually”.
Rating Item:
For the rating item, the numerator for the top box score is the number of respondents who responded 9 or 10 on the 0-10 scale (where 10 is the “Best” and 0 is the “Worst”). For example, if 600 out of 1,000 total respondents answered “9” or “10” to a rating item, the top box score for that item would be 60 percent [i.e., (600 ÷ 1,000)*100 = 60%].
Lower proportion and middle proportion rating scores can also be calculated where the lower proportion is the proportion answering 0-6 on the 0-10 scale and the middle proportion is the proportion answering 7 or 8.
Users may also choose to calculate mean scores or linearized mean scores.
Note the survey includes screener items to identify respondents who meet the target process for each measure, such as whether the individual needed care right away. Measures are only calculated using respondents who experienced a particular service/process.
Users can also case-mix adjust the results for characteristics such as respondent age, education, general health status, and mental health status. The CAHPS Analysis Program—often referred to as the CAHPS Macro—is a free program written in SAS (version 6.0 or later) that enables survey users to case-mix adjust their data. The program also generates a distribution of survey results for each of the measures, calculates the mean score for both individual survey items and composite measures, and indicates whether an entity’s scores are statistically different from the average. The results presented in these analyses are based on unadjusted top box scores unless otherwise noted.
More information about the calculation of proportion scores and mean scores can be found in these documents:
- Instructions for Preparing Data for Analysis: https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/surveys-guidance…
- Instructions for Analyzing Data from CAHPS Survey: https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/surveys-guidance…
The measure is not stratified.
Users should choose a data collection protocol that maximizes the survey response rate at an acceptable cost. Some sponsors, as well as researchers conducting field tests, have found that the mail with telephone follow-up method is most effective or email with mail or telephone follow-up.
AHRQ provides protocols for collecting responses though users can adapt it to meet their needs. The protocols include mail only, telephone only, mail with phone follow-up, or email (web) with mail or phone follow-up. AHRQ provides detailed instructions for these different protocols in the “Guidelines for Using the CAHPS Clinician & Group Survey” document survey available on the AHRQ CAHPS website: https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/surveys-guidance…;
There is no minimum response rate requirement on the CG-CAHPS Survey. The CAHPS consortium has found that higher response rates are achievable if users take steps to ensure the accuracy of the sample frame and carefully follow the recommended data collection protocol, including one or more attempts to follow up with non-respondents.
In its simplest form, the response rate is the total number of completed questionnaires divided by the total number of individuals selected for the sample. Calculating the response rate is helpful in determining a more accurate starting sample size for future survey administration. For the CG-CAHPS Survey, the goal is a response rate of at least 40 percent or at least 50 completed surveys per provider.
To calculate the response rate, use the following formula: Number of completed returned questionnaires divided by the total number of respondents selected minus the sum of deceased + ineligibles.
AHRQ makes the CG-CAHPS Survey available in English and Spanish.
The sample is drawn from a list of individuals (adults aged 18 and older, or children 17 and younger) who have received care from a given provider, practice site, or medical group during a six-month time period. The list is called a sample frame.
The source of sample information will vary by survey sponsor. The data to identify individual patients may be found in the records of medical practices or health systems.
Defining the Sample Frame: Eligibility Guidelines:
- The adult questionnaires include all adults 18 years or older.
- The child questionnaire includes all children 17 years or younger.
- Include only patients who have had at least one visit to the selected practice in the last 6 months. This time frame is also known as the look back period.
- To identify the sampling frame, use the anticipated start date of data collection to determine the reference period. For example, if your anticipated start date is September 1, 2026, include all those who have had at least one visit since March 1, 2026.
- The sampling frame is a person-level list and not a visit-level list. Therefore, patients should appear only once in the sampling frame regardless of how many visits they have had in the look back period.
- Draw the sample irrespective of reason for visit and duration of patient-provider relationship, so that the full range of patients is represented.
- Include all patients who meet the sampling criteria even if they are no longer receiving care from the practice, site/clinic or provider.
- Allow the sample frame to include multiple individuals from the same household, but the sample you draw should not have more than one person (adult or child) per household. In other words, the sample that is selected for data collection should be de-duplicated to ensure that only one person per household receives a survey. The final sample must contain only one respondent per household. Where a duplicate household is sampled, it is discarded and replaced by another random draw from the frame.
- All CAHPS survey items have been designed for the general population. Appropriate screening items are included for items targeted to assess a specific experience. In order to ensure that results are comparable to those produced by other sponsors and vendors, targeted sampling, such as selecting only patients with particular conditions or experiences, is not recommended. Targeted sampling should only be used to supplement the general population sample, if desired.
- In order to administer the survey, the name of the provider must be available, even if surveying at the site/clinic or practice level. If the sampling frame does not accurately identify the provider that the patient saw, select a larger sample to account for errors in connecting health care received to a specific provider. For example, errors can occur if administrative billing data are used for the sampling frame and visits with physician assistants or nurse practitioners are billed under the supervisory physician.
Calculating the Sample Size for the Adult (Child) Questionnaire
The sample size varies depending on whether sampling is done at the level of the individual provider, practice, or medical group. In general, to produce statistically valid comparisons, the sample needs to be large enough to yield 50 completed questionnaires per provider or 300 completed questionnaires per medical group. The recommended sample size when sampling at the practice level depends on the number of providers at each site. The document Guidelines for Using the CAHPS Clinician & Group Survey has a table for recommended sample sizes based on the number of providers, and it ranges from 50 to 300 completed questionnaires. (https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/surveys-guidance…)
Data are not reported for any item or measure with fewer than 10 valid responses and practice sites with fewer than 10 responses were not included. AHRQ recommends that there needs to be approximately 50 completed questionnaires per provider to have a sufficient number of responses for results to be statistically reliable.
Proxy Respondents
The CG- CAHPS Survey Plan does allow for proxy respondents for mail and web-based mode. At the end of the survey, there is an item that asks “Did someone help you complete this survey?” If the answer is Yes, the follow-up question is “How did that person help you?” and they are to mark one or more of these response items:
- Read the questions to me
- Wrote down the answers I gave
- Answered the questions for me
- Translated the questions into my language
- Helped in some other way
However, these the last two questions of the core questionnaire are not included in telephone scripts because telephone interviews should not be conducted with proxy respondents.
Supplemental Attachment
Point of Contact
CAHPS® is a registered trademark of the U.S. Department of Health and Human Services and managed by AHRQ
Karen Chaves
Rockville, MD
United States
Naomi Yount
Westat
Rockville, MD
United States
Importance
Evidence
The CG-CAHPS Survey measures key components of patient experience, such as provider communication and ease of access, that are consistent with patient-centered care. The CAHPS Surveys focus on aspects of care that consumers have identified as important and for which patients are the best or only source of information. Measuring patients’ perceptions of their healthcare experience is not just a means to improve services—it’s a recognition that the patient’s voice matters in and of itself. Listening to patients affirms their role as active participants in their care, and their insights are essential to truly understanding the quality and impact of healthcare delivery. In 2023, over 238,000 patients completed the CG-CAHPS Survey as part of the PCF PECS survey. It is possible that more practices are administering and using the CG-CAHPS survey beyond the practices participating in PCF. We reviewed the literature on the determinants of patient care experiences measured by CAHPS and their associations with other indicators of health care quality. CAHPS is also an actionable measure that helps clinicians and health plans target interventions that will improve the quality and patient-centeredness of care.
Review of the Evidence
Prior research has identified several features of healthcare delivery structure, including clinic accessibility, patient flow, and management, that are associated with patient experiences. Two major systematic reviews have examined the relationships among patient experience, clinical processes, and patient outcomes. A systematic review performed by researchers in the U.K. found that patient experience is favorably associated with adherence to recommended medications and treatments, preventive care such as screenings and immunizations, patient-reported health outcomes, clinical outcomes, reduced hospitalizations and primary care visits, and reduced adverse events (Doyle et al., 2013). Anhang Price et al. (2014) reviewed evidence on the association between patient experiences and other measures of health care quality in the U.S. They similarly found that better patient care experiences are associated with higher levels of adherence to recommended prevention and treatment processes, better clinical outcomes, and less health care utilization.
Health Care Process and Quality Improvement Influence on Patient Experience
Providers routinely use patient-reported measures such as CAHPS to guide quality improvement (QI) activities to improve their patients’ experience with care (Friedberg et al., 2011; Davies, Shaller, et al., 2013; Quigley et al., 2015).
For example, to improve scores for “how often the office staff were as helpful as you thought they should be”, Dean Clinic, a large integrated health care delivery system in Wisconsin, further surveyed their patients to ask them what “helpfulness” means to them and to ask how office staff could be more helpful. From this feedback, the Clinic learned about ways that the office staff can be more welcoming, friendly, and appreciative of patients. With input from both staff and management, Dean Clinic developed action plans to improve patient experience. The service department shadowed staff and provided feedback. To improve consistency in service across all sites, the Clinic developed an orientation for all new employees on customer service expectations. They also offered ongoing training in the form of service workshops, videos, and Webinars, as well as targeted interventions for the lowest scoring offices (AHRQ, 2013). Several quality improvement initiatives to improve provider communication and engagement include shadowing, coaching, and training (AHRQ, 2013; Hardee & Kasper, 2008; Quigley, Palimaru, et al., 2017). Lastly, Friedberg et al. (2011) found that physician groups that aim to improve access, communication with patients, and customer service do so by addressing office workflow, providing additional training for nonclinical staff, and adopting or enhancing an electronic health record.
Using data from the CAHPS survey and a newly installed electronic health record system, in 2015, leaders of Northeast Valley Health Corporation (Los Angeles County) pinpointed interventions including reallocating staff resources and not scheduling well-child visits first thing in the morning. System leaders found that when the first appointments of the day ran long, there was a cascading effect on the rest of the day's schedule. By 2015, NEVHC instituted changes systemwide. By 2017, total average cycle time was reduced from 82 minutes to 65 minutes; the proportion of patients with a cycle time under 60 minutes rose from 34 percent to 48 percent; and the proportion of patients seen within 15 minutes of appointment time rose from 38 percent to 47 percent. Case study found at https://www.ahrq.gov/news/newsroom/case-studies/201718.html
More examples of interventions to improve patient experience with primary and specialty care as measured by the CG-CAHPS Survey can be found in the CAHPS Improvement Guide available at https://www.ahrq.gov/cahps/quality-improvement/index.html. The Guide also includes information on analyzing survey results and identifying root causes of performance problems.
The Centers for Medicare and Medicaid Services Merit-based Incentive Payment System (MIPS) Quality Payment Program measures Medicare Part B providers in four performance categories to derive a score that could affect a provider's Medicare reimbursement positively or negatively starting at 4% in 2019 (based on 2017 performance). MIPS has a performance category called “Improvement Activities” that includes an inventory of activities that assess how a physician group can improve care processes, enhance patient engagement in care, and increase access to care. Several of these activities are geared to improve patient experience of care as measured by CG-CAHPS. A list of MIPS activities can be found at https://qpp.cms.gov/mips/improvement-activities.
Structure
Physician clinic hours of operation and availability for appointments have been found to predict patient experience in several studies. In a study with survey data from 61,839 patients of 1729 primary care physicians in California, system-level factors, such as belonging to a larger medical group and the physician’s zip code-based Primary Care Services Areas, explained between 28% to 48% of variation in patient care experience, with the highest proportion explained for the access to care composite (Rodriguez et al., 2009). Improving the infrastructure supporting certain aspects of care may have broad effects because system changes can influence multiple outcomes (Cleary, 2016).
“Expanded practice access” is a highly weighted CMS MIPS improvement activity. To improve the patient experience in access to care, practices may consider providing 24/7 access to clinicians, groups, or care teams for advice about urgent and emergent care (e.g., eligible clinician and care team access to medical record, cross-coverage with access to medical record, or protocol-driven nurse line with access to medical record). Access expansion can include one of the following: 1) expanded hours in evenings and weekends with access to the patient medical record (e.g., coordinate with small practices to provide alternate hour office visits and urgent care); 2) use of alternatives to increase access to care team by clinicians and groups, such as e-visits, phone visits, group visits, home visits and alternate locations (e.g., senior centers and assisted living centers); and/or 3) provision of same-day or next-day access to a consistent clinician, group or care team when needed for urgent care or transition management. Another example is implementation of “open access” scheduling, in which some physician time is always reserved for same-day appointments, to improve patient access to care (Murray & Tantau, 1998).
Health-related Patient Behavior and Disease Management
One dimension of the CG-CAHPS measure captures the patients’ perceptions of how well providers communicate with them. Better patient-provider communication promotes healthcare-related patient behaviors (Fuertes, Boylan, et al., 2009). A 2009 meta-analysis of 127 studies assessing the link between patient treatment adherence and physician-patient communication found a 19% higher risk of non-adherence among patients whose physician communicated poorly (Zolnierek and Dimatteo, 2009). Doyle’s (2013) meta-analysis showed positive associations between the quality of clinician-patient communications and adherence to medical treatment in 125 of 127 studies analyzed. Studies using the CAHPS measure have found that better provider communication is positively associated with adherence to hypoglycemic medications among diabetics (Ratanawongsa, Karter, et al., 2013), adherence to tamoxifen among breast cancer patients (Liu, Malin, et al., 2013), and higher rates of colorectal cancer screening among adults in the US (Carcaise-Edinboro and Bradley, 2008). Sequist and colleagues (2008) found that measures of patient experience, including doctor-patient communication, clinical team interactions, and health promotion support, were positively associated with some prevention and disease management clinical process measures in clinical practices and among individual clinicians.
Outcomes
Out of 40 evidence papers with outcome measures, Doyle’s (2013) meta- analysis found 29 studies with positive associations between patient experience and clinical outcomes, 11 with no associations, and none with negative associations. The lack of more evidence may be due to complexity between a patient’s illness level, their level of care, and their likelihood for a poor outcome such as mortality, morbidity or a readmission. Often, such associations have more than one plausible direction of causality. For example, clinicians may be especially attentive to the needs of sicker patients (Kahn et al., 2007) and patients near the end of life (Elliott, Haviland, et al., 2013; Xu et al., 2014).
Moreover, substantial evidence points to a positive association between various components of patient experience, such as good communication between clinicians and patients, and several important processes and outcomes. These include lower utilization of unnecessary healthcare services; better patient adherence to medical advice; better process of care measures for acute myocardial infarction (AMI), congestive heart failure, pneumonia and surgery; lower inpatient mortality among acute myocardial infarction (AMI) patients; lower infection rates (Anhang Price, et al., 2014); and better clinician and staff perceptions of patient safety culture (Sorra et al., 2012).
Utilization
Research suggests an association between better patient experiences and lower healthcare utilization. Children with asthma were less likely to visit the emergency department, make urgent office visits, or be hospitalized if their physicians had reviewed a long-term therapeutic plan with their parents (Clark, Cabana, et al., 2008). Among African Americans with Type 2 diabetes, those who reported that doctors or nurses usually listened carefully or spent enough time with them were significantly less likely to visit the emergency department in the 12 months following completion of a patient experience survey (Gary, Maiese, et al., 2005). Children whose parents report longer waits for primary care visits were more likely to visit the emergency department for non-urgent reasons than those who report waiting for less time (Brousseau, Bergholte, et al., 2004).
References
Agency for Healthcare Research and Quality (AHRQ). How Two Provider Groups Are Using the CAHPS® Clinician & Group Survey for Quality Improvement. Brief available at https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/quality-improvem….
Anhang Price, R, Elliott, MN, Zaslavsky, AM, Hays, RD, Lehrman, WG, Rybowski, L, Edgman-Levitan, S, Cleary, PD. (2014). Examining the role of patient experience surveys in measuring health care quality. Med Care Res Rev. 71(5), 522-54.
Brousseau, DC., Bergholte, J., et al. (2004). The effect of prior interactions with a primary care provider on nonurgent pediatric emergency department use. Archives of Pediatrics & Adolescent Medicine. 158(1), 78-82.
Carcaise-Edinboro, P. and Bradley, CJ. (2008). Influence of patient-provider communication on colorectal cancer screening. Medical Care. 46(7), 738-745.
Clark, NM., Cabana, MD., et al. (2008). The clinician-patient partnership paradigm: Outcomes associated with physician communication behavior. Clinical Pediatrics. 47(1), 49-57.
Cleary, PD. (2016). Evolving concepts of patient-centered care and the assessment of patient care experiences; optimism and opposition. J Health Pol, Policy & Law. 41(4), 675-696.
Davies, E, Shaller, D, Edgman-Levitan, S, Safran, DG, Oftedahl, G, Sakowski, J, Cleary, PD. (2008). Evaluating the use of a modified CAHPS survey to support improvements in patient-centered care: lessons from a quality improvement collaborative. Health Expect. 11(2), 160-76.
Doyle, C., Lennox, L., et al. (2013). A systematic review of evidence on the links between patient experience and clinical safety and effectiveness. BMJ Open. 3(1). http://bmjopen.bmj.com/content/3/1/e001570.full
Elliott, MN., Haviland, AM., et al. (2013). Care experiences of managed care Medicare enrollees near the end of life. Journal of the American Geriatrics Society 61(3), 407-412.
Friedberg, MV, SteelFisher, GK, Karp, M, and Schneider, EC. (2011). Physician groups’ use of data from patient experience surveys. J Gen Intern Med. 26(5), 498-504.
Fuertes, J N., Boylan, LS., et al. (2009). Behavioral indices in medical care outcome: The working alliance, adherence, and related factors. Journal of General Internal Medicine. 24(1), 80-85.
Gary, TL., Maiese, EM., et al. (2005). Patient satisfaction, preventive services, and emergency room use among African-Americans with type 2 diabetes. Disease Management. 8(6), 361-371.
Hardee, JT, Kasper, IK. (2008). A clinical communication strategy to enhance effectiveness and CAHPS scores: The ALERT model. Perm J. Summer. 12(3), 70-4.
Kahn, KL., Tisnado, DM., et al. (2007). Does ambulatory process of care predict health-related quality of life outcomes? Health Services Research. 42, 63-83.
Liu, Y., Malin, JL., et al. (2013). Adherence to adjuvant hormone therapy in low-income women with breast cancer: The role of provider-patient communication. Breast Cancer Research and Treatment. 137(3), 829-836.
Murray, M and Tantau, C. (1998). Must patients wait? Jt Comm J Qual Improv. 24(8), 423-5.
Quigley, DD, Mendel, PJ, Predmore, ZS, Chen, AY, Hays, RD. (2015). Use of CAHPS™ patient experience survey data as part of a patient-centered medical home quality improvement initiative. J Healthc Leadersh. 7, 41-54.
Quigley, DD., Palimuru, AI., Chen AY., & Hays, RD. (2017). Implementation of practice transformation: Patient experience according to practice leaders. Quality Management in Health Care. 26 (3), 140-151.
Ratanawongsa, N., Karter, AJ., et al. (2013). Communication and medication refill adherence: the Diabetes Study of Northern California. JAMA Internal Medicine. 173(3), 210-218.
Rodriguez, HP, Scoggins, JF, von Glahn, T, Zaslavsky, AM, Safran, DG. (2009). Attributing sources of variation in patients' experiences of ambulatory care. Med Care. 47(8), 835-41.
Sequist, TD., Schneider, EC., et al. (2008). Quality monitoring of physicians: linking patients' experiences of care to clinical quality and outcomes. Journal of General Internal Medicine. 23(11), 1784-1790.
Sorra, J, Khanna, K, Dyer, N, Mardon, R, Famolaro, T. (2012). Exploring relationships between patient safety culture and patients’ assessments of hospital care. Journal of Patient Safety. 8(3), 131–139.
Xu, X., Buta, E., Anhang Price, R., Elliott. NN., Hays, RD., & Cleary, PD. (2014). Methodological considerations when studying the association between patient-reported care experiences and mortality. Health Services Research. 50(4), 1146-61.
Zolnierek, KB. and Dimatteo, MR. (2009). Physician communication and patient adherence to treatment: a meta-analysis. Medical Care. 47(8), 826-834.
It is important that the Child CG-CAHPS Survey reflects the aspects of care that parents/guardians associate with high-quality healthcare for their child. Through a literature review, focus groups, a Technical Expert Panel, and numerous other development activities, the CAHPS Consortium found that items regarding Care Coordination are important components of pediatric care for parents/guardians. This measure assesses how often providers know important information about medical history and follow-up on test, x-ray, or other results. AHRQ alone has funded 47 projects focused on care coordination that has demonstrated better care coordination improves diagnostic accuracy, medication safety, and communication during care transitions (https://www.ahrq.gov/sites/default/files/wysiwyg/patient-safety/highlig…). Kern et al. (2024) noted that patients often see multiple providers, and this can lead to errors, avoidable hospitalizations, and worse care if there is not a provider coordinating the care.
Although most research has focused on adult patients, coordinated care is a critical component of high-quality care for both adults and children. Key provider actions—such as knowing important information about the patient’s medical history and following up on test results—are essential across both populations. Coordinated care helps ensure patients receive the right care at the right time and prevents duplication of tests. The threshold for care coordination for children may be lower than for adults. The CAHPS Consortium noted that early CAHPS qualitative work with Medicaid mothers revealed a higher priority on care for their children than themselves. Despite these differences in sensitivity, the underlying response mechanism or how the questions in this measure are interpreted are also consistent between the Adult and Child survey respondent populations. This consistency supports the importance of care coordination regardless of patient age.
Reference
Kern, LM, Bynum, JPW, Pincus, HA. (2024). Care fragmentation, care continuity, and care coordination – How they differ and why it matters. JAMA Intern Med, 184(3),236-237.
Measure Impact
The CAHPS measures, developed to complement more technical quality measures, are measures for which the patients are the best or only source of information and/or perspective, such as the degree to which care felt patient-centered (Anhang Price et al., 2014). Several studies provide evidence that patients value the CAHPS measures and find them meaningful. For example, Safran et al. (2001) found that patients who reported the poorest-quality relationships with their physicians were three times more likely to voluntarily leave the physicians’ practice than patients with the highest-quality relationships.
Collins et al. (2017) found that a patient’s “most important CAHPS domain” varied across subgroups; racial and ethnic patient subgroups differentially valued various aspects of the care experience. To efficiently reduce disparities and improve quality, Collins et al recommend tailoring quality improvement programs to the factors most important to the racial, ethnic, and language mix of the patient population of interest. Quigley et al. (2014) found that the importance of provider communication varied significantly by practice specialty type, yet respectful treatment was consistently import across all specialties.
Patients also use information from patient experience measures to make decisions about their healthcare providers and plans. One study found that seeing publicly reported quality information was a determinant of choosing higher quality-rated health plans, although the weight given to quality information also depended on other features, such as cost and provider choice (Faber et al., 2009).
References
Anhang Price, R, Elliott, MN, Zaslavsky, AM, Hays, RD, Lehrman, WG, Rybowski, L, Edgman-Levitan, S, Cleary, PD. (2014) Examining the role of patient experience surveys in measuring health care quality. Med Care Res Rev 71(5), 522-54.
Collins, RL, Haas, A, Haviland, AM, Elliott, MN. (2017). What matters most to whom: Racial, ethnic, and language differences in the health care experiences most important to patients. Med Care. Nov;55(11), 940-947.
Faber, M, Bosch, M., Wollersheim, H, Leatherman, S, and Grol, R. (2009). Public reporting in health care: how do consumers use quality-of-care information? A systematic review. Med Care 47(1), 1-8.
Quigley, DD, Elliott, MN., Burkhart, Q, Farley, DO., Skootsky, SA., & Hays, RD. (2014). Specialties differ in which aspects of doctor communication predict overall physician ratings. Journal of General Internal Medicine, 29, 447-454.
Safran, DG, Montgomery, JE, Chang, H, Murphy, J, Rogers, WH. (2001). Switching doctors: predictors of voluntary disenrollment from a primary physician’s practice. J Fam Practice. 50(2), 130–6.
The Child CG-CAHPS questions focus on aspects of care for which the parent/guardian of the child is the best or only source of information and that reflect elements of care that are most meaningful to patients/parents/guardians. Published research indicates that individuals use information from patient experience measures to make decisions about their healthcare providers and plans. Patient experiences with care are also linked to their persistence with the provider. At the provider level, patients who reported the poorest-quality relationships with their physicians are three times more likely to voluntarily leave the physicians’ practice than patients with the highest-quality relationships (Safran et al., 2001).
The Care Coordination composite measure is a vital indicator of patient experience because it reflects how well their providers work together to manage and support the child’s care. Coordinated care means fewer repeat tests, and a smoother experience navigating the healthcare system. Care Coordination has been shown to be significant and uniquely associated with the overall rating of the provider (Hays et al., 2014). Chang et al. (2018) conducted interviews with adult patients about care coordination and found that patients considered it to be very important and were most dissatisfied when care was not coordinated. This measure is valuable to patients/parents/guardians in helping them make informed decisions about their child’s care and offering a meaningful way to express concerns or positive experiences with performance related to care coordination.
References
Chang, L., Wanner, KJ, Kovalsky, D, Smith, KL, Rhodes, KV. (2018). “It’s really overwhelming”: Patient perspectives on care coordination. J Am Board Fam Med, 31, 682-690.
Hays, RD., Berman, LJ., Kanter, MH., Hugh, M., Oglesby, RR., Kim, CY., Cui, M., & Brown, J. (2014). Evaluating the psychometric properties of the CAHPS patient-centered medical home survey. Clin Ther, 36(5), 689-696. doi: 10.1016/j.clinthera.2014.04.004.
Safran, DG, Montgomery, JE, Chang H, Murphy, J, Rogers, WH. (2001). Switching doctors: Predictors of voluntary disenrollment from a primary physician’s practice. J Fam Practice. 50(2), 130–6.
Performance Gap
Due to the lack of access to Child CG-CAHPS data, we are extrapolating data from Adult CG-CAHPS Survey. Both the Adult and Child surveys capture the experience of care from an adult perspective — either directly from the patient (Adult CG-CAHPS) or the parent/guardian (Child CG-CAHPS). Additionally, results from the 2019 measure endorsement, when both Child and Adult data were available, demonstrated that the outcomes related to performance gaps were consistent across the two populations. This strong alignment supports the appropriateness of extrapolating adult data in the absence of child data.
The analyses were based on data from CMMI’s Primary Care First (PCF) Patient Experience of Care (PEC) survey, which is a version of the AHRQ Adult CG-CAHPS 3.1 Survey. The 2023 survey was administered from October 3, 2023, through December 19, 2023, and includes 2,486 practice sites and 238,204 respondents.
To examine the performance gap over time, we also analyzed the 2022 survey data. The 2022 survey was administered from October 6, 2022, through December 19, 2022, and includes 2,801 practice sites and 241,521 respondents.
Deciles in the Performance Scores by Decile tables are based on the performance scores; and for the “mean performance score” row in the tables show the average unadjusted top box scores across practice sites. For cases where the performance score was tied across decile boundaries, all practice sites with that score were assigned to the same decile.
As shown in Table 1 and the attached Table 2.4a.4, for the Care Coordination composite measure:
- the 2023 mean top box score = 72.8; number of measured entities = 2,486; and number of respondents was 233,988.
- the 2022 mean top box score = 72.0; number of measured entities = 2,801; and number of respondents was 236,560.
The mean top box score increased slightly (72.0% to 72.8%), as did the maximum score, which was 95.6% in 2023 versus 93.8% in 2022. The scores are lower than ideal (100% is ideal), given that about three-quarters of respondents felt their provider was coordinating their care. These results show that while there are small increases from 2022 to 2023, indicating that the performance gap may be decreasing over time, practices need to continue to aim to improve their care coordination.
| Overall | Minimum | Decile_1 | Decile_2 | Decile_3 | Decile_4 | Decile_5 | Decile_6 | Decile_7 | Decile_8 | Decile_9 | Decile_10 | Maximum | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Mean Performance Score | 72.8 | 37.8 | 58.7 | 65.3 | 68.4 | 70.7 | 72.6 | 74.3 | 75.9 | 77.7 | 79.9 | 84.2 | 95.6 |
| N of Entities | 2,486 | 1 | 249 | 248 | 249 | 249 | 249 | 247 | 250 | 247 | 249 | 247 | 1 |
| N of Persons / Encounters / Episodes | 233,988 | 68 | 19,571 | 22,846 | 23,191 | 23,551 | 24,434 | 24,267 | 23,580 | 24,273 | 24,704 | 23,488 | 15 |
Care Gaps
Closing Care Gaps
Optional for Fall 2025.
Feasibility
Feasibility
The CG-CAHPS Survey is a standardized instrument designed to assess patient experience of care. As these patient-experience data are collected from patients, the structured data are not available in electronic sources outside of the data collection by the practice site or medical group.
The data are collected through a survey instrument that is administered directly to patients, not during care delivery. Surveys are generally mailed to the sampled patients, and those survey results can be entered into structured databases (e.g., Excel, SPSS, SAS). No proprietary platform is required to administer the survey. Though mixed-mode administration (i.e., mail and phone) is a viable strategy for the collection of CAHPS surveys, mail continues to be the most frequent mode for most CAHPS surveys. Users then create electronic databases of results after receipt of the completed hard copy survey through scanning or data entry. However, vendors may set up their database before data collection by populating the frame to assist in identifying nonresponse.
Traditionally, the rationale for not using electronic sources more broadly is that mail and telephone are the best ways to obtain representative samples of patients based on the contact information that is available for sampling and data collection. E-mail has been added as a mixed mode strategy for physician groups with reliable email addresses for their patient population.
Structured or unstructured fields. All items are structured and on a 4 -item Likert-type response option scale (1-4) or for the rating items on a 0-10 scale. All responses are numeric.
Electronic feasibility. CG-CAHPS Survey users can offer a web survey to respondents to complete the survey though that option should not be the only option as it may exclude patients who have limited or no access to the web and/or who do not have an email address to send an electronic version of the survey.
Missing data. Item level missing data is low on the CG-CAHPS Survey however, some items will have fewer response than others due to gate or filter questions. For example, if a respondent did not need urgent care in the last 6 months, they are skipped through items about their experiences with getting urgent care. As a result, some CG-CAHPS Survey items have higher percentages of missing data overall, but when skip patterns are considered, the percentages of inappropriate missing data are much lower (<5%).
Measure susceptibility to inaccuracies and ability to audit data: The CG-CAHPS Survey is self-reported perceptions or experiences with the care received and therefore cannot be assessed to determine if the results are accurate. The protocol for administering CG-CAHPS Survey is outlined by AHRQ and vendors normally collect the data in a standardized format. The data that has been collected can be audited to ensure no data entry errors have occurred, there are no out of range values, and skip patterns are followed.
Change to the Instrument. Since the instrument was last endorsed, there was only one change to the survey and that was to add text to the instrument to allow for respondents to include care that was done virtually (i.e., phone or by video) to account for changes in care delivery due to the COVID-19 pandemic. For example, instructions now included “by phone, or by video”: “The questions in this survey will refer to the provider named in Question 1 as “this provider.” As you answer these questions, please think of the in-person, phone, and video visits you had with that person in the last 6 months.” This change did not impact data structure or availability.
Data Collection Burden
For respondents: The survey takes approximately 15 minutes to complete, dependent on the individual.
For medical groups and practice sites: Survey sampling uses administrative enrollment data that is maintained by all medical groups/practice sites and easily accessible to produce a sampling frame. Practice sites/medical groups generally hire a survey vendor to administer, track, and analyze their survey data resulting in lower burden for the practice sites/medical groups.
Cost Considerations
The CG-CAHPS Survey is freely available for use with no proprietary fees.
The cost to hire a vendor varies based on the size of the medical group and desired number of completed surveys. AHRQ provides guidance for hiring a vendor and resources for finding a certified vendor (https://www.ahrq.gov/cahps/surveys-guidance/helpful-resources/hiring/in…).
Impact on Clinician Workflow
The CG-CAHPS Survey does not interfere with diagnostic thought processes or patient -physician interactions as the survey is retrospective after care has been given, not during the visit.
Potential Barriers and Mitigation Strategies
Achieving a desired response rate may be difficult for users. Phone is not optimal as the only mode of survey administration, but it is commonly used as a follow-up for CAHPS mail surveys. Phone follow-up can improve CAHPS response rates compared to mail-only (Burkhart et al., 2014; Fowler et al., 2002; Gallagher et al., 2005; Klein et al., 2011). A study of Medicare beneficiaries found that response rates continue to improve when up to 4 follow-up calls are made (Burkhart et al., 2014). In addition, phone follow-up calls help to achieve better representation of patients in terms of income, literacy/education, health status, age, gender, and race/ethnicity, above and beyond mail surveys alone (Tesler and Sorra, 2017). The CAHPS Consortium continues to conduct research to develop and test survey administration methods that can improve the efficiency of data collection, enhance response rates, and gather more information about the experiences of those segments of the patient population that are hard to reach through more traditional means. This research includes: 1) studies comparing the effect of administration modes on response rates, survey scores, and data collection costs (e.g., mode comparisons have included in-office distribution vs. mail; email vs. mail); 2) studies assessing the effect of survey length on response rates and survey scores; 3) studies examining the impact of incentives on response rates; and 4) studies comparing the effect of different survey formats and design on survey responses. AHRQ also provided a webinar on how to achieve higher response rates (https://www.ahrq.gov/cahps/news-and-events/events/webinar-011124.html).
Analysis and Reporting: AHRQ makes available many resources to assist with analysis and reporting. For instance, there is a free CAHPS Analysis Program which is written for SAS that enables survey users to conduct the analyses needed to produce valid comparisons of performance across similar health care organizations. Users can also review documentation on how to prepare data for analysis (https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/surveys-guidance…). Further, vendors usually conduct all analyses and reports.
References
Burkhart, Q, Haviland, A, Kallaur, P, et al. (2014). How much do additional mailings and telephone calls contribute to response rates in a survey of Medicare beneficiaries. Field Methods. 27(4):409-25.
Fowler, FJ, Gallagher, PM, Stringfellow, VL, et al. (2002). Using telephone interviews to reduce nonresponse bias to mail surveys of health plan members. Med Care 40(3):190-200.
Gallagher, PM, Fowler, FJ, Stringfellow, VL. (2005). The nature of nonresponse in a Medicaid survey: causes and consequences. J Off Stat 21(1):73-87.
Klein, DJ, Elliott, MN, Haviland, AM, et al. (2011). Understanding nonresponse to the 2007 Medicare CAHPS survey. Gerontologist. 51(6):843-55.
Tesler, R. and Sorra, J. CAHPS Survey Administration: What We Know and Potential Research Questions. (Prepared by Westat, Rockville, MD, under Contract No. HHSA 290201300003C). Rockville, MD: Agency for Healthcare Research and Quality: October 2017. AHRQ Publication No. 18-0002-EF. Accessible at https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/about-cahps/rese….
Most vendors have established methods for tracking the sample. The Consortium suggests setting up a system to track the returned surveys by the unique ID number that is assigned to each respondent in the sample. This ID number should be placed on every questionnaire that is mailed and/or on the call record of each telephone case.
To maintain respondent confidentiality, the tracking system should not contain any of the survey responses. The survey responses should be entered in a separate data file linked to the sample file by the unique ID number. (This system will generate the weekly progress reports that should be review closely.) Data should be stored securely—preferably on encrypted or password-protected systems—with access limited. If paper responses are used, they should be shredded following de-identified data entry.
The CG-CAHPS Survey data is therefore de-identified upon data collection with a focus on protecting the confidentiality of respondents. Vendors are trained on maintaining confidentiality.
The CG-CAHPS Survey has a long history of use dating back to 2007. The CG-CAHPS Survey has gone through two main revisions since that time, using field and psychometric testing conducted by multiple partners and other stakeholders to increase the scientific rigor and relevance of the survey and the usability of the data (for more development information refer to https://www.ahrq.gov/cahps/surveys-guidance/cg/about/Develop-CG-Surveys…)
Steps which have contributed to the content and design of the CG-CAHPS Survey over time have included:
- Literature review and review of existing measures
- Development and consultation with technical expert panels
- Focus groups with consumers
- Cognitive testing of survey questions to ensure they will be understood by respondents
- Field testing to assess the reliability of the survey results
- Cognitive testing of measure labels to ensure that survey results are communicated clearly to providers and the public
- Public comment
- On-going collaboration and harmonization with key partners and stakeholders
The CAHPS Consortium continues to conduct research to develop and test survey administration methods that can improve the efficiency of data collection, enhance response rates, and gather more information about the experiences of those segments of the patient population that have been hard to reach through more traditional means. This research includes: 1) studies comparing the effect of administration modes on response rates, survey scores, and data collection costs (e.g., mode comparisons have included in-office distribution vs. mail; email vs. mail); 2) studies assessing the effect of survey length on response rates and survey scores; 3) studies examining the impact of incentives on response rates; and 4) studies comparing the effect of different survey formats and design on survey responses.
To address data collection efficiency and to improve response rates, the CAHPS Consortium endorsed e-mail notification for web-based surveys as an additional mode of data collection. The CAHPS Consortium recommends a mixed mode that would have two e-mail reminders and a follow-up by mail or telephone to all who are in the survey sample. The follow-up to the entire sample is necessary to get a representative set of responses from a practice’s population, as not all patients may have e-mail.
Proprietary Information
Scientific Acceptability
Testing Data
The data used for these analyses are from the CMMI Primary Care First (PCF) Patient Experience of Care (PEC) Survey) which includes a combination of items from the Adult CG-CAHPS Survey as well as from the Patient-Centered Medical Home CAHPS Supplement. The survey is available on the CMMI PCF website: https://pcfpecs.org/Survey-and-Protocols.
For evidence of performance gap demonstrating persistent gaps over time, we also include top box statistics on the 2022 Adult CG-CAHPS Survey data administered as part of the PCF PEC survey.
The data used to support extrapolation of Adult data to Child data, is based the 2019 maintenance endorsement submission, when both Adult and Child data were available. Using data from surveys administered from January 2016- March 2017, the Child survey included 77 practice sites and over 12,000 responses and the Adult survey included 635 practice sites and over 110,000 respondents. To support extrapolating adult data in the absence of child data, we compared characteristics between respondents from the prior child and adult CG-CAHPS Surveys.
Data were included in the analysis if they had at least one reportable item from the CG-CAHPS survey.
Unless noted otherwise, the top box scores presented are unadjusted since the results are not being used to compare entities, but rather for descriptive and scientific acceptability purposes.
The 2023 survey data was collected between October 3, 2023, through December 19, 2023.
The 2022 survey data was collected between October 6, 2022, through December 19, 2022.
None.
The CG-CAHPS Survey is specified for both individual clinician and group/practice. However, we only have testing for the group or “practice site” because the clinician-level identification in the CG-CAHPS database is not available. The site-level measured entity is referred to as a “practice site.” The practice site is an outpatient facility in a specific location. Practice site level survey results are calculated across the respondents within a specific site.
The data included 2,490 practice sites. For testing and performance scores, practice sites with less than 10 surveys were excluded. After the restriction, the number of sites is 2,486.
The 2022 data used exclusively in the performance gap section to demonstrate persistent gaps over time included 2,806 practice sites. For testing and performance scores, practice sites with less than 10 surveys were excluded. After the restriction, the number of sites is 2,801.
Practices in the 2023 data come from 23 states and the District of Columbia, as shown in 5.1.3a available in the Supplemental 7.1 zip file.
Due to the lack of access to Child CG-CAHPS data, we present the data for Adult CG-CAHPS Survey measures which can be extrapolated to provide evidence for the Child CG-CAHPS Survey measures. The Adult CG-CAHPS Survey and Child CG-CAHPS Survey are equivalent with only minor wording changes (refer to the crosswalk between Adult and Child measures included with the instruments in Attachment 1.21). The wording changes directed attention to the experiences for the child rather than the respondent.
For example, on the Adult Survey one question is worded:
- “In the last 6 months, when you contacted this provider’s office to get an appointment for care you needed right away, how often did you get an appointment as soon as you needed?
On the Child Survey, the question is worded (bolded added to differentiate):
- “In the last 6 months, when you contacted this provider’s office to get an appointment for care your child needed right away, how often did you get an appointment as soon as your child needed?”
Both the Adult and Child survey capture the experience of care from an adult perspective — either directly from the patient (Adult CG-CAHPS) or the parent/guardian (Child CG-CAHPS). Both surveys have adult respondents; the child survey is completed by the child’s parent, relative, or legal guardian.
The data includes 238,204 adult respondents. For testing and performance scores, practice sites with less than 10 surveys were excluded. After the restriction (N=4 sites), the total number of respondents was 238,181, the average number of respondents per site was 96, ranging from 14 to 272 respondents per site.
Tables 5.1.4a through g, available in the Supplemental 7.1 zip file, show descriptive characteristics of the respondents (sex, race/ethnicity, age, self-reported health status, survey mode, and survey language). Practice sites had adult respondents that were predominantly white and non-Hispanic (79%) and older than 54 (83%). Over 40% of respondents had at least a 4-year college degree (41%). Most responded via mail (89%) compared to by phone (11%). Most respondents completed the survey in English (99%), and only 1% completed it in Spanish.
Both the Adult and Child survey capture the experience of care from an adult perspective — either directly from the patient (Adult CG-CAHPS) or the parent/guardian (Child CG-CAHPS).
Both surveys have adult respondents; the child survey is completed by the child’s parent, relative, or legal guardian.
In the prior maintenance endorsement in 2019, using data from surveys administered from January 2016- March 2017, we had access to data from 77 practice sites and over 12,000 responses for the Child survey and 635 practice sites and over 110,000 respondents for the Adult survey. To support extrapolating adult data in the absence of child data, we compared characteristics between respondents from the prior child and adult CG-CAHPS Surveys.
Respondent education distributions were similar across child and adult CAHPS surveys. For example, in the 2016-2017 CG-CAHPS data, 65% of respondents to the Adult Survey had at least some college and 76% of respondents to the child survey had at least some college. Additionally, 21% of respondents to the Adult survey and 25% of respondents to the Child survey had more than a 4-year college degree. The Adult survey respondents had a slightly higher percentage of respondents with lower levels of education (27% high school graduate or GED or less versus 16% for the Child survey). Overall, these results support the appropriateness of extrapolating adult data in the absence of child data.
Respondents to both the Adult and Child 2016-2017 CG-CAHPS Surveys tended to be female (over 50% for both), with more females responding to the Child Survey (82%) than the Adult Survey (53%).
Patient general and mental health status distributions differ between the child and adult CG-CAHPS surveys when examining the 2016-2017 CG-CAHPS child survey data. For the child survey, 78% of respondents reported the general health status of the child to be very good or excellent while only 43% of respondents to the adult survey reported their general health status as very good or excellent. Similarly, 77% of respondents reported the mental health status of the child to be very good or excellent compared with 62% of respondents to the adult CG-CAHPS survey reported their mental health status as very good or excellent.
More respondents of Child CG-CAHPS Survey were aged 25-44 than respondents of the Adult surveys (61% versus 14%).
Keeping these differences in mind, we extrapolate adult data in the absence of child data. Further, as shown in the other sections, the scientific acceptability results (reliability and validity) between Adult and Child tended to align in the past maintenance endorsement package, further justifying extrapolation. Additionally, both surveys have the same usability and goal of assessing and improving the quality of care by gathering patient or parent/legal guardian feedback on key aspects of patient experience.
Reliability
We estimated internal consistency reliability using the Cronbach’s coefficient alpha for each composite measure. A reliability of at least 0.70 is considered acceptable for group-level comparisons (Nunnally and Bernstein, 1994). For composites with more than two items, we show the impact on Cronbach’s alpha of deleting one of the items from the composite. However, CAHPS scores are designed to evaluate care across units of care such as plans, physician groups, and hospitals, not individual patients.
For the Access, Provider Communication, and Office Staff composite measures, all items had less than 3% of records with missing values. For the Care Coordination composite measure, all items had less than 4% of records with missing values. We ran the Cronbach’s alpha excluding all missing data as well as with listwise deletion and the results were the same, except for the Access composite measure in which excluding all missing data increased Cronbach’s alpha by .01. Given the similarity of results, we have presented the Cronbach’s alpha values with the inclusion of cases with missing values (listwise deletion) in section 5.2.3.
Reference
Nunnally JC, Bernstein IH. (1994). Psychometric Theory. New York: McGraw Hill.
Table 5.2.3a (attached in 5.2.3a) shows the Cronbach’s alpha for each composite measure in the Adult CG-CAHPS Survey. For items within composite measures consisting of 3 or more items, the Cronbach’s alpha if the item were deleted is provided to determine if there was room for improving coefficient alpha by dropping an item. The table also shows the standardized correlation for the standardized item to total correlations.
All Cronbach’s alphas in the adult survey were above 0.70 except for the Care Coordination composite measure with a Cronbach’s alpha of 0.61. As shown in Table 5.2.3a, removal of any questions in this composite measure would not result in a higher Cronbach's alpha and Care Coordination is an important concept to patients in their experience of care. Further, all the item to total correlations were above 0.40. While Cronbach’s alpha fell below the conventional threshold for several composite measures, it is not the most critical metric in this context. More important is the reliability at the unit level (e.g., plan-level reliability), which better reflects the measure’s utility for quality improvement. Nonetheless, internal consistency remains a relevant consideration in health care, and the Consortium will keep this in mind when implementing future revisions of the instrument.
We assess reliability at the site level, which is the most relevant level of analysis for publicly reported CAHPS measures (Hays & Arnold, 1986, pp. 144-145). Since CAHPS surveys are used to compare groups/units, site-level reliability (which is directly related to the standard error of measurement) is used to determine the number of responses needed to obtain reliable information (Hays, Shaul, et al., 1999). Site reliability, which partitions within- and between-site variance, was calculated from the ICC and the Spearman-Brown prophecy formula in SAS version 9.4. Higher levels of site reliability correspond to more accurate performance measurement and a better ability to distinguish performance among practices. Reliability is not calculated when only one site is included in the decile or max/min value, since there would be no between-site variation. For cases where the number of respondents was tied across decile boundaries, all practice sites with that number of respondents were assigned to the same decile.
Like internal consistency reliability (i.e., Cronbach’s alpha), values of 0.70 and higher are considered acceptable for site-level reliability (Nunnally and Bernstein, 1994) and group comparisons. For example, CMS does not report (labeled as “Not available”) any score with reliability below 0.60, as that is considered low reliability. CMS reports scores that meet the sample size threshold and for which reliability falls between 0.60 and 0.70 but flags these scores as having low reliability and alerts consumers to interpret such scores with caution. Scores with reliability 0.70 or greater are reported without comment. Reliabilities of 0.85 or higher, where possible, are appropriate for applications such as pay-for-performance or actions that reward or classify individual practices.
The CAHPS Consortium has reported the reliability of the CAHPS measures at the appropriate unit of comparison since the beginning of the project over 25 years ago and for measure development throughout the project (e.g., Hays, Martino, et al., 2014; Hays, Berman, et al., 2014; Price, Stucky, et al., 2018).
Due to the lack of access to Child CG-CAHPS data, we are extrapolating data from Adult CG-CAHPS Survey. Both the Adult and Child survey capture the experience of care from an adult perspective — either directly from the patient (Adult CG-CAHPS) or the parent/guardian (Child CG-CAHPS). Additionally, summary results from the 2019 maintenance endorsement, based on 2016-2017 data when both Child and Adult data were available, are shared because they demonstrate that the reliability outcomes were consistent across the two populations. This strong alignment supports the appropriateness of extrapolating adult data in the absence of child data.
For the Care Coordination composite measure, no items had more than 4% of respondents missing values. We have presented the site reliability, excluding missing cases where data are missing.
References
Hays, R, Arnold, S. (1986). Patient and family satisfaction with care for the terminally ill. Hospice Journal, 7, 129-150.
Hays, R, Berman, L, Kanter, M, et al. (2014). Evaluating the psychometric properties of the CAHPS Patient-Centered Medical Home Survey. Clin Ther. 36(5), 689–696.e1.
Hays, RD, Martino, S, Brown, J, Cui, M, Cleary, P, Gaillot, S, Elliott, M. (2014). Evaluation of a care coordination measure for the Consumer Assessment of Healthcare Providers and System (CAHPS®) Medicare Survey. Medical Care Research and Review, 71, 192-202.
Hays, R, Shaul, J, William, V, et al. (1999). Psychometric Properties of the CAHPS™ 1.0 Survey measures. Medical Care. Volume 37(3) SUPPLEMENT, MS22-MS31.
Nunnally, JC, Bernstein, IH. (1994). Psychometric Theory. New York: McGraw Hill.
Price, RA, Stucky, B, Parast, L, Elliott, MN, Haas, A, Bradley, M, Teno, JM. (2018). Development of valid and reliable measures of patient and family experiences of hospice care for public reporting. Journal of Palliative Medicine, 21, 924-932.
The Care Coordination composite measure’s overall site reliability is 0.79. The information in Table 2 provides overall and decile-level reliability. Deciles for this table are based on the number of respondents per site, and the “mean performance score” row in this table shows the mean top box scores averaged across practice sites.
As a reminder, we present results for the Adult CG-CAHPS Survey due to a lack of access to Child CG-CAHPS Survey data. Refer to Section 5.2.2 for more information.
For reference, results from the 2019 maintenance endorsement for the Care Coordination composite measure were:
- Adult Survey: Site reliability was 0.85, with an average of 174 respondents per site
- Child Survey: Site reliability was 0.78, with an average of 158 respondents per site
| | Overall | Minimum | Decile_1 | Decile_2 | Decile_3 | Decile_4 | Decile_5 | Decile_6 | Decile_7 | Decile_8 | Decile_9 | Decile_10 | Maximum |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Reliability | 0.790 | NA | 0.758 | 0.792 | 0.794 | 0.761 | 0.791 | 0.797 | 0.781 | 0.808 | 0.803 | 0.852 | NA |
| Mean Performance Score | 72.8 | 67.5 | 69.8 | 70.8 | 72.7 | 73.1 | 73.0 | 73.3 | 73.4 | 74.2 | 73.8 | 73.6 | 77.1 |
| N of Entities | 2,486 | 1 | 257 | 240 | 269 | 233 | 250 | 259 | 238 | 251 | 249 | 238 | 1 |
| N of Persons / Encounters / Episodes | 233,988 | 13 | 11,705 | 14,845 | 19,279 | 18,668 | 21,877 | 24,719 | 24,688 | 28,487 | 31,644 | 37,795 | 268 |
The site reliability assessment for the Care Coordination composite measure indicates that overall and across all deciles, the measure maintains acceptable reliability (e.g., greater than 0.7) indicating the measure effectively detects systematic variation among sites relative to random variation.
Results from the 2019 measure maintenance endorsement submission showed similar reliability outcomes, where the measure maintained acceptable reliability overall (e.g., greater than 0.7), and demonstrated the reliability results were consistent across the Adult and Child surveys.
This strong alignment supports the appropriateness of extrapolating adult data in the absence of child data.
Validity
Several model fit indices were examined to determine how well the hypothesized factor structure, or composite measures fit the data, including chi-square divided by its degrees of freedom (𝝌𝟐/𝒅𝒇 ) (criteria: values less than 5.0; Schumacker & Lomax, 2004), comparative fit index (CFI) (criteria: values 0.95 or greater; Hu & Bentler, 1999), root mean square error of approximation (RMSEA) (criteria: values less than 0.06; Kline, 2005), and the standardized root mean square residual (SRMR) (criteria: values less than 0.08; Kenny, 2020).
We examined standardized factor loadings for each item on its respective composite measure. Factor loadings above 0.40 indicate that the item’s relationship to the composite measure is acceptable (Stevens, 2002).
References
Hu, L., & Bentler, PM. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55. http://dx.doi.org/10.1080/10705519909540118
Kenny, DA. (2020, June 5). Measuring model fit. Available at http://davidakenny.net/cm/fit.htm. Accessed October 2025.
Kline, RB. (2005). Principles and practice of structural equation modeling (2nd ed.) New York: The Guilford Press.
Schumacker R., & Lomax, R. (2004). A beginner’s guide to structural equation modeling
(2nd ed.). Lawrence Erlbaum.
Stevens, JP. (2002). Applied multivariate statistics for the social sciences (4th ed.). Mahwah, NJ: Lawrence Erlbaum
Tables 5.3.4a and 5.3.4b (attached in section 5.3.4a) show results for the model fit indices and standardized factor loadings for the adult dataset.
The Confirmatory Factor Analysis results for the Adult CG-CAHPS survey demonstrate strong model fit based on established criteria. The chi-square divided by degrees of freedom (χ²/df) was 127.28, exceeding the recommended threshold of less than 5.0, which is common in large samples due to the sensitivity of this index. However, the other fit indices all fall well within acceptable ranges: the Comparative Fit Index (CFI) was 0.98, surpassing the criterion of 0.95 or greater, indicating excellent model fit. The Root Mean Square Error of Approximation (RMSEA) was 0.05, meeting the standard of less than 0.06, and the Standardized Root Mean Square Residual (SRMR) was 0.04, comfortably below the threshold of 0.08.
The estimates for each standardized factor loading on the items in the composite measures assess convergent validity. All standardized factor loadings are at least 0.5, with the majority above 0.8, and all are statistically significant (p < 0.001), demonstrating the convergent validity of the measures.
These results support the hypothesized factor structure for the measures in the survey.
At the individual and site level, we examined the relationships between each composite measure and item’s top box score and the top box score for the provider rating measure using Spearman rank-order correlations. We expect the composite measures to be moderately to strongly related to the overall rating (Quigley et al., 2024).
We also examined Spearman rank-order intercorrelations among the composite measures to assess the extent to which they measure different constructs. As measures of patient experience, we expected the composite measures to be positively and significantly correlated. However, very large intercorrelations (e.g., > 0.80) suggest that the composite measures may not be sufficiently distinct to be considered separate measures (O’Brien, 2007).
One rule of thumb for correlations is:
0.10 is a small correlation
0.30 is a medium correlation and
0.50 is a large correlation.
For the Care Coordination composite measure, we expect a positive and large relationship with the rating of the provider since coordinating is a key component of the patience experience. That is, if the provider does seem to know a patient’s medical history, and follow up on test results, we expect the rating for that provider will also be higher. We also expect Care Coordination to show positive and large correlations with the other CG-CAHPS composite measures with the strongest relationship with Provider Communication, since both measures reflect the provider’s behavior, communication, and knowledge for the patient’s care.
Due to the lack of access to Child CG-CAHPS data, we are extrapolating data from Adult CG-CAHPS Survey. Both the Adult and Child survey capture the experience of care from an adult perspective — either directly from the patient (Adult CG-CAHPS) or the parent/guardian (Child CG-CAHPS).
Coordinated care is an essential aspect of high-quality care for both adults and children. The actions of providers to coordinate care, such as knowing important information about the patient’s medical history and following up with test results are consistent across adult patient populations and child populations. Having coordinated care helps ensure that adult or child patients receive the right care at the right time, and helps to avoid duplication of tests. The threshold for care coordination for children may be lower than for adults. The CAHPS Consortium noted that early CAHPS qualitative work with Medicaid mothers revealed a higher priority on care for their children than themselves. In sum, care coordination operates through similar mechanisms for adults and children because both require follow-up on test results and having the provider know about their medical history, and the impact is consistent across age groups. While the impact may be heightened for children due to parental/guardian involvement and developmental needs, the underlying response mechanism remains consistent, supporting extrapolation of findings between Adult and Child survey populations.
Additionally, summary results from the 2019 maintenance endorsement, based on 2016-2017 data when both Child and Adult data were available, are shared because they demonstrate that the validity outcomes were consistent across the two populations. This strong alignment supports the appropriateness of extrapolating adult data in the absence of child data.
References
Quigley, DD., Elliott, MN., Qureshi, N., Predmore, Z., Hays, RD. (2024). Associations of the Consumer Assessment of Healthcare Providers and Systems (CAHPS) Clinician and Group
Survey scores with interventions and site, provider, and patient factors: A systematic review of the evidence. Journal of Patient Experience, 11, 1-14.
O’Brien, RM. (2007). A caution regarding rules of thumb for variance inflation factors. Qual Quant. 41, 673–690.
The Care Coordination composite measure has a large correlation of 0.73 with the overall provider rating at the site level and a medium correlation of 0.46 at the individual level.
The Care Coordination composite measure’s correlations with the other CG-CAHPS composite measures were 0.54 with Access, 0.75 with Provider Communication, and 0.52 with Office Staff at the site level. At the individual level, it was 0.40 with Access, 0.56 with Provider Communication, and 0.38 with Office Staff.
Please see attachment 5.3.4a for validity testing results, which includes the site-level and individual-level correlations for the Care Coordination composite measure and items with the rating measure and intercorrelations among the Adult CG-CAHPS composite measures, all assessed using Spearman rank-order correlations.
As a reminder, we present results for the Adult CG-CAHPS Survey due to a lack of access to Child CG-CAHPS Survey data. Refer to Section 5.3.3 for more information.
For reference, results from the 2019 maintenance endorsement for the Care Coordination composite measure showed similar patterns of statistically significant correlations with:
Rating of Provider
- Adult Survey: Site level was 0.60, individual level was 0.39
- Child Survey: Site level was 0.47, individual level was 0.39
Access (site level correlations)
- Adult Survey: 0.47
- Child Survey: 0.41
Provider Communication (site level correlations)
- Adult Survey: 0.68
- Child Survey: 0.59
Care Coordination (site level correlations)
- Adult Survey: 0.42
- Child Survey: 0.36
In this analysis, all correlations between the Care Coordination composite measure and the other composite measures fell within an acceptable range (none exceeded 0.80), ranging from 0.52 to 0.75 at the site level and 0.38 to 0.56 at the individual level. The Care Coordination composite measure was significantly and positively correlated with the Rating of Provider (0.73 at the site level, 0.46 at the individual level). Overall, these results support the distinctiveness of the Care Coordination composite measure as a measure of patient experience, while also confirming its meaningful relationship to broader aspects of care.
Results from the 2019 measure maintenance endorsement submission showed similar correlations and demonstrated the validity results were consistent for the Adult survey and across the Adult and Child surveys. Correlations between the Care Coordination Composite measure and the other composite measures for the Adult and Child survey fell within an acceptable range (none exceeded 0.80) and ranged from 0.42 to 0.68 for the Adult survey and from 0.36 to 0.59 for the Child survey. The correlations between the Care Coordination composite measure and the Rating of Provider measure were also positive and statistically significant for the Child survey and Adult survey.
This strong alignment supports the appropriateness of extrapolating adult data in the absence of child data.
Risk Adjustment
The Child CG-CAHPS Survey results are not required to be risk adjusted by users. However, survey users, including public reporting entities, may voluntarily choose to adjust the data to account for patient/respondent case-mix differences if comparing practices. Guidance for this process is available in two key documents: “Preparing Data from CAHPS Surveys for Analysis” (available at https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/surveys-guidance…) and “Instructions for Analyzing Data from CAHPS Surveys” dated June 2025 (available at https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/surveys-guidance…). These resources provide instructions for coding the adjuster variables, imputing missing data for the adjusters, and incorporating adjustments in analyses using the CAHPS Analysis Program in SAS. The selection of adjuster variables and the calculation of risk-adjusted scores are user-defined. Users must also decide whether to impute missing data for the adjusters using each adjuster’s entity-level mean.
The CAHPS Analysis Program is a set of free SAS programs that enable survey users to conduct risk adjustment. The programs with test modules are available for download at https://www.ahrq.gov/cahps/surveys-guidance/helpful-resources/analysis/….
The CAHPS Analysis Program adjusts the data for case mix, generates a distribution of survey results for each of the measures, calculates the average score for both individual survey items and composite measures, and indicates whether an entity’s scores are statistically different from the average. AHRQ’s CAHPS Consortium developed the CAHPS Analysis Program to work with all CAHPS surveys. It is updated periodically to add functionality, produce additional output types, and correct or debug issues with previous versions.
This section describes the rationale for case-mix adjustment that is not required but that CAHPS users may elect to use. The standard methodology used is case-mix adjustment via regression in a linear model. Without an adjustment, differences in CAHPS scores between entities could be due to case-mix differences rather than true differences in quality.
The current CAHPS Analysis Program suggests adjusting for the child’s general health status, the child’s mental health status, respondent’s age, and respondent’s education. Studies have found that patient and consumer survey responses about experiences and satisfaction with healthcare correlate with personal characteristics like general health, mental health/depression, education, and age (Simon et al., 2009; Rahmqvist and Bara, 2010; Zaslavsky et al. 2001; Martino et al. 2011; Eselius et al., 2008; Elliott et al. 2009).
Health status and age are two respondent characteristics frequently associated with reports of the quality of medical care. People in worse health tend to report lower patient experience satisfaction and more problems with care perhaps because sicker patients have more complex health care needs and may tend to report more problems with coordination or communication. Older patients tend to report greater satisfaction, better patient experience, and fewer problems than younger patients, although this association is usually not as strong as that between health status and ratings (Hatfield and Zaslavsky, 2017; Eselius et al., 2008).
Education is self-reported by respondents who take the CAHPS surveys. Studies have shown that more educated respondents report more problems, perhaps because they have higher expectations rather than because they receive lower quality of care (Sofaer and Firminger, 2005). However, in a multivariate analysis using Medicare Advantage CAHPS data, Hatfield and Zaslavsky (2017) found that education had less influence on CAHPS dimension scores than self-reported general and mental health.
Different CAHPS surveys adjust for different variables, and the variables included here are not the only adjustment factors, even for the practice setting. It should be noted that race/ethnicity is not typically included as case-mix adjuster variables as doing so may mask disparities in care. CAHPS data can also be adjusted for other factors such as survey administration mode (Peipert et al., 2017). For example, a study by Drake and colleagues (2014) found that telephone respondents gave more positive responses than mail respondents did. The AHRQ CG-CAHPS Database did not adjust for survey mode.
The AHRQ CG-CAHPS Database adjusted only for respondent age, respondent education, the child’s general health status, and the child’s mental health status.
References
Drake, KM, Hargraves, JL, Lloyd, S, Gallagher, PM, Cleary, PD. (2014). The effect of response scale, administration mode, and format on responses to the CAHPS Clinician and Group Survey. Health Serv Res. 49(4), 1387-99. doi: 10.1111/1475-6773.12160.
Elliott, MN, Zaslavsky, AM, Goldstein, E, Lehrman, W, Hambarsoomians, K, Beckett, MK, Giordano, L. (2009). Effects of survey mode, patient mix, and nonresponse on CAHPS hospital survey scores. Health Serv Res. 44(2 Pt 1), 501-18. doi: 10.1111/j.1475-6773.2008.00914.x.
Eselius, LL, Cleary, PD, Zaslavsky, AM, Huskamp, HA, Busch, SH. (2008). Case-mix adjustment of consumer reports about managed behavioral health care and health plans. Health Research and Educational Trust. 43(6), 2014-2032.
Hatfield, LA, Zaslavsky, AM. (2017). Implications of variation in the relationships between beneficiary characteristics and Medicare Advantage CAHPS measures. Health Serv Res. 52(4), 1310-1329.
Martino, SC., Elliott, MN., Kanouse, DE., Farley, DO., Burkhart, Q., Hays., RD.(2011). Depression and the health care experiences of Medicare beneficiaries. Health Services Research. 46(6pt1), 1883–904.
Peipert, JD, Brown, JA, Cui, M, Hays, RD (2017). Differences in mail and telephone responses to the CAHPS In-Center Hemodialysis Survey. Ann Clin Nephrol. 1(1), 1.
Rahmqvist, M, Bara, AC. (2010). Patient characteristics and quality dimensions related to patient satisfaction. International Journal for Quality in Health Care. 22(2), 86–92.
Simon, GC, Rutter, M, Crosier, J, Scott, BH, Operskalski, Ludman E. (2009). Are comparisons of consumer satisfaction with providers biased by nonresponse or case-mix differences? Psychiatric Services. 60(1), 67–73.
Sofaer, S, Firminger, K. (2005). Patient perceptions of the quality of health services.
Annual Review of Public Health. 26, 513–59.
Zaslavsky, A, Zaborski, L, Ding, L, Shaul, JA, Cioffi, MJ, Cleary, PD. (2001). Adjusting performance measures to ensure equitable plan comparisons. Health Care Financing Review. 22(3), 109–26.
Tables 5.4.3a through 5.4.3c in Attachment 5.4.3 show descriptive characteristics for the risk/case mix variables tested: age, general health status, mental health status, and education. Practice sites had adult respondents who were predominantly 55 or older (83%). Forty-one percent (41%) of respondents reported having at least a 4-year college degree. Forty-one percent (41%) of respondents reported being in excellent or very good general health, and 54% of respondents reported being in excellent or very good mental health.
As a reminder, we present results for the Adult CG-CAHPS Survey due to a lack of access to Child CG-CAHPS Survey data. Both the Adult and Child survey capture the experience of care from an adult perspective — either directly from the patient (Adult CG-CAHPS) or the parent/guardian (Child CG-CAHPS).
Case-mix adjustment was conducted using the AHRQ CAHPS Analysis Program (https://www.ahrq.gov/cahps/surveys-guidance/helpful-resources/analysis/…). In the CAHPS Analysis Program, we set the program to impute the site mean if data were missing for that variable to avoid losing observations because of missing data. This is generally acceptable because, the size of the adjustment and the amount of missing data on adjusters are typically small. The case-mix adjusted scores are based on regression analyses and the results include case-mix adjusted coefficients. For detailed information on case-mix adjustment refer to page 46 of “Instructions for Analyzing CAHPS Data” document (https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/surveys-guidance…).
To quantify the effect of case-mix adjustment on the ranking of practices, using data from the Adult CG-CAHPS Survey we calculate the Pearson product moment correlation coefficient, Kendall Tau correlation coefficient, and the maximum difference between the adjusted and unadjusted practice mean scores, as shown in Attachment 5.4.5a. The adjustment factors include age, education, mental health status, and general health status. Appropriate case-mix adjusters are patient and respondent-level variables that are not under the control of the provider (“exogenous”) but result in different scores even when the quality of care is the same. For example, parent/guardian education and age are not under the control of the provider.
Eselius and colleagues (2008) published results of an analysis of case-mix adjustments for the CAHPS Health Plan Survey, a survey with similar composite measures to the CG-CAHPS Survey about access to care, provider communication and customer service. After selecting appropriate adjusters based on explanatory power in separate linear regression models, the authors determined the impact of case-mix adjustment on their sample health plans. Specifically, they examined the size of the adjustments and the extent to which adjustments impacted the ranking of health plans. Case-mix adjustments had only modest effects on health plan ratings and rankings. The authors also found that mental health status was a strong predictor of patient-reported experience. For this purpose, this question was added to CG-CAHPS: “In general, how would you rate your overall mental or emotional health?”
The Pearson product-moment correlation coefficient is widely used and understood. It assesses the linear association between the adjusted and unadjusted scores and ranges between -1 to 1. Because the ranking of scores is often important in public reports of CAHPS results, we also calculate Kendall’s Tau (Kendall rank correlation coefficient). Tau is the correlation between the rank orders of the adjusted and unadjusted scores. The Kendall Tau statistic also has a range of -1 to +1, so that it has a range comparable to other correlation coefficients. Tau can be interpreted as the percentage of pairs of units (e.g., practice sites) that switched ordering because of case-mix adjustment [100*(1-Tau)/2].
We also calculate the maximum absolute value difference between adjusted and unadjusted mean scores among practices.
Reference
Eselius, LL., Cleary, PD., Zaslavsky, AM., Huskamp, HA., & Busch, SH. (2008). Case-mix adjustment of consumer reports about managed behavioral health care and health plans. Health Research and Educational Trust. 43(6), 2014-2032.
The selection of case-mix adjusters is based on prior research across multiple datasets of the CG-CAHPS Database and is consistent with how scores are calculated. Results are found in Attachment 5.4.4a and 5.4.5a.
All adjusters were statistically significant in the regression models, except for one age category and two education categories:
- ages 65-74 for two of the three items in this measure.
- education Some high school or less for the item “In the last 6 months, when this provider ordered a blood test, x-ray, or other test for you, how often did someone from this provider’s office follow up to give you those results?”
- education High school graduate or GED for the item “In the last 6 months, when this provider ordered a blood test, x-ray, or other test for you, how often did someone from this provider’s office follow up to give you those results?”
The adjusters resulted in similar top box scores, with the Pearson correlation of unadjusted and adjusted mean scores being close to 1, at 0.99 and the Kendall Tau correlation at 0.91. Further, the maximum difference in mean scores for this measure was 0.15, and 0.22 for the items within the measure. This suggests that adjusting for case-mix can help level the playing field for this measure.
The final model includes four adjusters: respondent age, respondent education, child’s general health status, and child’s mental health status.
Use & Usability
Use
Provides patient experience, clinical quality and total cost of care ratings for medical groups in 44 CA counties. The patient experience ratings come from the Patient Assessment Survey (PAS) based on CG-CAHPS for adult and child practices. The program is reportedly being sunset in 2025 (https://www.pbgh.org/program/patient-assessment-survey/).
A total of 204 medical groups are included in the 44 California counties. Only 14 counties in the state do not have any medical groups reported. Number of patients: Information not available.
Medical group practice level of analyses, outpatient primary care practices
Through MIPS, clinicians can earn performance-based payment adjustments for services they provide to Medicare patients. The CG-CAHPS for MIPS survey is an optional quality measure that groups participating in MIPS can administer to their adult population.
MIPS is a national program that includes most physicians and group practices in the U.S who provide services to Medicare patients. The CAHPS sample frame includes all patients from a practice-supplied roster from groups participating in MIPS. Number of patients: Information not available.
Group practice level
Medicare Shared Savings Program ACOs collaborate to give coordinated high-quality care to people with Medicare. They are required to report the CAHPS for MIPS Survey, which is a version of the Adult CG-CAHPS Survey.
Over 470 ACOs are participating in the Shared Savings Program for Performance Year 2025. ACOs are located in almost all states and the District of Columbia. Number of patients: Information not available.
Medicare Shared Savings Program Accountable Care Organizations (ACOs)
NCQA recognizes clinicians and practices in key areas of performance. The recognition program includes optional reporting for practices that use the Adult CG-CAHPS survey with PCMH items.
The NCQA PCMH Recognition program is the most widely adopted PCMH evaluation program in the country. More than 13,000 practices (with more than 67,000 clinicians) are Recognized by NCQA. Number of patients: Information not available.
Group practice level
Primary Care First was a voluntary alternative payment model (through December 2025) that rewarded value and quality by offering an innovative payment structure to support the delivery of advanced primary care and administered a version of the Adult CG-CAHPS survey.
26 US regions and approximately 1,700 participant practices. Number of patients: Information not available.
Group practice level
UCLA Health, a large health system in southern California, administers the CAHPS Clinician & Group Survey on an ongoing basis to adult patients and the parents or guardians of pediatric patients (12 and younger) (Adult and Child). The organization has developed a set of data displays to manage and convey the tremendous amount of information they collect for quality improvement purposes.
Southern California as part of the University of California system. They collect data from over 1300 reporting physicians, 279 reporting offices, and 28 reporting clinical departments. Number of patients: Information not available.
Physician and office/clinic level
Usability
Actions to Improve Patient Experience
CAHPS® surveys play an important role as a quality improvement (QI) tool for healthcare organizations that use the standardized data to:
- Identify relative strengths and weaknesses in their performance.
- Determine where they need to improve.
- Track their progress over time.
AHRQ has made available a CAHPS Ambulatory Care Improvement Guide which is a comprehensive resource for health plans, medical groups, and other providers seeking to improve their performance in the domains of patient experience measured by CAHPS surveys. AHRQ also has created a short video to help improve patient experience https://www.ahrq.gov/cahps/quality-improvement/index.html#:~:text=CAHPS….
The steps are:
- Compare CAHPS survey scores to other health care organizations to determine how the plan is doing in comparison to others.
- Examine how CAHPS scores are changing over time.
- Identify priorities based on these comparisons
- Confirm these priorities based on other sources of information (e.g., patient complaints, patient comments)
- Find out what is actually happening with patients and why.
- Brainstorm with staff to determine the best strategies for improvement.
In addition, AHRQ held a research meeting in 2020 to discuss how to improve patient experience and provided summaries of the presentations: https://www.ahrq.gov/cahps/news-and-events/events/2020-meeting-summary…
Difficulty in Increasing Response Rates
Users are also provided advice for improving response rates:
- Improve initial contact rates by making sure that addresses and phone numbers are current and accurate (e.g., identify sources of up-to-date sample information, run a sample file through a national change-of-address database, send a sample to a phone number look-up vendor).
- Use all available tracking methods (e.g., Lexis-Nexis, Internet database services and directories).
- Improve contact rates after data collection has begun (e.g., increase maximum number of calls, ensure that calls take place at different day and evening times over a period of days, mail second reminders, use experienced and well-trained interviewers).
- Consider using a mixed-mode protocol. In field tests, the combined approach was more likely to achieve a desired response rate than did one mode alone.
- Train interviewers on how to deal with gatekeepers.
- Train interviewers on refusal aversion/conversion techniques.
AHRQ has several resources for practice sites to improve performance on the CAHPS Surveys. They created the CAHPS Ambulatory Care Improvement Guide, which is a comprehensive resource for organizations seeking to improve their performance: https://www.ahrq.gov/cahps/quality-improvement/improvement-guide/improv…. There are also case studies and webcasts that share insights and best practices in improving patient experience with care: https://www.ahrq.gov/cahps/surveys-guidance/hp/improve/index.html.
To improve on Care Coordination, practices can establish who is responsible for each aspect of a patient’s care by using care coordinators, assign specific coordination tasks to staff, and/or use a team-based approach to care. They can create protocols for timely updates between providers, use shared electronic health records to ensure all providers have access to the same information, and review the medication list for every visit. Practices can help ensure physicians have time to review the medical history during visits and ask parents/guardians if the child has have seen other providers to discuss that care. A literature review by Quigley, et al. (2024) examined 24 interventions that used CG-CAHPS data to assess improvements in patient experience. One study showed that activating a patient portal enhanced Care Coordination scores (Fareed, et al., 2022). Hays and Skootsky (2022) found that telehealth visits were associated with higher Care Coordination scores compared to in-office visits, highlighting the value of offering flexible care delivery options.
To support usability, the extrapolated adult data were evaluated for applicability to child measures. Both the Adult and Child survey capture the experience of care from an adult perspective — either directly from the patient (Adult CG-CAHPS) or the parent/guardian (Child CG-CAHPS). The adult endpoint “how often the provider knew important information about the patient’s medical history” translates directly to pediatric populations, as parents/guardians can reliably report on this experience for their children. This alignment demonstrates that the adult-derived measure is feasible for implementation in child populations and that results from the adult survey data can be extrapolated for the child survey results in the absence of child data.
References
Fareed, N, MacEwan, SR, Vink, S. et al. (2022). Relationships between patient portal activation and patient satisfaction scores among CG-CAHPS and HCAHPS respondents. The American Journal of Managed Care, 28(1).
Hays, RD. & Skootsky, SA. (2022). Patient experience with in-person and telehealth visits before and during the COVID-19 pandemic at a large integrated health system in the United States, Journal of General Internal Medicine, 37, 847-852.
Quigley, DD., Elliott, MN, Qureshi, N, Predmore, Z, Hays, RD. (2024). How the CAHPS Clinician and Group patient experience survey data have been used in research: A systematic review. Journal of Patient-Centered Research and Reviews, 11, 88-96. doi: 10.17294/2330-0698.2056.
Throughout the development process, the CAHPS Consortium has incorporated the data or input from these various sources in an incremental process of revision and refinement to develop measurement that is more precise and to produce survey data that would better meet the information needs of consumers and other stakeholders. Between versions 1.0 and 2.0 of Child CG-CAHPS, pediatric experts felt that the Child version of the CG-CAHPS Survey would benefit from a more comprehensive measurement of development and prevention. The CAHPS Consortium worked with the American Academy of Pediatrics and other key stakeholders to develop 11 new items that address development and prevention. These items were grouped into new two composite measures for the Child Survey version 2.0 (Gallagher et al., 2009).
The CG-CAHPS Survey was initially developed with the standard CAHPS 4-point response scale. The field test data that was used for the initial endorsement consisted of several testing sites in the US. One of the larger field tests was conducted in Massachusetts as part of the Massachusetts Health Quality Partners (MHQP) statewide surveying initiative. At that time, MHQP used a 6-point response scale instead of the standard CAHPS 4-point scale. Based on that evidence, the CG-CAHPS Survey was endorsed with a 6-point response scale. Affirmed by significant user feedback, additional testing was conducted to add to the other field test data to confirm the properties and function of the standard CAHPS 4-point response scale and the CG-CAHPS Survey was updated to the 4-point response scale so that results could be aligned across other CAHPS surveys (e.g., CAHPS Health Plan Survey, CAHPS Hospital Survey (HCAHPS)). Drake et al. (2014) examined how different response scales affect responses to the CG-CAHPS survey among 6,500 patients. They found that compared to the 4-category response options surveys, respondents to the 6-category response options surveys had 41 percent more missing items. There were no significant differences between the 4-category and 6-category response option surveys’ average composite measure score or provider-level reliability.
For the 3.0 version, the survey switched from a 12-month recall period to a 6-month recall period to improve accuracy of recall and focus on more recent visits.
The CAHPS Consortium hears user feedback during research studies and development. Users can contact the CAHPS Database team with questions or comments by phone at 888-808-7108 or email at [email protected]. The CAHPS consortium also solicits feedback via focus groups with patients in developing survey content and design. When changes are proposed to the survey, the changes also often go through a public comment period and those comments are summarized and posted on the AHRQ site (e.g., https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/surveys-guidance…)
References
Drake, KM, Hargraves, JL, Lloyd, S, Gallagher, PM, Cleary, PD. (2014). The effect of response scale, administration mode, and format on responses to the CAHPS Clinician and Group Survey. Health Serv Res. 49(4), 1387-99.
Gallagher, P, Ding, L, Ham, HP, Schor, EL, Hays, RD, Cleary, PD. (2009). Development of a new patient-based measure of pediatric ambulatory care. Pediatrics. 124(5), 1348-54.
No additional detail.
In late 2009, version 2.0 addressed the fact that many people receive care in the ambulatory setting from non-physician providers such as nurse practitioners and physicians´ assistants. This change to “this provider” was in response to requests from the medical community for a survey instrument that would allow patients to report on their experiences with all their health care practitioners.
The items designed to identify chronic conditions were moved from the core survey version 2.0 to a supplemental item set.
After extensive testing and receiving feedback from users, with the release of CG-CAHPS Version 2.0, the CAHPS consortium endorsed e-mail notification for web-based surveys as an additional mode of data collection. The CAHPS consortium recommends a mixed mode that would have two e-mail reminders and a follow-up by mail or telephone to all who are surveyed.
Based on the stakeholder and user feedback obtained through public comment, technical expert review, and further testing, the following key changes were made in the release of the CG-CAHPS 3.0 version:
- One instrument, in contrast to the three instruments available for the 2.0 version.
- Use of a 6-month reference time period rather than a 12-month reference period.
- New and modified composite measures:
- New composite measure for "Care Coordination."
- Modified the composite measure for "Access."
- Modified the composite measure for "Communication."
- A modified Patient-Centered Medical Home Item Set.
- Shift of development and prevention items from the core Child Survey to the Patient-Centered Medical Home Item Set.
- Overall reduced length.
For the 3.1 version, with the COVID-19 pandemic changing how some health care was delivered (e.g., video or phone rather than in-person), the instrument was updated again to change instructions and gate question wording to include these types of visits.
No additional detail.
Due to the lack of access to Child CG-CAHPS data, we are extrapolating data from Adult CG-CAHPS Survey. Both the Adult and Child survey capture the experience of care from an adult perspective — either directly from the patient (Adult CG-CAHPS) or the parent/guardian (Child CG-CAHPS). To examine improvement, we used two years of data from the PCF-PEC Survey and then, for reference, we also refer to trends in Adult CG-CAHPS Survey data provided in the 2019 AHRQ CG-CAHPS Survey Chartbook (AHRQ’s reporting of Child data ended in 2016). The PCF-PEC Survey includes the items on the core CG-CAHPS Survey and includes Patient-Centered Medical Home (PCMH) and other supplemental items. The results from the PCF-PEC Survey and the CG-CAHPS Survey are comparable because they include the same core items and most sites (81%) in the 2019 AHRQ CG-CAHPS Survey Chartbook also included the PCMH supplemental items. Further, Beckett et al. (2025) found that adding supplemental items did not affect CAHPS scores.
Care Coordination have shown minimal improvement between 2022 and 2023, increasing only slightly from 72.0% to 72.8%. In comparison, the 2019 AHRQ CG-CAHPS Chartbook reported an average of 75% in 2015, decreasing to 73% in 2018 and then increasing to 74% in 2019 (https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/cahps-database/2…). This trend suggests a variable performance since 2015, with scores still below levels in 2015. These results underscore a need for focused efforts to enhance coordinated care for patients.
Reference
Beckett, MK, Hambarsoomian, K, Brown, J, Cleary, PD, Abel, GA, Giordano, L, Elliott, MN. (2025). Supplemental items reduce HCAHPS response rates, but response rates do not affect HCAHPS scores: A randomized experiment, Medical Care, 63(10), 764-770. DOI: https://doi.org/10.1097/MLR.0000000000002197.
No unexpected findings.
No unexpected findings.
Comments
Staff Preliminary Assessment
CBE #0005-9 Staff Preliminary Assessment
Importance
Strengths
- In the absence of Child CG-CAHPS data, the developer extrapolated from Adult CG-CAHPS data to provide support for the pediatric measures, and justified this extrapolation by citing construct equivalence (both instruments capture the care experience via adult respondents) and cross-population consistency in prior performance gap results across child and adult populations. The Endorsement & Maintenance Guidebook (see Appendix H in v.3.2) allows for the extrapolation of adult data for pediatric-focused measures, provided certain requirements are met for Importance, Validity, and Use & Usability.
If adult CG-CAHPS respondents are deemed acceptable proxies for a parent in a parent-child dyad, then data from the 2023 Primary Care First Patient Experience Survey, in which CG-CAHPS is embedded, show a performance gap, with decile ranges from 58.7% to 84.2%, indicating variation in measure performance and less than optimal performance across the target population. Findings improved slightly from those in 2022.
The developers cited two recent peer-reviewed journal articles that contained information about how lack of care coordination can lead to errors and hospitalization, and how care coordination is associated with higher quality of care.
Limitations
- The developer used adult CG-CAHPS data as a proxy for parent responses to the Child CG-CAHPS survey. That is, the CG-CAHPS adult may have been, or not been, a parent with a child receiving health care services. Performance gaps are based on adult data rather than parent data.
The logic model provided does not clearly articulate the relationships between inputs, activities, and outcomes for the Care Coordination composite. For example, actions cited within the logic model are patient-centric, but do not include actions that can be taken by providers to improve care coordination for parents of children. Adding such information would strengthen the logic model.
In terms of meaningfulness to patients, the developer cites a 2018 article by Chang et al., but this study involved interviews with adult emergency department patients about care coordination, which is not the same population that responds to adult or adult (child) CG-CAHPS (patients in outpatient primary care and specialty groups). A second article by Hays et al. (2014) was referenced, but the content focused on the CAHPS Patient-Centered Medical Home survey, not CG-CAHPS.
Rationale
- This maintenance measure is rated as 'Not Met But Addressable' for importance due to insufficient evidence that the Care Coordination composite is meaningful to patients. The submission could be strengthened by engaging patients directly to elicit feedback on the meaningfulness of the Care Coordination composite and/or citing empiric evidence within the past 5 years that indicates care coordination is a meaningful concept for patients and contributes to their overall experience of care. Also, the logic model could be strengthened by adding provider actions that can improve care coordination for patients.
There is at least moderate confidence that the business case is adequate, i.e., the anticipated impacts of the measure on patient experience justify the use of the measure, due to the demonstrated performance gap.
Closing Care Gaps
The developer did not address this optional domain.
Feasibility Assessment
Strengths
- The developer indicated that the only change in the Adult CG CAHPS instrument was the addition of language to expand the context for questions to include virtual and telehealth visits.
The AHRQ website contains the documentation on CG CAHPS administration, confidentiality, scoring, and other topics.
There are no fees, licensing, or other requirements to use any aspect of the Adult CG CAHPS instrument or the resulting composites.
Limitations
- The feasibility domain can be strengthened by providing the median cost of vendor engagement to administer CAHPS (or a similar metric).
The developer's did not use Child CG CAHPS data from parents/guardians or children receiving care. This may point to limitations in data availability for quality improvement or reporting programs.
Rationale
- This maintenance measure meets the criteria for 'Met' due to availability of data and low degree of missingness, clear and implementable data collection strategies, and transparent handling of patient confidentiality, burden, licensing, and fees. These factors collectively ensure that the measure can be implemented effectively and sustainably in a real-world healthcare setting.
Scientific Acceptability
Strengths
- The developer gives a rationale for extrapolating the reliability results from the Adult CG-CAHPS survey to the Child CG-CAHPS survey. The rationale includes: the Adult and Child surveys are equivalent with only minor wording changes; the Adult and Child surveys both capture experience of care from an adult perspective – as a patient in the Adult survey and as a parent/guardian in the Child survey; and similar respondent education distributions across surveys.
Limitations
- The developer did not perform the required accountable entity-level reliability testing for this maintenance measure.
The developer stated they did not have access to the Child CG-CAHPS data.
The developer refers to reliability testing data from the 2019 maintenance endorsement submission that used survey data collected from January 2016 to March 2017 which is not within the past 5 years.
Although the respondents to the Child and Adult CG-CAHPS surveys are all adults, the Child CG-CAHPS survey respondents are experiencing the health care provided as a parent/guardian of a patient rather than as a patient for Adult CG-CAHPS survey respondents. This difference could lead to different assessments of health care provided and influence reliability of the measure.
The Child CG-CAHPS survey respondents are much younger on average than Adult CG-CAHPS survey respondents (61% of Child survey respondents were aged 25-44 while 15% of Adult survey respondents were aged 25-44).
The Child CG-CAHPS survey respondents tended to be female more often (82%) than for Adult CG-CAHPS survey respondents (53%).
Rationale
- This maintenance measure is rated as ‘Not Met’ for reliability because the required reliability testing was not performed.
Strengths
- The developer performed the required validity testing for this maintenance measure, namely, they conducted person (“data element”) validity testing for all critical data elements and accountable entity-level (“measure score”) validity testing at the clinician group level. The data source used for validity analysis was Primary Care First (PCF) Patient Experience of Care (PEC) Survey administered to adults in October through December of 2023. Validity analyses included 2,486 practices in 23 states.
The developer calculated model fit indices and standardized factor loadings to evaluate how well the set of four composite measures in this instrument-derived measure (IDM) set fit the data. All but one fit index met the required threshold, and factor loadings all three Care Coordination ("Providers’ Use of Information to Coordinate Patient Care") items were well above 0.40, the standard cited as acceptable, indicating that each item strongly contributes to the composite measure, providing evidence of convergent validity.
The developer conducted empirical validity testing at the accountable entity level using Spearman’s rank-order correlation on unadjusted top box scores. The developer hypothesized that Care Coordination would be positively and significantly correlated with Rating of Provider and the other composites, positing that each are key components of the care experience, but not so highly correlated as to suggest the measures are not distinct, citing rho > 0.80 as an example of a very large correlation. Care Coordination was significantly, positively correlated at the clinician group level with the composites Access (rho = 0.543) and Office Staff (rho = 0.517), in line with hypotheses. The developer used data from the PFC-PEC, a survey of adults, to extrapolate results for the child version of the survey, and they justified the extrapolation by noting the similarity in the survey questions and composite in the child and adult versions of the survey, and the similarity between child and adult validity testing results reported in the 2019 submission.
The developer conducted statistical case-mix adjustment, selecting case-mix indicators that are present at the start of care and have a significant correlation with the outcome.
Limitations
- The data from the PFC-PEC survey of adults may not adequately represent the population to whom the child survey is administered, in either distribution and range of practice types or characteristics of respondents (e.g., a child's guardian).
With respect to entity-level validity testing, while the developer has indicated that quality improvement activities can affect more than one IDM, their hypothesis regarding the mechanisms involved and the degree to which these mechanisms are shared between IDMs is not clearly supported by the evidence review and logic model provided. In the absence of an external gold standard against which to validate at least one of the IDMs, this submission would be strengthened by additional support guiding development of hypotheses about expected magnitudes of each correlation. For example, it is unclear from the submission whether mechanisms for improving children's experience of care are similar to those for adults.
The developer used unadjusted scores for validity testing when use of adjusted scores might help rule out known sources of confounding.
The developer states that case-mix adjustment of the measure is optional by the user, and does not provide guidance or supporting rationale stating when adjustment is or is not appropriate. The developer did not provide evidence demonstrating variation in the prevalence of case mix factors across accountable entities. The statistical testing results provided by the developer do not reflect the impact of adjustment on providers at the high or low extremes of the case mix.
Rationale
- This maintenance measure is rated as ‘Not Met But Addressable’ for validity because the accountable-entity validity testing results partially support an inference of validity for the measure, suggesting that the measure somewhat accurately reflects performance on patient experience of care and can distinguish good from poor performance to a limited extent. This submission would be strengthened by explicitly ruling in mechanisms and ruling out confounders for the effect of quality of care on survey respondents' experience with Care Coordination, and additional information to support the extrapolation from data on adults' experience of care to children's experience of care.
The developer employed a statistical case-mix adjustment approach, utilizing a conceptual model designed to account for demographic case-mix factors. Variation in the prevalence of case-mix indicators across different entities was not shown and the model testing results provided do not reflect whether case-mix differences are being appropriately accounted for.
Use and Usability
Strengths
- The developer noted that users can contact the CAHPS Database team with questions or comments by phone at 888-808-7108 or email at [email protected]. Also, when changes are proposed to CAHPS surveys, requests for public comment is posted on the AHRQ website.
The developer does provides some, albeit limited, information about how measured entities can improve their Care Coordination composite measure score: patient portals, telehealth visits, shared EHRs, and use of care coordinators.
The Care Coordination composite is currently used in the California Health Care Quality Report Cards program, CMS Merit-based Incentive Payment System (MIPS) under the Quality Payment Program, the Medicare Shared Savings Program Accountable Care Organizations (ACOs), and the CMMI Primary Care First Model.
Limitations
- The developer reports the Care Coordination composite scores from 2022 (72.0%) to 2023 (72.8%). The Care Coordination composite scores have declined over time. The scores from 2015, 2019, and 2023 were 75%, 74%, and 73% respectively. The developer does not provide an explanation for the lack of improvement over time.
Rationale
- This maintenance measure is rated 'Not Met But Addressable' for Use and Usability. The measure is actively used in at least one accountability application and the developer provides a small number of actions that providers can take to improve their Care Coordination composite scores. However, the submission can be strengthened by including evidence-based actions with citations and an explanation of the lack of improvement in Care Coordination composite scores from 2015 to 2023.
Committee Independent Review
The Child CG-CAHPS Care…
Importance
Care coordination is a core element of patient- and family-centered care and is highly meaningful to parents and guardians of pediatric patients. The Child CG-CAHPS Care Coordination composite captures whether providers know a child’s medical history and whether families receive timely follow-up on test and imaging results—experiences that directly affect safety, trust, and adherence to care plans. When coordination breaks down, caregivers are often forced to bridge gaps themselves, increasing burden and risk, particularly for children with complex needs. Although adult CG-CAHPS data were used as a proxy, this approach is allowable and supported by evidence showing consistent performance gap patterns across adult and child populations. The identified gaps are addressable and do not diminish the importance of continued measurement.
Closing Care Gaps
Closing gaps in care coordination is essential to improving safety, outcomes, and equity for pediatric patients and their families. The Child CG-CAHPS Care Coordination composite identifies breakdowns that caregivers experience most directly, including lack of provider awareness of medical history and failures in follow-up on tests and imaging. These gaps disproportionately affect children with complex or chronic conditions and shift the burden of coordination onto families. While the developer did not explicitly address this optional domain, available performance data demonstrate variation across practices and less than optimal performance, indicating meaningful opportunity for improvement. These gaps are addressable through clearer accountability, closed-loop communication, and proactive follow-up workflows, supporting continued use of this measure.
Feasibility Assessment
The Child CG-CAHPS Care Coordination measure meets feasibility criteria. The survey is standardized, freely available, and does not require proprietary systems or integration into clinical workflows. Data collection occurs retrospectively and does not interfere with care delivery. Mail-based administration with phone follow-up is feasible at scale and supports representative sampling, including families with limited digital access. Item-level missing data are low and attributable to appropriate skip patterns. Established vendor support, analytic tools, and clear guidance from AHRQ further support consistent, sustainable implementation in real-world settings.
Scientific Acceptability
The reliability criterion is not fully met for this maintenance review because required accountable entity–level reliability testing was not conducted using child-specific CG-CAHPS data within the past five years. While the developer provides a rationale for extrapolating reliability findings from Adult CG-CAHPS data, differences in respondent context, age distribution, and caregiver perspective may influence reliability estimates. Prior reliability results referenced were based on data outside the required timeframe. These limitations indicate that, although the construct is stable, the specific reliability evidence required for maintenance endorsement is incomplete and should be addressed through updated, entity-level testing using appropriate data sources.
The validity criterion is partially met and addressable. The developer conducted required person-level and accountable entity–level validity testing and demonstrated acceptable factor loadings and model fit for the Care Coordination composite. Correlations with related composites and Rating of Provider support convergent validity. However, validity testing relied on adult CG-CAHPS data as a proxy for child caregiver responses, which may not fully represent the pediatric care context. In addition, hypotheses regarding mechanisms linking care coordination to outcomes were not fully articulated, and the impact of case-mix adjustment was not clearly demonstrated across entities. These gaps are addressable through clearer causal pathways, additional justification for extrapolation, and expanded analysis of confounding factors.
Use and Usability
The Care Coordination composite is currently used in multiple accountability and reporting programs, including CMS quality initiatives and state reporting efforts, demonstrating baseline usability. However, the submission provides limited guidance on how measured entities can meaningfully improve performance and does not sufficiently explain the lack of sustained improvement in scores over time. Strengthening this domain would include clearer, evidence-based actions practices can take to improve care coordination, such as closed-loop test result communication, defined follow-up accountability, and structured care coordination roles. Additionally, contextualizing stagnant or declining scores in light of increasing care complexity and fragmentation would enhance interpretability. These gaps are addressable and do not negate the measure’s current utility or relevance.
Summary
The Child CG-CAHPS Care Coordination composite measures an aspect of care that families experience directly and that has meaningful implications for safety, trust, and adherence to care plans. While the submission has areas that would benefit from strengthening—particularly in documenting child-specific reliability testing, clarifying mechanisms in the logic model, and expanding evidence-based guidance for improvement—these gaps are addressable and do not undermine the importance or relevance of the measure.
The measure demonstrates feasibility, is actively used in accountability programs, and shows persistent variation across practice sites, indicating continued opportunity for improvement. Care coordination failures disproportionately burden caregivers, especially those supporting children with complex needs, and capturing caregiver-reported experience remains essential to understanding real-world performance. Preserving inclusive survey administration methods is critical to ensuring representative patient and caregiver voice.
With targeted enhancements to reliability evidence, clearer articulation of provider actions, and strengthened usability guidance, this measure can continue to support meaningful quality improvement and accountability. Overall, the Care Coordination composite remains an important and appropriate component of patient experience measurement in ambulatory pediatric care.
CBE0005-9
Importance
This is extremely important given the broadening landscape of the healthcare spectrum, making sure the providers and their staff are coordinating care is mission critical for patient safety, to provide the best care, and improving outcomes.
Closing Care Gaps
I do think the complex care situations and conditions that affect children relative to the adult population makes this something that needs additional attention to.
Feasibility Assessment
Data collection seems fairly straight forward.
Scientific Acceptability
Agree with assessments below
Agree with assessments below
Use and Usability
Reviewing the staff assessment, I agree with that feedback.
Summary
Some aspects are not met but addressable, but there are clear gaps in other components of the measure that are not met. With focused improvements, the measure will be an improved tool going forward.
(No subject)
Importance
Question is unclear in that it does not specify how the provider communicated and seems to imply that the provider called on the phone. In the era of electronic messaging, question should clarify the means of communication that count (phone, message, in person etc)
Closing Care Gaps
-
Feasibility Assessment
-
Scientific Acceptability
-
-
Use and Usability
-
Summary
-
Nothing
Importance
No Comments
Closing Care Gaps
No Comments
Feasibility Assessment
No Comments
Scientific Acceptability
No Comments
No Comments
Use and Usability
No Comments
Summary
No Comments
High Importance, Case-Mix Adjustment Approach Acceptable
Importance
Disagree with staff assessment. The wide distribution of scores overall suggests that patients have identified clear issues with the health care system reflected in their responses here, and several studies are provided supporting the link between overall patient experience with care coordination and clinical quality outcomes.
Closing Care Gaps
N/A
Feasibility Assessment
Agree with staff assessment. Measure is clearly feasible and widely used.
Scientific Acceptability
Disagree with staff assessment. I do not anticipate issues with reliability of the measure score if it was re-ran with case mix adjusted figures, as the developer provided evidence that case mix adjustment has only a modest effect on the measure score, and in any case is an optional “add-on” to the existing measure score calculation. A mitigation strategy is presented for low reliability entities.
Disagree with staff assessment. I do not find that the issues presented with the case mix adjustment specific to validity are sufficient to affect the rating of the criterion, as these are optional adjustments and largely under the discretion of the individual adjuster.
Use and Usability
Disagree with staff assessment. Although there is little explanation provided for changes in performance, the changes in performance are slight and could be consistent with a positive upward drift.
Summary
Although the submission could be strengthened in some areas, the specific weaknesses are not sufficient to threaten the continued endorsement of this measure. This measure is a rare source of patient-reported data about the health care system, and reflects the performance of entities that are increasingly critical in guiding the course of health care in the United States, as health plans assume ever greater levels of control over provisioning care for their members.
Not Met
Importance
I agree with the staff that use of Adult CAHPS data for child data is a real problem, especially considering the period over which NCQA has been using it. It seems to me that the issue will have implications across all areas of review.
Closing Care Gaps
not addressed
Feasibility Assessment
Given that the developer was unable to provide Child CAHPS data, perhaps this measure is not feasible. If it was, logic suggests people would use it and data would be available. The developer needs to provide data to support feasibility.
Scientific Acceptability
I am in complete concurrence with the staff assessment.
I am in complete concurrence with the staff assessment.
Use and Usability
I am in complete concurrence with the staff assessment.
Summary
Considering how long this measure has been in use, the developers ought to be able to produce measure data to support importance, feasibility, acceptability, and use and usability. They do not. This measure is not acceptable in its current form.
(No subject)
Importance
-
Closing Care Gaps
-
Feasibility Assessment
-
Scientific Acceptability
-
-
Use and Usability
-
Summary
-
Home Child CG-CAHPS Survey – Rating of the Provider
Importance
-
Closing Care Gaps
Needs to be more parent centric if they are the one responding to survey
Survey was a 4 point Lickert name schedule -- which if it was scored average - there was no average to report
Feasibility Assessment
-
Scientific Acceptability
-
-
Use and Usability
-
Summary
Not helpful
Support
Importance
No additional comments.
Closing Care Gaps
Optional item not addressed by measure steward.
Feasibility Assessment
No additional comments.
Scientific Acceptability
Measure developer can address the items noted in staff preliminary assessment.
Measure developer can address the items noted in staff preliminary assessment.
Use and Usability
No additional comments
Summary
No additional comments.
summary
Importance
.
Closing Care Gaps
.
Feasibility Assessment
.
Scientific Acceptability
.
.
Use and Usability
.
Summary
agree that patient experience in the health care setting influences likelihood of patient participation in their own healthcare needs and outcomes. Care coordination is an important aspect of meeting healthcare goals however this is not within absolute control of the clinic or provider. Other factors- where the patient lives, insurance, availability/access of other procedures/providers, etc- also influence outcomes and the overall rating of this measure.
Public Comments
Verbal Comment Shared During December 10 Listening Session
This is Traci Petrino. I am a director with the Pennsylvania Clinical Network and the ACO. Practices that we have in our network are saying that it seems like the patients are confused when it says “provider.” Could there be more clarification around providers, nurse practitioners? They [patients] think it's just an MD or the DO. Is this an opportunity to maybe change that verbiage a little bit? It [this question] does apply to all of them [the measures] because I think there's a lot of confusion, they think it's just the doctor, and they may be seeing a certified nurse practitioner, so I think it's kind of relevant to all of them.
Response to Traci Petrino
Cognitive testing with patients demonstrated that participants appropriately understood the term “provider.” The term “provider” was intentionally selected to include not only physicians (MDs, Dos) but also nurse practitioners. In addition, the CG-CAHPS Survey instructions further clarify who “this provider” refers to. The first question asks respondents to confirm their specific provider’s name and explains that all subsequent questions will refer to that person as “this provider.” Here is the wording from Question 1:
“1. A health care provider can care for patients in person, by phone, or by video. Our records show that you got care from the provider named below in the last 6 months.
Name of provider label goes here
Is that right? (Yes/No).
The questions in this survey will refer to the provider named in Question 1 as “this provider.” As you answer these questions, please think of the in-person, phone, and video visits you had with that person in the last 6 months.”
If vendors or practice sites omit or alter these instructions, clarity may be lost. However, when the survey wording is followed, the term “provider” is defined for respondents and they are responding about the specific individual (MD, DO, NP, etc.) that they got care from.
0005 Consumer Assessment of Healthcare Providers and Systems (CA
The American Medical Association (AMA) appreciates the opportunity to comment on this survey and its associated composite measures. We are extremely concerned over the lack of any performance scores and testing provided for the child survey and composite measures and extrapolation of the adult data to this version. The child survey and composite measures are currently in use, and it is not clear why the developer was unable to access these data for this submission. Without information on the degree to which the child version continues to demonstrate a gap in care and remains reliable and valid, we do not believe that endorsement should be maintained.
In addition, it is our understanding that measures that undergo endorsement maintenance should provide testing at the measure score level for any of the composite measures but none of this information is included in the submission. We believe that it is important for health plans, physicians, patients and caregivers, and others to know how the measures perform, particularly since they are used in many accountability programs and this information has previously been submitted for review. In addition, one of the composite measures for the adult version produced Cronbach’s alpha internal consistency reliability results below 0.7, which is less than desirable, and we do not know if results for the child version would be comparable.
Given the lack of any data and testing for the child version, the omission of testing at the measure score level, and lower Cronbach’s alpha results for some of the composite measures, we do not believe that the minimum endorsement criteria have been met and endorsement should be reconsidered until these concerns are addressed.
Response to AMA
While we know that the Child CG-CAHPS Survey is in use, we did not have access to recent child data other than the child data we included in the 2019 maintenance endorsement, which we also shared in the submission as an additional reference point.
As outlined in Appendix H in the CBE Endorsement and Maintenance Guidebook (https://p4qm.org/sites/default/files/2025-11/Del-3-6-Endorsement-and-Ma…), when access to pediatric data are limited, measure developers may submit comparable data for an adult population. The measure developers met with CBE staff to discuss the acceptability of the extrapolation approach and were informed that it would be acceptable. Therefore, extrapolating child CG-CAHPS results based on the adult CG-CAHPS results was an approved approach. As additional support for extrapolation, both the Adult and Child surveys have almost identical wording and capture experience of care from an adult perspective — either directly from the patient (Adult CG-CAHPS) or the parent/guardian (Child CG-CAHPS). Additionally, summary results from the 2019 maintenance endorsement, based on 2016-2017 data when both Child and Adult data were available, show similar patterns or results for scientific acceptability across the two populations. This alignment supports the appropriateness of extrapolating adult data in the absence of more recent child data.
Regarding the internal consistency reliability falling below 0.70 for one measure, this measure includes only two (2) items. Reliability is influenced by both item correlation and the number of items; a measure with a greater number of items would likely have higher reliability but would also increase respondent burden. In addition, internal consistency reliability is only one of the criteria for determining scientific acceptability, and the measure meets other required criteria, so it is important to examine the totality of the evidence rather than a single metric in isolation.