When this measure comes back for maintenance (3 years), the developer should: Provide entity-level empirical testing of reliability and validity using child survey data to support the extrapolation argument.
The Consumer Assessment of Healthcare Providers and Systems (CAHPS) Clinician & Group Survey 3.1 (CG-CAHPS) is a standardized survey instrument that asks patients to report on their experiences with primary or specialty care received from providers and their staff in ambulatory care settings over the preceding 6 months. CG-CAHPS Survey Version 1.0 was endorsed by NQF in July 2007 (NQF #0005) and version 2.0 received maintenance endorsement in early 2015. Version 3.0 was released in July 2015 and was last endorsed in 2019. The 3.1 version of the survey updates the 3.0 version to prompt respondents to consider in-person, phone, and video visits when they answer the questions and to report which type(s) of visits they had. The survey is part of the CAHPS family of patient experience surveys and is available at https://www.ahrq.gov/cahps/surveys-guidance/cg/index.html
The Adult CG-CAHPS Survey 3.1 is administered to patients aged 18 and over who had at least one visit to a selected provider during the past 6 months. The survey has 32 questions including one overall rating of the provider and 12 questions used to create four (4) composite measures.
The Child CG-CAHPS Survey 3.1 is administered to the parents or guardians of pediatric patients under the age of 18. The survey has 40 questions including one overall rating of the provider and 11 questions used to create four (4) composite measures.
The composite measures are:
- Getting Timely Appointments, Care, and Information (Access)
- How Well Providers Communicate With Patients (Provider Communication)
- Helpful, Courteous, and Respectful Office Staff (Office Staff)
- Providers’ Use of Information to Coordinate Patient Care (Care Coordination)
The survey also has a single-item rating measure:
- Rating of Provider
A guidance document is available on the AHRQ CAHPS website (https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/surveys-guidance…) which explains how to administer the survey including how to choose the sample, maintain confidentiality, collect the data, track returned questionnaires, and calculate the response rate.
The Child CG-CAHPS Survey Rating of the Provider measure assesses the overall rating of the child’s provider on a scale of 0 to 10, with 0 being the worst and 10 being the best.
Measure Specs
General Information
The CAHPS Clinician & Group (CG-CAHPS) Survey assesses aspects of health care delivery that are important to patients and for which patients are the best or only source of information (Cleary, 2016). Further, the CG-CAHPS survey focuses on patient-centered care (Cleary, Edgman-Levitan, 1997; Cleary, 2016), one of the six central aims identified by the Institute of Medicine for improving the health care system (IOM, 2001). A focus on the patient experience has the potential to enhance clinical outcomes, improve patient safety, and reduce unnecessary medical services. Moreover, assessing patient experience through surveys that include data on the demographic characteristics of respondents, such as race and ethnicity, can help identify the extent to which positive experiences are distributed equitably across patients (Haviland et al., 2003).
Use of this measure will benefit both patients, providers, and medical groups:
- Patients can use information from the measures to help make more informed choices about which practice or medical group to use.
- Medical groups and their providers can use data from the surveys for quality improvement initiatives and incentives.
- Researchers can use data files from the surveys to help answer important health services research questions.
Patient experience encompasses the range of interactions that patients have with the healthcare system. The terms patient satisfaction and patient experience are often used interchangeably, but they are not the same. CAHPS surveys ask patients to report on what they experienced in a healthcare encounter—for example, whether something happened or how often it happened. Patient experience of care surveys provide actionable, objective information for quality improvement. Patient satisfaction surveys, on the other hand, use ratings to measure whether a patient’s expectations about a health encounter were met.
The CG-CAHPS Survey is a standardized survey instrument for measuring patients’ perspectives on their care. The survey is generally administered annually to patients who have received care in the last 6 months.
References
Cleary, PD, Edgman-Levitan, S. (1997). Health care quality. Incorporating consumer perspectives. JAMA. 278(19), 1608-12.
Cleary, PD. (2016). Evolving concepts of patient-centered care and the assessment of patient care experiences; optimism and opposition. J Health Pol, Policy & Law, 41 (4), 675-696.
Haviland, M. et al, (2003). Do health care ratings differ by race or ethnicity? Joint Commission Journal on Quality and Safety. 29(3), 134-145.
Institute of Medicine. (2001). Crossing the Quality Chasm: A New Health System for the 21st Century. Accessible at https://nap.nationalacademies.org/catalog/10027/crossing-the-quality-ch….
The Child CG-CAHPS Survey Rating of the Provider captures the overall impression of the provider. The provider often serves as the main point of contact for care coordination, preventive services, and chronic condition management. This rating reflects the perceived quality of care provided by the provider. Improving performance on this measure fosters stronger relationships with the provider, which enhances continuity of care, promotes adherence to treatment plans, and reduces unnecessary hospitalizations and emergency visits. These improvements lead to better health outcomes, lower healthcare costs, and increased satisfaction.
The data were obtained from the Centers for Medicare & Medicaid Services (CMS) Center for Medicare and Medicaid Innovation (CMMI) Primary Care First (PCF) Patient Experience of Care Survey (PECS), which is based on the Adult CG-CAHPS Survey and adds Patient-Centered Medical Home Supplemental items (https://pcfpecs.org/Survey-and-Protocols) The survey is administered annually via mail with phone follow-up. The 2023 survey data included 2,490 practice sites and 238,204 respondents. For testing and performance scores, practice sites with less than 10 surveys were excluded. After the restriction, the number of sites is 2,486.
Numerator
The response options for this measure range from 0 to 10, where higher scores indicate more positive ratings. AHRQ calculates the score for this item using a top box scoring method. For the top box or “top proportion” score, the numerator is the number of respondents who answered “9” or “10.”
The rating question is as follows:
- Q25: Using any number from 0 to 10, where 0 is the worst provider possible and 10 is the best provider possible, what number would you use to rate this provider?
No additional detail, refer to 1.14.
Denominator
The measure’s denominator is the number of respondents to the survey item. The target population for the survey is patients who have had at least one visit to the selected provider/practice in the target 6-month time frame. This time frame is also known as the look back period. The sampling frame is a person-level list and not a visit-level list.
No additional detail, refer to 1.15.
Exclusions
Individuals are excluded from the denominator if:
- They are deceased.
- They did not receive care from the participating medical group or practice in the last 6 months.
The denominator is the total number of surveys fielded minus the total number of ineligible surveys. The total number of ineligible surveys includes sample cases deemed ineligible: does not meet the eligible population criteria (refer to Section 1.15b). No other cases are excluded from the denominator, but cases are excluded from the denominator of the measure if they did not answer any item within the measure.
Measure Calculation
Respondents report on their experiences accessing and using care over the past 6 months.
AHRQ calculates CG-CAHPS Survey this composite measure scores using a top box scoring method.
Composite Measures:
There are two basic steps to calculating a composite measure score for a practice site:
- Calculate the proportion of responses in the top box or most positive response category for each question in a composite measure.
- Calculate the mean or average top box scores across all questions in a composite measure to determine the composite measure’s top box score.
For the top box or “top proportion” score, the numerator is the number of respondents who answered that they “Always” received the desired care or service for a given measure. For example, if 400 out of 1,000 total respondents answered “Always” to a composite measure item, the top box score for that item would be 40 percent [i.e., (400 ÷ 1,000)*100 = 40%].
Lower proportion and middle proportion composite measure scores can also be calculated following the same methodology where the lower proportion is the proportion answering “Never” or “Sometimes” and the middle proportion is the proportion answering “Usually”.
Rating Item:
For the rating item, the numerator for the top box score is the number of respondents who responded 9 or 10 on the 0-10 scale (where 10 is the “Best” and 0 is the “Worst”). For example, if 600 out of 1,000 total respondents answered “9” or “10” to a rating item, the top box score for that item would be 60 percent [i.e., (600 ÷ 1,000)*100 = 60%].
Lower proportion and middle proportion rating scores can also be calculated where the lower proportion is the proportion answering 0-6 on the 0-10 scale and the middle proportion is the proportion answering 7 or 8.
Users may also choose to calculate mean scores or linearized mean scores.
Note the survey includes screener items to identify respondents who meet the target process for each measure, such as whether the individual needed care right away. Measures are only calculated using respondents who experienced a particular service/process.
Users can also case-mix adjust the results for characteristics such as respondent age, education, general health status, and mental health status. The CAHPS Analysis Program—often referred to as the CAHPS Macro—is a free program written in SAS (version 6.0 or later) that enables survey users to case-mix adjust their data. The program also generates a distribution of survey results for each of the measures, calculates the mean score for both individual survey items and composite measures, and indicates whether an entity’s scores are statistically different from the average. The results presented in these analyses are based on unadjusted top box scores unless otherwise noted.
More information about the calculation of proportion scores and mean scores can be found in these documents:
- Instructions for Preparing Data for Analysis: https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/surveys-guidance…
- Instructions for Analyzing Data from CAHPS Survey: https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/surveys-guidance…
The measure is not stratified.
Users should choose a data collection protocol that maximizes the survey response rate at an acceptable cost. Some sponsors, as well as researchers conducting field tests, have found that the mail with telephone follow-up method is most effective or email with mail or telephone follow-up.
AHRQ provides protocols for collecting responses though users can adapt it to meet their needs. The protocols include mail only, telephone only, mail with phone follow-up, or email (web) with mail or phone follow-up. AHRQ provides detailed instructions for these different protocols in the “Guidelines for Using the CAHPS Clinician & Group Survey” document survey available on the AHRQ CAHPS website: https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/surveys-guidance…;
There is no minimum response rate requirement on the CG-CAHPS Survey. The CAHPS consortium has found that higher response rates are achievable if users take steps to ensure the accuracy of the sample frame and carefully follow the recommended data collection protocol, including one or more attempts to follow up with non-respondents.
In its simplest form, the response rate is the total number of completed questionnaires divided by the total number of individuals selected for the sample. Calculating the response rate is helpful in determining a more accurate starting sample size for future survey administration. For the CG-CAHPS Survey, the goal is a response rate of at least 40 percent or at least 50 completed surveys per provider.
To calculate the response rate, use the following formula: Number of completed returned questionnaires divided by the total number of respondents selected minus the sum of deceased + ineligibles.
AHRQ makes the CG-CAHPS Survey available in English and Spanish.
The sample is drawn from a list of individuals (adults aged 18 and older, or children 17 and younger) who have received care from a given provider, practice site, or medical group during a six-month time period. The list is called a sample frame.
The source of sample information will vary by survey sponsor. The data to identify individual patients may be found in the records of medical practices or health systems.
Defining the Sample Frame: Eligibility Guidelines:
- The adult questionnaires include all adults 18 years or older.
- The child questionnaire includes all children 17 years or younger.
- Include only patients who have had at least one visit to the selected practice in the last 6 months. This time frame is also known as the look back period.
- To identify the sampling frame, use the anticipated start date of data collection to determine the reference period. For example, if your anticipated start date is September 1, 2026, include all those who have had at least one visit since March 1, 2026.
- The sampling frame is a person-level list and not a visit-level list. Therefore, patients should appear only once in the sampling frame regardless of how many visits they have had in the look back period.
- Draw the sample irrespective of reason for visit and duration of patient-provider relationship, so that the full range of patients is represented.
- Include all patients who meet the sampling criteria even if they are no longer receiving care from the practice, site/clinic or provider.
- Allow the sample frame to include multiple individuals from the same household, but the sample you draw should not have more than one person (adult or child) per household. In other words, the sample that is selected for data collection should be de-duplicated to ensure that only one person per household receives a survey. The final sample must contain only one respondent per household. Where a duplicate household is sampled, it is discarded and replaced by another random draw from the frame.
- All CAHPS survey items have been designed for the general population. Appropriate screening items are included for items targeted to assess a specific experience. In order to ensure that results are comparable to those produced by other sponsors and vendors, targeted sampling, such as selecting only patients with particular conditions or experiences, is not recommended. Targeted sampling should only be used to supplement the general population sample, if desired.
- In order to administer the survey, the name of the provider must be available, even if surveying at the site/clinic or practice level. If the sampling frame does not accurately identify the provider that the patient saw, select a larger sample to account for errors in connecting health care received to a specific provider. For example, errors can occur if administrative billing data are used for the sampling frame and visits with physician assistants or nurse practitioners are billed under the supervisory physician.
Calculating the Sample Size for the Adult (Child) Questionnaire
The sample size varies depending on whether sampling is done at the level of the individual provider, practice, or medical group. In general, to produce statistically valid comparisons, the sample needs to be large enough to yield 50 completed questionnaires per provider or 300 completed questionnaires per medical group. The recommended sample size when sampling at the practice level depends on the number of providers at each site. The document Guidelines for Using the CAHPS Clinician & Group Survey has a table for recommended sample sizes based on the number of providers, and it ranges from 50 to 300 completed questionnaires. (https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/surveys-guidance…)
Data are not reported for any item or measure with fewer than 10 valid responses and practice sites with fewer than 10 responses were not included. AHRQ recommends that there needs to be approximately 50 completed questionnaires per provider to have a sufficient number of responses for results to be statistically reliable.
Proxy Respondents
The CG- CAHPS Survey Plan does allow for proxy respondents for mail and web-based mode. At the end of the survey, there is an item that asks “Did someone help you complete this survey?” If the answer is Yes, the follow-up question is “How did that person help you?” and they are to mark one or more of these response items:
- Read the questions to me
- Wrote down the answers I gave
- Answered the questions for me
- Translated the questions into my language
- Helped in some other way
However, these the last two questions of the core questionnaire are not included in telephone scripts because telephone interviews should not be conducted with proxy respondents.
Supplemental Attachment
Point of Contact
CAHPS® is a registered trademark of the U.S. Department of Health and Human Services and managed by AHRQ
Karen Chaves
Rockville, MD
United States
Naomi Yount
Westat
Rockville, MD
United States
Importance
Evidence
The CG-CAHPS Survey measures key components of patient experience, such as provider communication and ease of access, that are consistent with patient-centered care. The CAHPS Surveys focus on aspects of care that consumers have identified as important and for which patients are the best or only source of information. Measuring patients’ perceptions of their healthcare experience is not just a means to improve services—it’s a recognition that the patient’s voice matters in and of itself. Listening to patients affirms their role as active participants in their care, and their insights are essential to truly understanding the quality and impact of healthcare delivery. In 2023, over 238,000 patients completed the CG-CAHPS Survey as part of the PCF PECS survey. It is possible that more practices are administering and using the CG-CAHPS survey beyond the practices participating in PCF. We reviewed the literature on the determinants of patient care experiences measured by CAHPS and their associations with other indicators of health care quality. CAHPS is also an actionable measure that helps clinicians and health plans target interventions that will improve the quality and patient-centeredness of care.
Review of the Evidence
Prior research has identified several features of healthcare delivery structure, including clinic accessibility, patient flow, and management, that are associated with patient experiences. Two major systematic reviews have examined the relationships among patient experience, clinical processes, and patient outcomes. A systematic review performed by researchers in the U.K. found that patient experience is favorably associated with adherence to recommended medications and treatments, preventive care such as screenings and immunizations, patient-reported health outcomes, clinical outcomes, reduced hospitalizations and primary care visits, and reduced adverse events (Doyle et al., 2013). Anhang Price et al. (2014) reviewed evidence on the association between patient experiences and other measures of health care quality in the U.S. They similarly found that better patient care experiences are associated with higher levels of adherence to recommended prevention and treatment processes, better clinical outcomes, and less health care utilization.
Health Care Process and Quality Improvement Influence on Patient Experience
Providers routinely use patient-reported measures such as CAHPS to guide quality improvement (QI) activities to improve their patients’ experience with care (Friedberg et al., 2011; Davies, Shaller, et al., 2013; Quigley et al., 2015).
For example, to improve scores for “how often the office staff were as helpful as you thought they should be”, Dean Clinic, a large integrated health care delivery system in Wisconsin, further surveyed their patients to ask them what “helpfulness” means to them and to ask how office staff could be more helpful. From this feedback, the Clinic learned about ways that the office staff can be more welcoming, friendly, and appreciative of patients. With input from both staff and management, Dean Clinic developed action plans to improve patient experience. The service department shadowed staff and provided feedback. To improve consistency in service across all sites, the Clinic developed an orientation for all new employees on customer service expectations. They also offered ongoing training in the form of service workshops, videos, and Webinars, as well as targeted interventions for the lowest scoring offices (AHRQ, 2013). Several quality improvement initiatives to improve provider communication and engagement include shadowing, coaching, and training (AHRQ, 2013; Hardee & Kasper, 2008; Quigley, Palimaru, et al., 2017). Lastly, Friedberg et al. (2011) found that physician groups that aim to improve access, communication with patients, and customer service do so by addressing office workflow, providing additional training for nonclinical staff, and adopting or enhancing an electronic health record.
Using data from the CAHPS survey and a newly installed electronic health record system, in 2015, leaders of Northeast Valley Health Corporation (Los Angeles County) pinpointed interventions including reallocating staff resources and not scheduling well-child visits first thing in the morning. System leaders found that when the first appointments of the day ran long, there was a cascading effect on the rest of the day's schedule. By 2015, NEVHC instituted changes systemwide. By 2017, total average cycle time was reduced from 82 minutes to 65 minutes; the proportion of patients with a cycle time under 60 minutes rose from 34 percent to 48 percent; and the proportion of patients seen within 15 minutes of appointment time rose from 38 percent to 47 percent. Case study found at https://www.ahrq.gov/news/newsroom/case-studies/201718.html
More examples of interventions to improve patient experience with primary and specialty care as measured by the CG-CAHPS Survey can be found in the CAHPS Improvement Guide available at https://www.ahrq.gov/cahps/quality-improvement/index.html. The Guide also includes information on analyzing survey results and identifying root causes of performance problems.
The Centers for Medicare and Medicaid Services Merit-based Incentive Payment System (MIPS) Quality Payment Program measures Medicare Part B providers in four performance categories to derive a score that could affect a provider's Medicare reimbursement positively or negatively starting at 4% in 2019 (based on 2017 performance). MIPS has a performance category called “Improvement Activities” that includes an inventory of activities that assess how a physician group can improve care processes, enhance patient engagement in care, and increase access to care. Several of these activities are geared to improve patient experience of care as measured by CG-CAHPS. A list of MIPS activities can be found at https://qpp.cms.gov/mips/improvement-activities.
Structure
Physician clinic hours of operation and availability for appointments have been found to predict patient experience in several studies. In a study with survey data from 61,839 patients of 1729 primary care physicians in California, system-level factors, such as belonging to a larger medical group and the physician’s zip code-based Primary Care Services Areas, explained between 28% to 48% of variation in patient care experience, with the highest proportion explained for the access to care composite (Rodriguez et al., 2009). Improving the infrastructure supporting certain aspects of care may have broad effects because system changes can influence multiple outcomes (Cleary, 2016).
“Expanded practice access” is a highly weighted CMS MIPS improvement activity. To improve the patient experience in access to care, practices may consider providing 24/7 access to clinicians, groups, or care teams for advice about urgent and emergent care (e.g., eligible clinician and care team access to medical record, cross-coverage with access to medical record, or protocol-driven nurse line with access to medical record). Access expansion can include one of the following: 1) expanded hours in evenings and weekends with access to the patient medical record (e.g., coordinate with small practices to provide alternate hour office visits and urgent care); 2) use of alternatives to increase access to care team by clinicians and groups, such as e-visits, phone visits, group visits, home visits and alternate locations (e.g., senior centers and assisted living centers); and/or 3) provision of same-day or next-day access to a consistent clinician, group or care team when needed for urgent care or transition management. Another example is implementation of “open access” scheduling, in which some physician time is always reserved for same-day appointments, to improve patient access to care (Murray & Tantau, 1998).
Health-related Patient Behavior and Disease Management
One dimension of the CG-CAHPS measure captures the patients’ perceptions of how well providers communicate with them. Better patient-provider communication promotes healthcare-related patient behaviors (Fuertes, Boylan, et al., 2009). A 2009 meta-analysis of 127 studies assessing the link between patient treatment adherence and physician-patient communication found a 19% higher risk of non-adherence among patients whose physician communicated poorly (Zolnierek and Dimatteo, 2009). Doyle’s (2013) meta-analysis showed positive associations between the quality of clinician-patient communications and adherence to medical treatment in 125 of 127 studies analyzed. Studies using the CAHPS measure have found that better provider communication is positively associated with adherence to hypoglycemic medications among diabetics (Ratanawongsa, Karter, et al., 2013), adherence to tamoxifen among breast cancer patients (Liu, Malin, et al., 2013), and higher rates of colorectal cancer screening among adults in the US (Carcaise-Edinboro and Bradley, 2008). Sequist and colleagues (2008) found that measures of patient experience, including doctor-patient communication, clinical team interactions, and health promotion support, were positively associated with some prevention and disease management clinical process measures in clinical practices and among individual clinicians.
Outcomes
Out of 40 evidence papers with outcome measures, Doyle’s (2013) meta- analysis found 29 studies with positive associations between patient experience and clinical outcomes, 11 with no associations, and none with negative associations. The lack of more evidence may be due to complexity between a patient’s illness level, their level of care, and their likelihood for a poor outcome such as mortality, morbidity or a readmission. Often, such associations have more than one plausible direction of causality. For example, clinicians may be especially attentive to the needs of sicker patients (Kahn et al., 2007) and patients near the end of life (Elliott, Haviland, et al., 2013; Xu et al., 2014).
Moreover, substantial evidence points to a positive association between various components of patient experience, such as good communication between clinicians and patients, and several important processes and outcomes. These include lower utilization of unnecessary healthcare services; better patient adherence to medical advice; better process of care measures for acute myocardial infarction (AMI), congestive heart failure, pneumonia and surgery; lower inpatient mortality among acute myocardial infarction (AMI) patients; lower infection rates (Anhang Price, et al., 2014); and better clinician and staff perceptions of patient safety culture (Sorra et al., 2012).
Utilization
Research suggests an association between better patient experiences and lower healthcare utilization. Children with asthma were less likely to visit the emergency department, make urgent office visits, or be hospitalized if their physicians had reviewed a long-term therapeutic plan with their parents (Clark, Cabana, et al., 2008). Among African Americans with Type 2 diabetes, those who reported that doctors or nurses usually listened carefully or spent enough time with them were significantly less likely to visit the emergency department in the 12 months following completion of a patient experience survey (Gary, Maiese, et al., 2005). Children whose parents report longer waits for primary care visits were more likely to visit the emergency department for non-urgent reasons than those who report waiting for less time (Brousseau, Bergholte, et al., 2004).
References
Agency for Healthcare Research and Quality (AHRQ). How Two Provider Groups Are Using the CAHPS® Clinician & Group Survey for Quality Improvement. Brief available at https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/quality-improvem….
Anhang Price, R, Elliott, MN, Zaslavsky, AM, Hays, RD, Lehrman, WG, Rybowski, L, Edgman-Levitan, S, Cleary, PD. (2014). Examining the role of patient experience surveys in measuring health care quality. Med Care Res Rev. 71(5), 522-54.
Brousseau, DC., Bergholte, J., et al. (2004). The effect of prior interactions with a primary care provider on nonurgent pediatric emergency department use. Archives of Pediatrics & Adolescent Medicine. 158(1), 78-82.
Carcaise-Edinboro, P. and Bradley, CJ. (2008). Influence of patient-provider communication on colorectal cancer screening. Medical Care. 46(7), 738-745.
Clark, NM., Cabana, MD., et al. (2008). The clinician-patient partnership paradigm: Outcomes associated with physician communication behavior. Clinical Pediatrics. 47(1), 49-57.
Cleary, PD. (2016). Evolving concepts of patient-centered care and the assessment of patient care experiences; optimism and opposition. J Health Pol, Policy & Law. 41(4), 675-696.
Davies, E, Shaller, D, Edgman-Levitan, S, Safran, DG, Oftedahl, G, Sakowski, J, Cleary, PD. (2008). Evaluating the use of a modified CAHPS survey to support improvements in patient-centered care: lessons from a quality improvement collaborative. Health Expect. 11(2), 160-76.
Doyle, C., Lennox, L., et al. (2013). A systematic review of evidence on the links between patient experience and clinical safety and effectiveness. BMJ Open. 3(1). http://bmjopen.bmj.com/content/3/1/e001570.full
Elliott, MN., Haviland, AM., et al. (2013). Care experiences of managed care Medicare enrollees near the end of life. Journal of the American Geriatrics Society 61(3), 407-412.
Friedberg, MV, SteelFisher, GK, Karp, M, and Schneider, EC. (2011). Physician groups’ use of data from patient experience surveys. J Gen Intern Med. 26(5), 498-504.
Fuertes, J N., Boylan, LS., et al. (2009). Behavioral indices in medical care outcome: The working alliance, adherence, and related factors. Journal of General Internal Medicine. 24(1), 80-85.
Gary, TL., Maiese, EM., et al. (2005). Patient satisfaction, preventive services, and emergency room use among African-Americans with type 2 diabetes. Disease Management. 8(6), 361-371.
Hardee, JT, Kasper, IK. (2008). A clinical communication strategy to enhance effectiveness and CAHPS scores: The ALERT model. Perm J. Summer. 12(3), 70-4.
Kahn, KL., Tisnado, DM., et al. (2007). Does ambulatory process of care predict health-related quality of life outcomes? Health Services Research. 42, 63-83.
Liu, Y., Malin, JL., et al. (2013). Adherence to adjuvant hormone therapy in low-income women with breast cancer: The role of provider-patient communication. Breast Cancer Research and Treatment. 137(3), 829-836.
Murray, M and Tantau, C. (1998). Must patients wait? Jt Comm J Qual Improv. 24(8), 423-5.
Quigley, DD, Mendel, PJ, Predmore, ZS, Chen, AY, Hays, RD. (2015). Use of CAHPS™ patient experience survey data as part of a patient-centered medical home quality improvement initiative. J Healthc Leadersh. 7, 41-54.
Quigley, DD., Palimuru, AI., Chen AY., & Hays, RD. (2017). Implementation of practice transformation: Patient experience according to practice leaders. Quality Management in Health Care. 26 (3), 140-151.
Ratanawongsa, N., Karter, AJ., et al. (2013). Communication and medication refill adherence: the Diabetes Study of Northern California. JAMA Internal Medicine. 173(3), 210-218.
Rodriguez, HP, Scoggins, JF, von Glahn, T, Zaslavsky, AM, Safran, DG. (2009). Attributing sources of variation in patients' experiences of ambulatory care. Med Care. 47(8), 835-41.
Sequist, TD., Schneider, EC., et al. (2008). Quality monitoring of physicians: linking patients' experiences of care to clinical quality and outcomes. Journal of General Internal Medicine. 23(11), 1784-1790.
Sorra, J, Khanna, K, Dyer, N, Mardon, R, Famolaro, T. (2012). Exploring relationships between patient safety culture and patients’ assessments of hospital care. Journal of Patient Safety. 8(3), 131–139.
Xu, X., Buta, E., Anhang Price, R., Elliott. NN., Hays, RD., & Cleary, PD. (2014). Methodological considerations when studying the association between patient-reported care experiences and mortality. Health Services Research. 50(4), 1146-61.
Zolnierek, KB. and Dimatteo, MR. (2009). Physician communication and patient adherence to treatment: a meta-analysis. Medical Care. 47(8), 826-834.
It is important that the Child CG-CAHPS Survey reflects the aspects of care that parents/guardians associate with high-quality healthcare for their child. Through a literature review, focus groups, a Technical Expert Panel, and numerous other development activities, the CAHPS Consortium found that an overall rating of the provider is an important component of ambulatory care. This measure asks the respondent to rate the provider on a scale from worst (0) to best (10). The overall rating of the provider captures a comprehensive view of the patient’s experience. The simple summary rating score can help make comparisons easier for parents/guardians of the child. The overall rating measure is associated with the other composite measures (Hays et al., 2018).
Although much of the research has focused on adult patients, providers often serve as the main point of contact for care coordination, preventive services, and chronic condition management for both adult and children. This rating reflects the perceived quality of care delivered by the provider and offers a comprehensive view of the patient’s, or parent/guardian’s, experience. As these behaviors are valued similarly across populations, the underlying response mechanism or how the questions in this measure are interpreted are also consistent between the Adult and Child survey respondent populations. This consistency supports the importance of an overall rating of provider interactions regardless of patient age.
References
Hays, RD, Mallett, JS, Haas, A, et al. (2018). Associations of CAHPS composites with global ratings of the doctor vary by medicare beneficiaries’ health status. Med Care. 56, 736–739.
Measure Impact
The CAHPS measures, developed to complement more technical quality measures, are measures for which the patients are the best or only source of information and/or perspective, such as the degree to which care felt patient-centered (Anhang Price et al., 2014). Several studies provide evidence that patients value the CAHPS measures and find them meaningful. For example, Safran et al. (2001) found that patients who reported the poorest-quality relationships with their physicians were three times more likely to voluntarily leave the physicians’ practice than patients with the highest-quality relationships.
Collins et al. (2017) found that a patient’s “most important CAHPS domain” varied across subgroups; racial and ethnic patient subgroups differentially valued various aspects of the care experience. To efficiently reduce disparities and improve quality, Collins et al recommend tailoring quality improvement programs to the factors most important to the racial, ethnic, and language mix of the patient population of interest. Quigley et al. (2014) found that the importance of provider communication varied significantly by practice specialty type, yet respectful treatment was consistently import across all specialties.
Patients also use information from patient experience measures to make decisions about their healthcare providers and plans. One study found that seeing publicly reported quality information was a determinant of choosing higher quality-rated health plans, although the weight given to quality information also depended on other features, such as cost and provider choice (Faber et al., 2009).
References
Anhang Price, R, Elliott, MN, Zaslavsky, AM, Hays, RD, Lehrman, WG, Rybowski, L, Edgman-Levitan, S, Cleary, PD. (2014) Examining the role of patient experience surveys in measuring health care quality. Med Care Res Rev 71(5), 522-54.
Collins, RL, Haas, A, Haviland, AM, Elliott, MN. (2017). What matters most to whom: Racial, ethnic, and language differences in the health care experiences most important to patients. Med Care. Nov;55(11), 940-947.
Faber, M, Bosch, M., Wollersheim, H, Leatherman, S, and Grol, R. (2009). Public reporting in health care: how do consumers use quality-of-care information? A systematic review. Med Care 47(1), 1-8.
Quigley, DD, Elliott, MN., Burkhart, Q, Farley, DO., Skootsky, SA., & Hays, RD. (2014). Specialties differ in which aspects of doctor communication predict overall physician ratings. Journal of General Internal Medicine, 29, 447-454.
Safran, DG, Montgomery, JE, Chang, H, Murphy, J, Rogers, WH. (2001). Switching doctors: predictors of voluntary disenrollment from a primary physician’s practice. J Fam Practice. 50(2), 130–6.
The Child CG-CAHPS questions focus on aspects of care for which the parent/guardian of the child is the best or only source of information and that reflect elements of care that are most meaningful to patients/parents/guardians. Published research indicates that individuals use information from patient experience measures to make decisions about their healthcare providers and plans. Patient experiences with care are also linked to their persistence with the provider. At the provider level, patients who reported the poorest-quality relationships with their physicians are three times more likely to voluntarily leave the physicians’ practice than patients with the highest-quality relationships (Safran et al., 2001).
The Rating of Provider measure is meaningful to patients/parents/guardians because it reflects their overall experience with the provider who often serves as their primary point of contact in the healthcare system. For many, the provider is someone they rely on for ongoing care, guidance, and support in managing their health. This measure captures the overall experience with the provider and helps support making decision about with whom to seek care. It also offers parents/guardians of the child a meaningful way to express concerns or positive experiences with their provider.
Reference
Safran, DG, Montgomery, JE, Chang H, Murphy, J, Rogers, WH. (2001). Switching doctors: Predictors of voluntary disenrollment from a primary physician’s practice. J Fam Practice. 50(2),130–6.
Performance Gap
Due to the lack of access to Child CG-CAHPS data, we are extrapolating data from Adult CG-CAHPS Survey. Both the Adult and Child surveys capture the experience of care from an adult perspective — either directly from the patient (Adult CG-CAHPS) or the parent/guardian (Child CG-CAHPS). Additionally, results from the 2019 measure endorsement, when both Child and Adult data were available, demonstrated that the outcomes related to performance gaps were consistent across the two populations. This strong alignment supports the appropriateness of extrapolating adult data in the absence of child data.
The analyses were based on data from CMMI’s Primary Care First (PCF) Patient Experience of Care (PEC) survey, which is a version of the AHRQ Adult CG-CAHPS 3.1 Survey. The 2023 survey was administered from October 3, 2023, through December 19, 2023, and includes 2,486 practice sites and 238,204 respondents.
To examine the performance gap over time, we also analyzed the 2022 survey data. The 2022 survey was administered from October 6, 2022, through December 19, 2022, and includes 2,801 practice sites and 241,521 respondents.
Deciles in the Performance Scores by Decile tables are based on the performance scores; and for the “mean performance score” row in the tables show the average unadjusted top box scores across practice sites. For cases where the performance score was tied across decile boundaries, all practice sites with that score were assigned to the same decile.
As shown in Table 1 and the attached Table 2.4a.5 for the Rating of Provider measure:
- the 2023 mean top box score = 80.3; number of measured entities = 2,486; and number of respondents was 232,292.
- the 2022 mean top box score = 79.4; number of measured entities = 2,801; and number of respondents was 234,582.
The mean top box score increased slightly (79.4% to 80.3%). The maximum score decreased slightly from 100.0% in 2022 to 98.8% in 2023. The scores are lower than ideal (100% is ideal), given that about 80% of respondents rated their provider as a 9 or 10 on a scale of 0 to 10. These results show that while there are small increases from 2022 to 2023, indicating that the performance gap may be decreasing over time, practices need to continue to aim to improve their provider ratings.
| Overall | Minimum | Decile_1 | Decile_2 | Decile_3 | Decile_4 | Decile_5 | Decile_6 | Decile_7 | Decile_8 | Decile_9 | Decile_10 | Maximum | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Mean Performance Score | 80.3 | 40.0 | 64.1 | 72.1 | 75.5 | 78.2 | 80.4 | 82.2 | 84.0 | 86.0 | 88.3 | 92.2 | 98.8 |
| N of Entities | 2,486 | 1 | 248 | 250 | 248 | 248 | 248 | 249 | 250 | 247 | 249 | 247 | 1 |
| N of Persons / Encounters / Episodes | 232,292 | 30 | 18,243 | 22,721 | 22,989 | 23,468 | 23,892 | 23,010 | 24,623 | 24,451 | 24,790 | 23,992 | 83 |
Care Gaps
Closing Care Gaps
Optional for Fall 2025.
Feasibility
Feasibility
The CG-CAHPS Survey is a standardized instrument designed to assess patient experience of care. As these patient-experience data are collected from patients, the structured data are not available in electronic sources outside of the data collection by the practice site or medical group.
The data are collected through a survey instrument that is administered directly to patients, not during care delivery. Surveys are generally mailed to the sampled patients, and those survey results can be entered into structured databases (e.g., Excel, SPSS, SAS). No proprietary platform is required to administer the survey. Though mixed-mode administration (i.e., mail and phone) is a viable strategy for the collection of CAHPS surveys, mail continues to be the most frequent mode for most CAHPS surveys. Users then create electronic databases of results after receipt of the completed hard copy survey through scanning or data entry. However, vendors may set up their database before data collection by populating the frame to assist in identifying nonresponse.
Traditionally, the rationale for not using electronic sources more broadly is that mail and telephone are the best ways to obtain representative samples of patients based on the contact information that is available for sampling and data collection. E-mail has been added as a mixed mode strategy for physician groups with reliable email addresses for their patient population.
Structured or unstructured fields. All items are structured and on a 4 -item Likert-type response option scale (1-4) or for the rating items on a 0-10 scale. All responses are numeric.
Electronic feasibility. CG-CAHPS Survey users can offer a web survey to respondents to complete the survey though that option should not be the only option as it may exclude patients who have limited or no access to the web and/or who do not have an email address to send an electronic version of the survey.
Missing data. Item level missing data is low on the CG-CAHPS Survey however, some items will have fewer response than others due to gate or filter questions. For example, if a respondent did not need urgent care in the last 6 months, they are skipped through items about their experiences with getting urgent care. As a result, some CG-CAHPS Survey items have higher percentages of missing data overall, but when skip patterns are considered, the percentages of inappropriate missing data are much lower (<5%).
Measure susceptibility to inaccuracies and ability to audit data: The CG-CAHPS Survey is self-reported perceptions or experiences with the care received and therefore cannot be assessed to determine if the results are accurate. The protocol for administering CG-CAHPS Survey is outlined by AHRQ and vendors normally collect the data in a standardized format. The data that has been collected can be audited to ensure no data entry errors have occurred, there are no out of range values, and skip patterns are followed.
Change to the Instrument. Since the instrument was last endorsed, there was only one change to the survey and that was to add text to the instrument to allow for respondents to include care that was done virtually (i.e., phone or by video) to account for changes in care delivery due to the COVID-19 pandemic. For example, instructions now included “by phone, or by video”: “The questions in this survey will refer to the provider named in Question 1 as “this provider.” As you answer these questions, please think of the in-person, phone, and video visits you had with that person in the last 6 months.” This change did not impact data structure or availability.
Data Collection Burden
For respondents: The survey takes approximately 15 minutes to complete, dependent on the individual.
For medical groups and practice sites: Survey sampling uses administrative enrollment data that is maintained by all medical groups/practice sites and easily accessible to produce a sampling frame. Practice sites/medical groups generally hire a survey vendor to administer, track, and analyze their survey data resulting in lower burden for the practice sites/medical groups.
Cost Considerations
The CG-CAHPS Survey is freely available for use with no proprietary fees.
The cost to hire a vendor varies based on the size of the medical group and desired number of completed surveys. AHRQ provides guidance for hiring a vendor and resources for finding a certified vendor (https://www.ahrq.gov/cahps/surveys-guidance/helpful-resources/hiring/in…).
Impact on Clinician Workflow
The CG-CAHPS Survey does not interfere with diagnostic thought processes or patient -physician interactions as the survey is retrospective after care has been given, not during the visit.
Potential Barriers and Mitigation Strategies
Achieving a desired response rate may be difficult for users. Phone is not optimal as the only mode of survey administration, but it is commonly used as a follow-up for CAHPS mail surveys. Phone follow-up can improve CAHPS response rates compared to mail-only (Burkhart et al., 2014; Fowler et al., 2002; Gallagher et al., 2005; Klein et al., 2011). A study of Medicare beneficiaries found that response rates continue to improve when up to 4 follow-up calls are made (Burkhart et al., 2014). In addition, phone follow-up calls help to achieve better representation of patients in terms of income, literacy/education, health status, age, gender, and race/ethnicity, above and beyond mail surveys alone (Tesler and Sorra, 2017). The CAHPS Consortium continues to conduct research to develop and test survey administration methods that can improve the efficiency of data collection, enhance response rates, and gather more information about the experiences of those segments of the patient population that are hard to reach through more traditional means. This research includes: 1) studies comparing the effect of administration modes on response rates, survey scores, and data collection costs (e.g., mode comparisons have included in-office distribution vs. mail; email vs. mail); 2) studies assessing the effect of survey length on response rates and survey scores; 3) studies examining the impact of incentives on response rates; and 4) studies comparing the effect of different survey formats and design on survey responses. AHRQ also provided a webinar on how to achieve higher response rates (https://www.ahrq.gov/cahps/news-and-events/events/webinar-011124.html).
Analysis and Reporting: AHRQ makes available many resources to assist with analysis and reporting. For instance, there is a free CAHPS Analysis Program which is written for SAS that enables survey users to conduct the analyses needed to produce valid comparisons of performance across similar health care organizations. Users can also review documentation on how to prepare data for analysis (https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/surveys-guidance…). Further, vendors usually conduct all analyses and reports.
References
Burkhart, Q, Haviland, A, Kallaur, P, et al. (2014). How much do additional mailings and telephone calls contribute to response rates in a survey of Medicare beneficiaries. Field Methods. 27(4):409-25.
Fowler, FJ, Gallagher, PM, Stringfellow, VL, et al. (2002). Using telephone interviews to reduce nonresponse bias to mail surveys of health plan members. Med Care 40(3):190-200.
Gallagher, PM, Fowler, FJ, Stringfellow, VL. (2005). The nature of nonresponse in a Medicaid survey: causes and consequences. J Off Stat 21(1):73-87.
Klein, DJ, Elliott, MN, Haviland, AM, et al. (2011). Understanding nonresponse to the 2007 Medicare CAHPS survey. Gerontologist. 51(6):843-55.
Tesler, R. and Sorra, J. CAHPS Survey Administration: What We Know and Potential Research Questions. (Prepared by Westat, Rockville, MD, under Contract No. HHSA 290201300003C). Rockville, MD: Agency for Healthcare Research and Quality: October 2017. AHRQ Publication No. 18-0002-EF. Accessible at https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/about-cahps/rese….
Most vendors have established methods for tracking the sample. The Consortium suggests setting up a system to track the returned surveys by the unique ID number that is assigned to each respondent in the sample. This ID number should be placed on every questionnaire that is mailed and/or on the call record of each telephone case.
To maintain respondent confidentiality, the tracking system should not contain any of the survey responses. The survey responses should be entered in a separate data file linked to the sample file by the unique ID number. (This system will generate the weekly progress reports that should be review closely.) Data should be stored securely—preferably on encrypted or password-protected systems—with access limited. If paper responses are used, they should be shredded following de-identified data entry.
The CG-CAHPS Survey data is therefore de-identified upon data collection with a focus on protecting the confidentiality of respondents. Vendors are trained on maintaining confidentiality.
The CG-CAHPS Survey has a long history of use dating back to 2007. The CG-CAHPS Survey has gone through two main revisions since that time, using field and psychometric testing conducted by multiple partners and other stakeholders to increase the scientific rigor and relevance of the survey and the usability of the data (for more development information refer to https://www.ahrq.gov/cahps/surveys-guidance/cg/about/Develop-CG-Surveys…)
Steps which have contributed to the content and design of the CG-CAHPS Survey over time have included:
- Literature review and review of existing measures
- Development and consultation with technical expert panels
- Focus groups with consumers
- Cognitive testing of survey questions to ensure they will be understood by respondents
- Field testing to assess the reliability of the survey results
- Cognitive testing of measure labels to ensure that survey results are communicated clearly to providers and the public
- Public comment
- On-going collaboration and harmonization with key partners and stakeholders
The CAHPS Consortium continues to conduct research to develop and test survey administration methods that can improve the efficiency of data collection, enhance response rates, and gather more information about the experiences of those segments of the patient population that have been hard to reach through more traditional means. This research includes: 1) studies comparing the effect of administration modes on response rates, survey scores, and data collection costs (e.g., mode comparisons have included in-office distribution vs. mail; email vs. mail); 2) studies assessing the effect of survey length on response rates and survey scores; 3) studies examining the impact of incentives on response rates; and 4) studies comparing the effect of different survey formats and design on survey responses.
To address data collection efficiency and to improve response rates, the CAHPS Consortium endorsed e-mail notification for web-based surveys as an additional mode of data collection. The CAHPS Consortium recommends a mixed mode that would have two e-mail reminders and a follow-up by mail or telephone to all who are in the survey sample. The follow-up to the entire sample is necessary to get a representative set of responses from a practice’s population, as not all patients may have e-mail.
Proprietary Information
Scientific Acceptability
Testing Data
The data used for these analyses are from the CMMI Primary Care First (PCF) Patient Experience of Care (PEC) Survey) which includes a combination of items from the Adult CG-CAHPS Survey as well as from the Patient-Centered Medical Home CAHPS Supplement. The survey is available on the CMMI PCF website: https://pcfpecs.org/Survey-and-Protocols.
For evidence of performance gap demonstrating persistent gaps over time, we also include top box statistics on the 2022 Adult CG-CAHPS Survey data administered as part of the PCF PEC survey.
The data used to support extrapolation of Adult data to Child data, is based the 2019 maintenance endorsement submission, when both Adult and Child data were available. Using data from surveys administered from January 2016- March 2017, the Child survey included 77 practice sites and over 12,000 responses and the Adult survey included 635 practice sites and over 110,000 respondents. To support extrapolating adult data in the absence of child data, we compared characteristics between respondents from the prior child and adult CG-CAHPS Surveys.
Data were included in the analysis if they had at least one reportable item from the CG-CAHPS survey.
Unless noted otherwise, the top box scores presented are unadjusted since the results are not being used to compare entities, but rather for descriptive and scientific acceptability purposes.
The 2023 survey data was collected between October 3, 2023, through December 19, 2023.
The 2022 survey data was collected between October 6, 2022, through December 19, 2022.
None.
The CG-CAHPS Survey is specified for both individual clinician and group/practice. However, we only have testing for the group or “practice site” because the clinician-level identification in the CG-CAHPS database is not available. The site-level measured entity is referred to as a “practice site.” The practice site is an outpatient facility in a specific location. Practice site level survey results are calculated across the respondents within a specific site.
The data included 2,490 practice sites. For testing and performance scores, practice sites with less than 10 surveys were excluded. After the restriction, the number of sites is 2,486.
The 2022 data used exclusively in the performance gap section to demonstrate persistent gaps over time included 2,806 practice sites. For testing and performance scores, practice sites with less than 10 surveys were excluded. After the restriction, the number of sites is 2,801.
Practices in the 2023 data come from 23 states and the District of Columbia, as shown in 5.1.3a available in the Supplemental 7.1 zip file.
Due to the lack of access to Child CG-CAHPS data, we present the data for Adult CG-CAHPS Survey measures which can be extrapolated to provide evidence for the Child CG-CAHPS Survey measures. The Adult CG-CAHPS Survey and Child CG-CAHPS Survey are equivalent with only minor wording changes (refer to the crosswalk between Adult and Child measures included with the instruments in Attachment 1.21). The wording changes directed attention to the experiences for the child rather than the respondent.
For example, on the Adult Survey one question is worded:
- “In the last 6 months, when you contacted this provider’s office to get an appointment for care you needed right away, how often did you get an appointment as soon as you needed?
On the Child Survey, the question is worded (bolded added to differentiate):
- “In the last 6 months, when you contacted this provider’s office to get an appointment for care your child needed right away, how often did you get an appointment as soon as your child needed?”
Both the Adult and Child survey capture the experience of care from an adult perspective — either directly from the patient (Adult CG-CAHPS) or the parent/guardian (Child CG-CAHPS). Both surveys have adult respondents; the child survey is completed by the child’s parent, relative, or legal guardian.
The data includes 238,204 adult respondents. For testing and performance scores, practice sites with less than 10 surveys were excluded. After the restriction (N=4 sites), the total number of respondents was 238,181, the average number of respondents per site was 96, ranging from 14 to 272 respondents per site.
Tables 5.1.4a through g, available in the Supplemental 7.1 zip file, show descriptive characteristics of the respondents (sex, race/ethnicity, age, self-reported health status, survey mode, and survey language). Practice sites had adult respondents that were predominantly white and non-Hispanic (79%) and older than 54 (83%). Over 40% of respondents had at least a 4-year college degree (41%). Most responded via mail (89%) compared to by phone (11%). Most respondents completed the survey in English (99%), and only 1% completed it in Spanish.
Both the Adult and Child survey capture the experience of care from an adult perspective — either directly from the patient (Adult CG-CAHPS) or the parent/guardian (Child CG-CAHPS).
Both surveys have adult respondents; the child survey is completed by the child’s parent, relative, or legal guardian.
In the prior maintenance endorsement in 2019, using data from surveys administered from January 2016- March 2017, we had access to data from 77 practice sites and over 12,000 responses for the Child survey and 635 practice sites and over 110,000 respondents for the Adult survey. To support extrapolating adult data in the absence of child data, we compared characteristics between respondents from the prior child and adult CG-CAHPS Surveys.
Respondent education distributions were similar across child and adult CAHPS surveys. For example, in the 2016-2017 CG-CAHPS data, 65% of respondents to the Adult Survey had at least some college and 76% of respondents to the child survey had at least some college. Additionally, 21% of respondents to the Adult survey and 25% of respondents to the Child survey had more than a 4-year college degree. The Adult survey respondents had a slightly higher percentage of respondents with lower levels of education (27% high school graduate or GED or less versus 16% for the Child survey). Overall, these results support the appropriateness of extrapolating adult data in the absence of child data.
Respondents to both the Adult and Child 2016-2017 CG-CAHPS Surveys tended to be female (over 50% for both), with more females responding to the Child Survey (82%) than the Adult Survey (53%).
Patient general and mental health status distributions differ between the child and adult CG-CAHPS surveys when examining the 2016-2017 CG-CAHPS child survey data. For the child survey, 78% of respondents reported the general health status of the child to be very good or excellent while only 43% of respondents to the adult survey reported their general health status as very good or excellent. Similarly, 77% of respondents reported the mental health status of the child to be very good or excellent compared with 62% of respondents to the adult CG-CAHPS survey reported their mental health status as very good or excellent.
More respondents of Child CG-CAHPS Survey were aged 25-44 than respondents of the Adult surveys (61% versus 14%).
Keeping these differences in mind, we extrapolate adult data in the absence of child data. Further, as shown in the other sections, the scientific acceptability results (reliability and validity) between Adult and Child tended to align in the past maintenance endorsement package, further justifying extrapolation. Additionally, both surveys have the same usability and goal of assessing and improving the quality of care by gathering patient or parent/legal guardian feedback on key aspects of patient experience.
Reliability
We estimated internal consistency reliability using the Cronbach’s coefficient alpha for each composite measure. A reliability of at least 0.70 is considered acceptable for group-level comparisons (Nunnally and Bernstein, 1994). For composites with more than two items, we show the impact on Cronbach’s alpha of deleting one of the items from the composite. However, CAHPS scores are designed to evaluate care across units of care such as plans, physician groups, and hospitals, not individual patients.
For the Access, Provider Communication, and Office Staff composite measures, all items had less than 3% of records with missing values. For the Care Coordination composite measure, all items had less than 4% of records with missing values. We ran the Cronbach’s alpha excluding all missing data as well as with listwise deletion and the results were the same, except for the Access composite measure in which excluding all missing data increased Cronbach’s alpha by .01. Given the similarity of results, we have presented the Cronbach’s alpha values with the inclusion of cases with missing values (listwise deletion) in section 5.2.3.
Reference
Nunnally JC, Bernstein IH. (1994). Psychometric Theory. New York: McGraw Hill.
Table 5.2.3a (attached in 5.2.3a) shows the Cronbach’s alpha for each composite measure in the Adult CG-CAHPS Survey. For items within composite measures consisting of 3 or more items, the Cronbach’s alpha if the item were deleted is provided to determine if there was room for improving coefficient alpha by dropping an item. The table also shows the standardized correlation for the standardized item to total correlations.
All Cronbach’s alphas in the adult survey were above 0.70 except for the Care Coordination composite measure with a Cronbach’s alpha of 0.61. As shown in Table 5.2.3a, removal of any questions in this composite measure would not result in a higher Cronbach's alpha and Care Coordination is an important concept to patients in their experience of care. Further, all the item to total correlations were above 0.40. While Cronbach’s alpha fell below the conventional threshold for several composite measures, it is not the most critical metric in this context. More important is the reliability at the unit level (e.g., plan-level reliability), which better reflects the measure’s utility for quality improvement. Nonetheless, internal consistency remains a relevant consideration in health care, and the Consortium will keep this in mind when implementing future revisions of the instrument.
We assess reliability at the site level, which is the most relevant level of analysis for publicly reported CAHPS measures (Hays & Arnold, 1986, pp. 144-145). Since CAHPS surveys are used to compare groups/units, site-level reliability (which is directly related to the standard error of measurement) is used to determine the number of responses needed to obtain reliable information (Hays, Shaul, et al., 1999). Site reliability, which partitions within- and between-site variance, was calculated from the ICC and the Spearman-Brown prophecy formula in SAS version 9.4. Higher levels of site reliability correspond to more accurate performance measurement and a better ability to distinguish performance among practices. Reliability is not calculated when only one site is included in the decile or max/min value, since there would be no between-site variation. For cases where the number of respondents was tied across decile boundaries, all practice sites with that number of respondents were assigned to the same decile.
Like internal consistency reliability (i.e., Cronbach’s alpha), values of 0.70 and higher are considered acceptable for site-level reliability (Nunnally and Bernstein, 1994) and group comparisons. For example, CMS does not report (labeled as “Not available”) any score with reliability below 0.60, as that is considered low reliability. CMS reports scores that meet the sample size threshold and for which reliability falls between 0.60 and 0.70 but flags these scores as having low reliability and alerts consumers to interpret such scores with caution. Scores with reliability 0.70 or greater are reported without comment. Reliabilities of 0.85 or higher, where possible, are appropriate for applications such as pay-for-performance or actions that reward or classify individual practices.
The CAHPS Consortium has reported the reliability of the CAHPS measures at the appropriate unit of comparison since the beginning of the project over 25 years ago and for measure development throughout the project (e.g., Hays, Martino, et al., 2014; Hays, Berman, et al., 2014; Price, Stucky, et al., 2018).
Due to the lack of access to Child CG-CAHPS data, we are extrapolating data from Adult CG-CAHPS Survey. Both the Adult and Child survey capture the experience of care from an adult perspective — either directly from the patient (Adult CG-CAHPS) or the parent/guardian (Child CG-CAHPS). Additionally, summary results from the 2019 maintenance endorsement, based on 2016-2017 data when both Child and Adult data were available, are shared because they demonstrate that the reliability outcomes were consistent across the two populations. This strong alignment supports the appropriateness of extrapolating adult data in the absence of child data.
For the Rating of Provider measure, fewer than 3% of respondents had missing values. We have presented the site reliability, excluding missing cases where data are missing.
References
Hays, R, Arnold, S. (1986). Patient and family satisfaction with care for the terminally ill. Hospice Journal, 7, 129-150.
Hays, R, Berman, L, Kanter, M, et al. (2014). Evaluating the psychometric properties of the CAHPS Patient-Centered Medical Home Survey. Clin Ther. 36(5), 689–696.e1.
Hays, RD, Martino, S, Brown, J, Cui, M, Cleary, P, Gaillot, S, Elliott, M. (2014). Evaluation of a care coordination measure for the Consumer Assessment of Healthcare Providers and System (CAHPS®) Medicare Survey. Medical Care Research and Review, 71, 192-202.
Hays, R, Shaul, J, William, V, et al. (1999). Psychometric Properties of the CAHPS™ 1.0 Survey measures. Medical Care. Volume 37(3) SUPPLEMENT, MS22-MS31.
Nunnally, JC, Bernstein, IH. (1994). Psychometric Theory. New York: McGraw Hill.
Price, RA, Stucky, B, Parast, L, Elliott, MN, Haas, A, Bradley, M, Teno, JM. (2018). Development of valid and reliable measures of patient and family experiences of hospice care for public reporting. Journal of Palliative Medicine, 21, 924-932.
The Rating of Provider measure’s overall site reliability is 0.81. The information in Table 2 provides overall and decile-level reliability. Deciles for this table are based on the number of respondents per site, and the “mean performance score” row in this table shows the mean top box scores averaged across practice sites.
As a reminder, we present results for the Adult CG-CAHPS Survey due to a lack of access to Child CG-CAHPS Survey data. Refer to Section 5.2.2 for more information.
For reference, results from the 2019 maintenance endorsement for the Rating of the Provider measure were:
- Adult Survey: Site reliability was 0.84, with an average of 167 respondents per site
- Child Survey: Site reliability was 0.82, with an average of 155 respondents per site
| | Overall | Minimum | Decile_1 | Decile_2 | Decile_3 | Decile_4 | Decile_5 | Decile_6 | Decile_7 | Decile_8 | Decile_9 | Decile_10 | Maximum |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Reliability | 0.807 | NA | 0.735 | 0.785 | 0.769 | 0.766 | 0.813 | 0.807 | 0.775 | 0.824 | 0.809 | 0.840 | NA |
| Mean Performance Score | 80.3 | 75.0 | 76.8 | 77.7 | 79.8 | 80.1 | 80.3 | 80.8 | 81.3 | 82.1 | 81.9 | 82.2 | 88.7 |
| N of Entities | 2,486 | 1 | 248 | 269 | 242 | 254 | 235 | 244 | 248 | 260 | 244 | 240 | 1 |
| N of Persons / Encounters / Episodes | 232,292 | 12 | 11,060 | 16,533 | 17,260 | 20,216 | 20,460 | 23,066 | 25,464 | 29,308 | 30,827 | 37,820 | 266 |
The site reliability assessment for the Rating of Provider measure indicates that overall and across all deciles, the measure maintains acceptable reliability (e.g., greater than 0.7) indicating the measure effectively detects systematic variation among sites relative to random variation.
Results from the 2019 measure maintenance endorsement submission showed similar reliability outcomes, where the measure maintained acceptable reliability overall (e.g., greater than 0.7), and demonstrated the reliability results were consistent across the Adult and Child surveys.
This strong alignment supports the appropriateness of extrapolating adult data in the absence of child data.
Validity
Several model fit indices were examined to determine how well the hypothesized factor structure, or composite measures fit the data, including chi-square divided by its degrees of freedom (𝝌𝟐/𝒅𝒇 ) (criteria: values less than 5.0; Schumacker & Lomax, 2004), comparative fit index (CFI) (criteria: values 0.95 or greater; Hu & Bentler, 1999), root mean square error of approximation (RMSEA) (criteria: values less than 0.06; Kline, 2005), and the standardized root mean square residual (SRMR) (criteria: values less than 0.08; Kenny, 2020).
We examined standardized factor loadings for each item on its respective composite measure. Factor loadings above 0.40 indicate that the item’s relationship to the composite measure is acceptable (Stevens, 2002).
References
Hu, L., & Bentler, PM. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55. http://dx.doi.org/10.1080/10705519909540118
Kenny, DA. (2020, June 5). Measuring model fit. Available at http://davidakenny.net/cm/fit.htm. Accessed October 2025.
Kline, RB. (2005). Principles and practice of structural equation modeling (2nd ed.) New York: The Guilford Press.
Schumacker R., & Lomax, R. (2004). A beginner’s guide to structural equation modeling
(2nd ed.). Lawrence Erlbaum.
Stevens, JP. (2002). Applied multivariate statistics for the social sciences (4th ed.). Mahwah, NJ: Lawrence Erlbaum
Tables 5.3.4a and 5.3.4b (attached in section 5.3.4a) show results for the model fit indices and standardized factor loadings for the adult dataset.
The Confirmatory Factor Analysis results for the Adult CG-CAHPS survey demonstrate strong model fit based on established criteria. The chi-square divided by degrees of freedom (χ²/df) was 127.28, exceeding the recommended threshold of less than 5.0, which is common in large samples due to the sensitivity of this index. However, the other fit indices all fall well within acceptable ranges: the Comparative Fit Index (CFI) was 0.98, surpassing the criterion of 0.95 or greater, indicating excellent model fit. The Root Mean Square Error of Approximation (RMSEA) was 0.05, meeting the standard of less than 0.06, and the Standardized Root Mean Square Residual (SRMR) was 0.04, comfortably below the threshold of 0.08.
The estimates for each standardized factor loading on the items in the composite measures assess convergent validity. All standardized factor loadings are at least 0.5, with the majority above 0.8, and all are statistically significant (p < 0.001), demonstrating the convergent validity of the measures.
These results support the hypothesized factor structure for the measures in the survey.
At the individual and site level, we examined the relationships between each composite measure and item’s top box score and the top box score for the provider rating measure using Spearman rank-order correlations. We expect the composite measures to be moderately to strongly related to the overall rating (Quigley et al., 2024).
We also examined Spearman rank-order intercorrelations among the composite measures to assess the extent to which they measure different constructs. As measures of patient experience, we expected the composite measures to be positively and significantly correlated. However, very large intercorrelations (e.g., > 0.80) suggest that the composite measures may not be sufficiently distinct to be considered separate measures (O’Brien, 2007).
One rule of thumb for correlations is:
0.10 is a small correlation
0.30 is a medium correlation and
0.50 is a large correlation.
For the Rating of the Provider measure, we expect a positive and medium to large relationship with the CG-CAHPS composite measures with the strongest relationship with Provider Communication and Care Coordination, since all these measures reflect the provider’s behavior, communication, and knowledge for the patient’s care.
Due to the lack of access to Child CG-CAHPS data, we are extrapolating data from Adult CG-CAHPS Survey. Both the Adult and Child survey capture the experience of care from an adult perspective — either directly from the patient (Adult CG-CAHPS) or the parent/guardian (Child CG-CAHPS).
The provider often serves as the main point of contact for care coordination, preventive services, and chronic condition management for both adult and child patient populations. This rating reflects the perceived quality of care provided by the provider. The overall rating of the provider captures a comprehensive view of the patient’s experience or that of the parent/guardian. In sum, the rating of the provider operates through similar mechanisms for adults and children because both populations rely on the provider as the central point for care coordination, preventive services, and chronic condition management. The impact of provider performance on patient experience is consistent across age groups. The underlying response mechanism is consistent, supporting extrapolation of findings between Adult and Child survey populations.
Additionally, summary results from the 2019 maintenance endorsement, based on 2016-2017 data when both Child and Adult data were available, are shared because they demonstrate that the validity outcomes were consistent across the two populations. This strong alignment supports the appropriateness of extrapolating adult data in the absence of child data.
Reference
Quigley, DD., Elliott, MN., Qureshi, N., Predmore, Z., Hays, RD. (2024). Associations of the Consumer Assessment of Healthcare Providers and Systems (CAHPS) Clinician and Group
Survey scores with interventions and site, provider, and patient factors: A systematic review of the evidence. Journal of Patient Experience, 11, 1-14.
O’Brien, RM. (2007). A caution regarding rules of thumb for variance inflation factors. Qual Quant. 41, 673–690.
The Rating of Provider measure has medium to large correlations with all four composite measures. The site-level correlations between Rating of Provider and the composite measures range from 0.43 (Office Staff) to 0.82 (Provider Communication). The individual-level correlations between Rating of Provider and the composite measures range from 0.31 (Access) to 0.58 (Provider Communication).
Please see attachment 5.3.4a for validity testing results, which includes the site-level and individual-level correlations for the Rating of Provider measure and the composite measure and items in the Adult CG-CAHPS Survey, all assessed using Spearman rank-order correlations.
As a reminder, we present results for the Adult CG-CAHPS Survey due to a lack of access to Child CG-CAHPS Survey data. Refer to Section 5.3.3 for more information.
For reference, results from the 2019 maintenance endorsement for the Rating of Provider measure showed similar patterns of statistically significant correlations. The correlations between the composite measures and the Rating of Provider measure were positive and statistically significant for the
- Adult survey (ranged from r = 0.42 to 0.72 at the site level and r = 0.26 to 0.51 at the individual level).
- Child survey (ranged from r = 0.35 to 0.66 at the site level and r = 0.23 to 0.50 at the individual level)
In this analysis, all correlations between the Rating of Provider measure and the composite measures fell within an acceptable medium to large range at the site and individual level (between 0.30 and 0.80), except for the correlation with the Provider Communication composite measure which had a correlation of 0.82 at the site-level, just above 0.80. This stronger correlation is expected given both measures focus on the experience with the provider. Overall, the composite measures were positively and significantly correlated with the Rating of Provider measure. The results support the distinctiveness of the Rating of the Provider measure as a measure of patient experience, while also confirming its meaningful relationship to broader aspects of care.
Results from the 2019 measure maintenance endorsement submission showed similar correlations and demonstrated the validity results were consistent for the Adult survey and across the Adult and Child surveys. The correlations were positive and statistically significant for the Child survey (ranged from r = 0.35 to 0.66 at the site level) and had similarly positive statistically significant correlations for the Adult survey (ranged from r = 0.42 to 0.72 at the site level).
This strong alignment supports the appropriateness of extrapolating adult data in the absence of child data.
Risk Adjustment
The Child CG-CAHPS Survey results are not required to be risk adjusted by users. However, survey users, including public reporting entities, may voluntarily choose to adjust the data to account for patient/respondent case-mix differences if comparing practices. Guidance for this process is available in two key documents: “Preparing Data from CAHPS Surveys for Analysis” (available at https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/surveys-guidance…) and “Instructions for Analyzing Data from CAHPS Surveys” dated June 2025 (available at https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/surveys-guidance…). These resources provide instructions for coding the adjuster variables, imputing missing data for the adjusters, and incorporating adjustments in analyses using the CAHPS Analysis Program in SAS. The selection of adjuster variables and the calculation of risk-adjusted scores are user-defined. Users must also decide whether to impute missing data for the adjusters using each adjuster’s entity-level mean.
The CAHPS Analysis Program is a set of free SAS programs that enable survey users to conduct risk adjustment. The programs with test modules are available for download at https://www.ahrq.gov/cahps/surveys-guidance/helpful-resources/analysis/….
The CAHPS Analysis Program adjusts the data for case mix, generates a distribution of survey results for each of the measures, calculates the average score for both individual survey items and composite measures, and indicates whether an entity’s scores are statistically different from the average. AHRQ’s CAHPS Consortium developed the CAHPS Analysis Program to work with all CAHPS surveys. It is updated periodically to add functionality, produce additional output types, and correct or debug issues with previous versions.
This section describes the rationale for case-mix adjustment that is not required but that CAHPS users may elect to use. The standard methodology used is case-mix adjustment via regression in a linear model. Without an adjustment, differences in CAHPS scores between entities could be due to case-mix differences rather than true differences in quality.
The current CAHPS Analysis Program suggests adjusting for the child’s general health status, the child’s mental health status, respondent’s age, and respondent’s education. Studies have found that patient and consumer survey responses about experiences and satisfaction with healthcare correlate with personal characteristics like general health, mental health/depression, education, and age (Simon et al., 2009; Rahmqvist and Bara, 2010; Zaslavsky et al. 2001; Martino et al. 2011; Eselius et al., 2008; Elliott et al. 2009).
Health status and age are two respondent characteristics frequently associated with reports of the quality of medical care. People in worse health tend to report lower patient experience satisfaction and more problems with care perhaps because sicker patients have more complex health care needs and may tend to report more problems with coordination or communication. Older patients tend to report greater satisfaction, better patient experience, and fewer problems than younger patients, although this association is usually not as strong as that between health status and ratings (Hatfield and Zaslavsky, 2017; Eselius et al., 2008).
Education is self-reported by respondents who take the CAHPS surveys. Studies have shown that more educated respondents report more problems, perhaps because they have higher expectations rather than because they receive lower quality of care (Sofaer and Firminger, 2005). However, in a multivariate analysis using Medicare Advantage CAHPS data, Hatfield and Zaslavsky (2017) found that education had less influence on CAHPS dimension scores than self-reported general and mental health.
Different CAHPS surveys adjust for different variables, and the variables included here are not the only adjustment factors, even for the practice setting. It should be noted that race/ethnicity is not typically included as case-mix adjuster variables as doing so may mask disparities in care. CAHPS data can also be adjusted for other factors such as survey administration mode (Peipert et al., 2017). For example, a study by Drake and colleagues (2014) found that telephone respondents gave more positive responses than mail respondents did. The AHRQ CG-CAHPS Database did not adjust for survey mode.
The AHRQ CG-CAHPS Database adjusted only for respondent age, respondent education, the child’s general health status, and the child’s mental health status.
References
Drake, KM, Hargraves, JL, Lloyd, S, Gallagher, PM, Cleary, PD. (2014). The effect of response scale, administration mode, and format on responses to the CAHPS Clinician and Group Survey. Health Serv Res. 49(4), 1387-99. doi: 10.1111/1475-6773.12160.
Elliott, MN, Zaslavsky, AM, Goldstein, E, Lehrman, W, Hambarsoomians, K, Beckett, MK, Giordano, L. (2009). Effects of survey mode, patient mix, and nonresponse on CAHPS hospital survey scores. Health Serv Res. 44(2 Pt 1), 501-18. doi: 10.1111/j.1475-6773.2008.00914.x.
Eselius, LL, Cleary, PD, Zaslavsky, AM, Huskamp, HA, Busch, SH. (2008). Case-mix adjustment of consumer reports about managed behavioral health care and health plans. Health Research and Educational Trust. 43(6), 2014-2032.
Hatfield, LA, Zaslavsky, AM. (2017). Implications of variation in the relationships between beneficiary characteristics and Medicare Advantage CAHPS measures. Health Serv Res. 52(4), 1310-1329.
Martino, SC., Elliott, MN., Kanouse, DE., Farley, DO., Burkhart, Q., Hays., RD.(2011). Depression and the health care experiences of Medicare beneficiaries. Health Services Research. 46(6pt1), 1883–904.
Peipert, JD, Brown, JA, Cui, M, Hays, RD (2017). Differences in mail and telephone responses to the CAHPS In-Center Hemodialysis Survey. Ann Clin Nephrol. 1(1), 1.
Rahmqvist, M, Bara, AC. (2010). Patient characteristics and quality dimensions related to patient satisfaction. International Journal for Quality in Health Care. 22(2), 86–92.
Simon, GC, Rutter, M, Crosier, J, Scott, BH, Operskalski, Ludman E. (2009). Are comparisons of consumer satisfaction with providers biased by nonresponse or case-mix differences? Psychiatric Services. 60(1), 67–73.
Sofaer, S, Firminger, K. (2005). Patient perceptions of the quality of health services.
Annual Review of Public Health. 26, 513–59.
Zaslavsky, A, Zaborski, L, Ding, L, Shaul, JA, Cioffi, MJ, Cleary, PD. (2001). Adjusting performance measures to ensure equitable plan comparisons. Health Care Financing Review. 22(3), 109–26.
Tables 5.4.3a through 5.4.3c in Attachment 5.4.3 show descriptive characteristics for the risk/case mix variables tested: age, general health status, mental health status, and education. Practice sites had adult respondents who were predominantly 55 or older (83%). Forty-one percent (41%) of respondents reported having at least a 4-year college degree. Forty-one percent (41%) of respondents reported being in excellent or very good general health, and 54% of respondents reported being in excellent or very good mental health.
As a reminder, we present results for the Adult CG-CAHPS Survey due to a lack of access to Child CG-CAHPS Survey data. Both the Adult and Child survey capture the experience of care from an adult perspective — either directly from the patient (Adult CG-CAHPS) or the parent/guardian (Child CG-CAHPS).
Case-mix adjustment was conducted using the AHRQ CAHPS Analysis Program (https://www.ahrq.gov/cahps/surveys-guidance/helpful-resources/analysis/…). In the CAHPS Analysis Program, we set the program to impute the site mean if data were missing for that variable to avoid losing observations because of missing data. This is generally acceptable because, the size of the adjustment and the amount of missing data on adjusters are typically small. The case-mix adjusted scores are based on regression analyses and the results include case-mix adjusted coefficients. For detailed information on case-mix adjustment refer to page 46 of “Instructions for Analyzing CAHPS Data” document (https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/surveys-guidance…).
To quantify the effect of case-mix adjustment on the ranking of practices, using data from the Adult CG-CAHPS Survey we calculate the Pearson product moment correlation coefficient, Kendall Tau correlation coefficient, and the maximum difference between the adjusted and unadjusted practice mean scores, as shown in Attachment 5.4.5a. The adjustment factors include age, education, mental health status, and general health status. Appropriate case-mix adjusters are patient and respondent-level variables that are not under the control of the provider (“exogenous”) but result in different scores even when the quality of care is the same. For example, parent/guardian education and age are not under the control of the provider.
Eselius and colleagues (2008) published results of an analysis of case-mix adjustments for the CAHPS Health Plan Survey, a survey with similar composite measures to the CG-CAHPS Survey about access to care, provider communication and customer service. After selecting appropriate adjusters based on explanatory power in separate linear regression models, the authors determined the impact of case-mix adjustment on their sample health plans. Specifically, they examined the size of the adjustments and the extent to which adjustments impacted the ranking of health plans. Case-mix adjustments had only modest effects on health plan ratings and rankings. The authors also found that mental health status was a strong predictor of patient-reported experience. For this purpose, this question was added to CG-CAHPS: “In general, how would you rate your overall mental or emotional health?”
The Pearson product-moment correlation coefficient is widely used and understood. It assesses the linear association between the adjusted and unadjusted scores and ranges between -1 to 1. Because the ranking of scores is often important in public reports of CAHPS results, we also calculate Kendall’s Tau (Kendall rank correlation coefficient). Tau is the correlation between the rank orders of the adjusted and unadjusted scores. The Kendall Tau statistic also has a range of -1 to +1, so that it has a range comparable to other correlation coefficients. Tau can be interpreted as the percentage of pairs of units (e.g., practice sites) that switched ordering because of case-mix adjustment [100*(1-Tau)/2].
We also calculate the maximum absolute value difference between adjusted and unadjusted mean scores among practices.
Reference
Eselius, LL., Cleary, PD., Zaslavsky, AM., Huskamp, HA., & Busch, SH. (2008). Case-mix adjustment of consumer reports about managed behavioral health care and health plans. Health Research and Educational Trust. 43(6), 2014-2032.
The selection of case-mix adjusters is based on prior research across multiple datasets of the CG-CAHPS Database and is consistent with how scores are calculated. Results are found in Attachment 5.4.4a and 5.4.5a.
All adjusters were statistically significant in the regression models for this measure, except for education level 4-year college graduate.
The adjusters resulted in very similar top box scores, with the Pearson correlation of unadjusted and adjusted mean scores being close to 1, at 0.98 and the Kendall Tau correlation at 0.88. Further, the difference in mean scores for this measure was 0.28. This suggests that adjusting for case-mix can help level the playing field for this measure.
The final model includes four adjusters: respondent age, respondent education, child’s general health status, and child’s mental health status.
Use & Usability
Use
Provides patient experience, clinical quality and total cost of care ratings for medical groups in 44 CA counties. The patient experience ratings come from the Patient Assessment Survey (PAS) based on CG-CAHPS for adult and child practices. The program is reportedly being sunset in 2025 (https://www.pbgh.org/program/patient-assessment-survey/).
A total of 204 medical groups are included in the 44 California counties. Only 14 counties in the state do not have any medical groups reported. Number of patients: Information not available.
Medical group practice level of analyses, outpatient primary care practices
Through MIPS, clinicians can earn performance-based payment adjustments for services they provide to Medicare patients. The CG-CAHPS for MIPS survey is an optional quality measure that groups participating in MIPS can administer to their adult population.
MIPS is a national program that includes most physicians and group practices in the U.S who provide services to Medicare patients. The CAHPS sample frame includes all patients from a practice-supplied roster from groups participating in MIPS. Number of patients: Information not available.
Group practice level
Medicare Shared Savings Program ACOs collaborate to give coordinated high-quality care to people with Medicare. They are required to report the CAHPS for MIPS Survey, which is a version of the Adult CG-CAHPS Survey.
Over 470 ACOs are participating in the Shared Savings Program for Performance Year 2025. ACOs are located in almost all states and the District of Columbia. Number of patients: Information not available.
Medicare Shared Savings Program Accountable Care Organizations (ACOs)
NCQA recognizes clinicians and practices in key areas of performance. The recognition program includes optional reporting for practices that use the Adult CG-CAHPS survey with PCMH items.
The NCQA PCMH Recognition program is the most widely adopted PCMH evaluation program in the country. More than 13,000 practices (with more than 67,000 clinicians) are Recognized by NCQA. Number of patients: Information not available.
Group practice level
Primary Care First was a voluntary alternative payment model (through December 2025) that rewarded value and quality by offering an innovative payment structure to support the delivery of advanced primary care and administered a version of the Adult CG-CAHPS survey.
26 US regions and approximately 1,700 participant practices. Number of patients: Information not available.
Group practice level
UCLA Health, a large health system in southern California, administers the CAHPS Clinician & Group Survey on an ongoing basis to adult patients and the parents or guardians of pediatric patients (12 and younger) (Adult and Child). The organization has developed a set of data displays to manage and convey the tremendous amount of information they collect for quality improvement purposes.
Southern California as part of the University of California system. They collect data from over 1300 reporting physicians, 279 reporting offices, and 28 reporting clinical departments. Number of patients: Information not available.
Physician and office/clinic level
Usability
Actions to Improve Patient Experience
CAHPS® surveys play an important role as a quality improvement (QI) tool for healthcare organizations that use the standardized data to:
- Identify relative strengths and weaknesses in their performance.
- Determine where they need to improve.
- Track their progress over time.
AHRQ has made available a CAHPS Ambulatory Care Improvement Guide which is a comprehensive resource for health plans, medical groups, and other providers seeking to improve their performance in the domains of patient experience measured by CAHPS surveys. AHRQ also has created a short video to help improve patient experience https://www.ahrq.gov/cahps/quality-improvement/index.html#:~:text=CAHPS….
The steps are:
- Compare CAHPS survey scores to other health care organizations to determine how the plan is doing in comparison to others.
- Examine how CAHPS scores are changing over time.
- Identify priorities based on these comparisons
- Confirm these priorities based on other sources of information (e.g., patient complaints, patient comments)
- Find out what is actually happening with patients and why.
- Brainstorm with staff to determine the best strategies for improvement.
In addition, AHRQ held a research meeting in 2020 to discuss how to improve patient experience and provided summaries of the presentations: https://www.ahrq.gov/cahps/news-and-events/events/2020-meeting-summary…
Difficulty in Increasing Response Rates
Users are also provided advice for improving response rates:
- Improve initial contact rates by making sure that addresses and phone numbers are current and accurate (e.g., identify sources of up-to-date sample information, run a sample file through a national change-of-address database, send a sample to a phone number look-up vendor).
- Use all available tracking methods (e.g., Lexis-Nexis, Internet database services and directories).
- Improve contact rates after data collection has begun (e.g., increase maximum number of calls, ensure that calls take place at different day and evening times over a period of days, mail second reminders, use experienced and well-trained interviewers).
- Consider using a mixed-mode protocol. In field tests, the combined approach was more likely to achieve a desired response rate than did one mode alone.
- Train interviewers on how to deal with gatekeepers.
- Train interviewers on refusal aversion/conversion techniques.
AHRQ has several resources for practice sites to improve performance on the CAHPS Surveys. They created the CAHPS Ambulatory Care Improvement Guide, which is a comprehensive resource for organizations seeking to improve their performance: https://www.ahrq.gov/cahps/quality-improvement/improvement-guide/improv…. There are also case studies and webcasts that share insights and best practices in improving patient experience with care: https://www.ahrq.gov/cahps/surveys-guidance/hp/improve/index.html.
To improve on Rating of the Provider, practices can encourage providers to treat patients/parents/guardians with empathy and respect, make eye contact, take the time to listen carefully, and ask if all questions have been addressed. Further, they can ensure they have reviewed the patient’s medical history during the visit and have staff that are coordinating their care. Essentially, all ways listed for improving the other CG-CAHPS composite measures should help improve the rating of the provider.
To support usability, the extrapolated adult data were evaluated for applicability to child measures. Both the Adult and Child survey capture the experience of care from an adult perspective — either directly from the patient (Adult CG-CAHPS) or the parent/guardian (Child CG-CAHPS). The adult endpoint capturing “the overall rating of the provider” translates directly to pediatric populations, as parents/guardians can reliably report on this experience for their children. This alignment demonstrates that the adult-derived measure is feasible for implementation in child populations and that results from the adult survey data can be extrapolated for the child survey results in the absence of child data.
Throughout the development process, the CAHPS Consortium has incorporated the data or input from these various sources in an incremental process of revision and refinement to develop measurement that is more precise and to produce survey data that would better meet the information needs of consumers and other stakeholders. Between versions 1.0 and 2.0 of Child CG-CAHPS, pediatric experts felt that the Child version of the CG-CAHPS Survey would benefit from a more comprehensive measurement of development and prevention. The CAHPS Consortium worked with the American Academy of Pediatrics and other key stakeholders to develop 11 new items that address development and prevention. These items were grouped into new two composite measures for the Child Survey version 2.0 (Gallagher et al., 2009).
The CG-CAHPS Survey was initially developed with the standard CAHPS 4-point response scale. The field test data that was used for the initial endorsement consisted of several testing sites in the US. One of the larger field tests was conducted in Massachusetts as part of the Massachusetts Health Quality Partners (MHQP) statewide surveying initiative. At that time, MHQP used a 6-point response scale instead of the standard CAHPS 4-point scale. Based on that evidence, the CG-CAHPS Survey was endorsed with a 6-point response scale. Affirmed by significant user feedback, additional testing was conducted to add to the other field test data to confirm the properties and function of the standard CAHPS 4-point response scale and the CG-CAHPS Survey was updated to the 4-point response scale so that results could be aligned across other CAHPS surveys (e.g., CAHPS Health Plan Survey, CAHPS Hospital Survey (HCAHPS)). Drake et al. (2014) examined how different response scales affect responses to the CG-CAHPS survey among 6,500 patients. They found that compared to the 4-category response options surveys, respondents to the 6-category response options surveys had 41 percent more missing items. There were no significant differences between the 4-category and 6-category response option surveys’ average composite measure score or provider-level reliability.
For the 3.0 version, the survey switched from a 12-month recall period to a 6-month recall period to improve accuracy of recall and focus on more recent visits.
The CAHPS Consortium hears user feedback during research studies and development. Users can contact the CAHPS Database team with questions or comments by phone at 888-808-7108 or email at [email protected]. The CAHPS consortium also solicits feedback via focus groups with patients in developing survey content and design. When changes are proposed to the survey, the changes also often go through a public comment period and those comments are summarized and posted on the AHRQ site (e.g., https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/surveys-guidance…)
References
Drake, KM, Hargraves, JL, Lloyd, S, Gallagher, PM, Cleary, PD. (2014). The effect of response scale, administration mode, and format on responses to the CAHPS Clinician and Group Survey. Health Serv Res. 49(4), 1387-99.
Gallagher, P, Ding, L, Ham, HP, Schor, EL, Hays, RD, Cleary, PD. (2009). Development of a new patient-based measure of pediatric ambulatory care. Pediatrics. 124(5), 1348-54.
No additional detail.
In late 2009, version 2.0 addressed the fact that many people receive care in the ambulatory setting from non-physician providers such as nurse practitioners and physicians´ assistants. This change to “this provider” was in response to requests from the medical community for a survey instrument that would allow patients to report on their experiences with all their health care practitioners.
The items designed to identify chronic conditions were moved from the core survey version 2.0 to a supplemental item set.
After extensive testing and receiving feedback from users, with the release of CG-CAHPS Version 2.0, the CAHPS consortium endorsed e-mail notification for web-based surveys as an additional mode of data collection. The CAHPS consortium recommends a mixed mode that would have two e-mail reminders and a follow-up by mail or telephone to all who are surveyed.
Based on the stakeholder and user feedback obtained through public comment, technical expert review, and further testing, the following key changes were made in the release of the CG-CAHPS 3.0 version:
- One instrument, in contrast to the three instruments available for the 2.0 version.
- Use of a 6-month reference time period rather than a 12-month reference period.
- New and modified composite measures:
- New composite measure for "Care Coordination."
- Modified the composite measure for "Access."
- Modified the composite measure for "Communication."
- A modified Patient-Centered Medical Home Item Set.
- Shift of development and prevention items from the core Child Survey to the Patient-Centered Medical Home Item Set.
- Overall reduced length.
For the 3.1 version, with the COVID-19 pandemic changing how some health care was delivered (e.g., video or phone rather than in-person), the instrument was updated again to change instructions and gate question wording to include these types of visits.
No additional detail.
Due to the lack of access to Child CG-CAHPS data, we are extrapolating data from Adult CG-CAHPS Survey. Both the Adult and Child survey capture the experience of care from an adult perspective — either directly from the patient (Adult CG-CAHPS) or the parent/guardian (Child CG-CAHPS). To examine improvement, we used two years of data from the PCF-PEC Survey and then, for reference, we also refer to trends in Adult CG-CAHPS Survey data provided in the 2019 AHRQ CG-CAHPS Survey Chartbook (AHRQ’s reporting of Child data ended in 2016). The PCF-PEC Survey includes the items on the core CG-CAHPS Survey and includes Patient-Centered Medical Home (PCMH) and other supplemental items. The results from the PCF-PEC Survey and the CG-CAHPS Survey are comparable because they include the same core items and most sites (81%) in the 2019 AHRQ CG-CAHPS Survey Chartbook also included the PCMH supplemental items. Further, Beckett et al. (2025) found that adding supplemental items did not affect CAHPS scores.
Rating of Provider has shown minimal improvement between 2022 and 2023, increasing only slightly from 79.4% to 80.3%. In comparison, the 2019 AHRQ CG-CAHPS Chartbook reported an average of 84% in 2015, decreasing to 79% in 2019 (https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/cahps-database/2…). This trend suggests a gradual decrease in performance from 2015 to 2022, with only modest gains in 2023. These results underscore a need for focused efforts to enhance the overall patient care experience.
Reference
Beckett, MK, Hambarsoomian, K, Brown, J, Cleary, PD, Abel, GA, Giordano, L, Elliott, MN. (2025). Supplemental items reduce HCAHPS response rates, but response rates do not affect HCAHPS scores: A randomized experiment, Medical Care, 63(10), 764-770. DOI: https://doi.org/10.1097/MLR.0000000000002197.
No unexpected findings.
No unexpected findings.
Comments
Staff Preliminary Assessment
CBE #0005-10 Staff Preliminary Assessment
Importance
Strengths
- In the absence of Child CG-CAHPS data, the developer extrapolated from Adult CG-CAHPS data to provide support for the pediatric measures, and justified this extrapolation by citing construct equivalence (both instruments capture the care experience via adult respondents) and cross-population consistency in prior performance gap results across child and adult populations. The Endorsement & Maintenance Guidebook (see Appendix H in v.3.2) allows for the extrapolation of adult data for pediatric-focused measures, provided certain requirements are met for Importance, Validity, and Use & Usability.
If adult CG-CAHPS respondents are deemed acceptable proxies for a parent in a parent-child dyad, then data from the 2023 Primary Care First Patient Experience Survey, in which CG-CAHPS is embedded, show a performance gap, with decile ranges from 64.1% to 92.2% for the Rating of Provider score, indicating variation in measure performance and less than optimal performance across the target population.
Limitations
- The developer used adult CG-CAHPS data as a proxy for parent responses to the Child CG-CAHPS survey. That is, the CG-CAHPS adult may have been, or not been, a parent with a child receiving health care services. Performance gaps are based on adult data rather than parent data.
The logic model provided does not clearly articulate the relationships between inputs, activities, and outcomes for the Rating of Provider measure. For example, actions cited within the logic model are patient-centric, but do not include actions that can be taken by providers to improve their ratings. Adding such information would strengthen the logic model.
In terms of meaningfulness of the Rating of Provider measure to patients, the developer provides narrative content about how the measure can be useful to providers and to patients. However, no empiric evidence is cited and no input from parents or pediatric patients, for this endorsement cycle, was included.
The developer cited one article from 2018 that indicated rating of ones provider is associated with other composites. However, the other composites were not included in the submission.
Rationale
- This maintenance measure is rated as 'Not Met But Addressable' for importance due to insufficient evidence that the Rating of Provider measure is meaningful to parents of pediatric patients. The submission could be strengthened by engaging parents directly to elicit feedback on the meaningfulness of the Rating of Provider measure and/or citing empiric evidence within the past 5 years that indicates provider rating is a meaningful concept for parents and contributes to their overall experience of care.
The logic model could be strengthened by adding provider actions that can improve provider ratings by patients.
There is at least moderate confidence that the business case is adequate, i.e., the anticipated impacts of the measure on patient experience justify the use of the measure, due to the demonstrated performance gap.
Closing Care Gaps
The developer did not address this optional domain.
Feasibility Assessment
Strengths
- The developer indicated that the only change in the Adult CG CAHPS instrument was the addition of language to expand the context for questions to include virtual and telehealth visits.
The AHRQ website contains the documentation on CG CAHPS administration, confidentiality, scoring, and other topics.
There are no fees, licensing, or other requirements to use any aspect of the Adult CG CAHPS instrument or the resulting composites.
Limitations
- The feasibility domain can be strengthened by providing the median cost of vendor engagement to administer CAHPS (or a similar metric).
The developer's did not use Child CG CAHPS data from parents/guardians or children receiving care. This may point to limitations in data availability for quality improvement or reporting programs.
Rationale
- This maintenance measure meets the criteria for 'Met' due to availability of data and low degree of missingness, clear and implementable data collection strategies, and transparent handling of patient confidentiality, burden, licensing, and fees. These factors collectively ensure that the measure can be implemented effectively and sustainably in a real-world healthcare setting.
Scientific Acceptability
Strengths
- The developer gives a rationale for extrapolating the reliability results from the Adult CG-CAHPS survey to the Child CG-CAHPS survey. The rationale includes: the Adult and Child surveys are equivalent with only minor wording changes; the Adult and Child surveys both capture experience of care from an adult perspective – as a patient in the Adult survey and as a parent/guardian in the Child survey; and similar respondent education distributions across surveys.
Limitations
- The developer did not perform the required accountable entity-level reliability testing for this maintenance measure.
The developer stated they did not have access to the Child CG-CAHPS data.
The developer refers to reliability testing data from the 2019 maintenance endorsement submission that used survey data collected from January 2016 to March 2017 which is not within the past 5 years.
Although the respondents to the Child and Adult CG-CAHPS surveys are all adults, the Child CG-CAHPS survey respondents are experiencing the health care provided as a parent/guardian of a patient rather than as a patient for Adult CG-CAHPS survey respondents. This difference could lead to different assessments of health care provided and influence reliability of the measure.
The Child CG-CAHPS survey respondents are much younger on average than Adult CG-CAHPS survey respondents (61% of Child survey respondents were aged 25-44 while 15% of Adult survey respondents were aged 25-44).
The Child CG-CAHPS survey respondents tended to be female more often (82%) than for Adult CG-CAHPS survey respondents (53%).
Rationale
- This maintenance measure is rated as ‘Not Met’ for reliability because the required reliability testing was not performed.
Strengths
- The developer performed the required validity testing for this maintenance measure, namely, they conducted accountable entity-level (“measure score”) validity testing at the clinician group level. The data source used for validity analysis was Primary Care First (PCF) Patient Experience of Care (PEC) Survey administered to adults in October through December of 2023. Validity analyses included 2,486 practices in 23 states.
The developer conducted empirical validity testing at the accountable entity level using Spearman’s rank-order correlation on unadjusted top box scores.
The developer conducted empirical validity testing at the accountable entity level using Spearman’s rank-order correlation on unadjusted top box scores. The developer hypothesized that Rating of the Provider would be positively and significantly correlated with the four composites, positing that each are key components of the care experience, but not so highly correlated as to suggest the measures are not distinct, citing rho > 0.80 as an example of a very large correlation. Provider Rating was significantly, positively correlated at the clinician group level with the composites Access (rho = 0.49) and Office Staff (rho = 0.43), in line with hypotheses. The developer used data from the PFC-PEC, a survey of adults, to extrapolate results for the child version of the survey, and they justified the extrapolation by noting the similarity in the survey questions in the child and adult versions of the survey, and the similarity between child and adult validity testing results reported in the 2019 submission.
The developer conducted statistical case-mix adjustment, selecting case-mix indicators that are present at the start of care and have a significant correlation with the outcome.
Limitations
- The data from the PFC-PEC survey of adults may not adequately represent the population to whom the child survey is administered, in either distribution and range of practice types or characteristics of respondents (e.g., a child's guardian).
With respect to entity-level validity testing, while the developer has indicated that quality improvement activities can affect more than one IDM, their hypothesis regarding the mechanisms involved and the degree to which these mechanisms are shared between IDMs is not clearly supported by the evidence review and logic model provided. In the absence of an external gold standard against which to validate at least one of the IDMs, this submission would be strengthened by additional support guiding development of hypotheses about expected magnitudes of each correlation. For example, it is unclear from the submission whether mechanisms for improving children's experience of care are similar to those for adults.
The developer used unadjusted scores for validity testing when use of adjusted scores might help rule out known sources of confounding.
The developer states that case-mix adjustment of the measure is optional by the user, and does not provide guidance or supporting rationale stating when adjustment is or is not appropriate. The developer did not provide evidence demonstrating variation in the prevalence of case mix factors across accountable entities. The statistical testing results provided by the developer do not reflect the impact of adjustment on providers at the high or low extremes of the case mix.
Rationale
- This maintenance measure is rated as ‘Not Met But Addressable’ for validity because the accountable-entity validity testing results partially support an inference of validity for the measure, suggesting that the measure somewhat accurately reflects performance on patient experience of care and can distinguish good from poor performance to a limited extent. This submission would be strengthened by explicitly ruling in mechanisms and ruling out confounders for the effect of quality of care on survey respondents' Ratings of Provider, and additional information to support the extrapolation from data on adults' experience of care to children's experience of care.
The developer employed a statistical case-mix adjustment approach, utilizing a conceptual model designed to account for demographic case-mix factors. Variation in the prevalence of case-mix indicators across different entities was not shown and the model testing results provided do not reflect whether case-mix differences are being appropriately accounted for.
Use and Usability
Strengths
- The developer noted that users can contact the CAHPS Database team with questions or comments by phone at 888-808-7108 or email at [email protected]. Also, when changes are proposed to CAHPS surveys, requests for public comment is posted on the AHRQ website.
The developer does provides some, albeit limited, information about how measured entities can improve their Rating of Provider measure score: empathy, respect, listening, and preparedness for appointments.
The Rating of Provider measure is currently used in the California Health Care Quality Report Cards program, CMS Merit-based Incentive Payment System (MIPS) under the Quality Payment Program, the Medicare Shared Savings Program Accountable Care Organizations (ACOs), and the CMMI Primary Care First Model.
Limitations
- The developer reports the Rating of Provider scores from 2022 (79.4%) to 2023 (80.3%). The Rating of Provider composite scores declined initially and have remained flat in more recent years. The scores from 2015, 2019, and 2023 were 84%, 79%, and 80% respectively. The developer does not provide an explanation for the lack of improvement over time.
Rationale
- This maintenance measure is rated 'Not Met But Addressable' for Use and Usability. The measure is actively used in at least one accountability application and the developer provides a small number of actions that providers can take to improve their Rating of Provider composite scores. However, the submission can be strengthened by including evidence-based actions with citations and an explanation of the lack of improvement in Rating of Provider composite scores from 2015 to 2023.
Committee Independent Review
Approve
Importance
It is an important measure to gauge providers and the experience they provide, given positive experiences are likely to lead to more adherence and follow through with treatment plans, while negative experience likely leads to less compliance and engagement on the patient side.
Closing Care Gaps
Information gleaned can be used to close gaps and improve care delivery interactions with patients.
Feasibility Assessment
As long as there is a multi-modal approach to gather the information, then I think this is feasible.
Scientific Acceptability
Believe this is met
Believe this is met
Use and Usability
The rating seems relatively straight forward and actionable information.
Summary
Approve
Continued Endorsement of Child CG-CAHPS Rating of the Provider
Importance
The Rating of the Provider measure reflects an aspect of care that is intuitively meaningful to parents and guardians, as the provider is often the central figure in a child’s care experience. Available adult CG-CAHPS data demonstrate meaningful performance variation, suggesting opportunity for improvement. However, the evidence base supporting importance for the pediatric population relies primarily on extrapolation from adult data rather than direct parent-reported input.
While extrapolation is permitted under the Endorsement & Maintenance Guidebook, the submission would be strengthened by incorporating recent empirical evidence or direct engagement with parents to confirm that overall provider ratings meaningfully reflect their pediatric care experience. Additionally, the logic model would benefit from clearer articulation of provider-level actions that can influence Rating of Provider scores. These gaps are addressable and do not negate the underlying relevance of the construct.
Closing Care Gaps
The developer did not address the optional Closing Care Gaps domain for this measure. As a result, there is insufficient information to assess how use of the Rating of Provider measure directly contributes to identifying or reducing disparities or gaps in care for pediatric populations.
Feasibility Assessment
The measure is feasible to implement, supported by well-established CAHPS administration protocols, publicly available guidance, and transparent data collection processes. There are no licensing fees or proprietary barriers to use, and the survey infrastructure is already widely deployed across multiple federal and state programs.
Although the absence of recent child-specific data may limit some quality improvement applications, the measure itself remains implementable with low burden and minimal missingness. Overall, feasibility criteria are met.
Scientific Acceptability
Required accountable entity-level reliability testing was not conducted using child-specific data for this maintenance cycle. Reliance on extrapolated adult reliability results and older data from prior endorsement cycles does not meet current testing expectations.
While the rationale for extrapolation is clearly described, differences in respondent characteristics and context between adult patients and parents of pediatric patients may affect reliability. Additional child-specific reliability testing is needed to support endorsement.
The developer conducted appropriate validity testing at the accountable entity level using adult CG-CAHPS data and demonstrated expected correlations between Rating of Provider and related composites. These findings provide partial support for construct validity.
However, extrapolation from adult respondents to parents of pediatric patients introduces uncertainty, particularly given potential differences in care expectations and decision-making. The submission would be strengthened by clearer hypotheses linking provider actions to pediatric experience outcomes, use of adjusted scores in validity testing, and additional evidence supporting extrapolation. These limitations are addressable.
Use and Usability
The measure is actively used in multiple accountability and reporting programs and includes general guidance on improvement strategies. However, the submission provides limited evidence-based actions and does not adequately explain the lack of sustained improvement in scores over time.
To strengthen usability, the developer should include more specific, evidence-based interventions with citations and address observed score stagnation. Despite these gaps, the measure remains usable and relevant, and the identified issues are addressable.
Summary
I support continued endorsement of the Child CG-CAHPS Rating of the Provider measure. The construct captures an essential aspect of the pediatric care experience, as the provider plays a central role in care delivery, communication, and trust for parents and guardians. The measure is widely implemented, feasible to administer, and actively used across multiple accountability and quality improvement programs.
While the current submission relies on extrapolation from adult CG-CAHPS data due to limited access to recent child-specific data, this approach is permitted under the Endorsement & Maintenance Guidebook and is supported by prior evidence demonstrating consistency across adult and child populations. The identified limitations related to pediatric-specific reliability testing, parent engagement, and articulation of improvement mechanisms are important but addressable.
With targeted enhancements, including incorporation of more recent
pediatric data, clearer provider-level action pathways, and additional evidence-based guidance for improvement, this measure can continue to provide meaningful insight into pediatric patient experience and support accountability, transparency, and quality improvement efforts.
(No subject)
Importance
-
Closing Care Gaps
-
Feasibility Assessment
-
Scientific Acceptability
-
-
Use and Usability
-
Summary
-
No Comments
Importance
No Comments
Closing Care Gaps
No Comments
Feasibility Assessment
No Comments
Scientific Acceptability
No Comments
No Comments
Use and Usability
No Comments
Summary
No Comments
High Importance, Case-Mix Adjustment Approach Acceptable
Importance
Disagree with staff assessment. The wide distribution of scores overall suggests that patients have identified clear issues with the health care system reflected in their responses here, and several studies are provided supporting the link between overall patient experience with their providers and clinical quality outcomes.
Closing Care Gaps
N/A
Feasibility Assessment
Agree with staff assessment. Measure is clearly feasible and widely used.
Scientific Acceptability
Disagree with staff assessment. I do not anticipate issues with reliability of the measure score if it was re-ran with case mix adjusted figures, as the developer provided evidence that case mix adjustment has only a modest effect on the measure score, and in any case is an optional “add-on” to the existing measure score calculation. A mitigation strategy is presented for low reliability entities.
Disagree with staff assessment. I do not find that the issues presented with the case mix adjustment specific to validity are sufficient to affect the rating of the criterion, as these are optional adjustments and largely under the discretion of the individual adjuster.
Use and Usability
Disagree with staff assessment. Although there is little explanation provided for changes in performance, the changes in performance are slight and could be consistent with a positive upward drift.
Summary
Although the submission could be strengthened in some areas, the specific weaknesses are not sufficient to threaten the continued endorsement of this measure. This measure is a rare source of patient-reported data about the health care system, and reflects the performance of entities that are increasingly critical in guiding the course of health care in the United States, as health plans assume ever greater levels of control over provisioning care for their members.
Not met
Importance
I agree with the staff that use of Adult CAHPS data for child data is a real problem, especially considering the period over which NCQA has been using it. It seems to me that the issue will have implications across all areas of review.
Closing Care Gaps
not addressed
Feasibility Assessment
Given that the developer was unable to provide Child CAHPS data, perhaps this measure is not feasible. If it was, logic suggests people would use it and data would be available. The developer needs to provide data to support feasibility.
Scientific Acceptability
I am in complete concurrence with the staff assessment.
The developer does not provide validity data for the correct unit of analysis.
Use and Usability
I am in complete concurrence with the staff assessment.
Summary
Considering how long this measure has been in use, the developers ought to be able to produce measure data to support importance, feasibility, acceptability, and use and usability. They do not. This measure is not acceptable in its current form.
The Child CG-CAHPS Survey –…
Importance
Although the Child CG-CAHPS Survey – Rating of the Provider is presented as a tool to help parents and guardians compare providers and “shop around” for higher-quality care, the practical benefit of this function is limited for many families. In most real-world settings, parents face significant constraints on provider choice due to geography, insurance network restrictions, and limited availability of pediatric providers or specialists. As a result, many families are unable to act on publicly reported ratings, reducing the measure’s value as a consumer decision-making tool.
Additionally, several factors that strongly influence overall provider ratings—such as appointment availability, office hours, wait times, and the availability and performance of support staff—are typically determined at the practice or system level rather than by an individual clinician. Consequently, attributing survey results to a single provider may misrepresent provider performance and conflate clinical quality with operational characteristics outside the provider’s control.
For these reasons, the measure is most meaningful when applied at the group or practice level, where access, staffing, and workflow issues can be addressed through system-level interventions. Its primary value lies in supporting quality improvement rather than individual provider comparison.
Closing Care Gaps
n/a
Feasibility Assessment
While the CG-CAHPS survey instrument itself is publicly available at no cost, implementation of the measure imposes a nontrivial operational and financial burden on practices. Administering the survey typically requires the use of a third-party vendor or dedicated practice staff to identify eligible patients, contact respondents, ensure confidentiality, manage data collection, track response rates, and calculate results. These requirements demand time, infrastructure, and personnel resources beyond routine clinical operations.
For small or independent pediatric practices, this burden may be particularly significant and may limit the ability to implement the measure consistently or achieve adequate response volumes. As a result, feasibility may vary substantially by practice size and available administrative support, potentially affecting data completeness, reliability, and equitable participation across practice settings. Larger health systems with established survey infrastructure are more likely to absorb these requirements, which may introduce participation bias and limit the generalizability of results.
Scientific Acceptability
-
-
Use and Usability
The stated intent of the Child CG-CAHPS Survey – Rating of the Provider is to support quality improvement by providing actionable feedback to providers and facilities. However, the developer has not demonstrated that use of this specific rating measure has led to meaningful or sustained improvements in performance over time. Data reported by the developer show no significant improvement in overall provider ratings between 2015 and 2023 in places where the survey was implemented.
The absence of demonstrated improvement raises concerns about the practical usability of the single-item provider rating as a quality improvement tool.
Summary
The Child CG-CAHPS Survey – Rating of the Provider has limited real-world utility. Many families cannot choose providers freely due to geographic and insurance constraints, reducing the value of the measure as a consumer “shopping” tool. Survey results are also influenced by access and staffing factors largely outside the control of individual providers, making provider-level attribution problematic. Although the survey instrument is free, implementation requires vendor support or dedicated staff, creating a disproportionate burden for small practices. Finally, developer-reported data show no meaningful improvement in provider ratings between 2015 and 2023 in places where the survey was implemented, raising concerns about usability for quality improvement.
Support
Importance
No additional comments
Closing Care Gaps
Optional item not addressed by measure developer.
Feasibility Assessment
No additional comments.
Scientific Acceptability
Measure developer can address the items noted in staff preliminary assessment.
Measure developer can address the items noted in staff preliminary assessment.
Use and Usability
No additional comments.
Summary
No additional comments
Making this a meaningful measure
Importance
.
Closing Care Gaps
.
Feasibility Assessment
.
Scientific Acceptability
.
.
Use and Usability
.
Summary
Disagree with a rating scale of 1-10. All the various criteria that an individual would have to consider in distinguishing between the number ranges would make this survey ineffective. Allow clear options for individuals to choose and only limit to 3 choices- below average, average, above average. The purpose of any surveys like this on individual providers should be for internal use to improve how providers engage with patients.
cGCAPHS rating of provider
Importance
extrapolating adult data for children has gaps
Closing Care Gaps
not addressed
Feasibility Assessment
Agree with committee comments
Scientific Acceptability
Agree with committee comments
Agree with committee assessment
Use and Usability
Agree with committee
Summary
Seems that the focus of this survey should be assessing levels of trust in the provider; access is often not in the span of control of the provider.
There's more work needed to be done
Importance
I would love to see more information focused on the Child CG-CAHPS Survey. The developer did not address children and youths, despite differences in healthcare experience between adults and youths.
Closing Care Gaps
This is an important item that was ignored.
Feasibility Assessment
I agree with the staff assessment.
Scientific Acceptability
I agree with the staff assessment.
I agree with the staff assessment.
Use and Usability
I agree with the staff assessment. However, I think this is addressable since it is currently in use across a broad number of facilities.
Summary
The measure is highly useful and important, but the developers need to update the measure.
Public Comments
Verbal Comment Shared During December 10 Listening Session
This is Traci Petrino. I am a director with the Pennsylvania Clinical Network and the ACO. Practices that we have in our network are saying that it seems like the patients are confused when it says “provider.” Could there be more clarification around providers, nurse practitioners? They [patients] think it's just an MD or the DO. Is this an opportunity to maybe change that verbiage a little bit? It [this question] does apply to all of them [the measures] because I think there's a lot of confusion, they think it's just the doctor, and they may be seeing a certified nurse practitioner, so I think it's kind of relevant to all of them.
Response to Traci Petrino
Cognitive testing with patients demonstrated that participants appropriately understood the term “provider.” The term “provider” was intentionally selected to include not only physicians (MDs, Dos) but also nurse practitioners. In addition, the CG-CAHPS Survey instructions further clarify who “this provider” refers to. The first question asks respondents to confirm their specific provider’s name and explains that all subsequent questions will refer to that person as “this provider.” Here is the wording from Question 1:
“1. A health care provider can care for patients in person, by phone, or by video. Our records show that you got care from the provider named below in the last 6 months.
Name of provider label goes here
Is that right? (Yes/No).
The questions in this survey will refer to the provider named in Question 1 as “this provider.” As you answer these questions, please think of the in-person, phone, and video visits you had with that person in the last 6 months.”
If vendors or practice sites omit or alter these instructions, clarity may be lost. However, when the survey wording is followed, the term “provider” is defined for respondents and they are responding about the specific individual (MD, DO, NP, etc.) that they got care from.
0005 Consumer Assessment of Healthcare Providers and Systems (CA
The American Medical Association (AMA) appreciates the opportunity to comment on this survey and its associated composite measures. We are extremely concerned over the lack of any performance scores and testing provided for the child survey and composite measures and extrapolation of the adult data to this version. The child survey and composite measures are currently in use, and it is not clear why the developer was unable to access these data for this submission. Without information on the degree to which the child version continues to demonstrate a gap in care and remains reliable and valid, we do not believe that endorsement should be maintained.
In addition, it is our understanding that measures that undergo endorsement maintenance should provide testing at the measure score level for any of the composite measures but none of this information is included in the submission. We believe that it is important for health plans, physicians, patients and caregivers, and others to know how the measures perform, particularly since they are used in many accountability programs and this information has previously been submitted for review. In addition, one of the composite measures for the adult version produced Cronbach’s alpha internal consistency reliability results below 0.7, which is less than desirable, and we do not know if results for the child version would be comparable.
Given the lack of any data and testing for the child version, the omission of testing at the measure score level, and lower Cronbach’s alpha results for some of the composite measures, we do not believe that the minimum endorsement criteria have been met and endorsement should be reconsidered until these concerns are addressed.
Response to AMA
While we know that the Child CG-CAHPS Survey is in use, we did not have access to recent child data other than the child data we included in the 2019 maintenance endorsement, which we also shared in the submission as an additional reference point.
As outlined in Appendix H in the CBE Endorsement and Maintenance Guidebook (https://p4qm.org/sites/default/files/2025-11/Del-3-6-Endorsement-and-Ma…), when access to pediatric data are limited, measure developers may submit comparable data for an adult population. The measure developers met with CBE staff to discuss the acceptability of the extrapolation approach and were informed that it would be acceptable. Therefore, extrapolating child CG-CAHPS results based on the adult CG-CAHPS results was an approved approach. As additional support for extrapolation, both the Adult and Child surveys have almost identical wording and capture experience of care from an adult perspective — either directly from the patient (Adult CG-CAHPS) or the parent/guardian (Child CG-CAHPS). Additionally, summary results from the 2019 maintenance endorsement, based on 2016-2017 data when both Child and Adult data were available, show similar patterns or results for scientific acceptability across the two populations. This alignment supports the appropriateness of extrapolating adult data in the absence of more recent child data.
Regarding the internal consistency reliability falling below 0.70 for one measure, this measure includes only two (2) items. Reliability is influenced by both item correlation and the number of items; a measure with a greater number of items would likely have higher reliability but would also increase respondent burden. In addition, internal consistency reliability is only one of the criteria for determining scientific acceptability, and the measure meets other required criteria, so it is important to examine the totality of the evidence rather than a single metric in isolation.