This measure evaluates the extent primary care physicians (PCPs) provide care-based and procedural-based services core to primary care. For each PCP, the resulting value reflects an average of the weighted proportion of services within each category provided during the measurement period.
Primary care providers (PCPs) caring for at least 30 patients per measurement period (the performance year and the 12 months prior to the performance year) score between 0 and 100 in scaled scores of comprehensiveness. Scores are based on weighted averages of 19 care-based and 20 procedural-based core primary care services.
Measure Specs
General Information
The aim of this physician-level measure is to assess the extent to which a primary care physician provides services that are considered core to comprehensive primary care. Comprehensiveness of care – the “provision of integrated, accessible health care services by clinicians who are accountable for addressing a large majority of personal health care needs” – is one of the key defining features of primary care.1 Existing studies show that more comprehensive care by primary care physicians lowers patients’ health care costs, prevents hospitalization and emergency department visits.2,3 There is a strong consensus that comprehensiveness of care is one of the key ingredients in providing high-quality primary care to individuals, families, and communities. 1,4–6 Comprehensiveness is a multi-faceted concept that includes both the scope of services offered and the depth and breadth of health conditions managed by the PCP pending the needs of the population.7 Although this physician-level measure assesses one aspect of comprehensiveness – provision of a range of services – there is empirical evidence of its positive association with patient outcomes.2,3 Nonetheless, this measure is flexible enough to implement in most data environments (e.g. administrative claims, EHR or registry), is straight-forward to interpret, and preliminary analysis suggests positive association with patient outcomes.
The methodology for arriving at the list of core primary care services involved a high level of primary care insight from clinicians, patients, educators, and policy makers. The multi-stage process is outlined below:
1. The comprehensive list of primary care services and procedures was obtained from the published literature on scope of practice among family physicians and general practitioners (since they provide care to all age groups) and electronic health records (EHR) of a nationally-representative sample of primary care practices. For the literature, we closely followed Schultz and Glazier (2017) paper and a series of scope of practice papers by the research team at the American Board of Family Medicine (ABFM). 8–11 Based on the data available in the EHR dataset, included services were narrowed down based on relevance to an office-based or outpatient setting. Non-physicians were excluded from the measure solely due to limited data availability (there was unreliable specialty and credential information for these providers).
2. The services and procedures included are part of primary care medical education, fall under what primary care clinicians are board certified to perform, and data that the American Board of Family Medicine (ABFM) collects about family medicine scope of practice practiced across the United States in primary care practices. The core primary care services were divided into two categories: care-based and procedure-based. The care-based category included a broad range of areas of care activities associated with certain modality, population, health condition, or healthcare setting, while the procedure-based category focused on specific procedures provided in primary care. Initially, there were 29 care-based services and 28 procedure-based services.
3. A technical expert panel (TEP) consisting of primary care physicians, researchers, health system executives, Federally Qualified Health Center (FQHC) representatives, family medicine advocacy groups and educators, and patient/caregivers (see TEP participants below) was held to finalize the list of core primary care services. Weights were assigned by the TEP in degrees of importance (0 = not important, and 100 = very important), then these weights were averaged. Along with creating the weighting, the TEP also reduced the list of services.
4. The weights within each category were then re-scaled so that their sum equaled the total number of services in that category (19 for care type, 20 for procedure). Categories were separated so that they could be reflective of relative importance to other services within that category (i.e. the weights in the care services reflect relative importance to other care services, while the weights in the procedure services reflect relative importance to other procedure services).
Our TEP participants included:
Tyler Barreto MD, MPH - Sea Mar CHC, Seattle, WA
Sanjay Basu MD, PhD - Center for Primary Care, Harvard Medical School
Andrew Bazemore MD, MPH – The American Board of Family Medicine Center for Professionalism and Value in Health Care
Reid Blackwelder MD, FAAFP - Quillen College of Medicine, ETSU
Hoon Byun DrPH – American Academy of Family Physicians- The Robert Graham Center
Yoonie Chung PhD – American Academy of Family Physicians-The Robert Graham Center
Julea Garner MD - Baptists Health-UAMS Family Medicine Residency
Jackson Griggs MD - Heart of Texas Community Health Center Inc.
Yalda Jabbarpour MD – American Academy of Family Physicians - The Robert Graham Center
Vivian Jiang MD - University of Colorado Medicine
Susan Lowe - Patient Representative
Amy Mullins MD - American Academy of Family Physicians
Ann O'Malley MD MPH- Mathematica
References:
- Institute of Medicine (US), Division of Health Care Services, Committee on the Future of Primary Care. Primary Care: America’s Health in a New Era. (Donaldson MS, Yordy KD, Lohr KN, Vanselow NA, eds.). National Academies Press (US); 1996.
- Bazemore A, Petterson S, Peterson LE, Phillips RL. More Comprehensive Care Among Family Physicians is Associated with Lower Costs and Fewer Hospitalizations. Ann Fam Med. 2015;13(3):206-213. doi:10.1370/afm.1787
- O’Malley AS, Rich EC, Shang L, et al. New approaches to measuring the comprehensiveness of primary care physicians. Health Serv Res. 2019;54(2):356-366. doi:10.1111/1475-6773.13101
- World Health Organization. Primary Health Care. Geneva: World Health Organization. 1978.
- Haggerty JL, Beaulieu MD, Pineault R, et al. Comprehensiveness of care from the patient perspective: comparison of primary healthcare evaluation instruments. Healthc Policy Polit Sante. 2011;7(Spec Issue):154-166.
- Bitton A. The Necessary Return of Comprehensive Primary Health Care. Health Serv Res. 2018;53(4):2020-2026. doi:10.1111/1475-6773.12817
- O’Malley AS, Rich EC. Measuring Comprehensiveness of Primary Care: Challenges and Opportunities. J Gen Intern Med. 2015;30 Suppl 3: S568-575. doi:10.1007/s11606-015-3300-z
- Schultz SE, Glazier RH. Identification of physicians providing comprehensive primary care in Ontario: a retrospective analysis using linked administrative data. CMAJ Open. 2017;5(4):E856E863. doi:10.9778/cmajo.20170083
- O’Neill T, Peabody MR, Blackburn BE, Peterson LE. Creating the Individual Scope of Practice (ISOP) scale. J Appl Meas. 2014;15(3):227-239.
- Coutinho AJ, Cochrane A, Stelter K, Phillips RL, Peterson LE. Comparison of Intended Scope of Practice for Family Medicine Residents with Reported Scope of Practice Among Practicing Family Physicians. JAMA. 2015; 314(22):2364-2372. doi:10.1001/jama.2015.13734
- Peterson LE, Fang B, Pu er JC, Bazemore AW. Wide Gap between Preparation and Scope of Practice of Early Career Family Physicians. J Am Board Fam Med. 2018;31(2):181-182. doi:10.3122/jabfm.2018.02.170359
The measure can be used in any data source that captures individual-level health care encounters across care settings (such as medical claims) and/or clinical data (such as from electronic health records or a registry) regarding demographics, services and procedures from primary care visits. ABFM tested the measure using two separate data sources: claims data and EHR data. Information on the two tested data sources (MarketScan for Claims data and American Family Cohort for EHR data) can be found below:
MarketScan (Claims):
The MarketScan Research database captures individual-level healthcare utilization, expenditures, and enrollment across the patient care setting spectra. Claims for inpatient, outpatient, prescriptions drugs in the outpatient settings, and carve-out services are included in these data. The data come from a large selection of employers, health plans, and government/public organizations. The annual medical database includes private-sector health data from approximately 350 payers representing 20 billion service records. The data also represent the health care experiences of insured employees and their dependents of active employees, early retirees, and those receiving COBRA benefits (Consolidated Omnibus Budget Reconciliation Act). Included in these data are Medicare-eligible retirees that have employer-sponsored Medicare Supplemental plans.
The database has limited geographical location information: Specific zip code level information and address are not available for either the patient or provider in this dataset. These data have been reviewed by their internal and external statisticians and found to be incompliance with HIPPA Privacy Rules. The data contained for all enrolled beneficiaries are considered de-identified. The ABFM has a signed data use agreement with MarketScan which grants ABFM access to their claims extracts via the Stanford Department of Population Health Sciences (PHS), whose Data Core team reviews and approves access for the individual user. For those included and enrolled during the measurement period, all encounters with primary care physicians should be captured and included in these data.
American Family Cohort (EHR):
Derived from the ABFM PRIME Registry, the American Family Cohort is a clinical data repository of electric health record data from more than 2,000 primary care clinicians across the United States, with the electronic medical record data elements pulled from visits in the primary care setting. These data were established by the American Board of Family Medicine with the mission to measure clinical quality as well as develop specific clinical measure for quality reports and dashboards for primary care practices. The data contains a convenience sample of 1,000 practices representing over 5 million patients with the focus of care received by family physicians, general internists, general pediatricians, and advanced clinical practitioners, such as nurse practitioners and physician assistants. In the American Family Cohort, which is the research database representing extracts from the original PRIME Registry, there exist detailed demographics, diagnosis codes, procedures, some laboratory results, medication data including prescriptions, and free-text clinical notes.
Public Comments
Public Comment
Measuring the Value-Functions of Primary Care: Comprehensiveness
The American Medical Association (AMA) asks that the committee determine whether the care-based and procedure-based services used to define the comprehensiveness of care are appropriate. We do not believe that it is practical or reflective of current primary care practice for an individual physician to provide such a broad set of services, particularly if they are in a subspecialty. For example, many internal medicine physicians specialize in pediatric or geriatric care, and we do not believe that scoring a geriatrician lower because they do not also see children is useful nor does it reflect the quality of the services that this physician provides.
ACP Concerns with CBE #4290
The American College of Physicians (ACP) appreciates the opportunity to comment on measures under review for the Spring 2025 Endorsement and Maintenance (E&M) cycle. We have significant concerns with a measure submitted to the Primary Prevention Project.
Overview of Concerns:
ACP strongly opposes endorsement of CBE #4290, “Measuring the Value-Functions of Primary Care: Comprehensiveness of Care.” This measure ignores consensus views on what defines comprehensive primary care, specifically that these services should be made available by a primary care team, and instead defines (and measures) “comprehensiveness” as the provision by an individual clinician of 19 care-based and 20 procedure-based services. The evidence supporting this construct (these 39 services equal comprehensive primary care delivery) is limited and weak. CBE #4290 lacks face validity and construct validity.
CBE #4290 is not broadly applicable across primary care clinicians, as geographic factors, community norms, and employment status often dictate what services are generally provided. Additionally, CBE #4290 is not applicable writ large to internal medicine physicians who currently constitute the largest percentage of actively practicing primary care physicians. Nor should it apply to other primary care specialties, such as pediatrics and geriatrics. Physicians who individually provide care across all ages and stages of life, from newborns to elderly patients, will receive higher scores, which may result in the unintended consequences of overuse or performance by clinicians whose scope of practice is focused on either children or adults.
Furthermore, this measure may lead to unintended acceleration of workforce shortages. The U.S. faces a shortage of 48,000 primary care physicians, which is expected to grow. Rather than promoting improvement, CBE #4290 may inadvertently discourage clinicians from remaining in or entering primary care.
More detailed concerns are described below, based on the 5 domains in which ACP’s Performance Measurement Committee reviews measures.
Importance
ACP is skeptical that implementation of the measure will lead to measurable and meaningful improvements in clinical outcomes.
Comprehensiveness is an important concept, and one of Starfield’s four core pillars (4C’s) included in her primary care framework. Comprehensiveness refers to “Offering a comprehensive scope of services…by building teams of professionals including [general practitioners (GPs)], registered nurses and allied health professionals that are based in the primary care space. Primary care teams can reduce the need for specialist referrals and services particularly when specialty services are made accessible at the primary care level.”
Attribution at the physician level is counter to the spirit of a team-based approach to addressing comprehensive care at the practice level. This measure could be useful as a tool to assess the degree to which primary care practices within a system can address the needs of its population.
Appropriate Use
CBE #4290 evaluates quality by assessing the provision of services regardless of whether the services are appropriate. As a result, it may encourage overuse.
Clinical Evidence Base
While no guidelines support the measure, evidence from scoping and narrative reviews is cited that demonstrates a linkage of comprehensive care to desired outcomes.
The evidence supporting the selection of these 39 services and associated weighing is thin. The services and procedures included are within the scope of training and credentialing of a family medicine physician. Individual physician privileges may vary by physician preference and practice location.
A technical expert panel (TEP) advised on the final list of services included in the measure and the weighting of services. The TEP was comprised of family physicians and family medicine researchers with almost no representation from other primary care specialties. The TEP was not inclusive of other primary care physicians for whom the measure is intended. Without having representation from internal medicine, pediatrics, or geriatrics, the definition of comprehensiveness does not consider the perspectives of these important stakeholders who provide a significant percentage of primary care in the United States. This likely influenced the measure's perspective and relevance for broader primary care practice.
Specifications
Face Validity: ACP’s perspective
CBE #4290 does not evaluate quality. It attempts to measure comprehensiveness of care by calculating the number of services provided. A higher score results from an individual billing each of the 39 specified services at least once during the measured period, regardless of whether the service was warranted or could have been better or more safely performed by another clinician and/or in a different setting. As a result, the measure results cannot distinguish good from poor quality. The measure has limited applicability to other primary care physicians, as noted above. Does a geriatrician who provides 8 services provide lesser quality than a family physician who provides 25 services, particularly when the geriatrician does not offer several of the listed services (e.g., newborn care, prenatal care)?
The weights of the various services are questionable. Several of the procedure-based services have higher weights than the care-based ones. Pap smears and cervical cancer screening have the highest weight across all services. Joint and tendon aspirations are weighted higher than chronic disease management.
The measure demonstrates poor content validity due to:
CBE #4290 risks exacerbating healthcare disparities. Medicaid-insured patients face 28% fewer preventive services than privately insured counterparts. Rural providers, who comprise 12% of the primary care workforce, report 45% fewer resources for reproductive health services due to funding gaps and community norms. Penalizing these providers for systemic failures will deepen access issues.
Face Validity: Developer’s data
The developers state that they surveyed 26 primary care physicians who participate in the PRIME registry and represent physicians across the United States. 80.77% of physicians surveyed agreed that scores obtained from the measure can be used to distinguish good quality of care from poor quality. The specialty of survey respondents is not known from the information provided. The developers should clarify the proportion of the surveyed physicians in specialties other than family medicine, if any.
Empiric Validity:
The measure developers performed additional validity testing by associating the physician-specific comprehensiveness of care index with no emergency department (ED) visits vs. any ED visits as a patient-level outcome, using descriptive analysis and multivariable hierarchical logistic regression. For the descriptive analysis, they separated the comprehensiveness of the care index by deciles and reported the proportion of patients with any ED visit vs. no ED visits for 2018-2019. They conclude that there is “a definitive decrease in the ED visit rates as the comprehensiveness of care increases.” However, their data do not support that conclusion (see graph below). Rather, their data suggest that a comprehensive score < 0.5 is associated with a higher ED visit rate and that further increases in comprehensiveness is not associated with further reductions in ED visits. (see graph in attachment)
Measure Feasibility and Applicability
CBE #4290 is not applicable to internal medicine physicians and has potential to lead to unintended consequences.
Appointment wait times for primary care physicians have increased 48% since 2004. To meet preventive care benchmarks alone, primary care physicians would need 26.7 hours daily, a logistical impossibility that risks clinic closures. Office-based procedures reimburse $34 below break-even costs, forcing providers to subsidize care. Compounding this, 78% of substance use treatment plans require specialized credentialing, which 67% of rural practices lack due to funding constraints. Additionally, the psychological burden is enormous. Subconsciously or even consciously, knowing that one will be "measured" on how many of these services are provided to patients can have a corrosive effect on the psyche of primary care physicians and the care they provide.
CBE #4290 overlooks patient autonomy and does not account for patient preference. Surveys indicate that 72% of patients prefer gynecological care from specialists over primary care physicians, even when primary care physicians are trained equivalently. Similarly, 60.3% of women in a large HMO study opted for OB/GYNs for basic services, citing comfort and perceived expertise. While integrated behavioral health is favored by 41% of patients, barriers like stigma and fragmented insurance coverage persist.