Primary Measure - Most and Moderately Effective Contraceptive Provision or Use: Percentage of patients ages 15-44 assigned female at birth with a live birth delivery who received a most or moderately effective contraceptive, or were documented to use a most or moderately effective contraceptive method, in the postpartum period. The primary measure captures new provision as well as current use of most and moderately effective contraceptive methods to accurately capture postpartum contraceptive utilization even if provided in a different calendar year or a different health care site.
Sub-Measure - LARC-SINC: Percentage of patients ages 15-44 assigned female at birth with a live birth delivery who received LARC in the postpartum period. The sub-measure captures LARC provision to ensure access to these methods by identifying low provision rates (i.e., below 2%).
For both measures: to focus on the population of postpartum patients interested in contraceptive services, the denominator excludes those individuals who did not receive or have documented use of a method if they indicated through a Self-Identified Need for Contraception screening question (SINC) they did not want these services
Measure Specs
General Information
Supporting postpartum patients to prevent pregnancy when they wish to do so has social and health benefits for individuals and their families (1,2). Achieving people’s reproductive goals depends on being able to achieve or prevent pregnancy when and how they want to (3). However, in 2015, based on National Survey of Family Growth (NSFG) data, only 47.9% of pregnancies were categorized as occurring at the desired time for the individual (4). In order to support patients to achieve their reproductive goals, facilities at which individuals receive their prenatal and postpartum care must ensure that contraceptive needs are assessed and met in the postpartum period. This includes ensuring that the most effective reversible methods of contraception – intrauterine devices (IUDs) and implants – are available in a timely fashion. Multiple commentaries have detailed how the use of performance measures related to contraceptive provision can improve health care quality and promote positive reproductive health outcomes (3–5). The University of California, San Francisco (UCSF) designed Contraceptive Use electronic clinical quality measure (eCQM, CU-SINC), Postpartum (CBE #3682e) to give health care organizations and facilities the opportunity to measure contraceptive use among postpartum patients who want contraceptive services. Specified for use with electronic health record (EHR) system data, CBE #3682e can be calculated in a wide array of health care settings, including systems that do not rely on administrative claims, and thus fills gaps of extant contraceptive provision measures (CBE #2902, #2903, and #2904), that rely on claims data. Specifically, administrative claims data have limitations affecting measure implementation in different care settings as well as assessment of previous contraceptive services received and patient preferences for contraception. The claims-based measures are designed for calculation in service delivery systems with a fee-for-services model. Thus, entities that use prospective payment systems, such as Federally Qualified Health Centers (FQHCs) easily employ CBE #2902, #2903, and #2904 to evaluate contraceptive services quality. Additionally, these measures of contraceptive provision do not always accurately identify which contraceptive method a woman is using following a visit (particularly LARC methods and sterilization, which are not captured in administrative claims if provided prior to the latest health care visit or during a previous measurement period), patient preferences for contraceptive services are not available in administrative data, and the claims-based measures cannot accurately parse which women need or want contraceptive services.
ECQMs offer a way to measure reproductive health care quality by utilizing EHR system data (6). Unlike administrative claims, EHR systems can capture patient need for contraceptive and other health services and are utilized in a wider array of health care settings. To focus the measure on the population of women interested in contraceptive services, CBE #3682e, i.e. CU-SINC, Postpartum uses the Self-Identified Need for Contraception (SINC) data element to remove people who are not interested in contraception from the measure denominator, which helps guard against the possibility of directive or coercive counseling towards contraception that may be an unintentional result of use of a contraceptive use performance measure (7,8). CU-SINC, Postpartum is structured to have a primary measure that is the proportion of those who desire contraception who are documented to have those needs met across all methods. It intentionally includes methods that may have been provided in previous calendar years, such as IUD and implants, and methods provided at other sites, in order to capture an overall assessment of how well people’s needs are being met. Recognizing that there are unique barriers to provision of IUDs and implants, including procedural training, availability of medical equipment, stocking of methods, and implementation of billing practices, the measure includes the Long-Acting Reversible Contraception (LARC) LARC-SINC submeasure that captures provision of these methods at the actual site. Designed as a floor measure, this submeasure assesses whether these methods are available to those who want them.
In summary, CU-SINC, Postpartum can be used in settings that cannot use the claims-based contraceptive provision measures and provides improved measurement of whether patient’s postpartum contraceptive needs are being fulfilled. CBE #3682e will inform in quality improvement initiatives that help health care organizations better meet patients’ needs by increasing patient-centered access to contraception, a step towards the goal of reproductive autonomy and well-being for all. Moreover, improvement in the quality of contraceptive care has been shown to improve people’s ability to identify methods that they can use over time and to promote engagement with health care across the reproductive life course, which will improve people’s reproductive outcomes and therefore would also be expected to have a positive impact on health care costs.
REFERENCES
1. Conde-Agudelo A, Rosas-Bermúdez A, Kafury-Goeta AC. Birth Spacing and Risk of Adverse Perinatal Outcomes: A Meta-analysis. JAMA. 2006 Apr 19;295(15):1809.
2. Conde-Agudelo A, Rosas-Bermúdez A, Kafury-Goeta AC. Effects of birth spacing on maternal health: a systematic review. American Journal of Obstetrics and Gynecology. 2007 Apr;196(4):297–308.
3. Gavin L, Frederiksen B, Robbins C, Pazol K, Moskosky S. New clinical performance measures for contraceptive care: their importance to healthcare quality. Contraception. 2017 Sep;96(3):149–57.
4. Gavin LE, Ahrens KA, Dehlendorf C, Frederiksen BN, Decker E, Moskosky S. Future directions in performance measures for contraceptive care: a proposed framework. Contraception. 2017 Sep 1;96(3):138–44.
5. Moniz MH, Gavin LE, Dalton VK. Performance Measures for Contraceptive Care: A New Tool to Enhance Access to Contraception. Obstetrics & Gynecology. 2017 Nov;130(5):1121–5.
6. Trussell J, Aiken ARA, Micks, E, Guthrie K. Efficacy, safety, and personal considerations. In: Contraceptive Technology. 21st ed. )Ayer Company Publishers, Inc.; 2018. p. 95–128.
7. Person-Centered Reproductive Health Program [Internet]. [cited 2024 Jul 18]. Self-Identified Need for Contraception (SINC). Available from: https://pcrhp.ucsf.edu/sinc
8. Dehlendorf C, Perry JC, Borrero S, Callegari L, Fuentes L, Perritt J. Meeting people’s pregnancy prevention needs: Let’s not force people to state an “Intention.” Contraception. 2024 Jul;135:110400.
CU-SINC, Postpartum uses electronically extracted data from structured fields within EHR systems, after data are collected from ambulatory, outpatient clinical encounters and entered into EHR structured fields.
We implemented and tested CU-SINC, Postpartum in primary care settings through a quality improvement learning collaborative among federally qualified health centers (FQHCs). All value sets utilized in our measure rely on standardized coding systems and are published on VSAC.
For more information on the feasibility of CU-SINC, Postpartum, see Section 4 Feasibility. To review our reliability and validity analyses methods and results, see Section 5, Scientific Acceptability.
Numerator
Primary measure: Of patients in the denominator, those who received or were documented to be using a most (i.e., sterilization, implants, intrauterine devices or systems (IUD/IUS) or moderately effective (i.e., injectables, oral pills, patch, or ring) contraceptive method within 90 days postpartum. If a live birth date is documented, provision or documentation of method use must occur in a visit following this date and prior to 90 days after this date. If no live birth date is documented, then the visit in which contraception was provided or documented must occur after 24 weeks of pregnancy as determined by the EDD, and prior to 90 days from the EDD.
Sub-Measure: Of patients in the denominator, those who were provided a long-acting reversible method (IUD or implant) within 90 days postpartum. If a live birth date is documented, provision of the method must occur in a visit following this date and prior to 90 days after this date. If no live birth date is documented, then the visit in which contraception was provided must occur after 24 weeks of pregnancy as determined by the EDD, and prior to 90 days from the EDD.
The numerator of the primary measure includes patients in the denominator who were provided or were documented to be using a most (i.e., sterilization, implants, intrauterine devices or systems) or moderately effective (i.e., injectables, oral pills, patch, or ring) method within 90 days after live birth delivery date. If a live birth date is documented, provision or documentation of method use must occur in a visit within 90 days after this date. If no live birth date is documented, then the visit in which contraception was provided or documented must occur after 24 weeks of pregnancy as determined by the estimated delivery date (EDD), and within 90 days after the EDD.
The numerator of the LARC-SINC submeasure includes patients in the denominator who were provided a LARC (i.e., IUD or implant) within 90 days after live birth delivery date. If a live birth date is documented, provision of the method must occur in a visit within 90 days after this date. If no live birth date is documented, method use or provision then the visit in which contraception was provided or documented must occur after 24 weeks of pregnancy as determined by the EDD and within 90 days after the EDD.
All data elements available to calculate the primary measure numerator are defined within the active value sets and codes in the Value Set Authority Center (VSAC) and included in the measure specification. For the list of value sets to calculate the primary measure numerator, see item 1.13a, Data Dictionary Attachment (3682e VSAC Value Sets_508.xlsx), worksheet named “Value Set List” with “Numerator (Primary Measure)” shown in Column B. The measure’s specification was developed in CMS’ Measure Authoring Development Integrated Environment (MADiE) system and uses direct reference codes for the SINC Screening question and its multiple response options. This eCQM specification utilizes Clinical Quality Language (CQL) and Quality Data Model (QDM 5.6). The primary measure assesses patients who were provided or were documented to be using most and moderately effective contraception; thus, the value sets include codes indicating surveillance of these contraceptive methods.
All data elements available to calculate the LARC-SINC submeasure numerator are defined within the active value sets and codes in the VSAC and included in the measure specification. For the list of value sets to calculate the submeasure numerator, see item 1.13a, Data Dictionary Attachment (3682e VSAC Value Sets_508.xlsx), worksheet named “Value Set List” with “Numerator (Submeasure)” shown in Column B. The submeasure assesses patients who were provided LARC methods and are not already using them; thus, the submeasure value sets exclude codes related to surveillance of LARC methods.
Denominator
Primary Measure and Sub-Measure: Patients assigned female at birth, aged 15-44 with a qualifying encounter and a prenatal care visit in the year prior to the calendar year through the first 9 months of the calendar year (i.e., 1/1/XX-1 through 9/30/XX) with a documented live birth delivery date, or a documented estimated delivery date (EDD) between 3 months prior to the calendar year and 9 months into the calendar year (i.e., 10/1/XX-1 through 9/30/XX)
The measurement period is 15 months: three months prior to the calendar year (i.e., 10/1/XX-1) through the end of the calendar year (i.e., 12/31/XX). For example, if the calendar year is 2023, then the measurement period for CU-SINC, Postpartum is 10/1/2022 through 12/31/2023. The measure uses two consecutive calendar years of data, is patient-based, and is calculated at the facility and clinician group/practice levels of analysis.
The measure denominator includes patients who meet the following criteria:
1) Assigned female at birth aged 15-44 years and had a qualifying encounter (QE) from three months prior to the start of the calendar year through the end of the calendar year (i.e., 10/1/XX-1 through 12/31/XX). Age is calculated with the start of the calendar year as an anchor date;
2) Had a live birth making them eligible for postpartum contraception: Those who had a prenatal care visit in the year prior to the calendar year through the first nine months of the calendar year (i.e., 1/1/XX-1 through 9/30/XX) with a documented live birth delivery date, or a documented EDD between three months prior to the calendar year and nine months into the calendar year (i.e., 10/1/XX-1 through 9/30/XX).
All data elements available to calculate this denominator are defined within the active value sets in the Value Set Authority Center (VSAC) and included in the measure specification. For the list of value sets to calculate the denominator, see item 1.13a, Data Dictionary Attachment (3682e VSAC Value Sets_508.xlsx), worksheet named “Value Set List” with “Denominator” shown in Column B.
Note that a single code from any of the value sets defining Qualifying Encounters (QE) that is documented during the measurement period counts as a QE.
Exclusions
Patients are excluded from the denominator if they were documented to not have had a live birth, including those with ectopic pregnancies, intrauterine fetal demise, early pregnancy loss, or abortion; or if they did not want to discuss their contraceptive needs and were not provided nor were documented to be using a most or moderately effective contraceptive method throughout the postpartum period (i.e., within 90 days after live birth delivery).
The measurement period is 15 months: three months prior to the calendar year (i.e., 10/1/XX-1) through the end of the calendar year (i.e., 12/31/XX). For example, if the calendar year is 2023, then the measurement period for CU-SINC, Postpartum is 10/1/2022 through 12/31/2023. The measure uses two consecutive calendar years of data, is patient-based, and is calculated at the facility and clinician group/practice levels of analysis.
Patients are excluded from the denominator if they meet one of the following criteria:
1) Had a non-live birth: Those who were documented to not have had a live birth, including those with ectopic pregnancies, intrauterine fetal demise, early pregnancy loss, or abortion from three months prior to the start of the calendar year through the end of the calendar year (i.e., 10/1/XX - 1 through 12/31/XX), or;
2) Answered “No” to all instances of the Self-Identified Need for Contraception (SINC) question: those who did not want to discuss their contraceptive needs every time they were asked the SINC question from among those who were not provided nor were documented to use a most or moderately effective method throughout the postpartum period (i.e., within 90 days after live birth delivery). Documented no responses to SINC must occur no earlier than 16 weeks before EDD and within 90 days after live birth delivery date.
All data elements available to calculate the denominator exclusions are defined within value sets active in the Value Set Authority Center (VSAC) and included in the measure specification. For the list of value sets to calculate the denominator exclusions, see item 1.13a, Data Dictionary Attachment (3682e VSAC Value Sets_508.xlsx), worksheet named “Value Set List” with “Denominator Exclusions” shown in Column B.
Measure Calculation
The measurement period is 15 months: three months prior to the calendar year (i.e., 10/1/XX-1) through the end of the calendar year (12/31/XX). For example, if the calendar year is 2023, then the measurement period for CU-SINC, Postpartum calendar years of data is patient-based, and is calculated at the facility and clinician group/practice levels of analysis.
Step 1: Identify all patients who are female assigned at birth, ages 15-44 who had:
- a qualifying encounter at the specified facility from three months prior to the calendar year and through the calendar year (i.e., 10/1/XX-1 through 12/31/XX), and;
- a live birth making them eligible for postpartum contraception: those who had a prenatal care visit in the year prior to the calendar year (i.e., 1/1/XX-1) through the first nine months of the calendar year (i.e., 1/1/XX-1 through 9/30/XX) with a documented live birth delivery date, or a documented EDD between the three months prior to the start of the calendar year and through the first nine months of the calendar year (i.e., 10/1/XX-1 through 9/30/XX).
Step 2: Define the denominator by excluding patients assigned female at birth who:
- Had a non-live birth from three months prior to the start of the calendar year through the end of the calendar year (i.e., 10/1/XX - 1 through 12/31/XX) (e.g. still birth, miscarriage, ectopic pregnancy, or induced abortion), or;
- Answered “No” to all instances of the SINC question and were not provided nor were documented to use a most or moderately effective method throughout the postpartum period (i.e., within 90 days after live birth delivery). Documented no responses to SINC must occur no earlier than 16 weeks before EDD and within 90 days after live birth delivery.
Step 3a: Define numerator 1 by using codes to identify patients in the denominator who were provided or documented to use a most or moderately effective method within 90 days after either their live birth delivery date, if available, or their EDD (primary measure).
Step 3b: Define numerator 2 by using codes to identify patients in the denominator who were provided a LARC, i.e., intrauterine device or subcutaneous implant, within 90 days after either their live birth delivery date or their EDD (submeasure).
If a live birth delivery date is documented, provision or documentation of most or moderately effective method use must occur in a visit within 90 days after this date. If no live birth delivery date is documented, then the visit in which contraception was provided or documented must occur after 24 weeks of pregnancy as determined by the EDD, and within 90 days after the EDD.
Step 4a: Calculate the primary measure rate by dividing numerator 1 by the denominator.
Step 4b: Calculate the submeasure rate by diving numerator 2 by the denominator.
In this application, we present CU-SINC, Postpartum scores stratified by the following age groups: ages 15-20 years and ages 21-44 years in addition to the full age range of the initial population, ages 15-44 years. These age groups align with the endorsed claims-based Contraceptive Care - Postpartum measure (CBE #2902) stewarded by the HHS Office of Population Affairs (OPA) and allow for measure rates among adolescents (ages 15-20 years) to be examined separately from adult (ages 21-44 years) patients for the purposes of quality improvement. Age is calculated using the start of the measurement period or calendar year as the anchor date; no value sets are required. See also Section 5.4.1 for more details.
Although we do not specify that the measure must be calculated with a minimum sample size, we suggest using (and present in this application) a minimum of at least 50 eligible patients per reporting unit.
Patient denominator minimums ensure that practices/facilities are large enough to have an adequate volume of patients across the year for consistent reporting. This minimum sample size also fields sufficient reliability as described in section 5.2.3.
Supplemental Attachment
Point of Contact
Not applicable
Christine Dehlendorf
San Francisco, CA
United States
Christine Dehlendorf
University of California, San Francisco
San Francisco, CA
United States
Importance
Evidence
Supporting postpartum patients to prevent pregnancy when they wish to do so has social and health benefits for individuals and their families (1,2). In order to support patients to achieve their reproductive goals, facilities at which individuals receive their prenatal and postpartum care must ensure that contraceptive needs are assessed and met in the postpartum period. However, in 2015, based on National Survey of Family Growth (NSFG) data, only 47.9% of pregnancies were categorized as occurring as desired at the time for the individual (3). Contraception is a highly effective clinical preventive service that can assist women in reaching their reproductive health goals (4,5). While most and moderately effective contraceptive methods have a failure rate of 1-23%, not using any method at all has a failure rate of 85% (5). The most commonly used, and most effective, contraceptive methods in the United States require contact with a health care provider (6). Moreover, access to sexual and reproductive healthcare has been associated with use of prescription contraceptive methods and contraceptive counseling has been linked to ongoing contraceptive use (7,8). Therefore, to support patients in achieving their reproductive goals, facilities where individuals receive their care must ensure that their contraceptive needs are assessed and met. This includes ensuring that long-acting reversible methods of contraception (LARC) - intrauterine devices (IUDs) and implants – are available. The Women’s Preventive Services Guidelines, issued by the American College of Obstetricians and Gynecologists and the Health Resources and Service Administration, recommend unhindered and affordable access to a full suite of contraceptive methods (9,10).
Barriers to access to contraceptive care exist. Despite this need for contraceptive services, many people who do not want to become pregnant do not use contraception. Data from the NSFG 2011-2017, for example, found that of those at risk of an unintended pregnancy and who were sexually active, 18% were not using any form of contraception, and, of those, 84% did not want to become pregnant in the following two years (11). One contributor to these statistics is lack of access to contraceptive care. NSFG analyses of 2011-2013 data found that only 46% of women at risk of unintended pregnancy receive contraceptive services in a year (12). Further, among ambulatory encounters with women of reproductive age in the United States, only 14% include any reproductive health services, including contraception (13). Among those who wanted to prevent pregnancy for at least 5 years, 19% reported that they were not using contraception because they could not access a method (11). Among postpartum individuals specifically, many do not use contraception, despite the fact that over 70% of individuals resume intercourse by 60 days postpartum.(14) For example, data from the 2006-2010 NSFG found that that 28% and 22% of individuals reported they did not use a method at 3 and 12 months after childbirth respectively.(15)
Gaps in care access are not equitably experienced and are increasing. One NSFG analysis of the 2006-2017 waves of data collection found that overall 18.3% of women of reproductive age received contraceptive counseling, and when looking at subgroups, found the Black and Latina heterosexual women had lower odds of receiving contraceptive counseling compared to white heterosexual women (with Latina sexual minority women having the lowest odds) (16). A separate analysis of these NSFG data reported disparities in receipt of contraceptive services by both race/ethnicity and age (with younger patients being less likely to receive contraceptive services) (12). Additionally, changing policy environments have led to a decrease in contraceptive care access and increased barriers to care. A recent study used four waves of cross-sectional study data to look at changes in contraceptive care access in four states between 2021 and 2023 and found a four-percentage point increase in reported barriers to accessing contraceptive care between the time points (17). Moreover, ten percent of respondents who were not using a preferred contraceptive method named access barriers, including difficulty accessing a facility, as a reason (18). These barriers infringe upon individuals’ ability to access and use contraception, compromising their ability to achieve their reproductive goals.
Contraceptive performance measures are an important tool to address gaps in access. Multiple commentaries have detailed how the use of performance measures related to contraceptive provision can improve health care quality and promote positive reproductive health outcomes (19–21). This impact can occur, for example, through encouraging providers and systems to prioritize reproductive health services, including in quality improvement initiatives, and follow national clinical recommendations related to this care and offer a full range of methods.
Existing claims-based measures of contraceptive provision have several limitations, including that they cannot be used in settings with prospective payment systems, such as Federally Qualified Health Centers (FQHCs), which are critical for contraceptive access. The National Quality Forum (NQF) endorsed the first clinical performance measures focused on contraception in October 2016, empowering health care organizations to assess contraceptive services to improve quality of care. Stewarded by the U.S. Health and Human Services Office of Population Affairs (OPA) and specified for calculation in administrative claims, the Contraceptive Care measures (Consensus-Based Entity [CBE] #2902, #2903, and #2904) estimate the percentage of women ages 15-44 years provided a most or moderately effective method of contraception in two populations in this age range: postpartum women and all fecund women. These CBE-endorsed measures also evaluate access LARC, which is a subset of most and moderately effective methods, by focusing on low (i.e., less than 2%) rates of use as a proxy for access (19–21) to these specific methods.
The Contraceptive Care measures provide reliable and valid metrics for health entities to evaluate the proportion of women receiving prescription contraceptive methods, but administrative claims data have limitations affecting measure implementation in different care settings as well as assessment of previous contraceptive services received and patient preferences for contraception. They are designed for calculation in service delivery systems with a fee-for-services model, which rely on claims. Thus, entities that use prospective payment systems, such as FQHCs, which are community-based health care providers that receive federal funds to provide primary care services in underserved areas, cannot easily employ CBE #2902, #2903, and #2904 to evaluate contraceptive services quality. These measures also do not always accurately identify which contraceptive method a woman is using following a visit (particularly LARC methods and sterilization, which are not captured in administrative claims if provided at a different site or prior to the measurement period). Furthermore, these claims-based measures cannot account for whether or not a person wants to be using contraception since this information is not available in administrative claims.
Electronic clinical quality measures (eCQMs) offer a way to measure contraceptive care quality by utilizing electronic health record (EHR) system data (5). Unlike administrative claims, EHR systems can capture patient need for contraceptive and other health services and can be utilized in care settings where claims-based measures are not feasible. EHR systems can also identify ongoing use of a method (e.g. IUD, implant, or sterilization) even if not provided in the measurement period or at the specific clinical site. Ideally, eCQMs are calculated with data captured in structured form during the process of patient care. CBE #3682e, UCSF’s Contraceptive Use eCQM (CU-SINC), Postpartum aims to document both contraceptive use and provision while defining the population in need of contraceptive services for the denominator more accurately through encounter-level EHR data.
CU-SINC promotes person-centered contraceptive care screening practices and better specifies the population in need of services to align with equity goals. To focus the measure on the population of women interested in contraceptive services, UCSF’s Person-Centered Reproductive Health Program (PCRHP) created the Self-Identified Need for Contraception (SINC) data element (22). SINC consists of a standardized question and response options in the LOINC code system. SINC acts as the basis of the process measure CBE #4655e (aka Contraceptive Care Screening eCQM) and serves as an exclusion criterion for the CBE #3682e denominator. Before SINC, no measure of patient desire for contraceptive services existed for consistent implementation across EHR systems (note that One Key Question® (23), a proprietary question that assesses desire for pregnancy in the next year, does not fulfill this need, in that it assesses future desires, rather than immediate need for services). Developed through our engagement with Reproductive Justice Consultants, patients, and industry stakeholders, this screening question asks patients for their desire for contraceptive services on the day of their visit. SINC helps refine the CBE #3682e denominator to exclude those individuals who did not receive or have documented use of a prescription contraceptive method if they indicated no desire for contraceptive services (24). Use of this novel data element helps guard against the possibility of directive or coercive counseling towards contraception that may be an unintentional result of implementation of a contraceptive use performance measure. This is particularly important given the (ongoing) history of reproductive oppression, contraceptive coercion, and biased counseling in the United States directed at women of color and low-income women (25–31).
CU-SINC encourages provision of contraceptive services. Like the currently endorsed measure of most and moderately effective contraceptive methods that relies on claims data (CBE #2903), the CBE #3682e primary measure is designed to encourage provision of these contraceptive methods to those who desire them. We recognize that some patients will prefer to use non-prescription methods that do not qualify as most or moderately effective methods, even when provided with full counseling. Currently this measure is limited to prescription methods because non-prescription methods are not routinely or consistently documented in EHRs. As a result, we do not have a currently identified benchmark for the CBE #3682e primary measure and do not expect scores to reach 100%. We hope for, and will continue to work toward, consistent EHR documentation of non-prescription contraceptive methods. At that point, we will re-assess the specifications and interpretation guidance of CU-SINC, Postpartum. The goal of the CBE #3682e submeasure related to IUD and implant provision (LARC-SINC submeasure) is to ensure access to these methods, and will be interpreted similarly to CBE #2904, in which the goal is to identify low rates of provision (i.e., below 2%) as an indication of barriers to access. We emphasize that it is important that contraceptive services are provided in a patient-centered manner that treats each person as a unique individual with respect, empathy, and understanding, providing accurate, easy-to-understand information based on the patient’s self-identified needs, goals, preferences, and values (32).
In summary, CU-SINC, Postpartum can support enhanced access to patient-centered contraceptive care, can be used in settings that cannot use the claims-based contraceptive provision measures, and provides improved measurement of whether patient’s contraceptive needs are being fulfilled compared to existing claims-based measures. By specifying the denominator as people who self-identify as needing contraceptive services, CBE #3682e shifts focus to people’s reproductive health needs as they define them. Implementing CBE #3682e will result in quality improvement initiatives that help health care organizations better meet patients’ needs by increasing patient-centered access to contraception, a step towards the goal of reproductive autonomy and well-being for all.
REFERENCES
1. Conde-Agudelo A, Rosas-Bermúdez A, Kafury-Goeta AC. Effects of birth spacing on maternal health: a systematic review. American Journal of Obstetrics and Gynecology. 2007 Apr;196(4):297–308.
2. Congdon JL, Baer RJ, Arcara J, Feuer SK, Gómez AM, Karasek D, et al. Interpregnancy Interval and Birth Outcomes: A Propensity Matching Study in the California Population. Matern Child Health J. 2022 May;26(5):1115–25.
3. Kost K, Zolna M, Murro R. Pregnancies in the United States by Desire for Pregnancy: Estimates for 2009, 2011, 2013, and 2015. Demography. 2023 Jun 1;60(3):837–63.
4. Mansour D, Inki P, Gemzell-Danielsson K. Efficacy of contraceptive methods: A review of the literature. The European Journal of Contraception & Reproductive Health Care. 2010 Feb;15(1):4–16.
5. Trussell J, Aiken ARA, Micks, E, Guthrie K. Efficacy, safety, and personal considerations. In: Contraceptive Technology. 21st ed. )Ayer Company Publishers, Inc.; 2018. p. 95–128.
6. Daniels K, Abma JC. Current Contraceptive Status Among Women Aged 15-49: United States, 2017-2019. NCHS Data Brief. 2020 Oct;(388):1–8.
7. Lee JK, Parisi SM, Akers AY, Borrerro S, Schwarz EB. The Impact of Contraceptive Counseling in Primary Care on Contraceptive Use. J GEN INTERN MED. 2011 Jul;26(7):731–6.
8. Kavanaugh ML, Pliskin E. Use of contraception among reproductive-aged women in the United States, 2014 and 2016. F&S Reports. 2020 Sep;1(2):83–93.
9. Committee opinion no. 615: Access to contraception. Obstet Gynecol. 2015 Jan;125(1):250–5.
10. Health Resouces and Services Administration and ACOG. Women’s Preventive Services Guidelines [Internet]. U.S. Department of Health and Human Services, Health Resources and Services Administration; 2019 Dec. Available from: https://www.hrsa.gov/womens-guidelines/index.html
11. Frederiksen BN, Ahrens K. Understanding the extent of contraceptive non-use among women at risk of unintended pregnancy, National Survey of Family Growth 2011-2017. Contracept X. 2020;2:100033.
12. Pazol K, Robbins CL, Black LI, Ahrens KA, Daniels K, Chandra A, et al. Receipt of Selected Preventive Health Services for Women and Men of Reproductive Age - United States, 2011-2013. MMWR Surveill Summ. 2017 Oct 27;66(20):1–31.
13. Bello JK, Rao G, Stulberg DB. Trends in contraceptive and preconception care in United States ambulatory practices. Fam Med. 2015 Apr;47(4):264–71.
14. Knutson AJ, Boyd SS, Long JB, Kjerulff KH. Early Resumption of Sexual Intercourse after First Childbirth and Unintended Pregnancy within Six Months. Women’s Health Issues. 2022 Jan;32(1):51–6.
15. White K, Teal SB, Potter JE. Contraception after delivery and short interpregnancy intervals among women in the United States. Obstet Gynecol. 2015 Jun;125(6):1471–7.
16. Agénor M, Pérez AE, Wilhoit A, Almeda F, Charlton BM, Evans ML, et al. Contraceptive Care Disparities Among Sexual Orientation Identity and Racial/Ethnic Subgroups of U.S. Women: A National Probability Sample Study. J Womens Health (Larchmt). 2021 Oct;30(10):1406–15.
17. Kavanaugh ML, Friedrich-Karnik A. Has the fall of Roe changed contraceptive access and use? New research from four US states offers critical insights. Health Affairs Scholar. 2024 Feb 1;2(2):qxae016.
18. Kavanaugh ML, Hussain R, Little AC. Unfulfilled and method‐specific contraceptive preferences among reproductive‐aged contraceptive users in Arizona, Iowa, New Jersey, and Wisconsin. Health Services Research. 2024 Jun;59(3):e14297.
19. Gavin L, Frederiksen B, Robbins C, Pazol K, Moskosky S. New clinical performance measures for contraceptive care: their importance to healthcare quality. Contraception. 2017 Sep;96(3):149–57.
20. Gavin LE, Ahrens KA, Dehlendorf C, Frederiksen BN, Decker E, Moskosky S. Future directions in performance measures for contraceptive care: a proposed framework. Contraception. 2017 Sep;96(3):138–44.
21. Moniz MH, Gavin LE, Dalton VK. Performance Measures for Contraceptive Care: A New Tool to Enhance Access to Contraception. Obstetrics & Gynecology. 2017 Nov;130(5):1121–5.
22. Person-Centered Reproductive Health Program [Internet]. [cited 2024 Jul 18]. Self-Identified Need for Contraception (SINC). Available from: https://pcrhp.ucsf.edu/sinc
23. Power to Decide. One Key Question online [Internet]. Available from: https://powertodecide.org/one-key-question
24. Person-Centered Reproductive Health Program. Assessing the Need for Contraceptive Services [Internet]. University of California, San Francisco; [cited 2023 Jul 14]. Available from: https://pcrhp.ucsf.edu/sites/g/files/tkssra4126/f/Implementation%20Guidance_5.7.21.pdf
25. Stern AM. Sterilized in the name of public health: race, immigration, and reproductive control in modern California. Am J Public Health. 2005 Jul;95(7):1128–38.
26. Downing RA, LaVeist TA, Bullock HE. Intersections of Ethnicity and Social Class in Provider Advice Regarding Reproductive Health. Am J Public Health. 2007 Oct;97(10):1803–7.
27. Becker D, Tsui AO. Reproductive Health Service Preferences And Perceptions of Quality Among Low-Income Women: Racial, Ethnic and Language Group Differences. Perspectives on Sexual and Reproductive Health. 2008 Dec;40(4):202–11.
28. Borrero S, Schwarz EB, Creinin M, Ibrahim S. The Impact of Race and Ethnicity on Receipt of Family Planning Services in the United States. Journal of Women’s Health. 2009 Jan;18(1):91–6.
29. Gomez AM, Wapman M. Under (implicit) pressure: young Black and Latina women’s perceptions of contraceptive care. Contraception. 2017 Oct;96(4):221–6.
30. Gomez AM, Fuentes L, Allina A. Women or LARC First? Reproductive Autonomy And the Promotion of Long-Acting Reversible Contraceptive Methods. Perspect Sex Repro H. 2014 Sep;46(3):171–5.
31. Gubrium AC, Mann ES, Borrero S, Dehlendorf C, Fields J, Geronimus AT, et al. Realizing Reproductive Health Equity Needs More Than Long-Acting Reversible Contraception (LARC). Am J Public Health. 2016 Jan;106(1):18–9.
32. Romer SE, Blum J, Borrero S, Crowley JM, Hart J, Magee MM, et al. Providing Quality Family Planning Services in the United States: Recommendations of the U.S. Office of Population Affairs (Revised 2024). American Journal of Preventive Medicine. 2024 Dec;67(6):S41–86.
Measure Impact
Contraception is a reproductive health technology relevant to many individuals and families throughout the life course. It can help enable individuals to build families and exercise reproductive autonomy aligned with their desires, preferences, values, and circumstances. Further, use of this technology is widespread; many patients who want to prevent pregnancy use contraceptive services to do so. In fact, in one survey of women of reproductive age, 82% reported having used some form of contraception in the past twelve months (1). Research shows desire to avoid pregnancy is strongly associated with contraceptive use (2–4) and use is prevalent; from 2015-2019, 26 million women received a contraceptive service in the United States (5). The most commonly used contraceptive methods in the United States require contact with a health care provider (6). Increased access to and use of preferred contraceptive methods (intermediate outcome) will better meet patients’ needs by increasing patient-centered access to contraception, a step towards the goal of reproductive autonomy and well-being for all (impact). In our development of our contraceptive performance measures, UCSF has worked closely with a patient stakeholder group (PSG) consisting of nine reproductive-aged individuals assigned female at birth, who endorsed the importance of access to patient-centered contraceptive care and guided our approach to measurement.
CBE #3682e uses SINC to define its denominator. This intentional inclusion serves to better center patient preferences for contraceptive care by limiting the denominator to only those who expressed interest in contraception or pregnancy prevention in a calendar year. SINC was designed as a patient-centered approach responsive to patient preferences of how they would like to be asked about reproductive health service needs. Studies have highlighted that patients prefer to be asked about their reproductive health needs through service-bound questions. In one qualitative study, patients desired contraceptive counseling availability in primary care, provided that their care team engaged in a manner that respects their autonomy and reproductive desires. To achieve this, they preferred a contraceptive care screening question that was open-ended, inclusive, and promoted autonomy, in particular a service-bound question akin to SINC (7). This finding was further tested and reinforced in a survey of over 1,000 FQHC patients in New York (8). UCSF developed SINC through our engagement with two Reproductive Justice thought leaders, nine patient advisors, and three industry stakeholders (the Coalition to Expand Contraceptive Access, the National Association of Community Health Centers, and the National Family Planning and Reproductive Health Association) to ensure the phrasing and approach resonated with patients and would not inadvertently pressure patients (9). The screening question language and response options were crafted intentionally and iteratively with patient and community input. These groups overall supported the development and utilization of this measure in clinical care and provided critical input for optimization of the SINC. PCRHP’s PSG uplifted the importance of the ability to opt out of receiving counseling through intentional inclusion of answer options such as “I am here for something else”. Further, they supported efforts to ensure the phrasing was succinct and interpretable for patients and provided key input on the appropriate frequency of asking SINC that would prevent undue pressure, leading to our decision to make this an annual screening question and the specification for the denominator of CBE #3682e to be any “Yes” response to SINC in a calendar year. Similarly, review by community groups, including the National Birth Equity Collaborative, provided input into wording and implementation guidance. Ultimately, the SINC screening tool takes a person-centered, service-bound approach that does not rely on assumptions about why patients are seeking contraception. Its role in defining the denominator for CBE #3682e serves to center patient priorities for their contraceptive care services.
REFERENCES
1. Frederiksen B, Diep K, Salganicoff A. Contraceptive Experiences, Coverage, and Preferences: Findings from the 2024 KFF Women’s Health Survey. KFF; 2024 Nov.
2. Samari G, Foster DG, Ralph LJ, Rocca CH. Pregnancy preferences and contraceptive use among US women. Contraception. 2020 Feb;101(2):79–85.
3. Rocca CH, Smith MG, Hale NL, Khoury AJ. Ranges of pregnancy preferences and contraceptive use: Results from a population‐based survey in the southeast United States. Perspect Sexual Reproductive. 2022 Sep;54(3):90–8.
4. Stulberg DB, Datta A, White VanGompel E, Schueler K, Rocca CH. One Key Question® and the Desire to Avoid Pregnancy Scale: A comparison of two approaches to asking about pregnancy preferences. Contraception. 2020 Apr;101(4):231–6.
5. Frost JJ, Mueller J, Pleasure ZH. Trends and Differentials in Receipt of Sexual and Reproductive Health Services in the United States: Services Received and Sources of Care, 2006–2019 [Internet]. Guttmacher Institute; 2021 Jun [cited 2024 Sep 10]. Available from: https://www.guttmacher.org/report/sexual-reproductive-health-services-in-us-sources-care-2006-2019
6. Daniels K, Abma JC. Current Contraceptive Status Among Women Aged 15-49: United States, 2017-2019. NCHS Data Brief. 2020 Oct;(388):1–8.
7. Manze MG, Romero DR, Sumberg A, Gagnon M, Roberts L, Jones H. Women’s Perspectives on Reproductive Health Services in Primary Care. Fam Med. 2020 Feb 7;52(2):112–9.
8. Jones HE, Calixte C, Manze M, Perlman M, Rubin S, Roberts L, et al. Primary care patients’ preferences for reproductive health service needs assessment and service availability in New York Federally Qualified Health Centers. Contraception. 2020 Apr;101(4):226–30.
9. Dehlendorf C, Perry JC, Borrero S, Callegari L, Fuentes L, Perritt J. Meeting people’s pregnancy prevention needs: Let’s not force people to state an “Intention.” Contraception. 2024 Jul;135:110400.
Performance Gap
To assess variation of our proposed eCQM in the ambulatory primary care setting, we implemented and tested the Contraceptive Use eCQM, Postpartum (CU-SINC) primary measure and submeasure in 135 Community Health Centers (CHCs) within two health care-controlled networks (HCCNs): HCCN 1 and HealthEfficient. HCCN 1 CHCs operate in states in the Midwest and West Coast of the United States, while HealthEfficient CHCs are in the Northeast region of the country. We used the following data sets from each HCCN to provide measure scores and descriptive statistics for the scores at the facility (CHC) and clinician group/practice (site) levels of analysis in the most recent measurement period (2023). We are only including facilities and clinical groups/practices that had patients who had a live birth during the measurement period.
- HCCN 1. The HCCN 1 dataset comprised all female patients aged 15-44 years from 411 outpatient sites nested in 132 CHCs in in 2023.
- HealthEfficient. The HealthEfficient dataset comprised all female clients aged 15-44 years from 14 outpatient sites nested in 3 CHCs in 2023.
For the purposes of this application, UCSF suggests that the CHC and outpatient site be considered proxies for the facility and clinician group/practice levels of analysis, respectively, since the outpatient sites operate as different locations of the same CHC.
When calculating performance scores, we excluded facilities and clinician groups/practices who had fewer than 50 patients in the #3682e denominator. As described in section 1.26, patient denominator minimums ensure that practices/facilities are large enough to have an adequate volume of patients across the year for consistent reporting. After the exclusion, HCCN 1 has 67 facilities (117 clinician groups/practices) and HealthEfficient has three facilities (three clinician groups/practices) included in the performance score calculation. See Section 2.4a Tables 1a – 1h for performance gap results.
Most or Moderately Effective Methods (Primary Measure)
The primary measure of #3682e captures new provision as well as current use of most and moderately effective contraceptive methods to accurately capture postpartum contraceptive utilization even if provided in a different calendar year or a different health care site. Across the 67 facilities within HCCN 1 (N of Persons=19,832), the overall performance score for most or moderately effective methods (primary measure) was 50.1%. The highest performing facility that provided or documented use of most or moderately effective contraceptive methods in 75.0% of its eligible postpartum patients; the lowest performing facility had a score of 15.9% during the measurement period. The overall performance score among the three facilities within HealthEfficient (N of Persons=469) was 48.2%, while the percentage of eligible patients with most or moderately contraceptive method provision or documented utilization ranged from 36.6% to 58.3%.
At the clinician group/practice (nested within facility) level of analysis, the lowest performing group/practice in HCCN 1 had a score of 15.5% for most or moderately contraceptive method provision or documented utilization during the measurement period, while the highest performing group/practice had a score of 75.7%. For HealthEfficient, the highest and lowest scores were 58.3% and 39.6%, respectively. The overall performance score for the primary measure in HCCN 1 clinician groups/practices was 52.4%; HealthEfficient’s overall performance score was 48.2%.
Long-Acting Reversible Contraception (Submeasure)
The submeasure of #3682e is a floor measure that captures provision of long-acting reversible contraception (LARC, e.g., intrauterine devices or systems, subcutaneous implants) in the postpartum period to ensure access to these methods by identifying low provision rates (i.e., below 2%). This measure therefore will not have a benchmark encouraging high rates of use. In addition, utilization in pay-for-performance or similar programs or setting a high benchmark will be explicitly defined as inappropriate, as doing so may incentivize coercive practices.
Within HCCN 1, the overall performance score for provision of long-acting reversible contraception (LARC, submeasure) was 21.7% at the facility level. The highest performing facility had a LARC provision rate of 39.8% among its eligible patients; the lowest performing facility had a measure rate of 1.9%. The overall submeasure performance score among the three facilities within HealthEfficient was 13.2%, while the percentage of eligible patients provided with LARC methods ranged from 9.7% to 15.0%.
At the clinician group/practice (nested within CHC) level of analysis, the overall performance score for the submeasure was 22.6% in HCCN 1 and 13.2% in HealthEfficient. In HCCN 1 the lowest performing group/practice did not provide LARC methods to any eligible patients (0%) during the measurement period. The highest performing HCCN 1 group/practice provided LARC methods to 50.8% of eligible patients. Among HealthEfficient clinician groups/practices, the lowest and maximum measure scores were 9.7% and 16.5%, respectively.
Our implementation and testing of CU-SINC, Postpartum in these HCCNs provide evidence of an existing performance gap for the primary measure in the primary care setting, given the variation in performance scores across facilities and clinician groups/practices within each network. The submeasure also identified some areas within our partner HCCNs in which postpartum patients experience lower access to LARC methods (i.e., submeasure rate below 2%).
Equity
Equity
Gaps in equitable contraceptive care access are well documented. One analysis of National Survey of Family Growth (NSFG) 2006-2017 waves of data collection found that overall 18.3% of women of reproductive age received contraceptive counseling, and when looking at subgroups, found that Black and Latina heterosexual women had lower odds of receiving contraceptive counseling compared to white heterosexual women (with Latina sexual minority women having the lowest odds) (1). A separate analysis of these NSFG data reported disparities in receipt of contraceptive services by both race/ethnicity and age (with younger patients being less likely to receive contraceptive services) (2). These barriers infringe upon individuals’ ability to access and use contraception, compromising their ability to achieve their reproductive goals. These disparities continue into the postpartum period – a retrospective cohort study of postpartum women found that Spanish-speaking, Hispanic foreign-born, women and women of lower socioeconomic status were less likely to receive high-quality contraceptive counseling postpartum (3). This may lead to differences in health outcomes; national surveillance data from 2014, for example, found that short interpregnancy intervals (sIPIs, i.e., shorter than twelve months), which are associated with adverse pregnancy outcomes (4), were most common among non-Hispanic Black mothers, with Hispanic mothers being more likely than white mothers to have sIPIs shorter than six months (5–7). Examining CU-SINC, Postpartum (CBE #3286e) scores overall and by subgroups is a means to evaluate for equitable access to contraceptive care.
In addition when reviewing CU-SINC, Postpartum scores, attention must also be given to larger rates of contraceptive provision, particularly long-acting reversible contraceptive (LARC) methods, to racial and ethnic minoritized patients and patients living on low incomes. Evidence has documented disparities in contraceptive counseling, including provider pressuring low-income and racially minoritized groups to use contraception, or to specifically use a LARC method, even when that is not their preference (8–11). This is tied to a long history of coercive reproductive healthcare practices grounded in structural racism and classism (12–14). Looking to the postpartum period specifically, studies show that racially minoritized patients are more likely to use LARC methods postpartum compared to white patients (15–17), while this is not hold true outside of pregnancy (18,19). Structural or interpersonal racism may play a role in these differences (15,20). Thus, the LARC-SINC submeasure of CBE #3682e, aligned with guidance for the claims-based measure CBE #2904 (Contraceptive Care – Access to LARC), is intended to be used a floor measure, monitoring for very low rates of provision (below 2%) that could indicate barriers to access. We can also examine the CBE #3682e LARC-SINC submeasure by sub-groups to identify both whether there is sufficient access to LARC methods across groups (>=2%) and for signs of potential overprovision to racially minoritized and low-income patients.
METHODS:
To evaluate differences in CU-SINC, Postpartum rates by demographic subgroups (age, race, and ethnicity), we first calculated measure rates at the facility level for each of our two data partners (Health Center Control Network [HCCN] 1 and HealthEfficient). Due to the small postpartum population in many facilities that leads to unstable facility-level rate estimates, we restricted our analysis to facilities with at least 50 eligible patients. Results are shown in Tables 1, 3, and 4 (Supplemental attachment). Then, using aggregated data at the facility level, we analyzed the differences in rates among subgroups across facilities using a multilevel regression model (21), controlling for nesting of subgroups in facilities. Age, race, and ethnicity groups are each analyzed in a separate model. Regression results are shown in Tables 2 and 5 (File located in Section 7 - Supplemental Information).
RESULTS
Most or moderately effective contraceptive methods primary measure:
Among 77 HCCN 1 facilities with a minimum of 50 eligible postpartum patients (Tables 1 & 2):
- Patients aged 15-20 years had a significantly higher rate (56.9%) compared to patients aged 21-44 years (49.4%, p=0.007).
- Compared to White patients (52.0%), Black/African American had significantly lower rates (48.2%, p=0.047).
- Compared to non-Hispanic patients (45.4%), Hispanic patients had a significantly lower rate (38.7%, p=0.001) and those who did not report ethnicity had a significantly higher rate (58.4%, p<0.001).
Among three HealthEfficient facilities, our results (Tables 3 & 5):
- Patients aged 15-20 years had a significantly higher rate (62.8%) compared to patients aged 21-44 years (46.1%, p<0.001).
- Compared to White patients (51.3%), Black/African American patients had a significantly lower rates (41.1%, p=0.005).
- Compared to non-Hispanic patients (44.0%), Hispanic patients had a slightly lower but significant rate (43.2%, p<0.001) and those who did not report ethnicity had a significantly higher rate (57.6%, p<0.001). However, due to the large number of patients in the “Unreported” group, these results may not represent reliable differences by ethnicity categories.
LARC-SINC submeasure
Among 77 HCCN 1 facilities with a minimum of 50 eligible postpartum patients (Tables 1 & 2):
- Patients aged 15-20 years had a significantly higher rate (27.6%) compared to patients aged 21-44 years (21.3%, p<0.001).
- Compared to White patients (23.4%), Black/African American patients had significantly lower rates (17.4%, p=0.001).
- Compared to non-Hispanic patients (18.5%), those who did not report ethnicity had significantly higher rates (26.1%, p<0.001).
Among three HealthEfficient facilities, (Tables 4 & 5):
- Patients aged 15-20 years had a significantly higher rate (17.9%) compared to patients aged 21-44 years (13.1%, p=0.002).
- Compared to White patients (12.5%), Black/African American patients had a significantly lower rates (7.1%, p=0.034) and those who did not report race had a higher rate (19.4%, p=0.015).
- Compared to non-Hispanic patients (11.4%), those who did not report ethnicity had a significantly higher rate (19.5%, p<0.001). Due to the large number of “Unreported” group patients for ethnicity in health center 2 and 3 and a 0 count of Hispanic patients in these health centers, these results may not represent reliable differences by ethnic categories.
INTERPRETATION AND ANTICIPATED IMPACT:
We found significantly lower rates of most or moderately effective contraceptive use among Black patients compared to white patients within both the HCCN 1 and HealthEfficient cohorts. This may indicate an issue with inequitable access, with non-White patients not able to obtain postpartum contraception at the same rate as white patients to support them in meeting their reproductive goals. This may result in a higher number of undesired pregnancies that do not align with their pregnancy preferences among non-White patients compared to White patients (5,22). Alternatively, this may reflect differences in preferences for prescription contraception by race/ethnicity. Assessing the LARC-SINC submeasure, all groups surpass the floor 2% at the HCCN level, suggesting suitable access across groups. However, it is worth noting that among the HCCN 1 cohort, Black patients did report significantly lower rates of LARC provision, reflecting the results of the most and moderately effective primary measure results. Accountable entities could utilize this finding as an indicator of an equity issue that would warrant further exploration to determine whether there are differential counseling practices directed toward different ethnic groups, such as through use of the visit-specific Person-Centered Contraceptive Counseling measure (CBE #3543). The differences by age in HCCN 1 may indicate higher levels of access to desired contraception among younger individuals as compared to those over the age of 20. Alternative explanations include that providers may be more directive with adolescents, as concern about their potential fertility may motivate less patient-centered care, or that there may be differences in preferences by age group. Ensuring that patients of all ages who have the potential for pregnancy have their needs assessed and met in a patient-centered manner can be facilitated by examining these data and assessing for differential practices.
REFERENCES
1. Agénor M, Pérez AE, Wilhoit A, Almeda F, Charlton BM, Evans ML, et al. Contraceptive Care Disparities Among Sexual Orientation Identity and Racial/Ethnic Subgroups of U.S. Women: A National Probability Sample Study. J Womens Health (Larchmt). 2021 Oct;30(10):1406–15.
2. Pazol K, Robbins CL, Black LI, Ahrens KA, Daniels K, Chandra A, et al. Receipt of Selected Preventive Health Services for Women and Men of Reproductive Age - United States, 2011-2013. MMWR Surveill Summ. 2017 Oct 27;66(20):1–31.
3. Coleman-Minahan K, Potter JE. Quality of postpartum contraceptive counseling and changes in contraceptive method preferences,. Contraception. 2019 Dec;100(6):492–7.
4. Wang Y, Zeng C, Chen Y, Yang L, Tian D, Liu X, et al. Short interpregnancy interval can lead to adverse pregnancy outcomes: A meta-analysis. Front Med (Lausanne). 2022;9:922053.
5. Thoma M, Copen C, Kirmeyer S. Short interpregnancy intervals in 2014: Differences by maternal demographic characteristics. Hyattsville, MD: National Center for Health Statistics; 2016. Report No.: NCHS data brief, no 240.
6. Brown J, Chang X, Matson A, Lainwala S, Chen MH, Cong X, et al. Health disparities in preterm births. Front Public Health. 2023;11:1275776.
7. Office of Disease Prevention and Health Promotion. Data Source: National Vital Statistics System - Natality (NVSS-N) [Internet]. n.d. Available from: https://health.gov/healthypeople/objectives-and-data/data-sources-and-methods/data-sources/national-vital-statistics-system-natality-nvss-n
8. Gomez AM, Wapman M. Under (implicit) pressure: young Black and Latina women’s perceptions of contraceptive care. Contraception. 2017 Oct;96(4):221–6.
9. Dehlendorf C, Ruskin R, Grumbach K, Vittinghoff E, Bibbins-Domingo K, Schillinger D, et al. Recommendations for intrauterine contraception: a randomized trial of the effects of patients’ race/ethnicity and socioeconomic status. Am J Obstet Gynecol. 2010 Oct;203(4):319.e1-8.
10. Downing RA, LaVeist TA, Bullock HE. Intersections of Ethnicity and Social Class in Provider Advice Regarding Reproductive Health. Am J Public Health. 2007 Oct;97(10):1803–7.
11. Borrero S, Schwarz EB, Creinin M, Ibrahim S. The Impact of Race and Ethnicity on Receipt of Family Planning Services in the United States. Journal of Women’s Health. 2009 Jan;18(1):91–6.
12. Stern AM. Sterilized in the name of public health: race, immigration, and reproductive control in modern California. Am J Public Health. 2005 Jul;95(7):1128–38.
13. Roberts D. Killing the Black Body. New York, NY: Penguin Random House; 1998.
14. Brandi K, Fuentes L. The history of tiered-effectiveness contraceptive counseling and the importance of patient-centered family planning care. American Journal of Obstetrics and Gynecology. 2020 Apr;222(4):S873–7.
15. Rodriguez MI, Meath THA, Watson K, Daly A, McConnell KJ, Kim H. Decomposition analysis of racial and ethnic differences in receipt of immediate postpartum, long-acting, reversible, and permanent contraception. Contraception. 2024 Oct;138:110512.
16. Steenland MW, Pace LE, Sinaiko AD, Cohen JL. Medicaid Payments For Immediate Postpartum Long-Acting Reversible Contraception: Evidence From South Carolina: Study examines South Carolina’s Medicaid program payments for the immediate postpartum placement of long-acting reversible contraception for women giving birth from 2010 to 2014. Health Affairs. 2021 Feb 1;40(2):334–42.
17. Liberty A, Yee K, Darney BG, Lopez-Defede A, Rodriguez MI. Coverage of immediate postpartum long-acting reversible contraception has improved birth intervals for at-risk populations. American Journal of Obstetrics and Gynecology. 2020 Apr;222(4):S886.e1-S886.e9.
18. Kavanaugh ML, Pliskin E. Use of contraception among reproductive-aged women in the United States, 2014 and 2016. F&S Reports. 2020 Sep;1(2):83–93.
19. Kavanaugh ML, Jerman J, Finer LB. Changes in Use of Long-Acting Reversible Contraceptive Methods Among U.S. Women, 2009–2012. Obstetrics & Gynecology. 2015 Nov;126(5):917–27.
20. Taylor JK. Structural Racism and Maternal Health Among Black Women. J Law Med Ethics. 2020;48(3):506–17.
21. Leyland AH, Groenewegen PP. Multilevel modelling and public health policy. Scand J Public Health. 2003 Aug;31(4):267–74.
22. Samari G, Foster DG, Ralph LJ, Rocca CH. Pregnancy preferences and contraceptive use among US women. Contraception. 2020 Feb;101(2):79–85.
Feasibility
Feasibility
Background
UCSF implemented and tested the Contraceptive Use eCQM (CU-SINC), Postpartum (CBE #3682e) primary measure and LARC-SINC submeasure in 425 locations within 136 Community Health Centers (CHCs), nested within two health care-controlled networks (HCCNs). Thus, CBE #3682e was tested and implemented in two different electronic health record (her) systems: HCCN 1 utilizes its customized version of Epic, and HealthEfficient uses eClinicalWorks (eCW).
The first facilities to implement CBE #3682e were the nine facilities within two HCCNs included in our analysis that also participated in Innovating Contraceptive Care in Community Health Centers (ICC in CHCs). ICC in CHCs was a 2020-2023 quality improvement (QI) initiative to improve the CHCs’ contraceptive services through the implementation of evidence-based strategies to incorporate person-centered contraceptive care. This project provided participating facilities and clinician group/practices with resources to effectively incorporate CBE #3682e primary measure and submeasure, including the Self-Identified Need for Contraception (SINC) question. SINC allows CHCs to ask patients about their contraceptive needs at their visit. Our QI initiative also included one additional CHC that implemented the data element into their AthenaOne EHR system. We excluded this additional CHC from measure testing because we do not have access to their data.
Availability of data elements
UCSF recommends implementation of SINC to center contraceptive care on the patients and their stated desire for contraceptive services, but CBE #3682e can still be calculated when SINC is not asked or implemented.
To assist our data partners to implement the SINC data element, UCSF first worked with ten CHCs using three different EHR systems (eCW, Epic, AthenaOne) for the ICC in CHCs QI initiative. In these instances, EHR changes were able to be implemented in about three months in collaboration with clinical and vendor teams. The implementation burden depends on the change process for the EHR at the health system to embed the SINC data element into the relevant EHR template so that care teams can use SINC. HCCN 1 uses its customized version of Epic, a centrally managed EHR system and easily implemented the SINC question, first across the ICC-participating CHCs and then for all remaining CHCs. In HealthEfficient, each CHC uses eCW and initially implemented SINC at the individual ICC-participating clinician group/practices before incorporating SINC at the non-ICC clinician group/practices. This de-centralized process required more coordination and time. During the final meeting for ICC in CHCs, one participating CHC using Epic stated, “(HCCN 1) did a phenomenal job with our SINC question (and) made it so user-friendly for our clinicians”. An ICC-participating CHC in HealthEfficient at the same meeting reported, “It was challenging to get the SINC implemented into our E[H]R; we use eCW and also train the staff, and (it was) more so a resource challenge, finding the time and staff to distribute the information about the changes.” In contrast, a different participating CHC in HealthEfficient, noted that implementing SINC screening into their EHR system facilitated their QI activities and stated, “I think that what made it easy for me … is that [the HIT staff] had already incorporated the [SINC] questions into our eCW. So it was something that was already running.” Most systems, from our experience, are able to implement the SINC EHR template changes in about three months, with some systems requiring less time. We note that clinical sites across the country have begun to implement and use the SINC data element since it was made available as a LOINC code in August 2021, demonstrating the feasibility of this implementation. These clinical sites include the Planned Parenthood Federation of America, which includes SINC as its recommended approach in Medical Standard and Guidelines to assessing reproductive needs, and sites affiliated with Upstream USA, including Cambridge Health Alliance. In 2024, HHS Health Resources Services Administration (HRSA) released guidance about reporting of reproductive needs screening in Uniform Data System (UDS) that highlighted SINC. Since that time, UCSF has received several requests for technical assistance from EHR vendors that work with FQHCs who are interested in incorporating this data element.
Maintenance updates to CBE #3682e specification impact on data structures and availability
UCSF’s updates to CBE #3682e include the following revisions: adding a new, active code set to replace the inactive, not maintained set used to create the denominator in the specification that was approved for trial use and updating the value sets that we author on VSAC to reflect annual code system edits to keep the code sets current. We also changed the time period for SINC to be asked of postpartum patients because prenatal care protocols often have providers ask pregnant patients about postpartum contraception during the second trimester of pregnancy. To align with this standard of prenatal care, SINC must be documented for postpartum patients no earlier than 16 weeks before estimated delivery date (EDD) (which should occur in the second trimester of pregnancy) and within 90 days after live birth delivery date (which aligns with the recommendations of the American College of Obstetricians and Gynecologists for postpartum visit timing).
These small changes did not affect the data structures and availability of electronic clinical quality measurement (eCQM) elements in our HCCN partners’ EHR systems.
Feasibility scorecard results
Our HCCN data partners completed the feasibility scorecards for the CU-SINC, Postpartum to provide feedback on the measure and its calculation in their respective EHR systems (see the attached file named “3682e_eCQM-Feasibility-Scorecard_508.xlsx”). The HCCNs evaluated all data elements (n=84) utilized in CBE #3682e in the feasibility scorecard to ensure a thorough assessment, but CBE #3682e does not require all 84 elements for calculation.
HealthEfficient indicated that 25% of the eCQM data elements scored 0 in the Data Availability feasibility domain (i.e., data are readily available in a structured format), 27% scored 0 in the Data Accuracy domain (i.e., information contained in the data is correct), 33% scored 0 in the Data Standards domain (i.e., data element is coded using a nationally accepted terminology standard and mapped to the QDM), and 27% scored 0 in the Workflow domain (i.e., extent to which data element capture impacts the workflow for the user). In contrast, HCCN 1 reported that 13% of eCQM data elements scored 0 for availability, 7% scored 0 for accuracy, 15% scored 0 for standards, and 17% scored 0 for workflow. The HCCN data partners provided their solutions to address the CU-SINC, Postpartum elements that did not achieve 100% across the four feasibility domains (see the tabs named, “Feasibility Plan” in the attached scorecard).
The 0 scores for Data Availability, Data Standards, and Data Workflow for both partners were reported mostly because neither EHR system uses all the code systems available in the Centers for Medicaid and Medicare (CMS) Measure Authoring Development Integrated Environment (MADiE) and National Library of Medicine (NLM) Value Set Authority Center (Value Set Authority Center (VSAC) web-based systems. HealthEfficient had 23 data elements scoring 0 for the Workflow domain while HCCN 1 had 14 elements scoring 0 for Workflow, indicating that these elements are not routinely collected during clinical care because both partners’ systems do not use SNOMED CT. For example, neither HealthEfficient’s nor HCCN 1’s system uses SNOMED CT or LOINC to define most or moderately effective contraceptive methods. However, this does not negatively impact the ability to calculate the measure, as the specifications take into account the fact that different EHR systems use multiple combinations of code systems. As a result, CU-SINC, Postpartum includes several terminologies that define the measure’s data elements to facilitate measure use and calculation, even if one or more code systems are excluded. In the case of contraceptive methods, HealthEfficient and HCCN 1 accomplish this using ICD-10-CM codes, which are included in our specifications and stated in the CBE #3682e feasibility scorecard plans to address the absence of SNOMED CT codes. One example of this redundancy is that we use six value sets to assess utilization of oral contraceptive pills in #3682e; these value sets contain codes from ICD-10-CM, CPT, HCPCS, RXNORM, LOINC, and SNOMED CT code systems. Patients in the #3682e denominator who have at least one code from any of these six oral contraceptive pill value sets for a visit in the appropriate date range are counted in the CU-SINC, Postpartum numerator.
Another example of this flexibility is the “Prenatal Care Visits Group” definition that defines the CBE #3682e denominator. The value sets that define prenatal care visits include codes from ICD-10-CM, CPT, HCPCS, and SNOMED CT. If a patient’s EHR record contains the prenatal care visit codes from ICD-10-CM and CPT, then that record can count as a prenatal visit, even if that EHR system does not use SNOMED CT codes. Additionally, both HCCNs had data elements that scored 0 in the Data Accuracy domain because the participating CHCs do not routinely offer services related to the data elements for live birth delivery, even if the partners’ EHR systems contain structured fields in which the information can reside. UCSF specified CU-SINC, Postpartum so that the measure allows for use of EDD when Live Birth Delivery Date is unavailable or missing from a patient’s record. Although our partners reported in the feasibility scorecard that Live Birth Delivery Date may not often be reported (because CHCs do not offer labor and delivery services), the EDD data element will be recorded for patients visiting CHCs for prenatal care services, making it available in both HCCNs’ EHR systems and for calculation of the primary measure and submeasure.
Finally, the new SINC data element was initially not integrated into the EHR systems of the nine CHCs participating in the ICC in CHCs QI initiative, but they all ultimately implemented it in order to both assess contraceptive needs and calculate CBE #3682e. Unlike HCCN 1, which easily implemented SINC in its centrally-managed EHR system across their CHCs, HealthEfficient had to implement the SINC question and response options in their EHR for each participating CHC as separate processes over a longer time period. While one HealthEfficient CHC reported a smooth implementation of SINC use, the other CHCs found it more challenging to add the new element. Both HCCN 1 and HealthEfficient CHCs have maintained their use of SINC in primary care visits after the end of ICC in CHCs and expanded SINC use to additional clinician groups/practices that did not originally participate in our QI initiative.
Estimates of the costs or burden of CU-SINC, Postpartum implementation
To implement SINC in partner EHR systems, our partner HCCNs changed their primary care EHR template to include the SINC question and its response options as structured fields. The EHR system programmers and analysts bore the burden of mapping EHR data required for the measure to the eCQM concepts according to the CHCs’ regular clinical workflows for the purposes of calculating and reporting the measure rates. After implementation, we estimate that data entry by clinical staff will require a minute or less to conduct. The SINC question and its response options are also specified in the LOINC code system and can be mapped as such for EHR systems that include LOINC.
With respect to clinical implementation, UCSF advocates for SINC to be included as part of a patient-centered workflow, enhancing access to contraceptive services for those who need it. Therefore, the added time burden on the clinical workflow will be offset by enhanced quality of care. The SINC Screening question is estimated to require a minute or less to implement within clinical workflows.
Data abstraction, measure calculation, and reporting
In order to calculate the measure, it is necessary to perform a data extraction of all the required elements for measure calculation for all encounters in two consecutive calendar years for the patients assigned female at birth. The burden of this process will depend on the structure of the clinical datasets and the experience of EHR programmers. During the ICC in CHCs project period, HCCN 1 and HealthEfficient performed these extracts, and we provided quality assurance (QA) checks of their datasets to calculate CU-SINC, Postpartum. Our QA involved confirming that the analysis datasets included both the required elements for measure calculation and all encounters occurring in two consecutive calendar years for patients assigned female at birth, ages 15-44 years. When questions about calculation arose, UCSF and Far Harbor collaborated with the HCCN Health Information Technology staff to investigate and agree on a solution. When a health system first calculates #3682e, the measure users and EHR programmers may require technical assistance and additional QA checks to assess the accuracy of their measure scores. As they gain familiarity with calculating the measure, the time and resources needed for calculation should be reduced. Furthermore, as health systems adopt the use of CU-SINC, Postpartum, these experienced measure users and EHR programmers could answer questions or assist other similar health systems who want to implement SINC and calculate #3682e. One partner implemented a new analytics and population health reporting platform system in CHCs working on the QI initiative; in our QA, we helped the partner troubleshoot this new query system to obtain accurate encounter records to calculate CU-SINC, Postpartum and assess the impact of their QI strategies on contraceptive use and LARC provision.
The data elements comprising CBE #3682e, including the SINC data element, reside within the clinical agency’s secure EHR system server. Each HCCN upholds the normal requirements to ensure that its EHR data elements, including SINC and the elements defining CU-SINC, Postpartum, are shared only with entities that have active data use agreements and the infrastructure to securely exchange Protected Health Information / Potentially Identifiable Information (PHI/PII).
When calculating CBE #3682e performance scores, the UCSF and Far Harbor team excluded facilities and clinician groups/practices who served fewer than 50 patients during the measurement period. We recommend this cutoff to measure users as well, who will calculate and report CBE #3699e rates. Patient denominator minimums ensure that practices/facilities are large enough to have an adequate volume of patients across the year for consistent reporting. Reporting measure rates only for units with 50 or more patients also helps to protect patient confidentiality. After the exclusion, HCCN 1 has 67 facilities (117 clinician groups/practices) and HealthEfficient has three facilities (three clinician groups/practices) included in the performance score calculation presented in this application.
Adjustments to CBE #3682e in response to feasibility assessment
UCSF first started developing CBE #3682e in 2019 and successfully obtained trial use approval for the eCQM from the previous consensus-based entity (CBE). During that initial development phase, UCSF heard from its potential HCCN data partners that their respective EHR systems contained only some code systems used to specify eCQMs. In anticipation of this challenge to calculate the measure in different EHR systems, we designed CBE #3682e and our other eCQMs to be flexible and usable across EHR systems regardless of the different combinations of code systems that each EHR system can employ. Recognizing that different EHR systems use multiple combinations of code systems, CU-SINC, Postpartum includes several, redundant terminologies that define the measure’s components to facilitate measure use and calculation, even if one code system is excluded. For example, as previously discussed in Section 4.1a, contraceptive use and LARC provision for the primary measure and submeasure numerators are specified in both ICD-10-CM and SNOMED CT. So even though HealthEfficient’s and HCCN 1’s EHR systems do not contain SNOMED CT, they can still use ICD-10-CM codes to assess contraceptive use and LARC provision. This adjustment is also stated in the CBE #3682e feasibility scorecard plans for HealthEfficient and HCCN 1 respectively.
Proprietary Information
Scientific Acceptability
Testing Data
Electronic health records (EHR) data from two networks were used for reliability and validity testing:
- Health Center Control Network (HCCN) 1. The HCCN 1 uses its customized Epic system and the dataset comprised all female patients aged 15-44 years from 411 outpatient sites (clinician groups/practices) nested in 132 Community Health Centers (facilities) located in Washington, Ohio, California, and Oregon in 2023.
- HealthEfficient. The HealthEfficient uses the eCW system, and the dataset comprised all female clients aged 15-44 years from 14 outpatient sites (clinician groups/practices) nested in three Community Health Centers (CHCs, facilities) located in Connecticut, New York, and New Jersey in 2023.
- HCCN 1. For accountable entity-level reliability and validity testing, the full dataset was used. For data element validity testing, we utilized data from a random sample of 150 female patients ages 15-44 who visited eight sites in calendar year 2023. Due to the low volume of patients who had a birth in HCCN 1, we oversampled eight additional women who had an estimated delivery date, making a total of 158 women for this element. As it was not feasible for HCCN1 to do data element validity testing at all sites, these eight sites were randomly selected to provide an overall assessment of the data validity in the network. The age distribution is comparable between the sample and the full dataset.
- HealthEfficient. For accountable entity-level reliability and validity testing, the full dataset was used. For data element validity testing, we utilized data from a random sample of 144 female patients ages 15-44 years who visited five sites in calendar year 2023. As it was not feasible for HealthEfficient to do data element validity testing at all sites, these five sites were randomly selected to provide an overall assessment of the data validity in the network. The age distribution is comparable between the sample and the full dataset.
Accountable Entity-Level Reliability and Validity
The measure was tested at the clinician group/practice-level and the facility-level for both data partners. Accountable entity-level reliability and validity testing used full denominator population in the datasets from HCCN 1 and HealthEfficient networks described above.
HCCN 1 includes 411 outpatient sites (clinician groups/practices) nested in 132 CHCs (facilities) located in Washington, Ohio, California, and Oregon. HealthEfficient includes 14 outpatient sites (clinician groups/practices) nested in 3 Community Health Centers (facilities) located in Connecticut, New York, and New Jersey.
Data Element Validity
Eight sites from HCCN 1 and five sites from HealthEfficient provided data and the analysis was conducted using aggregated numbers across all sites for each network. A sample of 150 patients were randomly selected from the eight sites in HCCN 1 (158 patients were selected for estimated delivery date) and 144 patients were randomly selected from the five sites in HealthEfficient.
See file “5.2.3a. Characteristics, reliability full tables and method_PP.docx” in section 5.2.3a, Tables 1-4 for tables of characteristics of units of the eligible population.
Reliability
Several methods have been suggested to assess the reliability of provider-level performance measures (1-3). These methods may focus on different facets of reliability such as consistency across time, consistency across raters or units, or variability at different levels of aggregation. PQM has suggested a signal-to-noise approach as one way to evaluate measure reliability (4). For this application, reliability was estimated from a Beta-binomial model using parametric empirical Bayes methods. This method is better able to produce reliability estimates when the performance measures fall in the extreme ranges of the distribution (below 10% or above 90%), whereas the signal-to-noise approach may produce unstable results under those conditions, especially when the patient size is small. Two distributional shape parameters (alpha and beta) were estimated from the observed quality scores, and reliability was then calculated as a function of alpha, beta, and total patient count for each unit of analysis. Overall reliability in this context represents the ability of the proposed measure to confidently distinguish the performance of one entity (e.g., site) from another. A detailed description of this method is demonstrated in Section 5.2.3a. Appendix, where we lay out the formulation of the method and describe how it improves upon the Beta-binomial approach applied in previous studies (4-8). This method is currently under revise and resubmit process for journal publication. A threshold of 0.6 is used based on the PQM Endorsement and Maintenance Guidebook.
Measure developers frequently recommended setting a minimum patient size for performance measurement when estimating at the facility or provider level because patient size has a large impact on reliability (9, 10). In this analysis, we tested reliability using 50 as a cutoff of total patients served at each unit of analysis to show how such threshold impacts reliability.
REFERENCES
1. Adams, J. L., Mehrotra, A., Thomas, J. W., & McGlynn, E. A. (2010). Physician cost profiling--reliability and risk of misclassification. The New England journal of medicine, 362(11), 1014–1021. https://doi.org/10.1056/NEJMsa0906323
2. Scholle, S. H., Roski, J., Adams, J. L., Dunn, D. L., Kerr, E. A., Dugan, D. P., & Jensen, R. E. (2008). Benchmarking physician performance: reliability of individual and composite measures. The American journal of managed care, 14(12), 833–838.
3. Fung, V., Schmittdiel, J. A., Fireman, B., Meer, A., Thomas, S., Smider, N., Hsu, J., & Selby, J. V. (2010). Meaningful variation in performance: a systematic literature review. Medical care, 48(2), 140–148. https://doi.org/10.1097/MLR.0b013e3181bd4dc3
4. Adams, John L. (2009) The Reliability of Provider Profiling: A Tutorial. Santa Monica, CA: RAND Corporation. https://www.rand.org/pubs/technical_reports/TR653.html.
5. Adams, J. L., & Paddock, S. M. (2017). Misclassification Risk of Tier-Based Physician Quality Performance Systems. Health services research, 52(4), 1277–1296. https://doi.org/10.1111/1475-6773.12561
6. Blair, R., Liu, J., Rosenau, M., Brannan, M., Hazelwood, N., Gray, K. F. & Schmitt, A. (2015). Development of Quality Measures for Inpatient Psychiatric Facilities. Development, 2, 04.
7. Kazis, L. E., Rogers, W. H., Rothendler, J., Qian, S., Selim, A., Edelen, M. O.& Butcher, E. (2017). Outcome performance measure development for persons with multiple chronic conditions. RAND.
8. Staggs, V. S., & Cramer, E. (2016). Reliability of Pressure Ulcer Rates: How Precisely Can We Differentiate Among Hospital Units, and Does the Standard Signal-Noise Reliability Measure Reflect This Precision?. Research in nursing & health, 39(4), 298–305. https://doi.org/10.1002/nur.21727
9. HEDIS, N. (2007). Technical Specifications for Physician Measurement. Washington DC: National Committee for Quality Assurance.
10. Safran, D. G., Karp, M., Coltin, K., Chang, H., Li, A., Ogren, J., & Rogers, W. H. (2006). Measuring patients' experiences with individual primary care physicians: results of a statewide demonstration project. Journal of General Internal Medicine, 21(1), 13-21.
Reliability analyses were conducted at two levels (facility and clinician group/practice), stratified by three age categories (15-20, 21-44, and 15-44 years). More detailed information including reliability estimates for each unit at each level for HealthEfficient can be found in Section 5.2.3a Tables 5-10. Detailed reliability results at the unit level for HCCN 1 are not included due to the large number of units.
Our tested reliability is consistently greater than 0.60 at the facility and clinician group/practice levels when using the patient size threshold of 50, showing adequate reliability at these levels. This was mostly driven by the large number of patients per unit at these levels.
For the most or moderately effective methods, reliability ranges from 0.854 to 1 for HCCN 1 and 0.985 to 0.994 for HealthEfficient for the 15-44 years age group at the facility level, when using the threshold of 50 patients per unit. At the clinician group/practice level, reliability ranges from 0.849 to 0.999 for HCCN 1 and 0.817 to 0.991 for HealthEfficient for the 15-44 years age group. HealthEfficient has a small number of units (n=3) at the facility level.
For the LARC-SINC submeasure, reliability ranges from 0.647 to 0.999 for HCCN 1 and 0.971 to 0.988 for HealthEfficient for the 15-44 years age group at the facility level, when using the threshold of 50 patients per unit. At the clinician group/practice level, reliability ranges from 0.640 to 0.996 for HCCN 1 and 0.598 to 0.975 for HealthEfficient for the 15-44 years age group.
Our results also show that with the minimum threshold of 50 patients, reliability improves greatly at the clinician group/practice level, compared to not using a threshold. This is because at this level patient size tends to be small. Measure developers frequently recommend the minimum patient size approach for performance and our analysis suggests that a minimum of 50 patients yields sufficient reliability for our measure.
Our tested reliability is consistently greater than 0.60 at the facility and clinician group/practice levels for all age groups, showing adequate reliability at these levels. Overall, testing results showed that the Contraceptive Use electronic clinical quality measure (CU-SINC), Postpartum measure, as currently specified, can distinguish the true contraceptive use performance in facilities and clinician groups/practices from one entity to another.
Validity
Data element level validity testing
For HCCN 1, a total of 150 female patients aged 15-44 years in 2023 were randomly sampled from eight sites. Due to the low volume of patients who had a birth in HCCN 1, we oversampled eight additional women who had an estimated delivery date documented in their medical record, making a total of 158 women for this element. For HealthEfficient, a total of 144 female patients aged 15-44 years in 2023 were randomly sampled from five sites. For each of these patients, data elements used for CU-SINC, Postpartum calculation were compared between the EHR records and the patient charts, and agreement numbers were summarized in a two by two table (yes/yes, yes/no, no/yes, and no/no) for each element. We compared 13 different data composites/fields reflecting combinations of multiple data elements to capture core constructs of the measures, such as use of contraceptive methods and exclusion conditions: female sterilization, implants, intrauterine devices, injectables, contraceptive pills, contraceptive patch, vaginal ring, had a qualifying encounter, SINC response, had a prenatal visit, had a live birth delivery date, had an estimated delivery date, and had a non-live birth. Using the patient chart as the authoritative source, we calculated sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) for each data element.
Empirical validity testing
We tested validity of CU-SINC, Postpartum by exploring whether they were correlated with a similar quality measure listed below:
- Postpartum care: Percentage of deliveries of live births on or between October eighth of the year prior to the measurement year and October seventh of the measurement year that had a postpartum visit on or between seven and 84 days after delivery.
We hypothesize that facilities and clinician groups/practices that perform well on postpartum contraceptive care services should perform well on postpartum care, and therefore, this related measure will be positively correlated to CU-SINC, Postpartum. This hypothesis is based on the assumption that these measure denominators represent the same group of women who received pregnancy-related clinical care over the same period of time. American College of Obstetricians and Gynecologists (ACOG) guidelines call for contraception to be part of the postpartum visit (1). On the other hand, literature suggests that a proportion of patients have unmet contraception needs even with postpartum visit attendance, suggesting other barriers to their contraception needs (2). Thus, we hypothesize that the positive correlation will be moderate. The claims-based Contraceptive Care – Postpartum measure also showed a moderately positive association with the postpartum care measure.
We used a cutoff of 50 eligible patients per unit to exclude facilities and clinician groups/practices who served only a small number of patients during the measurement year. As described in section 1.26, patient denominator minimums ensure that practices/facilities are large enough to have an adequate volume of patients across the year for consistent reporting.
We performed correlation analyses at the facility and clinician group/practice levels for HCCN 1. We are not reporting results for the 15-20 year age group at the clinician group/practice level due to the small number of units (n=5) after excluding groups/practices with fewer than 50 patients. Similarly, for HealthEfficient, due to their small number of facilities (n=3) and clinician groups/practices after applying the cutoff of 50 as the minimum patient number (n=3), we were not able to conduct the correlation analysis for this partner.
To test the correlation, we used two different approaches. In the first approach, we used the Pearson’s correlation test. This test estimates the strength of the linear association between two continuous variables. The correlation coefficient ranges from -1 to +1. A value of 1 indicates a perfect positive linear correlation between two variables. A value of 0 indicates no linear association. A value of -1 indicates a perfect negative linear relationship between two variables. We used a threshold of p < .05 to evaluate the statistical significance of test results.
Even though Pearson’s correlation test is widely used to evaluate the correlation between two measures, it is only optimal in cases where linearity can be assumed. Crucially, the bounded nature of the variation in the proportion of CU-SINC, Postpartum (i.e., 0 and 1) means that estimates of association that assume linearity on the measure rates may be biased. This is a particular concern when the count of service events is either very high or very low relative to the total number of patients in a cluster. In addition, the correlations captured by the Pearson correlation matrix are averaged over the “true” and error variances. As a result, Pearson's correlation could downwardly bias the correlation substantially in cases when the clusters are small with few patients and where the measurement error is high.
Given these limitations with Pearson’s correlation test, we present a novel alternative approach. We employ a multilevel correlation estimation method to test the relationship between the contraceptive care measure and the related measures. The model is based on a multivariate generalized linear mixed model framework (3). By employing a logit transformation of the binomial proportions, the model relaxes the linearity assumption on the original measurement scale. In addition, it analytically separates “true” score variance from measurement error by presenting measurement error as a random, binomial deviate, conditional on each cluster’s “true” quality measure. Thus, the multilevel correlation estimation approach captures the correlation more accurately when the cluster size is small.
In the present analyses, the parameters of the multilevel model were estimated using a hierarchical SAS 9.4 GLIMMIX procedure with a log link function and fully unstructured residual error. Parameters were estimated by pseudo-maximum-likelihood using the Laplace method. The error structure was reported as correlation coefficients and variances. We are also able to provide 95% confidence limits for the estimates using likelihood bounds, which is far more informative than the single p-value for statistical significance. Rather than estimating all possible pairwise associations simultaneously, we estimated each pairwise association in a separate model to speed up and improve model convergence. In the appendix of the application, we provide a detailed description of the model with example statistical programming code.
REFERENCE
1. Optimizing postpartum care. ACOG Committee Opinion No. 736. American College of Obstetricians and Gynecologists. Obstet Gynecol 2018;131:e140–50. https://doi.org/10.1097/AOG.0000000000002633
2. Congdon JL, Bardach NS, Franck LS, Brindis CD, Boscardin WJ, Carrasco Z, Cabana MD, Dehlendorf C. Postpartum Family Planning in Pediatrics: A Survey of Parental Contraceptive Needs and Health Services Preferences. Acad Pediatr. 2023 Sep-Oct;23(7):1417-1425. doi: 10.1016/j.acap.2023.03.009. Epub 2023 Mar 22. PMID: 36958531; PMCID: PMC11166476.
3. Coull BA, Agresti A. Random effects modeling of multiple binomial responses using the multivariate binomial logit-normal distribution. Biometrics. 2000 Mar;56(1):73-80.
Results of the data element validity analyses are shown in Section 5.3.4a Tables 1-2. Results of the empirical validity testing are shown in Section 5.3.4a Tables 3-6.
Data element level validity
Among both HCCNs, sensitivity, specificity, PPV, and NPV were above 0.7 for most of the data elements.
For both HCCNs, we found that the patient charts captured a small number of sterilization cases (five in HCCN 1 and three in HealthEfficient), whereas standardized medical codes for sterilization were not found in the EHR data extract. This is largely because the CHCs from which the data were extracted are outpatient centers and do not provide sterilization procedures themselves. When documenting sterilization that has occurred in other sites, this can be documented in the surgical history section of their EHR systems using unstructured fields (as opposed to using ICD-10 codes to document this in the problem list).
For HealthEfficient, we found one vaginal ring that was observed in the patient charts but not in the EHR data extract, resulting in sensitivity of zero for this element. Given the low use of vaginal ring, this validity testing result for this element is likely unreliable and we don’t expect it to have a meaningful impact on our measure calculation.
For HCCN 1, we also found three live birth dates and one non-live birth that were observed in the patient charts but not in the EHR data extract. This is due to the same reason as sterilization in that pregnancy outcome information was often entered into the system as history records, instead of standardized medical terminologies such as CPT or ICD codes. This level of discrepancy for these elements is expected in outpatient settings.
To compensate for the possibility of missing live birth date information in the outpatient settings, we specify our measure to use either live birth date or estimated delivery date to capture live births. For HCCN 1, even though the sensitivity for live birth date was low (0.4), estimated delivery date had a higher sensitivity (0.73). There were three estimated delivery dates that were observed in the patient charts and not in the EHR data extract, but two out of these three patients did not have any prenatal care visits. The measure specification requires the presence of both a prenatal care visit and a live birth date/estimated delivery date for a woman to be included in the denominator. Therefore, these two mismatched estimated delivery date records would not have impacted the measure calculation because these two women would not have been included in the denominator; only one out of the 158 sampled records had a true mismatch on estimated delivery date that would have impacted the measure score to a small degree. The combined use of these two elements allows us to more reliably capture women who have had a live birth.
Overall, our data provide fairly strong evidence for validity of CU-SINC, Postpartum at the data element level.
Empirical validity testing
Coefficients with absolute values of less than 0.3 are generally considered indicative of weak associations whereas absolute values of 0.3 or higher denote moderate to strong associations (1). For CU-SINC, Postpartum, the multilevel correlation estimation method showed statistically significant moderate to strong positive correlations with the postpartum care measure at both levels of analysis for HCCN 1. In comparison, Pearson’s correlation results indicated slightly weaker positive correlations. We did not present correlations between CU-SINC, Postpartum with the postpartum care measure for HealthEfficient due to the limited number of units (n=3 at both levels of analysis).
Generally, the magnitude of correlation was weaker using Pearson’s correlation, as expected since the distributional assumptions of this method are a poor fit to binary outcomes, resulting in underestimation. Although the Pearson correlation can be a rough approximation of correlation in binary outcomes for large units (facilities), cluster sizes become much smaller at the clinician group/practice level, resulting in further attenuation. We demonstrate that our generalized linear multilevel estimation more closely captures the “true” correlation between two measures and is better suited for binary outcomes and smaller units of analysis.
Overall, we observed moderate to strong positive correlations between CU-SINC, Postpartum with the postpartum care measure that (in theory) should be related, consistent with our hypothesis that health care facilities that perform well on postpartum care will also perform well on postpartum contraception services, because contraception care is a part of the postpartum care service. Our results provide reasonable evidence for validity of CU-SINC, Postpartum at the score level.
REFERENCE
1. Ratner, B. (2009) The correlation coefficient: Its values range between +1/−1, or do they?. J Target Meas Anal Mark 17, 139–142. https://doi.org/10.1057/jt.2009.5
Risk Adjustment
We do not believe that risk adjustment is justified. Our measure is explicitly designed to measure contraceptive use among those who have an interest in this use, as determined by the use of the SINC question to refine the denominator. Therefore, variations in contraceptive use by socio-demographic characteristics in this population will exist as a result of modifiable clinical and programmatic considerations and not patient preferences. These differences will be reduced if contraceptive services are offered in a patient-centered manner, as recommended by the Centers for Disease Control and Prevention (CDC) and the Office of Population Affairs (OPA) (1-4).
In this application, we present CU-SINC, Postpartum scores by age group so that adolescents (ages 15-20 years) can be examined separately from adult (ages 21-44 years) patients for the purposes of quality improvement. Though their current clinical guidelines report that most and moderately effective contraceptive methods are safe and recommended for teen and nulliparous populations who wish to use them, the American Academy of Pediatrics (AAP), ACOG, CDC, and OPA note that it can still be difficult for these populations to access these contraceptive methods (1-6); thus, it is important for facilities and clinician group/practices that want to improve the quality of their contraceptive services for adolescents and track progress to calculate CU-SINC measure rates by age group. We also utilize age groups that align with the OPA’s claims-based Contraceptive Care – Postpartum measure (CBE #2902).
REFERENCES
- Gavin, L., Moskosky, S., Carter, M., Curtis, K., Glass, E., Godfrey, E., Marcell, A., Mautone-Smith, N., Pazol, K., Tepper, N., Zapata, L., & Centers for Disease Control and Prevention (CDC) (2014). Providing quality family planning services: Recommendations of CDC and the U.S. Office of Population Affairs. MMWR. Recommendations and reports: Morbidity and mortality weekly report. Recommendations and reports, 63(RR-04), 1–54.
- Gavin, L., & Pazol, K. (2016). Update: Providing Quality Family Planning Services - Recommendations from CDC and the U.S. Office of Population Affairs, 2015. MMWR. Morbidity and mortality weekly report, 65(9), 231–234. https://doi.org/10.15585/mmwr.mm6509a3
- Gavin, L., Pazol, K., & Ahrens, K. (2017). Update: Providing Quality Family Planning Services - Recommendations from CDC and the U.S. Office of Population Affairs, 2017. MMWR. Morbidity and mortality weekly report, 66(50), 1383–1385. https://doi.org/10.15585/mmwr.mm6650a4
- Romer, S. E., Blum, J., Borrero, S., Crowley, J. M., Hart, J., Magee, M. M., Manzer, J. L., & Stern, L. (2024). Providing Quality Family Planning Services in the United States: Recommendations of the U.S. Office of Population Affairs (Revised 2024). American journal of preventive medicine, 67(6S), S41–S86. https://doi.org/10.1016/j.amepre.2024.09.007
- Committee Opinion No. 710: Counseling Adolescents About Contraception. (2017). Obstetrics and gynecology, 130(2), e74–e80. https://doi.org/10.1097/AOG.0000000000002234
- Chung, R. J., Lee, J. B., Hackell, J. M., Alderman, E. M., Committee on Adolescence, & Committee on Practice and Ambulatory Medicine (2024). Confidentiality in the Care of Adolescents: Policy Statement. Pediatrics, 153(5), e2024066326. https://doi.org/10.1542/peds.2024-066326
Use & Usability
Use
Usability
Health center agencies and networks can use the measure to assess whether eligible patients of reproductive age who want contraception are accessing methods in the postpartum period. Enhancing performance on this measure includes two key components: 1) optimizing the denominator by standardizing screening for contraceptive need, which will limit the denominator to the desired population of eligible patients who want contraception while ensuring that patients who want contraception are having their needs identified, and 2) optimizing surveillance (i.e. documentation of use) and provision of the full range of contraceptive methods. These two areas represent two intervention points along the reproductive health care pathway for patients.
To enhance screening for contraceptive need, entities would develop clear and standard workflows adapted to optimize the Self-Identified Need for Contraception (SINC) screening during peripartum care, including how to identify eligible patients for routine screening, identifying at which point and by whom SINC will be asked, and working with EHR vendor to optimize SINC location in the health record. As part of the intervention with community health centers (CHCs) described in section 4.1a, sites were trained by experts from the National Family Planning and Reproductive Health Association (NFPRHA) on how to integrate SINC into clinical workflows. During this training, a NFPHRA expert provided quality improvement strategies and workflow development support. They also provided participants with templates and worksheets to help sites more easily integrate contraceptive screening into their clinical workflow including sample workflow diagrams, process mapping flowcharts, tally sheets, and policy and procedure templates. Sites were able to utilize these templates to standardize their procedures; example activities from sites include integrating SINC into the rooming tab of their EHR, training medical assistants to record SINC responses during rooming, and programming SINC into check-in modules. We received feedback from sites that having these materials and training greatly helped teams seamlessly integrate SINC into their existing workflows, which will improve performance on the Contraceptive Use electronic clinical quality measure (eCQM) (CU-SINC), Postpartum (CBE #3682e) through both refining the denominator and improving identification of patients need for contraception It will also improve performance on the Contraceptive Care Screening eCQM (CBE #4655e).
To enhance performance on this measure in regard to contraceptive provision and surveillance, entities would prioritize comprehensive clinician and staff training, both on the importance of peripartum contraceptive care and contraceptive counseling skills as appropriate. To enhance EHR documentation of patient contraceptive use, particularly among patients who have received contraception from another healthcare location, health care settings can standardize procedures and fields for the collection of patient-reported contraceptive use. These strategies are further described on our quality improvement website (1).
REFERENCES
1. Person-Centered Reproductive Health Program. Innovating Contraceptive Care in Community Health Centers [Internet]. 2022. Available from: https://innovatingcontraceptivecare.ucsf.edu/
To develop the Contraceptive Use eCQM (CU-SINC), Postpartum (CBE #3682e), UCSF solicited a wide range of expert input, including convening three stakeholder panels to discuss how to optimize the measure specifications to capture the desired measure of the extent to which patient’s contraceptive needs are being addressed. In addition, we had a series of meetings with Dr. Joia Crear-Perry from National Birth Equity Collaborative and Dr. Jamila Perritt from Physicians for Reproductive Health to provide a Reproductive Justice and race-equity informed perspective on the measure development. Their consultation particularly informed the development and wording of SINC as a means to refine the denominator of the measure and decrease the potential to incentivize directive counseling towards individuals who are not interested in receiving contraceptive care. The inclusion of SINC in CBE #3682e increases the patient-centeredness of our measure and contributes to patient-centered workflows to identify and meet patients’ reproductive health needs. Given the documented disparities in reproductive health counseling experienced by patients of color (1–4), utilization of the SINC data element is also consistent with attention to race equity and health care disparities.
In addition, we participated in expert workgroup meetings to discuss measurement of contraceptive provision and care with measure users. This includes meetings held by the Office of Population Affairs (OPA) every other year and a semi-annual independent National Contraceptive Quality Measures Workgroup convened by the Coalition to Expand Contraceptive Access (CECA), Planned Parenthood Federation of America (PPFA), and NFPRHA. These workgroups bring together representatives from the Centers for Medicare and Medicaid Services (CMS), OPA, and PPFA, as well as relevant stakeholders and subject matter experts, including direct care providers, who could use CU-SINC, Postpartum to assess contraceptive use and long-acting reversible contraception (LARC) provision in their respective health systems. In these meetings, we received widespread support for the measure and input into its specifications, including around the necessity to refine the denominator.
We continue to participate in, and solicit feedback from, these expert working groups. Feedback on CU-SINC, Postpartum received from these meetings focused primarily on score interpretation, ensuring that the performance measure does not inadvertently incentivize practices that compromise reproductive autonomy and health equity goals. For example, at each meeting, we discuss ensuring that measure guidance highlights that neither the primary measure nor the submeasure should be used in the context of pay-for-performance.
CBE #3682e was approved for Trial Use in 2022. However, validity and reliability testing was necessary prior to moving toward full implementation and reporting, including by federal agencies. Thus, there has not yet been uptake of CBE #3682e at this time. We anticipate wider uptake of CU-SINC, Postpartum, and thus, additional feedback by the next maintenance cycle.
REFERENCES
1. Downing RA, LaVeist TA, Bullock HE. Intersections of Ethnicity and Social Class in Provider Advice Regarding Reproductive Health. Am J Public Health. 2007 Oct;97(10):1803–7.
2. Becker D, Klassen AC, Koenig MA, LaVeist TA, Sonenstein FL, Tsui AO. Women’s Perspectives on Family Planning Service Quality: An Exploration of Differences by Race, Ethnicity and Language. Perspectives on Sexual and Reproductive Health. 2009 Sep;41(3):158–65.
3. Borrero S, Schwarz EB, Creinin M, Ibrahim S. The Impact of Race and Ethnicity on Receipt of Family Planning Services in the United States. Journal of Women’s Health. 2009 Jan;18(1):91–6.
4. Dehlendorf C, Ruskin R, Grumbach K, Vittinghoff E, Bibbins-Domingo K, Schillinger D, et al. Recommendations for intrauterine contraception: a randomized trial of the effects of patients’ race/ethnicity and socioeconomic status. Am J Obstet Gynecol. 2010 Oct;203(4):319.e1-8.
Through our participation in the CECA Technical Expert Panels, National Contraceptive Quality Measures Workgroup, and OPA Expert Work Group, UCSF solicited and received input on how to incorporate patient choice more directly into an electronic clinical quality measure (eCQM) of contraceptive provision and use. The feedback obtained from these workgroups contributed to UCSF finalizing the following substantive decisions regarding our eCQM specifications prior to submitting the measure for CBE Approval for Trial Use:
- Incorporation of the SINC data element to account for patient choice in the CU-SINC, Postpartum denominator. We considered the use of One Key Question® (OKQ®) as an alternative. OKQ® asks patients whether they wish to get pregnant in the next year and has been proposed as a means of identifying patients in need of contraceptive services by excluding those desiring pregnancy (1). However, through discussion with stakeholders, we determined OKQ® was not optimal because it does not identify patients who desire contraception and/or pregnancy prevention at the current time (which is not incompatible with desiring pregnancy in a year). Use of SINC in the peripartum period allows exclusion from the denominator of patients who do want to discuss their contraceptive needs, such as those in same-sex partnerships.
- Definition of the postpartum period to 90 days after live birth delivery (as opposed to 60 days), aligned with the American College of Obstetrician & Gynecologists (ACOG) postpartum care guidelines (2). A 90-day window also provides a greater amount of time to meet a patient’s contraceptive needs in the postpartum period.
- Utilization of both live birth delivery date and estimated delivery date (EDD) as options for identifying the start of the postpartum period. Because delivery date is not always available in the electronic health record of the site of prenatal care, CBE #3682e uses EDD as entered into the prenatal care record when delivery date is not available.
- Including contraceptive provision from 24 weeks gestation (generally accepted as viability) per EDD for patients for whom the delivery date is not known. This accommodates the possibility of preterm birth by including contraceptive provision that could appear to occur before delivery but would be in the postpartum period for deliveries that occur before the EDD.
- Defining the measurement period over a 15-month period, with inclusion only of delivery dates within a 12-month period. This allows for flexible identification of contraceptive use and provision across the postpartum period and avoids double-counting of individual pregnancies in multiple measurement periods.
Additional specification updates have been made since receiving Approval for Trial Use designation to optimize and improve the measure:
- Updated time period for SINC “No” Responses for exclusions. In our specifications for Trial Use, we excluded postpartum patients who recorded “No” to SINC at any time during calendar year. We updated the time period for SINC “No” responses documented during the postpartum period, based on the anchor dates EDD or Live Birth Delivery Date. This creates a more precise measure of lack of desire for contraceptive counseling during the peripartum period.
- Two additional value sets were added to the definition of a qualifying encounter to align with current CMS eCQM specifications that use qualifying encounters.
REFERENCES
1. Power to Decide. One Key Question online [Internet]. Available from: https://powertodecide.org/one-key-question
2. ACOG Committee Opinion No. 736: Optimizing Postpartum Care. Obstet Gynecol. 2018 May;131(5):e140–50.
As CBE#3682e is not yet in use, due to its previous Approval for Trial Use designation, we do not yet have adequate data to demonstrate measure improvement.
As widespread uptake of the measure has not yet occurred due to our previous Approval for Trial Use designation, we do not have any unexpected findings to report.
Public Comments
Enter a comment below