When the measure returns for maintenance, the committee would like to see:
- Consider additional approaches for the reliability assessment that inform the reliability- validity (e.g. shrinkage) and reliability- usability (e.g. stability) tradeoffs
This measure was developed to improve the quality of care delivered to patients undergoing outpatient colonoscopy procedures. The Facility 7-Day Risk-Standardized Hospital Visit Rate after Outpatient Colonoscopy Measure, estimates a facility-level rate of risk-standardized, all-cause, unplanned hospital visits within seven days of a colonoscopy procedure performed at a hospital outpatient department (HOPD) or ambulatory surgical center (ASC) among Medicare Fee-for-Service (FFS) patients aged 65 years and older. An unplanned hospital visit is defined as an emergency department (ED) visit, observation stay, or unplanned inpatient admission. The measure is calculated separately for HOPDs and ASCs. The measure is also reported stratified by dual eligibility for the HOPD setting.
Measure Specs
- General Information(active tab)
- Numerator
- Denominator
- Exclusions
- Measure Calculation
- Supplemental Attachment
- Point of Contact
General Information
The Facility 7-Day Risk-Standardized Hospital Visit Rate after Outpatient Colonoscopy measure (CBE #2539) (hereafter “Colonoscopy measure”) captures unplanned hospital visits 7 days after a colonoscopy procedure performed at an HOPD, or separately, at an ASC. The measure focuses on the outcome of unplanned hospital visits because this is a broad, patient-centered outcome that captures the full range of hospital visits resulting from adverse events or poor care coordination following the procedure. By providing HOPDs and ASCs with detailed information about patients who have an unplanned hospital visit, this measure supports quality improvement at facilities, and through public reporting of the measure allows for assessment and illumination of the variation in risk-adjusted hospital visits following colonoscopy.
Colonoscopy is a common and costly procedure performed at outpatient facilities and is frequently performed among relatively healthy patients to screen for colorectal cancer (CRC). Given the widespread use of colonoscopy, understanding and minimizing procedure-related adverse events is a high priority. These adverse events, such as abdominal pain, bleeding, and intestinal perforation, can result in unanticipated hospital visits after the procedure. Furthermore, physicians performing colonoscopies may be unaware that patients seek acute care at hospitals following the procedure and thus underestimate such events. This risk-standardized quality measure addresses this information gap and promotes quality improvement by providing feedback to facilities and physicians, including patient-level details for each post-procedure visit. Public reporting transparency for patients on the rates of and variation across facilities in unplanned hospital visits after colonoscopy.
Patients may experience a range of potential adverse events after an outpatient colonoscopy, which could lead to unplanned hospital visits, including ED visits, observation stays, and unplanned inpatient admissions. This measure provides the opportunity to improve the quality of care and to lower rates of adverse events leading to hospital visits after an outpatient colonoscopy. Below we describe the complications that patients may experience; pathways for improvement by measured entities, and improvement through public reporting.
Colonoscopy Complications
Gastrointestinal complications are common and range from severe to mild. Complications such as colonic perforation and gastrointestinal (GI) bleeding are relatively rare but severe adverse events; other GI complications are considerably more common (Olaiya et al., 2020; Kothari et al., 2019). A recent meta-analysis found that perforation rates ranged from 1.6 to 11.9 per 10,000 procedures and that the risk of perforation was not associated with age; study authors suggested that characteristics of the procedure itself were underlying this adverse event (Kothari et al., 2019). Other adverse events are more common but less severe; among surveyed patients, the reported frequency of complications was 25% by the second day following the procedure (Sewitch et al., 2018). These complications include abdominal pain, abdominal distension, nausea, vomiting, and other nonspecific symptoms. In addition, the overall risk of a complication is higher in older adults (Causada-Calo et al., 2020).
Cardiopulmonary complications are rare (Kothari et al., 2019) but can occur as a complication of the sedation given at the time of the procedure. It has been shown that aspiration is more common with deep sedation with anesthesia assistance (0.14% in Medicare-aged patients) compared to moderate sedation without anesthesia assistance (0.10%) (Cooper et al., 2013).
Post-procedural infection can also occur as a result of a colonoscopy. For example, a 2018 study found rates of infection within 7 days of a screening colonoscopy performed at an ASC to be 1.1 per 1,000 colonoscopies (Wang et al., 2018). Furthermore, the study authors found that the rates of infection varied widely by ASC, from 0 to 115 per 1,000 colonoscopies. Infection can occur due to lapses in infection control procedures, as well as defective equipment (Petersen et al., 2017).
Hospital visits following colonoscopy
The symptoms described above can result in the need for acute care. A 2018 retrospectively review of 50,319 colonoscopies performed on 44,082 individuals (47% male, median age 59 years) reported an ED visit rate within 7 days of a colonoscopy of 0.76% (Grossberg et al., 2018), and a claims-based analysis found an average 7-day hospital visit rate (defined as an ED visit, observation stay, or inpatient hospitalization) of 1.63% (Ranasinghe et al., 2016). A recent study found that older patients are more likely to experience a hospital visit after colonoscopy and reported a rate of inpatient admission or an ED visit within 30 days of 6.8% in people aged 75 and older (Causada-Calo et al, 2020). The rate of hospitalization varies by type of complication; hospitalization rates were nearly 100% among patients who developed perforation and between 50.8% and 70.7% among patients who developed lower GI bleeding (Wang et al., 2018). In contrast, hospitalizations among patients with abdominal pain or nausea diagnosis were less common.
Studies have shown that many of the reasons for post-procedural hospital visits are related to the colonoscopy. For example, a 2018 single-center study examined the medical records (including medication information) of patients who experienced an emergency department (ED) visit within 7 days of an outpatient colonoscopy (Grossberg et al., 2018). The study authors extracted patients’ chief complaint from medical records, assigned the chief complaints as related or unrelated to the colonoscopy, and found that 68% of the reasons for the ED visit were due to the colonoscopy. The most common reasons for related ED visits were abdominal pain (38.2%), gastrointestinal bleeding (29.7%), cardiopulmonary disorders (12.7%), and nausea/vomiting (4.2%). In another study, the authors examined the most frequent diagnoses in claims data associated with an unplanned hospital visit within 7 days, which included hemorrhage (6.4% of all unplanned visits), accidental operative laceration (3.0%), abdominal pain (3.0%), GI hemorrhage (2.7%), chest pain (1.9%), and urinary tract infection (1.8%) (Ranasinghe et al., 2016). CORE’s updated analysis (see Section 4.3, Validity) shows a similar pattern of complications 7 days following a screening colonoscopy, with recent data.
Pathways for improvement
Provider- and facility-level factors can affect the outcome of complications and hospital visits related to a colonoscopy. For example, provider-level factors such as low provider volume and fellow involvement in the procedure were significantly associated with a higher risk of an ED visit in one study (Grossberg et al., 2018), and another study found that low procedure volume was associated with a higher risk of infection (Wang et al., 2018), suggesting facilities can influence the patients’ outcome through these modifiable pathways.
Providers may be unaware of complications for which patients visit the hospital, leading to understated complication rates and suggesting the need for better measurement to drive quality improvement. Both patients and providers can benefit from outcome measures that capture the full range of adverse experiences associated with outpatient colonoscopy and illuminate quality differences.
Public reporting and facility feedback
CMS provides the public with data to help people make more informed decisions about their healthcare. As of December 2017, measure results have been publicly available; results have been available to facilities since 2015 in the form of facility-specific quality reports. Thus, it is important to continue to make this information transparent to patients choosing among providers who offer this elective procedure, and to facilities that can use the detailed feedback for quality improvement.
Importantly, providing outcome rates to providers will make meaningful quality differences visible to clinicians, thus incentivizing improvement. In this submission we show improvement across the five performance periods captured by this measure since it was implemented. For example, the national rate of hospital visits per 1,000 colonoscopies among HOPDs declined from 16.4 in 2018 reporting to 14.8 in 2019 reporting, and 13.2 for 2023 reporting, and the distribution of risk-standardized rates also declined.
References
ASGE Standards of Practice Committee, Early, D. S., Lightdale, J. R., Vargo, J. J., 2nd, Acosta, R. D., Chandrasekhara, V., Chathadi, K. V., Evans, J. A., Fisher, D. A., Fonkalsrud, L., Hwang, J. H., Khashab, M. A., Muthusamy, V. R., Pasha, S. F., Saltzman, J. R., Shergill, A. K., Cash, B. D., & DeWitt, J. M. (2018a). Guidelines for sedation and anesthesia in GI endoscopy. Gastrointestinal endoscopy, 87(2), 327–337. https://doi.org/10.1016/j.gie.2017.07.018
ASGE Quality Assurance in Endoscopy Committee, Calderwood, A. H., Day, L. W., Muthusamy, V. R., Collins, J., Hambrick, R. D., 3rd, Brock, A. S., Guda, N. M., Buscaglia, J. M., Petersen, B. T., Buttar, N. S., Khanna, L. G., Kushnir, V. M., Repaka, A., Villa, N. A., & Eisen, G. M. (2018b). ASGE guideline for infection control during GI endoscopy. Gastrointestinal endoscopy, 87(5), 1167–1179. https://doi.org/10.1016/j.gie.2017.12.009
Bielawska B, Hookey LC, Sutradhar R, Whitehead M, Xu J, Paszat LF, Rabeneck L, Tinmouth J. Anesthesia Assistance in Outpatient Colonoscopy and Risk of Aspiration Pneumonia, Bowel Perforation, and Splenic Injury. Gastroenterology. 2018 Jan;154(1):77-85.e3. doi: 10.1053/j.gastro.2017.08.043. Epub 2017 Sep 1. PMID: 28865733.
Causada-Calo, N., Bishay, K., Albashir, S., Al Mazroui, A., & Armstrong, D. (2020). Association Between Age and Complications After Outpatient Colonoscopy. JAMA network open, 3(6), e208958. https://doi.org/10.1001/jamanetworkopen.2020.8958
Grossberg LB, Vodonos A, Papamichael K, Novack V, Sawhney M, Leffler DA. Predictors of post-colonoscopy emergency department use. Gastrointest Endosc. Feb 2018;87(2):517-525.
Kothari ST, Huang RJ, Shaukat A, Agrawal D, Buxbaum JL, Abbas Fehmi SM, Fishman DS, Gurudu SR, Khashab MA, Jamil LH, Jue TL, Law JK, Lee JK, Naveed M, Qumseya BJ, Sawhney MS, Thosani N, Yang J, DeWitt JM, Wani S; ASGE Standards of Practice Committee Chair. ASGE review of adverse events in colonoscopy. Gastrointest Endosc. 2019 Dec;90(6):863-876.e33. doi: 10.1016/j.gie.2019.07.033. Epub 2019 Sep 25. PMID: 31563271
Olaiya B, Adler DG. Adverse Events After Inpatient Colonoscopy in Octogenarians: Results From the National Inpatient Sample (1998-2013). J Clin Gastroenterol. 2020 Oct;54(9):813-818. doi: 10.1097/MCG.0000000000001288. PMID: 31764488
Ranasinghe I, Parzynski CS, Searfoss R, Montague J, Lin Z, Allen J, Vender R, Bhat K, Ross JS, Bernheim S, Krumholz HM, Drye EE. Differences in colonoscopy quality among facilities: Development of a post-colonoscopy risk-standardized rate of unplanned hospital visits. Gastroenterology. Jan 2016;150(1):103-13.
Petersen, B. T., Cohen, J., Hambrick, R. D., 3rd, Buttar, N., Greenwald, D. A., Buscaglia, J. M., Collins, J., & Eisen, G; Reprocessing Guideline Task Force,. (2017). Multisociety guideline on reprocessing flexible GI endoscopes: 2016 update. Gastrointestinal endoscopy, 85(2), 282–294.e1. https://doi.org/10.1016/j.gie.2016.10.002
Sewitch MJ, Azalgara VM, Sing MF. Screening Indication Associated With Lower Likelihood of Minor Adverse Events in Patients Undergoing Outpatient Colonoscopy. Gastroenterol Nurs. 2018 Mar/Apr;41(2):159-164. doi: 10.1097/SGA.0000000000000308. PMID: 29596130.
Wang P, Xu T, Ngamruengphong S, Makary MA, Kalloo A, Hutfless S. Rates of infection after colonoscopy and osophagogastroduodenoscopy in ambulatory surgery centres in the USA. Gut. 2018;67(9):1626–1636.
To calculate the measure score CMS uses final-action claims for Medicare FFS Part A and B and the Medicare Enrollment Database.
This is a claims-based measure and the measure score is calculated automatically from 100% final-action claims; claims data are routinely generated during the delivery of care. We did not encounter any difficulties with respect to data feasibility, reliability, or validity.
Numerator
The outcome for this measure is all-cause, unplanned hospital visits within seven days of a qualifying outpatient colonoscopy. The measure defines a hospital visit as any emergency department (ED) visit, observation stay, or unplanned inpatient admission.
The measure defines the outcome as any (one or more) hospital visit for acute care within seven days of an outpatient colonoscopy. A hospital visit includes any ED visit, observation stay, or unplanned inpatient admission. If more than one unplanned hospital visit occurs, only the first hospital visit within the outcome timeframe is counted in the outcome.
The measure defines ED visits and observation stays using billing codes or revenue center codes identified in Medicare Part B outpatient hospital claims. The data dictionary (tab “Colonos Outcome ED Obs Stay”) provides the specific codes used to identify ED visits and observation stays.
Outcome Time Frame
The measure limits the outcome of hospital visits to seven days, as existing literature suggests that the vast majority of adverse events after a colonoscopy occur within the first seven days following the procedure, and our empirical analyses during measure development indicated that the highest rates of hospital visits were within seven days of colonoscopy. Thus, based on existing literature and empirical findings, as well as input from the TEP and public comment, the measure development team concluded that unplanned hospital visits within seven days is the optimal outcome to ensure capture of procedure-related adverse events and to minimize capture of hospital visits unrelated to the procedure.
Identification of Planned Admissions
Inpatient admissions within the first 7 days after the colonoscopy are included unless the admission is deemed a “planned” admission as defined by the measure’s Planned Admission Algorithm (PAA). The Centers for Medicare & Medicaid Services (CMS) seeks to count only unplanned admissions in the measure outcome, because variation in “planned” admissions does not reflect quality differences. We based the PAA on the CMS Planned Readmission Algorithm (PRA), which CMS created for its hospital-wide readmission measure. The measure does not consider observation stays or ED visits as planned.
In brief, the algorithm identifies admissions that are typically planned and may occur after the patient’s index event. The algorithm always considers a few specific, limited types of care planned (e.g., major organ transplant, rehabilitation, or maintenance
chemotherapy). Otherwise, the algorithm defines a planned admission as a non-acute admission for a scheduled procedure (e.g., total hip replacement or cholecystectomy), and the algorithm never considers admissions for acute illness or for complications of care planned. For example, the algorithm considers hip replacement unplanned if hip fracture (an acute condition) is the discharge diagnosis but planned if osteoarthritis (a non-acute condition) is the discharge diagnosis. The algorithm considers admissions that include potentially planned procedures with acute diagnoses or that might represent complications of a colonoscopy unplanned and thus counts these admissions in the measure outcome.
For more information about the PAA, please see the Facility 7-day Risk-Standardized Hospital Visit Rate after Outpatient Colonoscopy Measure 2023 Measure Updates and Specifications Report.
Also see sheets ‘PAA1 Always Planned Px’, ‘PAA2 Always Planned Dx’, ‘PAA3 Post Planned Px’, and ‘PAA4 Acute Dx’ in the attached Data Dictionary for the most up-to-date sets of codes in the algorithm for ‘always planned procedures,’ ‘always planned diagnoses,’ ‘potentially planned procedures, and ‘acute’ diagnoses.
Denominator
The target population for this measure includes low-risk colonoscopies performed in the outpatient setting (HOPD or ASC) for Medicare FFS patients aged 65 years and older. The measure is calculated separately for procedures performed at HOPDs vs ASCs and reported stratified by dual eligibility for HOPDs.
The denominator includes colonoscopies performed at hospital outpatient departments (HOPDs) and (separately) ambulatory surgical centers (ASCs) for Medicare FFS patients aged 65 years and older.
The target population is patients aged 65 years and older who have a colonoscopy, to screen for colorectal cancer, biopsy or remove pre-cancerous lesions, or evaluate non-emergent symptoms and signs of disease. We limited the measure cohort to patients who are 65 and older, enrolled in Medicare FFS, and have been enrolled in Part A and Part B Medicare for the 12 months prior to the date of procedure since national data linking risk factors, procedures, and outcomes across care settings are only available for this group.
Eligible colonoscopies were identified using specified Current Procedural Terminology (CPT)/Healthcare Common Procedure Coding System (HCPCS) procedure codes in the Medicare professional claims (Part B Physician). Specifically, we identified outpatient colonoscopies using physician claims from ASCs, HOPDs and physician office settings. For ASCs, the facility claim (with a unique facility identifier) is included on the professional (physician claim). Physician claims for colonoscopies performed at HOPDs were linked to the corresponding facility claim.
The CPT and HCPCS procedure codes that define the cohort are in the attached Data Dictionary, sheet “Colonos Cohort.”
We considered all colonoscopy codes during development of the measure cohort. We did not include in the measure colonoscopy CPT procedure codes that reflected fundamentally higher-risk or different procedures. Those procedures billed with a qualifying colonoscopy procedure code and a high-risk colonoscopy procedure code (see attached Data Dictionary, sheet “Colonos Exclusions”) were not included in the measure.
Colonoscopy is not possible among patients who have had a prior total colectomy. Any claim for a colonoscopy in a patient with a prior total colectomy is therefore likely to be a coding error. We perform an error check to ensure the measure does not include these patients with a total colectomy recorded in their prior medical history. The CPT and HCPCS procedure codes and International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM) codes that define the total colectomy data reliability check are in the attached Data Dictionary, sheet “Colonos Exclusions.”
Capture of Colonoscopies Affected by the Medicare 3-Day Payment Window Policy:
Colonoscopies performed at HOPDs can be affected by the Medicare 3-day payment window policy. The policy states that outpatient services (including all diagnostic services such as colonoscopy) provided by a hospital or any Part B entity wholly owned or wholly operated by a hospital (such as an HOPD) in the three calendar days preceding the date of a beneficiary’s inpatient admission are deemed to be related to the admission. For outpatient colonoscopies affected, the facility claim (for the technical portion of the colonoscopy) is bundled with the inpatient claim, although the Medicare Part B physician claim for professional services rendered is still submitted. This policy has implications for the measure because it may lead to: (1) failure to completely capture outpatient colonoscopies performed at HOPDs; and (2) underreporting of outcomes for colonoscopies performed in the HOPD setting. To ensure the capture of HOPD colonoscopies, we identify physician claims for colonoscopy in the HOPD setting from Medicare Part B claims, which had an inpatient admission within three days and lacked a corresponding HOPD facility claim. We then attribute the colonoscopies identified as affected by this policy to the appropriate HOPD facility using the facility provider ID from the inpatient claim.
Exclusions
A diagram showing the inclusion and exclusion criteria for this measure is shown in Figure 1A (HOPDs) and 1B (ACSs) (see attachment).
We established the following exclusion criteria after reviewing the literature, examining existing measures, discussing alternatives with the working group and technical expert panel (TEP) members, reviewing feedback from the national dry run held in July 2015, and public reporting in 2018 and 2019, and annual re-evaluation of the measure. The goal was to be as inclusive as possible; we excluded only those high-risk procedures and patient groups for which risk adjustment would not be adequate or for which hospital visits were not typically a quality signal. The exclusions, based on clinical rationales, prevent unfair distortion of performance results.
Colonoscopy Measure Exclusion Criteria:
1) Colonoscopies for patients who lack continuous enrollment in Medicare FFS Parts A and B in the 7 days after the procedure.
Rationale: We exclude these patients to ensure full data availability for outcome assessment.
2) Colonoscopies that occur concurrently with high-risk upper gastrointestinal (GI) endoscopy procedures.
Rationale: Patients undergoing concurrent high-risk upper GI endoscopy procedures, such as upper GI endoscopy procedures for the control of bleeding or treatment of esophageal varices, and have a higher risk profile than typical colonoscopy patients. Therefore, these patients have a disproportionally higher risk for the outcome.
3) Colonoscopies for patients with a history of inflammatory bowel disease (IBD) or diagnosis of IBD at time of index colonoscopy or on the subsequent hospital visit outcome claim. Rationale: We exclude these patients because:
• IBD is a chronic condition; patients with IBD undergo colonoscopy both for surveillance due to increased cancer risk and for evaluation of acute symptoms. IBD is likely to be coded as the primary diagnosis prompting the procedure irrespective of whether the patients are undergoing a screening procedure or a diagnostic procedure in the setting of an acute exacerbation of IBD. Therefore, we may not be able to adequately risk adjust for these patients, as we cannot identify relatively well versus acutely unwell patients among visits coded as IBD.
• Our aim is to capture hospital visits which reflect the quality of care. Admissions for acutely ill IBD patients who are evaluated with an outpatient colonoscopy and are subsequently admitted for medical treatment of an IBD flare do not reflect the quality of the colonoscopy. During measure development (see the 2014 Facility 7-day Risk-Standardized Hospital Visit Rate after Outpatient Colonoscopy Measure Technical Report we found that more than one-third of IBD patients admitted to the hospital with colonoscopy had a discharge diagnosis of IBD, indicating their admission was for medical treatment of their IBD. We therefore excluded this group so that providers who treat a disproportionate number of IBD patients will not be disadvantaged in the measure.
• A post-index diagnosis of IBD, which represents a very small fraction of cases (less than 0.5% of the cohort) in the measure population, indicates that the condition was likely present at the time of the index colonoscopy but not coded.
4) Colonoscopies for patients with a history of diverticulitis or diagnosis of diverticulitis at time of index colonoscopy or on the subsequent hospital visit outcome claim.
Rationale: We exclude these patients because:
• It is unclear what the health status is of patients coded with a history or current diagnosis of diverticulitis, making it difficult to fully risk adjust for patients’ health. Colonoscopies performed on patients with a history or current diagnosis of diverticulitis are likely to be coded as diverticulitis as the primary diagnosis irrespective of whether the patients are undergoing a screening procedure or a diagnostic procedure (i.e., are acutely unwell with active disease). Furthermore, the codes for diverticulitis and diverticulosis may not be consistently used; patients with diverticulosis may be erroneously coded as diverticulitis. Therefore, we may not be able to adequately risk adjust as we cannot identify relatively well versus acutely unwell patients among visits coded as diverticulitis.
• Admissions for acutely ill patients with a history or current diagnosis of diverticulitis who are evaluated with an outpatient colonoscopy and are subsequently admitted for medical treatment of do not reflect the quality of the colonoscopy. During measure development (see the Facility 7-day Risk-Standardized Hospital Visit Rate after Outpatient Colonoscopy Measure Technical Report) we found that more than one-quarter of patients with a history or current diagnosis of diverticulitis admitted to the hospital post colonoscopy had a discharge diagnosis of diverticulitis, indicating they were admitted for medical treatment of the condition. These admissions are likely unrelated to the quality of the colonoscopy. We therefore excluded this group so that providers who treat a disproportionate number of diverticulitis patients will not be disadvantaged in the measure.
• A post-index diagnosis of diverticulitis, which represents a very small fraction of cases (less than 0.5% of the cohort) in the measure population, indicates that the condition was likely present at the time of the index colonoscopy but not coded.
5) Colonoscopies followed by a subsequent outpatient colonoscopy procedure within 7 days.
Rationale: In these situations, the two colonoscopies are considered part of a single episode of care, for which the subsequent colonoscopy is considered the index procedure.
In addition, for colonoscopies performed at HOPDs, we exclude:
6) Colonoscopies that occur on the same day and at the same hospital as an emergency department (ED) visit that is billed on a different claim than the index colonoscopy, unless the ED visit has a diagnosis indicative of a complication of care.
Rationale: It is unclear whether the colonoscopy or ED visit occurred first. If the ED visit is coded with a diagnosis indicative of a complication of care, the measure assumes the ED visit occurred after the colonoscopy procedure and is counted in the measure. It is unlikely that a patient would experience an ED visit for an acute diagnosis at 1 facility and then travel to another facility for a routine colonoscopy on the same day. Accordingly, ED visits billed on the same day as a colonoscopy but at a different facility are included because they likely represent a routine procedure followed by a complication of care.
7) Colonoscopies that are billed on the same hospital claim as an ED visit and that occur on the same calendar day, unless the ED visit has a diagnosis indicative of a complication of care.
Rationale: In these situations, it is not possible to use claims data to determine whether the colonoscopy was the cause of, subsequent to, or during the ED visit. However, if the ED visit is coded with a diagnosis for a complication, the assumption is that it occurred after the colonoscopy procedure.
8) Colonoscopies that are billed on the same hospital outpatient claim and that occur after the ED visit.
Rationale: In these situations, we assume that the colonoscopy was subsequent to the ED visit and may not represent a routine colonoscopy procedure. Timing of the ED visits is determined using revenue center dates from the outpatient claim.
9) Colonoscopies that are billed on the same hospital outpatient claim as an observation stay.
Rationale: In these situations, it is not possible to use claims data to determine whether the colonoscopy was the cause of, subsequent to, or during the observation stay.
1) Colonoscopies for patients who lack continuous enrollment in Medicare FFS Parts A and B in the 7 days after the procedure. Lack of continuous enrollment in Medicare FFS for 7 days after the procedure is determined by patient enrollment status in FFS Parts A and B using the Medicare Enrollment Database. The enrollment indicators must be appropriately marked for the month(s) which fall within 7 days of the procedure date.
2) Colonoscopies that occur concurrently with high-risk upper GI endoscopy procedures. The list of the CPT codes for the upper GI endoscopy procedures identified as “high-risk” are in attached Data Dictionary, sheet “Colonos Exclusions”
3) Colonoscopies for patients with a history of IBD or diagnosis of IBD at time of index colonoscopy or on the subsequent hospital visit outcome claim. The ICD-10-CM codes that define IBD are in the attached Data Dictionary, sheet “Colonos Exclusions.”
4) Colonoscopies for patients with a history of diverticulitis or diagnosis of diverticulitis at time of index colonoscopy or on the subsequent hospital visit outcome claim. The ICD-10-CM codes that define diverticulitis are in the attached Data Dictionary, sheet “Colonos Exclusions.”
5) Colonoscopies followed by a subsequent outpatient colonoscopy procedure within 7 days. For cases in which a colonoscopy is followed by another colonoscopy within 7 days, the measure will use the subsequent colonoscopy as the index colonoscopy.
The following are in addition to those above, but only for HOPDs:
6) Colonoscopies that occur on the same day and at the same hospital as an ED visit that is billed on a separate claim than the index colonoscopy, unless the ED visit has a diagnosis indicative of a complication of care. The billing and revenue center codes that define ED visits are in the attached Data Dictionary, sheet “Colonos Outcome ED Obs Stay.” The same facility is defined as having the same CMS Certification Number (CCN). Complications of care codes are shown in tab “Colonos Exclusions ED CoC” include the CCS categories such as “Complications of surgical procedures or medical care,” “Adverse effects of medical care,” and others.
7) Colonoscopies that are billed on the same hospital claim as an ED visit and that occur on the same calendar day, unless the ED visit has a diagnosis indicative of a complication of care.
The billing and revenue center codes that define ED visits are in the attached Data Dictionary, sheet “Colonos Outcome ED Obs Stay.” Complications of care codes are shown in tab “Colonos Exclusions ED CoC” include the CCS categories such as “Complications of surgical procedures or medical care,” “Adverse effects of medical care,” and others.
8) Colonoscopies that are billed on the same hospital outpatient claim and that occur after the ED visit. The billing and revenue center codes that define ED visits are in the attached Data Dictionary, sheet “Colonos Outcome ED Obs Stay.”
9) Colonoscopies that are billed on the same hospital outpatient claim as an observation stay.
The billing and revenue center codes that define observation stays are in the attached Data Dictionary, sheet “Colonos Outcome ED Obs Stay.”
Measure Calculation
The measure is calculated separately for HOPDs and ASCs.
The measure uses a two-level hierarchical logistic regression model to estimate facility-level risk-standardized hospital visit rates (RSHVRs). This approach accounts for the clustering of patients within facilities and variation in sample size across facilities The RSHVR is calculated as the ratio of the predicted to the expected number of post-procedural unplanned hospital visits among a facility’s patients, multiplied by the national observed rate of unplanned hospital visits. For each facility, the numerator of the ratio is the number of hospital visits predicted for the facility’s patients, accounting for its observed rate and case mix. The denominator is the number of hospital visits expected nationally for the facility’s case mix. As noted above, to calculate a facility’s predicted-to-expected (P/E) ratio, the measure uses a two-level hierarchical logistic regression model. The log-odds of the outcome for an index procedure is modeled as a function of the patient demographics, comorbidities, procedure characteristics, and a random facility-specific intercept. A ratio greater than one indicates that the facility’s patients have more post-procedural hospital visits than expected, compared to an average facility with similar patients. A ratio less than one indicates that the facility’s patients have fewer post-procedural hospital visits than expected, compared to an average facility with a similar patient mix. A facility’s P/E ratio is then multiplied by the overall national rate of unplanned hospital visits to calculate the facility-level RSHVR. This approach is analogous to an observed-to-expected ratio, but accounts for within-facility correlation of the observed outcome and sample size differences, accommodates the assumption that underlying differences in quality across facilities lead to systematic differences in outcomes, and is tailored to and appropriate for a publicly reported outcome measure as articulated in published scientific guidelines (Normand and Shahian, 2007; Krumholtz et al., 2007).
Below we outline the steps to calculate the measure score:
1. Identify colonoscopies meeting the inclusion criteria described above.
2. Exclude procedures meeting any of the exclusion criteria described above.
3. Identify and create a binary (0/1) flag for an unplanned hospital visit within 7 days of the colonoscopy.
4. Use patients’ historical and index procedure claims data to create risk adjustment variables.
5. Fit a hierarchical generalized linear model (HGLM) to produce a ratio of the number of “predicted” hospital visits to the number of “expected” hospital visits for each facility, given its case mix. The HGLM is adjusted for clinical risk factors that vary across patient populations, are unrelated to quality, and influence the outcome.
6. Multiply the ratio estimated in step 3 by the observed national 7-day hospital visit rate, and then by 1,000 to obtain a risk-standardized hospital visit (RSHV) rate per 1,000 colonoscopies for each facility.
7. Use bootstrapping to construct a 95% confidence interval estimate for each facility’s RSHV rate.
For more information about the measure methodology, please see the Facility 7-day Risk-Standardized Hospital Visit Rate after Outpatient Colonoscopy 2023 Measure Updates and Specifications Report for HOPDs and ASCs.
HOPDs: https://qualitynet.cms.gov/files/651b5fb570a30f001c388004?filename=2023_OQR_MeasureUpdatesRpt.pdf
ASCs: https://qualitynet.cms.gov/files/651b5f9970a30f001c388001?filename=2023_ASC_MeasureUpdatesRpt.pdf
References
Krumholz HM, Brindis RG, Brush JE, et al. Standards for Statistical Models Used for Public Reporting of Health Outcomes An American Heart Association Scientific Statement From the Quality of Care and Outcomes Research Interdisciplinary Writing Group: Cosponsored by the Council on Epidemiology and Prevention and the Stroke Council Endorsed by the American College of Cardiology Foundation. Circulation. 2006;113(3):456-462.
Normand S-LT, Shahian DM. Statistical and clinical aspects of hospital outcomes profiling. Statistical Science. 2007;22(2):206-226.
The HOPD version of this measure is currently stratified by dual eligibility (DE), reported confidentially to hospitals.
In the Calendar Year (CY) 2022 OPPS Proposed Rule, CMS described a plan to stratify reporting using two disparity methods, described below, in the HOPD setting and have identified the colonoscopy measure as one of six priority measures included in the Hospital Outpatient Quality Reporting (OQR) program for confidential disparity reporting stratified by patient DE.
The two stratification methods are:
1. The Within-Facility Disparity Method, highlights differences in outcomes for patient groups based on social risk factors within an HOPD, and
2. The Across-Facility Disparity Method, illuminates variation in healthcare quality for patients with social risk factors across facilities.
The two methods are described in more detail below and visually shown in Figure 2 (see attachment with tables and figures). Details of the methodology can be found here: https://qualitynet.cms.gov/files/652fd45a8be3e0001c0b5141?filename=CY23_OP_32_35_36_DsprtySpecs.pdf
The Within-Facility Disparity Method reports differences in health outcomes between patient populations in the same facility. The goal of this method is to assess the difference in outcomes for two patients with the same condition and medical history but with different social risks. This method can answer the question: “Does a patient with a social risk factor experience similar health outcomes as a patient without that social risk factor when cared for at the same facility?”
The Across-Facility Disparity Method reports facility outcome rates for one patient population with a particular social risk factor across facilities. This method can answer the question: “How does the outcome rate for patients with a social risk factor at a specific facility compared to the outcome rate for patients with that social risk factor at an average facility?”
Please see Section 5, Equity, for results for the HOPD colonoscopy method when applying these two disparity methods.
Not applicable. This measure is not based on a sample.
Supplemental Attachment
Point of Contact
Not applicable
Centers for Medicare & Medicaid Services (CMS)
7500 Security Boulevard
Windsor Mill, MD 21244
United States
Oscar Gonzalez
Acumen LLC
500 Airport Blvd., Suite 100
Burlingame, CA 94010
United States
Importance
Evidence
Below we review the evidence that supports our logic model (see Figure 3 in attachment) for the connection between both structure and process, outcomes (complications), and post-procedural acute hospital visits, the outcome captured by the Colonoscopy measure. Please see the “measure rationale” section for additional information.
Processes related to the procedure, such as adhering to clinical best practices for the procedure itself, and adhering to evidence-based infection control processes, are tied to outcomes such as complications. More common complications such as nausea and vomiting, as well as more rare complications such as cardiovascular complications, and pneumonia, can be related to anesthesia; excessive sedation may lead to hypoxia, hypotension, respiratory arrest, and aspiration pneumonia (Bielawska et al., 2018). For example, a 2018 retrospective claims-based analysis of more than 3 million outpatient colonoscopies found that the use of anesthesia assistance (sedation with agents that result in deeper sedation, such as Propofol, rather than conscious sedation), resulted in increased risk of aspiration pneumonia (OR, 1.63; 95% CI, 1.11–2.37) (Bielawska et al. 2018). Post-procedural infections can be caused by improper infection control processes as well as defective equipment, and ASCs have admitted to lapses in infection control procedures (Petersen et al., 2017). Facilities ensure that providers follow best practice guidelines for the procedure, outlined within practice guidelines of professional societies (ASGE, 2018a; ASGE 2018b).
Processes before, during, and after discharge can reduce complications and hospital visits following a procedure. For example, identifying patients at higher risk for a complication, good patient communication, medication reconciliation, and providing access to non-acute care, can reduce post-discharge complications and the need for hospital-based acute care. (Kothari et al., 2022; Thompson et al., 2016; Padmanabhan et al., 2016; Grossberg et al., 2018). Interventions around post-discharge communication have been shown to reduce post-discharge hospital care and improve patient satisfaction (Becker et al., 2021).
Structural (provider- and facility-level) factors can affect the outcome of complications and hospital visits related to a colonoscopy. For example, provider-level factors such as low provider volume and fellow involvement in the procedure were significantly associated with a higher risk of an ED visit in one study (Grossberg et al., 2018), and low procedure volume was associated with a higher risk of infection in another study (Wang et al., 2018), suggesting that facilities can influence patient outcomes through these modifiable pathways. Additionally, endoscopists trained as gastroenterologists and endoscopists with higher colonoscopy volumes have lower complication rates compared to non-gastroenterologists or low-volume endoscopists, respectively (Bielawska et a., 2014).
There is clear evidence that complications related to the processes and structures (or lack of those processes and structures) described above, are linked to post-procedural hospital visits. As described in more detail in the “measure rational section” and summarized here. A 2018 study that categorized chart-abstracted chief complaints from ED visits within 7 days after a hospital visit found that most (68%) were related to the colonoscopy (Grossberg et al., 2018). Other studies, and our own analysis of principal discharge diagnosis codes show that the most frequent reasons for a hospital visit are complications related to the procedure (Ranasinghe et al., 2016; see section 4.3, Validity in this submission).
Providers may be unaware of complications for which patients visit the hospital, leading to understated complication rates and suggesting the need for better measurement to drive quality improvement.
References
ASGE Standards of Practice Committee, Early, D. S., Lightdale, J. R., Vargo, J. J., 2nd, Acosta, R. D., Chandrasekhara, V., Chathadi, K. V., Evans, J. A., Fisher, D. A., Fonkalsrud, L., Hwang, J. H., Khashab, M. A., Muthusamy, V. R., Pasha, S. F., Saltzman, J. R., Shergill, A. K., Cash, B. D., & DeWitt, J. M. (2018a). Guidelines for sedation and anesthesia in GI endoscopy. Gastrointestinal endoscopy, 87(2), 327–337. https://doi.org/10.1016/j.gie.2017.07.018
ASGE Quality Assurance in Endoscopy Committee, Calderwood, A. H., Day, L. W., Muthusamy, V. R., Collins, J., Hambrick, R. D., 3rd, Brock, A. S., Guda, N. M., Buscaglia, J. M., Petersen, B. T., Buttar, N. S., Khanna, L. G., Kushnir, V. M., Repaka, A., Villa, N. A., & Eisen, G. M. (2018b). ASGE guideline for infection control during GI endoscopy. Gastrointestinal endoscopy, 87(5), 1167–1179. https://doi.org/10.1016/j.gie.2017.12.009
Becker, C., Zumbrunn, S., Beck, K., Vincent, A., Loretz, N., Müller, J., Amacher, S. A., Schaefert, R., & Hunziker, S. (2021). Interventions to Improve Communication at Hospital Discharge and Rates of Readmission: A Systematic Review and Meta-analysis. JAMA network open, 4(8), e2119346. https://doi.org/10.1001/jamanetworkopen.2021.19346
Bielawska B, Day AG, Lieberman DA, et al. Risk factors for early colonoscopic perforation include nongastroenterologist endoscopists: a multivariable analysis. Clin Gastroenterol Hepatol. 2014; 12:85–92.
Bielawska B, Hookey LC, Sutradhar R, et al. Anesthesia assistance in outpatient colonoscopy and risk of aspiration pneumonia, bowel perforation, and splenic injury. Gastroenterology. 2018; 154(1): 77–85. e3.
Grossberg LB, Vodonos A, Papamichael K, Novack V, Sawhney M, Leffler DA. Predictors of post-colonoscopy emergency department use. Gastrointest Endosc. Feb 2018;87(2):517-525.
Kothari ST, Huang RJ, Shaukat A, Agrawal D, Buxbaum JL, Abbas Fehmi SM, Fishman DS, Gurudu SR, Khashab MA, Jamil LH, Jue TL, Law JK, Lee JK, Naveed M, Qumseya BJ, Sawhney MS, Thosani N, Yang J, DeWitt JM, Wani S; ASGE Standards of Practice Committee Chair. ASGE review of adverse events in colonoscopy. Gastrointest Endosc. 2019 Dec;90(6):863-876.e33. doi: 10.1016/j.gie.2019.07.033. Epub 2019 Sep 25. PMID: 31563271
Padmanabhan, H., Rothnie, A., Higgins, A., Grewal, A., Arndtz, K., Nevill, A. M., Brookes, M. J., & Mathew, R. (2016). Preassessment Interview Improves the Efficacy and Safety of Bowel Preparation for Colonoscopy. Canadian journal of gastroenterology & hepatology, 2016, 7591637. https://doi.org/10.1155/2016/7591637
Petersen, B. T., Cohen, J., Hambrick, R. D., 3rd, Buttar, N., Greenwald, D. A., Buscaglia, J. M., Collins, J., & Eisen, G; Reprocessing Guideline Task Force,. (2017). Multisociety guideline on reprocessing flexible GI endoscopes: 2016 update. Gastrointestinal endoscopy, 85(2), 282–294.e1. https://doi.org/10.1016/j.gie.2016.10.002
Ranasinghe I, Parzynski CS, Searfoss R, Montague J, Lin Z, Allen J, Vender R, Bhat K, Ross JS, Bernheim S, Krumholz HM, Drye EE. Differences in colonoscopy quality among facilities: Development of a post-colonoscopy risk-standardized rate of unplanned hospital visits. Gastroenterology. Jan 2016;150(1):103-13.
Thompson BM, Stearns JD, Apsey HA, Schlinkert RT, Cook CB. Perioperative Management of Patients with Diabetes and Hyperglycemia Undergoing Elective Surgery. Curr Diab Rep. 2016 Jan;16(1):2.
Wang L, Mannalithara A, Singh G, Ladabaum U. Low rates of gastrointestinal and non-gastrointestinal complications for screening or surveillance colonoscopies in a population-based study. Gastroenterology. Oct 2018; 154: 540-555.
Measure Impact
This measure includes colonoscopies performed to screen for colorectal cancer (in addition to other qualifying colonoscopies). Screening for colorectal cancer is a U.S. Preventive Services Task Force (USPSTF) “A” recommendation for adults aged 50-75 and a “B” recommendation for adults aged 45-49 (Lin, JS et al., 2021), and millions of colorectal cancer screenings (a large proportion of them colonoscopies) are performed each year.
A hospital visit following a colonoscopy is an unexpected, undesired, and potentially preventable outcome that is important to patients and providers. Providers may be unaware of their patients’ hospital visits after surgery because patients often present to the ED or to different hospitals, leading to understated adverse event rates and suggesting the need for better measurement to drive quality improvement. Therefore, both patients and providers benefit from outcome measures of hospital visits – a broad, patient-centered outcome that reflects the full range of reasons leading to hospitalization among patients undergoing a colonoscopy.
Colorectal cancer screenings are elective, so patients must choose to receive the screening. However, due to patient concerns about the procedures (related to anticipated discomfort of the preparation or the procedure itself), efforts have been made to better understand the patients’ perspective related to screening. A 2022 study aimed to identify quality domains that map to patients’ experiences and concerns around colonoscopy; study authors found that, among other domains, “information” as a concept that fit within the model describing patients’ experiences and needs (Rosvall et al., 2022). Information about colonoscopy could include instructions on the preparation prior to and care after a colonoscopy, but also could include information on the quality of care provided by the facilities (and their providers) performing the service. In a study of patient self-reported complications, patients with complications rated the information they received about their surgery lower compared with patients who did not experience a complication. In addition, patients who reported experiencing a complication were more likely to experience post-operative pain, and lower quality of life (Woodfield et al., 2019).
The Colonoscopy measures is part of the Hospital Outpatient Quality Reporting (HOQR) and the Ambulatory Surgical Center Quality Reporting (ASCQR) programs; both pay-for-reporting programs (not pay-for-performance). Currently, there are no other publicly available quality reports related to this procedure which underscores the measurement gap that would exist without this measure. Thus, this measure addresses an important quality measurement area and enhances the information available to patients choosing among HOPDs/ASCs that provide screening colonoscopies. Furthermore, providing outcome rates to HOPDs and ASCs makes visible to clinicians and facilities meaningful quality differences and incentivizes improvement.
References
Lin JS, Perdue LA, Henrikson NB, et al. Screening for Colorectal Cancer: An Evidence Update for the U.S. Preventive Services Task Force [Internet]. Rockville (MD): Agency for Healthcare Research and Quality (US); 2021 May. (Evidence Synthesis, No. 202.) Available from: https://www.ncbi.nlm.nih.gov/books/NBK570913/
Rosvall, A., Annersten Gershater, M., Kumlien, C., Toth, E., & Axelsson, M. (2022). Patient-Reported Experience Measures for Colonoscopy: A Systematic Review and Meta-Ethnography. Diagnostics (Basel, Switzerland), 12(2), 242. https://doi.org/10.3390/diagnostics12020242
Woodfield, J., Deo, P., Davidson, A., Chen, T. Y., & van Rij, A. (2019). Patient reporting of complications after surgery: what impact does documenting postoperative problems from the perspective of the patient using telephone interview and postal questionnaires have on the identification of complications after surgery?. BMJ open, 9(7), e028561. https://doi.org/10.1136/bmjopen-2018-028561
Performance Gap
There remains variation in measure scores among facilities. Below we provide the distribution of measures scores, as well as evidence of variation through the median odds ratio. We have seen improvement in this measure score, for both HOPDs and ASCs, over time. Please see Section 6.2.4 on improvement for more information.
Measure score distribution
Tables 1A and 1B, and 2A and 2B, shows the distribution of measure scores, the risk-standardized hospital visit rate, or RSHVR, using the most recent testing data (2024 EM Dataset; July 1, 2020-December 31, 2022) for HOPDs, and ASCs (per 1,000 colonoscopies). Histograms are presented in Figure 4A (HOPD) and 4B (ASC), in the attachment.
For HOPDs, RSHVRs range from 9.13 to 18.14; the median is 13.13; the 25th percentile is 12.74 and the 75th percentile is 13.61. This indicates that the best performer (9.13 per 1,000 colonoscopies) is performing about 30 percent better than the median, and the worst performer (18.14) is performing about 38 percent worse than the median.
For ASCs, RSHVRs range from 6.29 to 14.16; the median is 9.85; the 25th percentile is 9.42 and the 75th percentile is 10.37. This indicates that the best performer (6.29 per 1,000 colonoscopies) is performing about 36 percent better than the median, and the worst performer (14.16) is performing about 44 percent worse than the median.
Median odds ratio
We provide further evidence of variation by calculating and interpreting the median odds ratio (MOR) (Merlo, et al., 2006). The MOR represents the median increase in odds of a hospital visit if a procedure on a single patient was performed at a higher risk facility compared to a lower risk facility. It is calculated by taking all possible combinations of facilities, always comparing the higher risk facility to the lower risk facility. The MOR is interpreted as a traditional odds ratio would be.
The median odds ratio for this updated analysis for HOPDs is 1.19 and for ASCs is 1.21. The median odds ratio suggests a meaningful increase in the risk of a hospital visit if a colonoscopy was performed at a higher risk facility compared to a lower risk facility. A value of 1.19 for HOPDs and ASCs indicates that a patient has a 19 percent increase, or a 21 percent increase in the odds of a hospital visit if the colonoscopy was performed at higher risk facility compared to a lower risk facility (for an HOPD and ASC, respectively) indicating the impact of quality on the outcome rate is meaningful.
References
Merlo J, Chaix B, Ohlsson H, Beckman A, Johnell K, Hjerpe P, Råstam L, Larsen K. (2006) A brief conceptual tutorial of multilevel analysis in social epidemiology: Using measures of clustering in multilevel logistic regression to investigate contextual phenomena. J Epidemiol Community Health, 60(4):290-7.
| Overall | Minimum | Decile_1 | Decile_2 | Decile_3 | Decile_4 | Decile_5 | Decile_6 | Decile_7 | Decile_8 | Decile_9 | Decile_10 | Maximum | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Mean Performance Score | 13.18 | 9.13 | 11.56 | 12.40 | 12.74 | 12,94 | 13.07 | 13.18 | 13.38 | 13.63 | 14.02 | 14.93 | 18.14 |
| N of Entities | 3,775 | 1 | 377 | 378 | 377 | 378 | 377 | 378 | 378 | 377 | 378 | 377 | 1 |
| N of Persons / Encounters / Episodes | 1,747,993 | 3,606 | 506,806 | 210,522 | 151,798 | 101,998 | 63,409 | 83,284 | 131,654 | 123,112 | 153,100 | 222,250 | 1,303 |
Equity
Equity
At the patient level, we know that patients with social risk factors (present in our conceptual model) may have higher unadjusted outcomes (hospital visit rates) following an outpatient colonoscopy, but differences vary depending on the social risk factor. For example, at the patient level, for HOPDs we found that patients with dual eligibility (DE) have an unadjusted hospital visit rate of 23.8%, compared with 12.6% for patients without DE. Patients with high ADI, however, have unadjusted hospital visit rates of 17.0%, vs. 12.7% for patients without high ADI. A similar pattern is observed for patients who received their colonoscopies at ASCs (see Table 12 in the attachment).
Measure Stratification
Due to these observed disparities in outcomes and the desire to shed light on and improve outcomes for all patients, CORE has developed for CMS (as described in the “stratification” section of this CBE submission) a disparities stratification methodology. Please see Section 1.19, “Measure Stratification Details,” for more information about the methodology; here we provide the most recent results for both the within-facility disparities method (which compares care within a hospital, comparing their DE with non-DE patients) and the across-facility method (which compares facility-level outcomes for DE patients to the national average for all DE patients). The full methodology and results are available in more detail in this report:
https://qualitynet.cms.gov/files/652fd45a8be3e0001c0b5141?filename=CY23_OP_32_35_36_DsprtySpecs.pdf
For the disparities analysis described below, we used data from July 1, 2019 through June 30, 2021.
For the within-hospital approach, we found that more hospitals have worse outcomes for their DE patients compared with their non-DE patients. In these analyses, there were 1,736,193 procedures overall; 86,620 (5%) of patients were dual eligible (DE); 976 of 3,379 facilities (29%) had sufficient volume for reporting. Eligible hospitals are defined as hospitals with at least one patient with DE and one patient without DE. Reporting HOPDs must have at least 25 patients with DE and 25 patients without DE.
We characterized performance using the within-facility approach at the facility level, using rate difference cutoffs of: “better” outcomes as rate differences of less than -1%; “worse” outcomes as rate differences of >1%, and “similar” outcomes for rate differences between -1% and +1%. Using these categories, we found that 151 hospitals (4.5%) were characterized as “better,” 275 hospitals (8.1%) were characterized as “worse” and the remaining 550 (16.3%) were characterized as “similar” for their outcomes for DE patients compared with non-DE patients. Most hospitals (71.1%) had too few cases to be evaluated. There was a 1.8 percent rate adjusted rate difference (mean facility difference between DE and non-DE patients) indicating overall worse performance for DE patients.
For the across-facility disparities approach, we found only one facility had outcomes for DE patients that were performed better than the national rate, and 2 facilities that were worse; 513 (15.1%) were no different than the national rate and most hospitals (84.8%) did not have sufficient number of DE patients. In this analysis there were 86,648 DE procedures, 3,392 eligible facilities (with at least one DE patient), and 516 facilities eligible for reporting (those with at least 45 DE patients) (15.2%).
Publicly reported measure
The version of the Colonoscopy measure that is publicly reported is not adjusted for social risk factors. We performed two analyses to explore the impact of adding either of two social risk factors (DE, and ADI) to the model, on measure scores. We found that adding either social risk factor to the model did not result in meaningful impacts on measure scores, suggesting that the variables in the risk model account for most of the differences we see in unadjusted patient-level outcome rates. Therefore, in this pay-for-reporting program, providers will not be unfairly profiled when assessed by the Colonoscopy measure as specified for public reporting. We describe the analyses and results below. All analyses described below used the 2024 EM Dataset.
To examine the impact of social risk factors on measure scores, we first examined correlations (Pearsons) between measure scores with and without either social risk factor and found that correlations were near 1 (HOPDs: 0.997, and 0.997, for DE and high ADI, respectively; ASCs: 0.996, and 0.998, for DE and high ADI, respectively) (Figures 7A, 7B, and 8A, 8B in attachment of tables/figures).
Second, we examined the association (Spearman) between the facility proportion of patients with each social risk factor and measure scores, focusing on the quartile of facilities with the highest proportion of patients with social risk factors (Figures 9A, 9B, and 10A, 10B in the attachment of tables/figures). We found that for HOPDs there no significant correlation (r=-0.024, p=.51) between the proportion of patients with DE and the measure score for the fourth quartile of facility-proportion of patients with DE, and no significant correlation for the high ADI variable (r=-0.02, p=.63). We found similar results for ASCs (r=-0.056, p=.22; r=0.014, p=.77).
We concluded therefore, that there is little to no impact of adding social risk factors on measure scores for this HOPD Surgery measure. As described above, however, CMS has implemented confidential reporting of the measure, stratified for DE.
Feasibility
Feasibility
This is a claims-based measure and as mentioned above, the measure score is calculated automatically from claims data which are routinely generated during the delivery of care. No data are collected by facilities; therefore, this measure imposes no burden on measured entities, and no implementation effort. CMS monitors feedback from the public and measured entities through CMS’s Q&A portal on QualityNet; there have been no concerns about burden related to this measure. There are no concerns about patient confidentiality because the measure is based on claims data submitted by facilities to CMS, and CMS then uses that data for both payment and calculation of the measure score.
We did not perform an analysis of missing data for the measure because it is based on a 100% sample of paid, final action claims submitted by facilities for payment. To ensure complete claims, we allow at least 3 months of time between accessing the data and the end of the performance period.
Because this is a claims-based measure the is no burden on the facility; rates are automatically calculated by CMS based on claims data submitted by facilities for payment.
Proprietary Information
Scientific Acceptability
Testing Data
We use paid, final action Medicare claims to identify colonoscopies performed in the outpatient setting at Hospital Outpatient Departments (HOPDs) and Ambulatory Surgical Centers (ASCs), including subsequent hospital visits. In addition, we use the Center for Medicare and Medicaid Services (CMS) enrollment and demographic data to determine inclusion and exclusion criteria. Patient history is assessed using claims data collected in the 12 months prior to the colonoscopy procedure. The measure is calculated separately for HOPDs and ASCs, and the results in this form are presented separately by facility type. Please see the numerator and denominator details sections for additional information. Please see Table 3 (attachment) for details on the different datasets used for testing (original development, and this endorsement maintenance submission).
Please see Table 3 (attachment) for details.
Please see Table 3 (attachment) for details.
Please see Table 3 (attachment) for details.
Reliability
We tested facility-level measure score reliability using the signal-to-noise method, using the formula presented by Adams and colleagues (Yu et al., 2013; Adams et al., 2010). Specifically, for each facility we calculate the reliability as:
Reliability=(σ_(facility-to-facility)^2)/(σ_(facility-to-facility)^2+ (σ_(facility error variance)^2)/n)
Where facility-to-facility variance is estimated from the hierarchical logistic regression model, n is equal to each facility’s observed case size, and the facility error variance is estimated using the variance of the logistic distribution (pi^2/3).
Signal-to-noise reliability scores can range from 0 to 1. A reliability of zero implies that all the variability in a measure is attributable to measurement error. A reliability of one implies that all the variability is attributable to real difference in performance.
We calculated the measure score reliability (using the 2024 EM Dataset) for all facilities, and for facilities with a volume cutoff of 30 procedures. Our rationale for this is described below.
In general, CMS sets the volume cutoff for publicly reporting facility measures scores based on two considerations. CMS considers the empiric results of reliability testing conducted on the dataset used for public reporting. CMS also considers the volume cutoff for score reporting used for related measures. CMS has empirically determined that measure scores for facilities with 30 or more procedures are reliable. Regardless of the score reporting volume cutoff, all facilities and their cases are used in calculating the measure scores. In the dry run and in public reporting CMS typically reports scores for facilities with fewer procedures than the volume cutoff as having “too few cases” to support a reliable estimate. In summary, the measure specifications do not prejudge the ideal volume cutoff. The minimum sample size for public reporting is a policy choice that balances considerations such as the facility-level reliability testing results on the reporting data and consistency across measures for consumers.
References
Yu, H, Mehrota, A, Adams J. (2013). Reliability of utilization measures for primary care physician profiling. Healthcare, 1, 22-29.
Adams J, Mehrota, A, Thoman J, McGlynn, E. (2010). Physician cost profiling – reliability and risk of misclassification. NEJM, 362(11): 1014-1021.
The distribution of measure score reliability (signal-to-noise reliability) for HOPDs and ASCs is shown in Table 4 and Table 5 (see attachment).
HOPDs: Using three years of performance data, the median facility-level reliability score is 0.748 (IQR, 0.555 - 0.872) for HOPDs with at least 30 cases (the public reporting threshold) representing moderate reliability (Table 4 and Table 6A in the attachment).
ASCs: Using three years of performance data, the median facility-level reliability score for ASCs is 0.822 (IQR, 0.752 - 0.948) for ASCs with at least 30 cases (the public reporting threshold) representing moderate reliability (Table 4 and Table 6B in the attachment).
We note that there is only one embedded table in the Battelle form, which we used for the HOPD data. The ASC results can be found in Table 6B in the attachment.
HOPDs
Using three years of performance data, the median facility-level reliability score is 0.748 (IQR, 0.555 - 0.872) for HOPDs with at least 30 cases (the public reporting threshold) representing moderate reliability.
At the current public reporting threshold, most facilities (about 75 percent of facilities with at least 30 cases) fall above the 0.6 minimum threshold stated in Battelle’s current CBE guidebook. If CMS were to increase the case volume minimum so that all facilities exceeded this threshold, it would remove publicly available information from about 205 facilities that are currently publicly reported.
We believe that median reliability of 0.6 (signal to noise) is sufficiently high for a facility-level publicly reported measure in a pay for reporting program. Increasing the minimum case volume for HOPDs has the tradeoff of removing important information available to the public on Care Compare.
ASCs
Using three years of performance data, the median facility-level reliability score for ASCs is 0.822 (IQR, 0.752 - 0.948) for ASCs with at least 30 cases (the public reporting threshold) representing moderate reliability.
At the current public reporting threshold of at least 30 cases, most facilities (all but the first decile) meet the 0.6 minimum threshold stated in Battelle’s current CBE guidebook.
| | Overall | Minimum | Decile_1 | Decile_2 | Decile_3 | Decile_4 | Decile_5 | Decile_6 | Decile_7 | Decile_8 | Decile_9 | Decile_10 | Maximum |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Reliability | 0.700 | 0.314 | 0.314 | 0.446 | 0.552 | 0.636 | 0.714 | 0.778 | 0.829 | 0.872 | 0.909 | 0.953 | 0.992 |
| Mean Performance Score | 13.18 | 13.24 | 13.24 | 13.27 | 13.28 | 13.27 | 13.31 | 13.39 | 13.22 | 13.24 | 13.09 | 12.52 | 13.45 |
| N of Entities | 3,270 | 16 | 329 | 320 | 329 | 330 | 327 | 327 | 328 | 325 | 328 | 327 | 1 |
| N of Persons / Encounters / Episodes | 1,741,827 | 480 | 14,554 | 24,790 | 38,896 | 55,511 | 78,332 | 109,905 | 153,040 | 212,800 | 318,259 | 735,740 | 11,595 |
Validity
For this endorsement maintenance submission, we provide evidence of validity through: face validity conducted during measure development, validation of the outcome, empiric validity through associations with volume, and evidence of improvement in the measure score. Below we describe the methodology; the results are provided in the subsequent section.
Face validity
We assessed measure validity through systematic assessment of measure face validity by a technical expert panel (TEP) of national experts and stakeholder organizations.
To convene the TEP, we released a public call for nominations and selected individuals to represent a range of perspectives including clinicians, patients, and individuals with experience in quality improvement, performance measurement, and healthcare disparities. We held three structured TEP conference calls consisting of presentation of key issues, our proposed approach, and relevant data, followed by open discussion among TEP members.
The TEP, made up of 17 members including patient representatives, expert clinicians, methodologists, researchers, and providers, were asked to formally assess the measure’s face validity. We systematically assessed the face validity of the measure score as an indicator of quality by soliciting TEP members’ agreement with the following statement: “The risk-standardized hospital visit rates obtained from the colonoscopy measure as specified can be used to distinguish between better and worse quality facilities.”
The 14 TEP members who responded to the survey indicated their agreement with the face validity of the measure on a six-point scale:
1=Strongly disagree
2=Moderately disagree
3=Somewhat disagree
4=Somewhat agree
5=Moderately agree
6=Strongly agree
List of TEP Members
- Joel Brill, MD; Predictive Health LLC (Chief Medical Officer); Fair Health (Medical Director)
- Zahid Butt, MD; Medisolv Inc. (CEO)
- David Chang, PhD, MPH, MBA; University of California San Diego (Director of Outcomes Research, Assistant Professor, Department of Surgery)
- Richard Dutton, MD, MBA; Anesthesia Quality Institute (Executive Director)
- Brian Fennerty, MD; Oregon Health and Science University (Professor of Medicine, Department of Internal Medicine, Section of Gastroenterology)
- Terry Golash, MD; Aetna, Inc. (Senior Medical Director)
- Claudia Gruss, MD; Arbor Medical Group, a division of ProHealth (Physician Partner)
- Cynthia Ko, MD, MS; University of Washington (Associate Professor, Division of Medicine; Adjunct Associate Professor, Department of Health Services)
- David Lieberman, MD; Oregon Health and Science University (Professor of Medicine; Chief, Division of Gastroenterology and Hepatology)
- Keith Metz, MD, JD, MSA; Great Lakes Surgical Center (Medical Director)
- Michael Morelli, MD, CPE; Indianapolis Gastroenterology and Hepatology (President)
- Philip Schoenfeld, MD, MSEd, MSc; University of Michigan (Professor of Medicine, Division of Gastroenterology)
- Anthony Senagore, MD, MS, MBA; Central Michigan University, School of Medicine (Chair, Surgical Disciplines)
- Joan Warren, PhD; Applied Research Program, NIH, National Cancer Institute (Epidemiologist)
- Jennifer Weiss, MD, MS; University of Wisconsin School of Medicine and Public Health (Assistant Professor, Department of Medicine – Division of Gastroenterology & Hepatology)
- 16, 17) Two patients
Validation of the Outcome
The outcome of an unplanned hospital visit following a low-risk elective procedure such as a colonoscopy, is intended to capture adverse events that occur as part of the care received before, during, and after the procedure. To validate the outcome, we examined identified the most commonly occurring principal discharge diagnosis codes associated with the post-procedure hospital visit. (For any hospitalization, a claim for the hospital visit is submitted to CMS that indicates the main reason for the hospitalization; there is only one such main reason, called the “principal diagnosis code” that is used to capture this information.) Based on previous research to validate the outcome during measure development, we know that the most frequent reasons for a post-colonoscopy hospital visit are complications from the procedure. We updated this analysis for this endorsement maintenance submission.
External Empiric Validity
One approach for assessing the validity of a quality measure is to show that performance on the test measure is associated with another quality measure in the same causal pathway. To do this, we needed to identify a comparator measure, however, as summarized below, we did not identify a suitable measure with currently publicly available data.
We first considered CMS’s two related CBE-endorsed measures, Facility-Level 7-Day Hospital Visits after General Surgery Procedures Performed at ASCs (ASC General Surgery), and Hospital Visits after Hospital Outpatient Surgery (HOPD Surgery). The outcome of both measures is nearly identical to that of the colonoscopy measure; an unplanned hospital visit is defined as an emergency department (ED) visit, observation stay, or unplanned inpatient admission. Hence, the measures target the same quality domains as the CMS colonoscopy measure. The patient cohort is also somewhat similar in that the measures target Medicare Fee-For-Service (FFS) patients aged 65 years and older. The cohorts, however, have no overlap with the colonoscopy measure, because they include patients undergoing general surgery, not colonoscopy procedures. Furthermore, the clinicians performing the procedures across the different cohorts are unlikely to be the same individuals, and in addition, most ASCs (about 70% as of 2022) subspecialize (MedPAC, 2024).
To identify non-CMS measures against which to validate, we first searched Battelle’s website for measures related to colonoscopy specified for the HOPD or ASC level and identified only one measure: Appropriate Follow-Up Interval for Normal Colonoscopy in Average Risk Patients. This measure identifies the percentage of patients who have received a screening colonoscopy and have a regular recommended follow-up of ten years. This measure excludes patients who are older than 66 or who have a life expectancy of fewer than ten years, as the follow-up colonoscopy is no longer deemed beneficial. This measure is also not risk-adjusted. Because this follow-up measure does not assess the domains of quality measured by the CMS colonoscopy measure we would expected it to correlate with facilities’ 7-Day Risk-Standardized Hospital Visit rate we therefore did not use it to externally validate the CMS measure.
In summary, none of the measures that we identified meet the criteria for a comparator measure that could be used for external validation.
Association with volume
Although the evidence is inconsistent, studies have shown a volume outcome relationship between colonoscopy volume (at the clinician level) and complications (Grossberg et. al., 2018; Wang et. al., 2018, Forbes et. al., 2020). We therefore assessed if higher facility colonoscopy volume was associated with the Colonoscopy measure score. We hypothesized that there would be a weak to moderate, negative relationship between facility procedural volume and colonoscopy measure scores, with higher volumes being associated with better (lower) Colonoscopy measure scores.
Evidence for improvement
The majority of hospitals download their facility-specific reports and claims-detail reports when they are released (three times per year). We also receive specific questions from facilities about these reports, so we know they are using them. As a result, we expect that hospitals, through their work with CMS’s Quality Improvement Organizations (QIOs) and their own internal quality improvement, have put in place processes and procedures to improve the quality and coordination of care following outpatient colonoscopy, resulting in improvement in outcome rates. To empirically examine changes in risk-adjusted outcomes over time, we provide density plots (histograms, but without the bars) for the risk-standardized outcomes for each reporting period, comparing the current data used for this endorsement maintenance submission (July 1, 2020-December 31, 2022) with prior reporting periods.
References
Forbes, N., Boyne, D. J., Mazurek, M. S., Hilsden, R. J., Sutherland, R. L., Pader, J., Ruan, Y., Shaheen, A. A., Wong, C., Lamidi, M., Lorenzetti, D. L., Brenner, D. R., & Heitman, S. J. (2020). Association Between Endoscopist Annual Procedure Volume and Colonoscopy Quality: Systematic Review and Meta-analysis. Clinical gastroenterology and hepatology : the official clinical practice journal of the American Gastroenterological Association, 18(10), 2192–2208.e12. https://doi.org/10.1016/j.cgh.2020.03.046
Grossberg LB, Vodonos A, Papamichael K, Novack V, Sawhney M, Leffler DA. Predictors of post-colonoscopy emergency department use. Gastrointest Endosc. Feb 2018;87(2):517-525.
MedPAC’s Report to Congress, Chapter 5, Ambulatory Surgical Center Services, March 2024. https://www.medpac.gov/wp-content/uploads/2024/03/Mar24_MedPAC_Report_To_Congress_SEC.pdf
Wang P, Xu T, Ngamruengphong S, Makary MA, Kalloo A, Hutfless S. Rates of infection after colonoscopy and osophagogastroduodenoscopy in ambulatory surgery centres in the USA. Gut. 2018;67(9):1626–1636.
Face Validity
The distribution of the responses is shown below:
- Mean rating=4.6.
- Frequency of Ratings of Agreement
- Rating, # (%) of Responses:
- 1 (Strongly disagree): 0 (0)
- 2 (Moderately disagree): 1 (7.1)
- 3 (Somewhat disagree): 1 (7.1)
- 4 (Somewhat agree): 2 (14.3)
- 5 (Moderately agree): 8 (57.1)
- 6 (Strongly agree): 2 (14.3)
Of the 14 TEP members who responded to the survey, 12 (86%) indicated they somewhat, moderately, or strongly agreed with the validity statement. In addition, one TEP member somewhat disagreed, and one TEP member moderately disagreed. The TEP member who moderately disagreed did not provide a reason. The reason for the other TEP member’s disagreement can no longer be accessed due to software restrictions.
Validation of the Outcome
Tables 7A and 7B show the most frequent (the top 25) principal diagnosis codes associated with the outcome (unplanned hospital visits within 7 days of a qualifying colonoscopy). For example, For HOPDs, the top four principal diagnoses are “Post procedural hemorrhage….,” “Hemorrhage of anus and rectum,” “Gastrointestinal hemorrhage, unspecified,” and “Urinary tract infection, site not specified.” Many of the codes in the list in Table 7A (see attachment) indicate a complication from the procedure.
External empiric validity
As noted above, we did not identify any measures that were suitable for comparison with the colonoscopy measure, therefore we examined the association of the measure score with facility volume. Pearson’s correlation coefficient for the association between measure scores and facility procedural volume was -0.224 for HOPDs; and -0.132 for ASCs.
Evidence of improvement
The colonoscopy measure went into public reporting in December 2017. Since that time, both the HOPD and ASC colonoscopy measures have shown improvement. For example, the national rate of hospital visits per 1,000 colonoscopies among HOPDs declined from 16.4 in 2018 reporting to 14.8 in 2019 reporting, and 13.2 in 2023 reporting (current data). The distribution of risk-standardized rates also show improvement; Figures 5A and 5B (see attachment) show this improvement graphically as density graphs (histograms without bars), showing an improvement in performance (current data shown in red solid line) in the distribution of risk-standardized rates (narrowing of the range of performance and a shift of performance to the left), for HOPDs (Figure 5A), and to a lesser extent for ASCs (Figure 5B). We note that in Figure 5A the results for 2016-2018 and 2017-2019 almost perfectly overlap so they appear superimposed.
Our results, shown above, demonstrate validity of the Colonoscopy measure at several levels for both ASCs and HOPDs: face validity as assessed by experts; validity of the outcome, shown by analysis of ICD-10 codes associated with the outcome; validity of the measure score as shown by association (in the expected direction) with procedural volume; and finally, evidence of improvement over five performance periods.
Risk Adjustment
Description of Risk Adjustment Method
We use a two-level hierarchical logistic regression model to estimate risk-standardized hospital visit rates (RSHVRs). This approach accounts for the clustering of patients within facilities and variation in sample size.
The risk-adjustment model includes 15 patient-level variables, including age, concomitant upper GI endoscopy, polypectomy during procedure, and 12 comorbidity variables obtained from inpatient, outpatient, and physician claims 12 months prior to index procedure. The 2023 Colonoscopy Measure Data Dictionary tab “Colonos Risk Factor CCs” presents the definition of these variables, based on CMS’s hierarchical CCs. The measure does not include certain diagnoses that occur only at the time of the colonoscopy procedure toward risk adjustment because these diagnoses may represent complications of care; see the 2023 Colonoscopy Measure Data Dictionary tab “Colonos CoC CCs” for a summary of these diagnoses.
Selection of Risk-Adjustment Variables during Measure Development
Candidate risk-adjustment variables were patient-level risk adjustors that are expected to be predictive of hospital visits following colonoscopy, based on prior literature, clinical judgment, and empirical analysis. We limited our initial selection of candidate variables for inclusion in our preliminary colonoscopy-specific risk-adjustment model to variables with a strong clinical rationale for inclusion as identified in the literature and through clinical expert input. These variables include age, sex, indicators of comorbidity and disease severity, and two procedural factors associated with an increased risk of adverse outcomes following colonoscopy (concomitant upper GI endoscopy and polypectomy during the procedure).
Variable Selection
To select the final variables to include in the risk-adjustment model, using Dataset #1, we fitted a logistic regression model to predict the outcome with the candidate variable set. To develop a parsimonious model, we then removed non-significant variables from the initial model using a stepwise purposeful selection method described by Hosmer and Lemeshow (2000). Our goal was to minimize the number of variables in the model while preserving model performance (as measured by the c-statistic). During this process, the least significant variable in the model was removed one at a time until only statistically significant (p<0.05, assessed using a likelihood ratio test) variables remained in the model. Interaction terms between variables were tested and were only retained in the model if significant at a level of p<0.01. The higher threshold for statistical significance ensured that only interactions that have a higher likelihood of being true interactions were included.
More detail about risk adjustment variable selection, including a list of candidate risk adjustment variables, can be found in the “Facility 7-Day Risk-Standardized Hospital Visit Rate after Outpatient Colonoscopy Measure Technical Report,” 2014: https://www.qualitynet.org/files/5d0d37ae764be766b010196e?filename=ClnscpyMsr_TechReport.pdf.
Social Risk Factors for Disparities Analyses
We selected variables representing social risk factors based on a review of literature, conceptual pathways, and feasibility. Below, we describe the pathways by which social risk factors may influence risk of the outcome.
Causal Pathways for Social Risk Variable Selection
Our conceptualization of the pathways by which patients’ social risk factors affect the outcome was informed by the literature and IMPACT Act–funded work by the National Academies of Sciences, Engineering and Medicine (NASEM) and the Department of Health and Human Services Assistant Secretary for Planning and Evaluation (ASPE) (ASPE, 2016; NASEM, 2016).
Literature Review of Social Risk Variables and Ambulatory Surgery Post-Procedure Hospital Visits
To inform a conceptual model for the relationship of social risk factors to the outcome we performed a literature search during development of the related Hospital Visits After Outpatient Surgery measure, which included articles that contained key words in the title or abstract related to outpatient surgeries or procedures, socioeconomic and sociodemographic disparities, and hospital visits (emergency department, observation, or hospital admission). A total of 176 studies were reviewed by title and abstract. There were no studies that addressed colonoscopy specifically, therefore we did not find any studies that suggested that variation in patients’ social risk factors affected variation in colonoscopy outcome risk across facilities.
For this 2024 submission we have reviewed our original conceptual model related to social risk factors and have determined that the original conceptual model remains valid. We note that for colonoscopy, which is a screening procedure, it is well documented that there are disparities for patients with social risk factors related to access and use of screening colonoscopy, in addition to higher mortality rates for colorectal cancer (Warren Andersen et al, 2019).
Regarding the outcome of hospital visits following a screening colonoscopy, we performed an updated focused literature search and found additional evidence for the impact of social risk factors on outcomes for patients undergoing outpatient procedures, but no studies specific to colonoscopy. For example, a 2019 study found that while patients with low income undergoing colectomy had higher rates of surgical-site infections compared with higher-income patients, there was no difference in surgical-site infection rates based on income for patients undergoing hysterectomy (Qi et al., 2019). A 2023 study in cancer patients undergoing surgery found that patients with psychosocial risk factors were more likely to experience complications following surgery (Leeds et al., 2019). Finally, a 2021 study found that for some procedures, people living in counties with high social vulnerability (SVI) were more likely to experience complications compared with patients who live in low SVI counties (Diaz et al., 2021).
We note that ASCs serve a low proportion of patients with social risk factors therefore there are disparities in access to care at ASCs, with Black patients and patients with public insurance being less likely to receive care at an ASC compared with others without those social risk factors (Janeway et al., 2020). A recent study found that there are disparities in the geographic distribution of ASCs, with counties with higher socioeconomic status having more ASCs per capita compared with counties with lower socioeconomic status (Chatterjee, Amen, & Khormaee, 2022).
Conceptual Pathways for Social Risk Factor Variable Selection
Although there is limited literature linking social risk factors and adverse outcomes for colonoscopy, we identified the following potential pathways through which social risk factors may influence the outcome of 7-day visits following a colonoscopy, based on the specific clinical consideration of the procedure and the broader social risk factor literature:
- Differential care within a facility or unmet differential needs. One pathway by which social risk factors may contribute to hospital visit risk is that patients may not receive equivalent care within a facility (Trivedi et al., 2014; ASPE, 2016). However, as noted above, studies of colonoscopy in the HOPD and ASC setting are lacking. Moreover, patients with social risk factors, such as lower education, may require differentiated care – e.g., provision of information at a lower health literacy level – to achieve outcomes comparable to those of patients without social risk factors. Facilities that do not identify the need for and provide such care could have worse outcome rates for their patients with social risk factors.
- Use of lower-quality facilities. Patients may differentially obtain care in lower quality facilities. With respect to inpatient hospital care, patients of lower income, lower education, or unstable housing have been shown not to have equitable access to high-quality facilities because such facilities are less likely to be found in geographic areas with large populations of poor patients (Jha et al. 2011). Thus, patients with low income may be likely to be seen in lower-quality facilities, which can contribute to increased risk of adverse outcomes following hospitalization. While analogous data for patients undergoing colonoscopies at HOPDs and ASCs is lacking, a similar pattern may exist, leading to higher (worse) outcome rates for patients with social risk factors. As described above, patients with social risk factors are less likely to have access to care at ASCs, in general (Chatterjee, Amen, & Khormaee, 2022; Janeway et al., 2020).
- Influence of social risk factors on hospital visit risk outside of facility quality. Some social risk factors, such as income or wealth, may affect the likelihood of post-procedure hospital visits without directly being associated with the quality of care received at the facility. For instance, while a colonoscopy provider and/or a facility may make appropriate care decisions and provide tailored care and education, we hypothesized that a lower-income patient may still have a worse outcome post-procedure due to their approach to preparation for the procedure, a limited understanding of the discharge plan, or a lack of home support, transportation or other resources for following discharge instructions. These factors, however, can be anticipated and addressed for outpatient elective procedures more readily than in more emergent care contexts.
- Relationship of social risk factors with patients’ health at admission. Patients with lower income/education/literacy or unstable housing may have a worse general health status and may present for their procedure with greater severity of underlying illness (ASPE, 2016). This causal pathway should be largely accounted for by current clinical risk-adjustment.
As indicated in the Section 5, Equity, the social risk variables that we examined are:
- Dual-eligible status (DE)
Dual eligibility for Medicare and Medicaid is available at the patient level in the Medicare Master Beneficiary Summary File. The eligibility threshold for over 65-year-old Medicare patients considers both income and assets. For the dual-eligible (DE) indicator, there is a body of literature demonstrating differential health care and health outcomes among beneficiaries, and the DE variable allows us to examine some of the pathways of interest (Department of Health and Human Services, 2016)
- Area deprivation index (ADI)
Area Deprivation index (ADI): The ADI, initially developed by Health Resources & Services Administration (HRSA), is based 17 measures across four domains: income, education, employment, and housing quality (Kind et al., 2018; Singh, 2003).
The 17 components are listed below:
- Population aged ≥ 25 y with < 9 y of education, %
- Population aged ≥ 25 y with at least a high school diploma, %
- Employed persons aged ≥ 16 y in white-collar occupations, %
- Median family income, $
- Income disparity
- Median home value, $
- Median gross rent, $
- Median monthly mortgage, $
- Owner-occupied housing units, % (home ownership rate)
- Civilian labor force population aged ≥16 y unemployed, % (unemployment rate)
- Families below poverty level, %
- Population below 150% of the poverty threshold, %
- Single-parent households with children aged < 18 y, %
- Households without a motor vehicle, %
- Households without a telephone, %
- Occupied housing units without complete plumbing, % (log)
- Households with more than 1 person per room, % (crowding)
ADI scores were derived using beneficiary’s 9-digit ZIP Code of residence, which is obtained from the Master Beneficiary Summary File, and is linked to 2017-2021 US Census/American Community Survey (ACS) data. In accordance with the ADI developers’ methodology, an ADI score is calculated for the census block group corresponding to the beneficiary’s 9-digit ZIP Code using 17 weighted Census indicators. Raw ADI scores were then transformed into a national percentile ranking ranging from 1 to 100, with lower scores indicating lower levels of disadvantage and higher scores indicating higher levels of disadvantage. Percentile thresholds established by the ADI developers were then applied to ADI percentile to dichotomize neighborhoods into more disadvantaged (high ADI areas=ranking equal to or greater than 85) or less disadvantaged areas (Low ADI areas= ranking of less than 85).
References
Chatterjee, A., Amen, T. B., & Khormaee, S. (2022). Trends in geographic disparities in access to ambulatory surgery centers in New York, 2010 to 2018. JAMA Health Forum, 3(10), Article e223608. https://doi.org/10.1001/jamahealthforum.2022.3608
Department of Health and Human Services, Office of the Assistant Secretary of Planning and Evaluation. Report to Congress: Social Risk factors and Performance Under Medicare’s Value-based Payment Programs. 2016; https://aspe.hhs.gov/pdf-report/report-congress-social-risk-factors-and-performance-under-medicares-value-based-purchasing-programs.
Department of Health and Human Services, Office of the Assistant Secretary of Planning and Evaluation. Report to Congress: Social Risk factors and Performance Under Medicare’s Value-based Payment Programs. 2016; https://aspe.hhs.gov/pdf-report/report-congress-social-risk-factors-and-performance-under-medicares-value-based-purchasing-programs. Accessed December 8, 2019.
Diaz, A., Hyer, J. M., Barmash, E., Azap, R., Paredes, A. Z., & Pawlik, T. M. (2021). County-level Social Vulnerability is Associated With Worse Surgical Outcomes Especially Among Minority Patients. Annals of surgery, 274(6), 881–891. https://doi.org/10.1097/SLA.0000000000004691
Hosmer D, Lemeshow S. Applied Logistic Regression. New York: John Wiley & Sons; 2000.
Janeway, M. G., Sanchez, S. E., Chen, Q., Nofal, M. R., Wang, N., Rosen, A., & Dechert, T. A. (2020). Association of race, health insurance status, and household income with location and outcomes of ambulatory surgery among adult patients in 2 US states. JAMA Surgery. https://doi.org/10.1001/jamasurg.2020.3318
Jha AK, Orav EJ, Epstein AM. Low-quality, high-cost hospitals, mainly in South, care for sharply higher shares of elderly black, Hispanic, and Medicaid patients. Health Aff. 2011; 30:1904-1911.
Krumholz HM, Brindis RG, Brush JE, et al. Standards for statistical models used for public reporting of health outcomes: An American Heart Association scientific statement from the Quality of Care and Outcomes Research Interdisciplinary Writing Group: cosponsored by the Council on Epidemiology and Prevention and the Stroke Council endorsed by the American College of Cardiology Foundation. Circulation. 2006; 113(3):456-462.
Leeds, I. L., Meyers, P. M., Enumah, Z. O., He, J., Burkhart, R. A., Haut, E. R., Efron, J. E., & Johnston, F. M. (2019). Psychosocial Risks are Independently Associated with Cancer Surgery Outcomes in Medically Comorbid Patients. Annals of surgical oncology, 26(4), 936–944. https://doi.org/10.1245/s10434-018-07136-3
National Academies of Sciences, Engineering, and Medicine (NASEM); Accounting for Social Risk Factors in Medicare Payment: Identifying Social Risk Factors. Washington DC: National Academies Press; 2016.
National Academies of Sciences, Engineering, and Medicine (NASEM); Accounting for Social Risk Factors in Medicare Payment: Data. Washington DC: National Academies Press; 2016.
Normand S-LT, Shahian DM. Statistical and clinical aspects of hospital outcomes profiling. Statistical Science. 2007; 22(2):206-226.
Qi, A. C., Peacock, K., Luke, A. A., Barker, A., Olsen, M. A., & Joynt Maddox, K. E. (2019). Associations Between Social Risk Factors and Surgical Site Infections After Colectomy and Abdominal Hysterectomy. JAMA network open, 2(10), e1912339. https://doi.org/10.1001/jamanetworkopen.2019.12339
Reames BN, Birkmeyer NJ, Dimick JB, et al. Socioeconomic disparities in mortality after cancer surgery: failure to rescue. JAMA Surg. 2014; 149:475-481.
Trivedi AN, Nsa W, Hausmann LR, et al. Quality and equity of care in U.S. hospitals. New Engl J Med. 2014; 371:2298-2308.
Warren Andersen, S., Blot, W. J., Lipworth, L., Steinwandel, M., Murff, H. J., & Zheng, W. (2019). Association of Race and Socioeconomic Status With Colorectal Cancer Screening, Colorectal Cancer Risk, and Mortality in Southern US Adults. JAMA network open, 2(12), e1917995. https://doi.org/10.1001/jamanetworkopen.2019.17995
Table 8A and 8B (see tables/figures attachment) shows the distribution of social risk factors identified in the conceptual model for the Colonoscopy measure. Across all HOPDs with at least one procedure during the performance period, the facility median proportion of patients with the DE and ADI variables is 3.9% and 9.1%, respectively (Table 8A). For ASCs, the facility median proportion of patients with the DE and ADI variables is 1.6% and 2.6%, respectively (Table 8B).
Tables 9A and 10A show the frequency of risk variables in the final Colonoscopy measure risk model (see attachment) for HOPDs and ASCs, respectively.
Tables 9B and 10B (see attachment) provide risk variable names, definitions, and odds ratios for the HOPD and ASC risk models, respectively. Codes that define the risk variables can be found in the attached data dictionaries for each measure (HOPD, ASC).
During risk variable selection, we iteratively removed non-significant variables from the initial model using a step-wise purposeful selection approach until only statistically significant (p<0.05, assessed using a likelihood ratio test) variables remained in the model. Interaction terms between variables were tested and were only retained in the model if significant at a level of p<0.01.
The final risk-adjustment model has 16 variables (age categories, age categorized x arrhythmia interaction, twelve comorbidity variables, and two surgical variables). With the exception of concomitant endoscopy and polypectomy during procedure, which we define using individual CPT® codes, we define comorbidity variables using CMS Condition Categories (CCs), which are clinically meaningful groupings of thousands of ICD-10 diagnosis codes, maintained by CMS.
See Tab 5, “Colono Risk Factor CCs” and Tab 6, “Colonos Risk Factor Codes” in the attached Data Dictionary for the list of CC and CPT codes used to define the colonoscopy model risk variables.
Final Model Variables:
- Age Categorized (years 65-69; 70-74; 75-79; 80-84; 85+)
- Concomitant Endoscopy
- Polypectomy during Procedure
- Congestive Heart Failure (CC 85)
- Ischemic Heart Disease (CC 86-89)
- Stroke/Transient Ischemic Attack (CC 99-101)
- Chronic Lung Disease (CC 111-113)
- Metastatic Cancer (CC 8-11)
- Liver Disease (CC 27-32)
- Iron Deficiency Anemia (CC 49)
- Disorders of Fluid, Electrolyte, Acid-Base (CC 24)
- Pneumonia (CC 114-116)
- Psychiatric Disorders (CC 57-59, 61-63)
- Drug and Alcohol Abuse/Dependence (CC 54-56; 202-203)
- Arrhythmia (CC 96-97)
- Age Categorized x Arrhythmia Interactions
The frequencies and odds ratios for each risk variable for HOPDs and ASCs are shown in Tables 9A, 9B, 10A and 10B in the attachment.
Our approach to risk adjustment is tailored to, and appropriate for, a publicly reported outcome measure as articulated in published scientific guidelines (Krumholz et al., 2006; Normand & Shahian, 2007). For example, we only adjust for risk factors that are present at the start of care. We do not risk adjust for conditions that are possible adverse events of care and that are only recorded at the time of the procedure. We do not adjust for factors related to the delivery of care that may reflect care quality.
The measure employs a hierarchical logistic regression model (a form of hierarchical generalized linear model [HGLM]) to calculate the measure score. This approach to modeling appropriately accounts for the structure of the data (patients clustered within facilities), the underlying risk due to patients’ procedures/comorbidities, and sample size at a given facility when estimating hospital visit rates. In brief, the approach simultaneously models two levels (patient and facility) to account for the variance in patient outcomes within and between facilities (Normand & Shahian, 2007). At the patient level, the model adjusts the log-odds of hospital visits within 7 days after the procedure for selected demographic, clinical, and procedure risk variables. The second level models the facility-specific intercepts as arising from a normal distribution. The facility intercept, or facility-specific effect, represents the facility contribution to the risk of 7-day hospital visits, after accounting for patient risk and sample size, and can be inferred as a measure of quality. If there were no differences among facilities, then after adjusting for patient risk, the facility intercepts would be identical across all facilities. We provide additional details in the attachment in 4.4.4a
References
Krumholz, H. M., Brindis, R. G., Brush, J. E., et al. (2006). Standards for statistical models used for public reporting of health outcomes: An American Heart Association scientific statement from the Quality of Care and Outcomes Research Interdisciplinary Writing Group: cosponsored by the Council on Epidemiology and Prevention and the Stroke Council endorsed by the American College of Cardiology Foundation. Circulation, 113(3), 456-462.
Normand, S.-L. T., & Shahian, D. M. (2007). Statistical and clinical aspects of hospital outcomes profiling. Statistical Science, 22(2), 206-226.
CORE’s measures undergo an annual measure reevaluation process, which ensures that the risk-standardized models are continually assessed and remain valid, given possible changes in clinical practice and coding standards over time. Modifications made to measure cohorts, risk models, and outcomes are informed by a review of the most recent literature related to measure conditions or outcomes, feedback from various stakeholders, and empirical analyses, including assessment of coding trends that reveal shifts in clinical practice or billing patterns. Input is solicited from a workgroup composed of up to 20 clinical and measure experts, inclusive of internal and external consultants and subcontractors.
To assess model performance, we computed three summary statistics to demonstrate model discrimination and calibration, and we provide risk-decile plots to demonstrate additional evidence of calibration. We describe the approach and results below; related tables and figures are in the attachment.
Discrimination statistics (c-statistic and predictive ability)
1.C-statistic: Area under the receiver operating characteristic (ROC) curve (the c-statistic) indicates the probability that predicting the outcome is better than chance, which is a measure of how accurately a statistical model can distinguish between a patient with and without an outcome. Potential values range from 0.5, meaning no better than chance, to 1.0, meaning perfect discrimination. The 2024 EM Dataset was used for this analysis. The c-statistic of 0.679 for HOPDs, and 0.638 for ASCs, respectively, indicates good model discrimination. Results are similar to the c-statistics submitted for initial CBE endorsement, and in line with readmission-type measures. 2.Predictive ability: Discrimination in predictive ability measures the ability to distinguish high-risk subjects from low-risk subjects; therefore, we want to see a wide range between the lowest decile and highest decile. Predictive ability was 0.47-3.66 for HOPDs, and 0.47-2.34 for ASCs; results are similar to those provided to support initial CBE endorsement.
Calibration statistics (overfitting)
3. Overfitting: Over-fitting refers to the phenomenon in which a model accurately describes the relationship between predictive variables and outcome in the development dataset but fails to provide valid predictions in new patients. Estimated calibration values of γ0 close to 0 and estimated values of γ1 close 1 provide evidence of good calibration of the model. We used Dataset 1a and 1b for this analysis, which were performed during initial measure development. Our results below show a calibration value of close to 0 at one end and close to 1 to the other end indicating good calibration of the model.
Results from original model/measure development:
2010 Medicare 20% FFS Development Sample (Dataset #1a):
Calibration: (0,1)
2010 Medicare 20% FFS Validation Sample (Dataset #1b) results:
Calibration: (-0.03, 0.99)
CORE notes that after initial measure development we do not re-test our risk models for overfitting using a dataset that is external to the testing sample. In our risk models, coefficients are updated each time the measure is calculated; we refit the model with new data each time the measure is calculated. Therefore, random statistical fluctuations in model coefficients across repeated reporting cycles are part of the overall random error in the facility performance estimates.
Risk-decile Plots
Risk decile plots assess calibration of the risk model by examining the alignment between predicted and observed outcomes across deciles of predicted risk. Risk-decile plots (Figures 6A and 6B in the attachment) show good calibration for both the HOPD and ASC models.
We describe the approach to risk variable selection in Section 4.4.2, and analysis and rationale for not including social risk factors in the final model in Section 5.1. In this section we provide the interpretation of the risk model testing results described in section 4.4.5.
The following results demonstrate that the risk-adjustment model adequately controls for differences in patient characteristics:
Discrimination Statistics
The calculated c-statistics were 0.670 (HOPD) and 0.638 (ASC), which indicates good model discrimination. The models also predicted a wide range between the lowest decile and highest decile, indicating the ability to distinguish high-risk subjects from low-risk subjects.
Calibration Statistics
The calibration values were consistently close to 0 at one end and close to 1 to the other, indicating good calibration of the models.
Risk Decile Plot
Higher deciles of the predicted outcomes are associated with higher observed outcomes, which show a good calibration of the models.
Overall Interpretation
Interpreted together with information provided in the aforementioned sections, our diagnostic results demonstrate the risk-adjustment model adequately controls for differences in patient characteristics (case mix).
Use & Usability
Use
The level of analysis is the hospital outpatient department, and the care setting is hospital outpatient.
The level of analysis is facility level, and the care setting is ambulatory surgery centers.
The level of analysis is the rural emergency hospital, and the care setting is outpatient.
Usability
As described earlier in Section 2, provider- and facility-level factors can affect the outcome of complications and hospital visits related to a colonoscopy. For example, provider-level factors such as low provider volume and fellow involvement in the procedure were significantly associated with a higher risk of an ED visit in one study (Grossberg et al., 2018), and low procedure volume was associated with a higher risk of infection in another study (Wang et al., 2018). In addition, the specialty of the provider performing the procedure was associated with outcomes; surgeons’ or endoscopists’ outcomes have been shown to be worse than gastroenterologists’ outcomes (Mazurek et al., 2022). These studies suggest that facilities can influence the patients’ outcome through these modifiable pathways.
In addition, processes that can be influenced by the facility or providers at the facility can improve patient outcomes. For example, the providers’ choice of sedation may influence complication rates as it has been shown that anesthesia assistance (sedation with agents that result in deeper sedation, such as Propofol, rather than conscious sedation) was associated with an increased risk of aspiration pneumonia (OR, 1.63; 95% CI, 1.11–2.37) (Bielawska et al., 2018). Another study found that patients who were taking more classes of medications were at higher risk of an ED visit or hospitalization within 7 days of a colonoscopy, suggesting that properly managing patients’ medications may improve outcomes (Grossberg et al., 2020).
Facilities and providers may be unaware of complications for which patients experience a post-procedural hospital visit, leading to understated complication rates and suggesting the need for this type of measure to drive quality improvement. To support quality improvement, CMS shares reports with measured entities that include measure results benchmarked against the state and nation (“facility-specific reports”), as well as reports that provide claim-level details for each claim that meets numerator and denominator criteria (“claims-detail reports”). These reports include, among other details, the principal diagnosis code associated with the post-procedure hospital visit, which allows facilities to tie their quality improvement efforts to the specific complications that are occurring.
References
Bielawska B, Hookey LC, Sutradhar R, et al. Anesthesia assistance in outpatient colonoscopy and risk of aspiration pneumonia, bowel perforation, and splenic injury. Gastroenterology. 2018;154(1):77–85.e3.
Grossberg LB, Vodonos A, Papamichael K, Novack V, Sawhney M, Leffler DA. Predictors of post-colonoscopy emergency department use. Gastrointest Endosc. Feb 2018;87(2):517-525.
Grossberg, L. B., Papamichael, K., Leffler, D. A., Sawhney, M. S., & Feuerstein, J. D. (2020). Patients over Age 75 Are at Increased Risk of Emergency Department Visit and Hospitalization Following Colonoscopy. Digestive diseases and sciences, 65(7), 1964–1970.
Mazurek, M., Murray, A., Heitman, S. J., Ruan, Y., Antoniou, S. A., Boyne, D., Murthy, S., Baxter, N. N., Datta, I., Shorr, R., Ma, C., Swain, M. G., Hilsden, R. J., Brenner, D. R., & Forbes, N. (2022). Association Between Endoscopist Specialty and Colonoscopy Quality: A Systematic Review and Meta-analysis. Clinical gastroenterology and hepatology : the official clinical practice journal of the American Gastroenterological Association, 20(9), 1931–1946. https://doi.org/10.1016/j.cgh.2021.08.029
Wang P, Xu T, Ngamruengphong S, Makary MA, Kalloo A, Hutfless S. Rates of infection after colonoscopy and osophagogastroduodenoscopy in ambulatory surgery centres in the USA. Gut. 2018;67(9):1626–1636.
CMS shares reports with measured entities that include measure results benchmarked against the state and nation (“facility-specific reports”), as well as reports that provide claim-level details for each claim that meets numerator and denominator criteria (“claims-detail reports). For the reporting period covered with data in this submission (July 1, 2020-December 31, 2022), facilities received two claims-detail reports (released in September 2022 with data on colonoscopies performed from July 1, 2020, through May 31, 2022; and one released in March 2022 with data on colonoscopies performed from July 1, 2020, through November 30, 2022) and one facility-specific report (for this data, released in the fall with data on colonoscopies performed from July 1, 2020, through December 31, 2022.
Stakeholders can submit questions and issues to CMS through an online tool (Q&A tool) available to the public on QualityNet. CMS responds to each question submitted by stakeholders. We describe how we consider this, and other feedback, in the next section.
Each year, issues raised through Q&A or in the literature related to this measure are considered by measure and clinical experts. Any issues that warrant additional analytic work due to potential changes in the measure specifications are addressed as a part of annual measure reevaluation. If small changes are indicated after additional analytic work is complete, those changes are usually incorporated into the measure in the next measurement period. If the changes are substantial, CMS may propose the changes through rulemaking and adopt the changes only after CMS received public comment and finalizes those changes in the OPPS or other rule. Each year we also review and consider changes to HCPCS and ICD-10 codes that are then incorporated into the measure. Those code set files are made available to the public on QualityNet.
During the time between the last maintenance submission and the measure’s re-endorsement in 2020 and this current submission, we have made no major changes to the Colonoscopy measure based on stakeholder feedback.
We note that data used in this endorsement maintenance submission CMS reduced the performance period to approximately 29 months (from the typical 36 months) in response to the COVID-19 public health emergency, which included a CMS decision to exclude claims data for January 1, 2020 – June 30, 2020 (Q1 and Q2 of 2020) from quality measurement. This also impacts the 12-month look-back period for risk adjustment. Consistent with this policy, the final performance period for the results in this report included procedures between 7/1/2020 – 12/31/2022, and the data period used for risk adjustment included claims from 1/1/2019 – 12/31/2019 and 7/1/2020 – 12/31/2022.
As shown in the section above (see section 4.3, Validity), for both the HOPD and ASCs versions of this measure, we have seen improvements in performance in both national unadjusted outcome rates as well as the risk-adjusted measure score. Please see Figures 5A and 5B in the attachment for histogram-like plots that show how facility-level performance has improved between 2016 and 2022, across five performance periods.
There have been no negative unexpected findings during or following implementation of this measure. Please see Section 4.3 (Validity) for a description of improvements we have seen for this measure over time.
Comments
Staff Preliminary Assessment
CBE #2539 Staff Assessment
Importance
Strengths:
- The developer cites evidence from the primary literature interventions that the accountable entity can implements to reduce hospitalizations, such choice of sedation, improving low volume providers and procedures, and proper management of patient medications. The developer further suggests facilities can implement a system of quality improvement and use CMS facility-level and claims-based reports, which include the principal diagnosis on admission, to tie quality improvement efforts to complications that are occurring.
Using data from July 1, 2020 through Dec 31, 2022, the developer reported a distribution of scores for both HOPDs and ASCs, with an overall mean score of 13.18 (national average) and 9.92 (national average), respectively. The developer reported decile ranges of 11.56 – 14.93 for HOPDs and 8.33 – 11.74 for ASCs, showing that ~30% of HOPDs and ASCs are higher than their respective national averages. The developer also notes that millions of colorectal cancer screenings (a large proportion of them colonoscopies) are performed each year and that a hospital visit following a colonoscopy is an unexpected, undesired, and potentially preventable outcome that is important to patients and providers.
Limitations:
- The developer does not provide direct evidence from patients about the meaningfulness of this measure focus. However, the developer posits that both patients and providers benefit from outcome measures of hospital visits, which is a broad, patient-centered outcome that reflects the full range of reasons leading to hospitalization among patients undergoing a colonoscopy. In addition, the developer does cite evidence in which the quality domain of “information” was related to patient experiences and concerns with colonoscopy. In a patient self-report study, patients experiencing complications after surgery rated information received lower compared to those not experiencing complications. Further, the developer argues that providing reports of this measure to providers will incentivize quality improvement efforts.
Rationale:
- The developer posits that this measure provides an opportunity for improvement in reducing unplanned hospital visits due to complications after colonoscopy in the outpatient setting.
The developer cites evidence from the primary literature interventions that the accountable entity can implements to improve hospitalizations, such choice of sedation, improving low volume providers and procedures, and proper management of patient medications. The developer further suggests facilities can implement a system of quality improvement and use CMS facility-level and claims-based reports, which include the principal diagnosis on admission, to tie quality improvement efforts to complications that are occurring.
Using data from July 1, 2020 through Dec 31, 2022, the developer reported a distribution of scores for both HOPDs and ASCs, with an overall mean score of 13.18 (national average) and 9.92 (national average), respectively. The developer reported decile ranges of 11.56 – 14.93 for HOPDs and 8.33 – 11.74 for ASCs, showing that ~30% of HOPDs and ASCs are higher than their respective national averages.
Although the developer does not provide direct evidence from patients about the meaningfulness of this measure focus. However, the developer posits that both patients and providers benefit from outcome measures of hospital visits, which is a broad, patient-centered outcome that reflects the full range of reasons leading to hospitalization among patients undergoing a colonoscopy. In addition, the developer does cite evidence in which the quality domain of “information” was related to patient experiences and concerns with colonoscopy. In a patient self-report study, patients experiencing complications after surgery rated information received lower compared to those not experiencing complications. Further, the developer argues that providing reports of this measure to providers will incentivize quality improvement efforts.
Feasibility Acceptance
Strengths:
- The developer did not conduct a feasibility assessment of an assessment of missing data for this measure, as it used a 100% sample of paid Medicare claims. The developer further states that since this is an claims-based measure, there is no added burden of reporting as all required data elements are routinely generated. The measure does not include any proprietary information.
Limitations:
- None identified.
Rationale:
- The developer did not conduct a feasibility assessment of an assessment of missing data for this measure, as it used a 100% sample of paid Medicare claims. The developer further states that since this is an claims-based measure, there is no added burden of reporting as all required data elements are routinely generated. The measure does not include any proprietary information.
Scientific Acceptability
Strengths:
- The measure is clear and well defined.
Entity level reliability (signal-to-noise) is calculated on 2.5 years of data (7/20-12/22) for the risk adjusted measure.
- For HOPDs entity level reliability is tested across 3270 entities and is >0.6 for somewhat less than 75% of the entities. (The estimated lower quartile is 0.555.)
- For ASCs entity level reliability is tested across 1948 entities and is >0.6 for more than 90% of the entities. (The estimated average reliability for the 2nd decile is 0.627.)
Limitations:
- Reliability is calculated using 2.5 years of data, so the results only apply to that timeframe. CMS CY2025 payment determination will be calculated based on 2 years (1/21-12/23), potentially reducing reliability below 0.6 for approximately 10% more of the entities. This is a concern especially for HOPDs where the reliability could be below 0.6 for about 65% of the entities.
Rationale:
- The measure is well defined. Reliability was assessed at the entity level. Reliability statistics are above the established thresholds for at least 70% of the entities across HOPDs and ASCs.
Strengths:
- The developer provided four sources of support for validity: (1) face validity, (2) association of measure scores with volume, (3) validity of the outcome based on an analysis in claims data of the reasons why patients experienced a hospital visit (complications based on ICD-10 codes), and (4) evidence of improvement in performance since the start of public reporting. First, the developer convened a technical expert panel (TEP) for face validity testing during the measure’s initial development. 12/14 (86.7%) of TEP respondents agreed that the measure is valid and useful and can improve the quality of care. Second, the developer showed that the top reasons for a hospital visit within 7 days of a colonoscopy were complications from the procedures such as, “Post procedural hemorrhage,” “Hemorrhage of anus and rectum,” “Gastrointestinal hemorrhage, unspecified,” and “Urinary tract infection, site not specified.” With respect to association with procedural volume, the developer conducted empirical testing, finding that the results were consistent with their hypothesis of a weak-to-moderate association between facility volume and measure scores. The developer reported a statistically significant correlation of -0.224 for HOPDs; and -0.132 for ASCs, consistent with their hypothesis. However, some limitations should be considered (see limitations).
The developer also conducted statistical risk adjustment, based on their conceptual model, and selecting factors that had a significant correlation to the outcome. The developer also explored social risk factors, such as dual eligibility and the Area Deprivation Index. The developer did not include these in the final models due to the minimal impact these social risk factors have on the measure scores and to the model overall. The developer also explained that the measure is stratified (for HOPDs) by dual eligibility. The developer reported a c-statistic of 0.699, indicating good model discrimination.
Limitations:
- The developer conducted accountable entity-level validity testing by associating measure score to facility volume. The developer considered conducting a correlation analysis to another quality measure but did not find any that “was suitable for comparison.” With respect to the association by facility volume, the developer hypothesized a weak-to-moderate, negative relationship meaning that facilities with higher volume are expected to have lower (better) measure scores. The developer based this hypothesis on prior literature showing a volume-outcome relationship with higher volume associated with better outcomes. The developer reported a statistically significant correlation of -0.224 for HOPDs; and -0.132 for ASCs, consistent with their hypothesis. However, it is unclear, based on the evidence provided, how facility procedural volume supports the validity of the measure, including the quality construct. The developer did not identify procedural volume within the measure's logic model. Volume may indicate higher number of surgeons or increased quality control and resources, but this is not clearly articulated in the submission. In addition, some facilities may not be able to control procedural volume, thus based on this analysis, facilities with low volumes will have worse outcomes. Overall, it is not clear how procedural volume is an appropriate covariate for validity analysis.
Rationale:
- The developer provided four sources of support for validity: (1) face validity, (2) association of measure scores with volume, (3) validity of the outcome based on an analysis in claims data of the reasons why patients experienced a hospital visit (complications based on ICD-10 codes), and (4) evidence of improvement in performance since the start of public reporting. The developer conducted accountable entity-level validity testing by associating measure score to facility volume. The developer considered conducting a correlation analysis to another quality measure but did not find any that “was suitable for comparison.” With respect to the association by facility volume, the developer hypothesized a weak-to-moderate, negative relationship meaning that facilities with higher volume are expected to have lower (better) measure scores. The developer based this hypothesis on prior literature showing a volume-outcome relationship with higher volume associated with better outcomes. The developer reported a statistically significant correlation of -0.224 for HOPDs; and -0.132 for ASCs, consistent with their hypothesis. However, it is unclear, based on the evidence provided, how facility procedural volume supports the validity of the measure, including the quality construct. The developer did not identify procedural volume within the measure's logic model. Volume may indicate higher number of surgeons or increased quality control and resources, but this is not clearly articulated in the submission. In addition, some facilities may not be able to control procedural volume, thus based on this analysis, facilities with low volumes will have worse outcomes. Overall, it is not clear how procedural volume is an appropriate covariate for validity analysis. The developer may be able to speak to this during the endorsement meeting.
The developer also convened a technical expert panel (TEP) for face validity testing during the measure’s initial development. 12/14 (86.7%) of TEP respondents agreed that the measure is valid and useful and can improve the quality of care.
The developer also conducted statistical risk adjustment, based on their conceptual model, and selecting factors that had a significant correlation to the outcome. The developer also explored social risk factors, such as dual eligibility and the Area Deprivation Index. The developer did not include these in the final models due to the minimal impact these social risk factors have on the measure scores. The developer reported a c-statistic of 0.699, indicating good model discrimination.
Equity
Strengths:
- The developer reported that for HOPDs, patients with dual eligibility (DE) have an unadjusted hospital visit rate of 23.8%, compared with 12.6% for patients without DE. Patients with high ADI, however, have unadjusted hospital visit rates of 17.0%, vs. 12.7% for patients without high AD. This pattern was also similar for colonoscopies received at ASCs (although the absolute differences in mean unadjusted rates between patients with and without social risk factors were smaller at ASCs compared with HOPDs). The developer also looked at within facility (HOPD only)and across facility rates for these subpopulations, finding that for the within-facility approach that a higher proportion of hospitals have worse outcomes for their DE patients overall, and specifically showed that 151 hospitals (4.5%) were characterized as “better,” 275 hospitals (8.1%) were characterized as “worse” and the remaining 550 (16.3%) were characterized as “similar” for their outcomes for DE patients compared with non-DE patients. Most hospitals (71.1%) had too few cases to be evaluated. Looking across hospitals, the developer reports that only one facility had outcomes for DE patients that performed better than the national rate, and 2 facilities that were worse; 513 (15.1%) were no different than the national rate and most hospitals (84.8%) did not have sufficient number of DE patients.
Limitations:
- None identified.
Rationale:
- The developer reported that for HOPDs, patients with dual eligibility (DE) have an unadjusted hospital visit rate of 23.8%, compared with 12.6% for patients without DE. Patients with high ADI, however, have unadjusted hospital visit rates of 17.0%, vs. 12.7% for patients without high AD. This pattern was also similar for colonoscopies received at ASCs. The developer also looked at within facility and across facility rates for these subpopulations, finding that hospitals have worse outcomes for their DE patients compared with their non-DE patients. Looking across hospitals, the developer reports that only one facility had outcomes for DE patients that performed better than the national rate.
Use and Usability
Strengths:
- The measure is currently used in the Hospital Outpatient Reporting program, Ambulatory Surgical Center Quality Reporting program, and the Rural Emergency Hospital Quality Reporting program.
The developer cites evidence of interventions that can improve the outcome (hospital visits, defined as an ED visit, observation stay, or inpatient admission), as noted previously.
The developer notes that stakeholders can submit questions about the measure through an online tool and CMS responds to each question. The developer notes that these questions are considered in addition to literature reviews related to the measure and engagement with clinical experts for updating the measure. The developer also updates procedure codes used in the measure accordingly and adds that there have been no major substantive changes to the measure since its last endorsement in 2020.
The developer reports that the national rate of hospital visits per 1,000 colonoscopies among patients with procedures performed at HOPDs declined from 16.4 in 2018 reporting to 14.8 in 2019 reporting, and 13.2 in 2023 reporting (current data). The developer further posits that facilities can use CMS reports on the measure to tie their quality improvement efforts to the specific complications that are occurring.
The developer reports no unexpected findings.
Limitations:
- None identified.
Rationale:
- The measure is currently used in the Hospital Outpatient Reporting program, Ambulatory Surgical Center Quality Reporting program, and the Rural Emergency Hospital Quality Reporting program.
The developer cites evidence of interventions that can improve readmissions, as noted previously.
The developer notes that stakeholders can submit questions about the measure through an online tool and CMS responds to each question. The developer notes that these questions are considered in addition to literature reviews related to the measure and engagement with clinical experts for updating the measure. The developer also updates procedure codes used in the measure accordingly and adds that there have been no major substantive changes to the measure since its last endorsement in 2020.
The developer reports that the national rate of hospital visits per 1,000 colonoscopies among HOPDs declined from 16.4 in 2018 reporting to 14.8 in 2019 reporting, and 13.2 in 2023 reporting (current data). The developer further posits that facilities can use CMS reports on the measure to tie their quality improvement efforts to the specific complications that are occurring.
The developer reports no unexpected findings.
Summary
Strengths:
Limitations:
Rationale:
Committee Independent Review
Do not support
Importance
Agree with staff
Feasibility Acceptance
agree with staff
Scientific Acceptability
Agree with staff
Agree with staff
Equity
Patients and their caregivers might find this measure more meaningful if it considered the corporatization of healthcare with associated increased out-of-pocket expenses and the shift from solo to group practices; the increase in minimally invasive procedures, imaging advancements, telemedicine, the impact of the COVID-19 pandemic, and included data from Medicare Advantage Plans.
Use and Usability
Patients and their caregivers might find this measure more respectful if more settings of follow-up were considered, such as home care, satellite clinics, portal communication, and telehealth, as well as transportation and appointment access.
Summary
Patients and their caregivers might find this measure more meaningful if it considered the corporatization of healthcare with associated increased out-of-pocket expenses and the shift from solo to group practices; the increase in minimally invasive procedures, imaging advancements, telemedicine, the impact of the COVID-19 pandemic, and included data from Medicare Advantage Plans.
Patients and their caregivers might find this measure more respectful if more settings of follow-up were considered, such as home care, satellite clinics, portal communication, and telehealth, as well as transportation and appointment access.
Support
Importance
agree with staff assessment
Feasibility Acceptance
agree with staff assessment
Scientific Acceptability
agree with staff assessment
agree with staff assessment; question whether measure should include a narrower list of admission diagnoses, and whether the inclusion of a narrower list relating to likely complications would significantly impact results.
Equity
agree with staff assessment
Use and Usability
agree with staff assessment
Summary
Another important measure considering common, relatively low risk procedure. Need to clarify whether limiting admission diagnosis would improve reliability.
Summary
Importance
The developer reported a distribution of scores for both HOPDs and ASCs, with an overall mean score of 13.18 (national average) and 9.92 (national average), respectively. Overall, large number of colonoscopies are performed yearly for cancer screening.
Feasibility Acceptance
Uses 100% Medicare claims and developer did not conduct feasibility.
Scientific Acceptability
Not clear on the impact of procedure volumes by facility. The minimum reliability with at least 30 cases was 0.239 for HOPDs and 0.265 for ASCs. This seems low for acceptable reliability.
Not clear on the impact by volume
Equity
The developer did report differences in dual eligibility in HOPDs with dual eligibility vs those without.
Use and Usability
Currently used within the Hospital Outpatient Reporting program, Ambulatory Surgical Center Quality Reporting program, and the Rural Emergency Hospital Quality Reporting program.
Summary
This is an important measure but there are areas that could be improved with additional information from the developer in the area of scientific acceptability and addressing the volume within the measure.
agree with staff assessment
Importance
agree with staff assessment
Feasibility Acceptance
agree with staff assessment
Scientific Acceptability
agree with staff assessment
agree with staff assessment
Equity
agree with staff assessment
Use and Usability
agree with staff assessment
Summary
no additional comments
Support
Importance
The developer highlighted the measure's connections to patient outcomes through procedural practices and structural factors. Also used a range of evidence.
Feasibility Acceptance
Agree with staff assessment; no burden on the facility since this is a claims-based measure; also no feasibility assessment was conducted.
Scientific Acceptability
agree with staff assessment
agree with staff assessment
Equity
equity was met
Use and Usability
agree with staff assessment
Summary
Support
Support with limitation
Importance
Agree w staff assessment
Feasibility Acceptance
Agree w staff assessment
Scientific Acceptability
Agree w staff assessment
Agree w staff assessment. Since developer identified a high percentage of admissions related to procedure then perhaps they can only include those complications associated with the procedure? Again a numerator issue. If care coordination is the issue then where is primary care in this measure?
Equity
Agree w staff assessment
Use and Usability
Agree w staff assessment
Summary
Would like to see the numerator addressed or at least convince me otherwise that all cause is appropriate
Support
Importance
Agree with the staff assessment.
Feasibility Acceptance
Agree with staff assessment.
Scientific Acceptability
Agree with the staff assessment.
There is some concern surrounding facilities with a reliability below 0.60. Given that the measure is used in public reporting, is there a distinct need to present data on facilities with low reliability? This may be confusing or misleading for consumers. Facilities could still receive facility-specific reports to guide quality improvement and for benchmarking without public report.
Agree with staff assessment.
Equity
Agree with staff assessment.
Use and Usability
Agree with staff assessment.
One improvement to the use and usability of the measures would be to provide stratified information on hospital visits plausibly related to the procedure. The developer provides analysis of the top 25 diagnoses and confirms that they are related to the elective colonoscopy (From the validity section: "For HOPDs, the top four principal diagnoses are “Post procedural hemorrhage….,” “Hemorrhage of anus and rectum,” “Gastrointestinal hemorrhage, unspecified,” and “Urinary tract infection, site not specified.”). This information could be helpful in understanding whether the improvements in hospital visit rates evaluated over the life of the measure are reducing related hospital visits specifically.
Summary
This measure addresses a high-volume elective procedure with room for improvement. The measure has shown improvement over time. Additional work can be done to show the validity of the measure in terms of the assessment of the quality-volume relationship and how this informs validity. One concern raised in public comment but seems addressable through modified reporting is assessing "related" hospital visits vs less plausibly related hospital visits. This modification could improve the use and usability of the measure. Finally, there is some concern regarding the public reporting for facilities with a reliability less than the minimum threshold of 0.60.
Public Comments
Public Comment Shared During May 29 Listening Session
So, these [CBE 3470, 3357, 2539, and 3366] are all ambulatory, which means you were not admitted to the hospital to have the surgery. I had to go into a hospital after a general surgery, but it was orthopedic, right. Mine was within that 7 days. But really any hospital visit after any kind of surgery is important. I totally recognize these and think it's very important, particularly when there's sort of a push to get people in and out of the hospital. So, thank you.
Yeah, I mean, they need to watch. There's a risk standardized hospital visit. They all are about some hospital visit, right? And that does not mean that actually means hospital. Either a stay or observational. I'm assuming. So it's important. And these are all very important. Because sometimes it's just check the box, but it’s important to know why the person ends up back in the hospital after some sort of procedure. Thank you.
Public Comment Shared During May 29 Listening Session
Well, Hello! So my question would just be regarding the age, the 65 years and older for the Medicare patients Fee-for-Service. I don't know if the previous maintenance [review] was still in the same age category, but if the age could be expanded for Medicare Fee-for-Service patients that are younger as well, too. Just widening the age range for these Medicare patients. I know they're looking at older patients, but I don't know if they're looking at older patients being able to bounce back after the surgery, but not because they have to be re-admitted or have an ED visit or an unplanned inpatient admission. So, to just kind of like understand whether it's the age-related readmission, or if it's the procedural readmission, or if they're having complications from the actual surgical event that they experienced. So, I just have a question on why do they [the developer] have the age at 65, and not maybe in the fifties, or just to kind of exclude whether it's age related or procedure related.
Endorsement Should Be Removed From This Measure
CBE ID: 2539
Facility 7-Day Risk-Standardized Hospital Visit Rate After Outpatient Colonoscopy
Endorsement should be removed from this measure. It is not a valid or reliable measure of the quality or efficiency of outpatient colonoscopy procedures due to problems with the numerator and the risk adjustment methodology. Continued public reporting of the results could mislead patients about where they should receive such procedures.
Problems with the Numerator
The stated goal of the measure is to improve the quality of care delivered to patients undergoing outpatient colonoscopy procedures. However, the numerator includes events that have nothing to do with the procedure, while excluding events that represent undesirable outcomes and avoidable costs.
The inclusion of so many visits unrelated to the procedure creates two different problems.
Problems with the Risk Adjustment Methodology
The actual visit rate is modified using a hierarchical logistical regression model that is intended to adjust for differences in the characteristics of the patients receiving colonoscopies that affect outcomes. However, there are serious flaws with the model that is used:
Poor Measure Reliability and an Inappropriately Low Minimum Case Threshold
The measure is inappropriately being used for ASCs that perform as few as 30 procedures. With that few cases, a single post-discharge visit results in a 3% visit rate. Since the average national rate is only 1%, a single visit at a small ASC could make the difference between having a “low” or “high” visit rate.
Because of the low volumes and high variability of post-discharge visits, this measure is not sufficiently reliable to use for public reporting. The developers report that the signal-to-noise reliability was below 0.6 for 10% of ASCs and 30% of hospitals, below 0.7 for 20% of ASCs and 40% of hospitals, and below 0.8 for 30% of ASCs and the majority of hospitals. It does not appear that any effort was made to assess the reliability of the measure over time.
Lack of Business Case for Using the Measure and Undesirable Effects of Doing So
The measure identified only 17 out of over 2,000 ASCs as being either better or worse than the national average. Over 200 were classified as “number of cases too small” and 90% were classified as “no different than the national average.” Moreover, it does not appear that the developers made any effort to examine the 8 facilities identified as “better than the national rate” or the 9 facilities identified as “worse than the national rate” to assess whether this was really a valid classification based on the number of cases, the reasons for post-discharge visits, and the consistency of the classification over time.
Since only a very small number of facilities are being classified as anything other than “no different than expected,” this measure is unlikely to encourage improvements in the quality of care delivery. On the other hand, since the weaknesses in the measure methodology could cause some facilities to be inappropriately labeled as “worse than expected” because of the kinds of patients they treat, use of the measure creates an undesirable incentive for facilities to avoid performing colonoscopies on patients with characteristics that are likely to result in higher numbers of visits, such as patients with multiple comorbidities and patients who do not have good access to primary care and chronic disease management services. It could also cause patients to inappropriately avoid obtaining a colonoscopy, or to try and obtain the procedure in a facility that is not adequately equipped to address their needs.
2539 Facility 7-Day Risk-Standardized Hospital Visit Rate after
The American Medical Association (AMA) questions whether this measure produces scores that can be considered sufficiently reliable. Specifically, the minimum reliability with at least 30 cases was 0.239 for hospital outpatient departments and 0.265 for ambulatory surgical centers, which falls well below what the AMA considers to be acceptable. We believe that measures must achieve a minimum reliability of at least 0.7 or higher in order to be achieve endorsement.
The American College of…
The American College of Gastroenterology (ACG) questions whether this measure produces scores that can be considered sufficiently reliable. Specifically, the minimum reliability with at least 30 cases was 0.239 for hospital outpatient departments and 0.265 for ambulatory surgical centers. We believe that measures must achieve a minimum reliability of at least 0.7 or higher in order to be achieve endorsement.
In addition, we ask that the committee discuss whether the measure inappropriately includes hospital visits that are unrelated to a complication after an outpatient colonoscopy. For example, if a patient is involved in a car accident during this 7-day time frame, it would count against the facility. It is not clear whether these occurrences could negatively impact performance in a meaningful way.