When the measure returns for maintenance, the committee would like to see:
- Explore methods to enable the evaluation of improvement over time; and
- Consider additional approaches for the reliability assessment that inform the reliability-validity (e.g. shrinkage) and reliability-usability (e.g. stability) tradeoffs
This measure was developed to improve the quality of care delivered to patients undergoing general surgery procedures in an ambulatory surgical center (ASC). To assess quality, the measure calculates the risk-standardized rate of return to a hospital for an acute, unplanned hospital visit within seven days of qualified general surgery procedures performed at an ambulatory surgical center (ASC) among Medicare Fee-For-Service (FFS) patients aged 65 years and older. An unplanned hospital visit is defined as an emergency department (ED) visit, observation stay, or unplanned inpatient admission.
Measure Specs
- General Information(active tab)
- Numerator
- Denominator
- Exclusions
- Measure Calculation
- Supplemental Attachment
- Point of Contact
General Information
The shift of surgical procedures from the inpatient setting to the outpatient setting has caused an exponential growth in the number of ASCs and procedures performed in this setting (Jain et al., 2020). The number of ASCs grew by 35% between 2006 to 2021. In 2022, about 6,100 ASCs treated 3.3 million fee-for-service (FFS) Medicare beneficiaries. FFS Medicare program spending and beneficiary cost sharing on ASC services was about $6.1 billion (MedPAC, 2024). The patient population served at ASCs has increased not only in volume but also in age and complexity, which can be partially attributed to improvements in anesthetic care and innovations in minimally invasive surgical techniques. In comparison to the traditional HOPD setting, ASCs can offer patients more convenient location, shorter wait times, lower cost sharing, easier scheduling, and the ability to return to work rapidly (Munnich et. al., 2014; MedPAC, 2024). Therefore, in the context of growth in volume and diversity of procedures performed at ASCs, evaluating the quality of care provided at ASCs is increasingly important.
Hospital visits following an ambulatory surgery vary from 0.5% to 9.0%, based on the type of surgery, outcome measured, and timeframe for measurement after surgery (Bongiovanni et al., 2021). Hospital visits can occur due to a range of potentially preventable adverse events including uncontrolled pain, urinary retention, surgical site infection, bleeding, septicemia, and venous thromboembolism. Patients also frequently report minor adverse events -- for example, uncontrolled pain, nausea, and vomiting -- that may result in unplanned acute care visits following surgery (Owens et al., 2014; Bongiovanni et al., 2021).
Several factors make unanticipated hospital visits a priority quality indicator. Because ASC providers are not aware of all post-surgical hospital visits that occur among their patients, reporting this outcome will help to illuminate problems that may not be currently visible (Zivanovic et al., 2020). In addition, the outcome of hospital visits is a broad, patient-centered outcome that reflects the full range of reasons leading to hospital use among patients undergoing same-day surgery. Public reporting of this outcome measure will provide ASCs with critical information and incentives to implement strategies to reduce unplanned hospital visits.
References
Bongiovanni T, Parzynski C, Ranasinghe I, Steinman MA, Ross JS. Unplanned hospital visits after ambulatory surgical care. PloS one. 2021 Jul 20;16(7):e0254039.
Jain S, Rosenbaum PR, Reiter JG, Ramadan OI, Hill AS, Silber JH, Fleisher LA. Assessing the Ambulatory Surgery Center Volume-Outcome Association. JAMA surgery. 2024 Apr 10.
Medicare Payment Advisory Commission (MedPAC). Report to Congress: Medicare Payment Policy. March 2024; https://www.medpac.gov/wp-content/uploads/2024/03/Mar24_MedPAC_Report_To_Congress_SEC.pdf
Munnich EL, Parente ST. Procedures take less time at ambulatory surgery centers, keeping costs down and ability to meet demand up. Health Affairs. 2014 May 1;33(5):764-9.
Owens PL, Barrett ML, Raetzman S, Maggard-Gibbons M, Steiner CA. Surgical site infections following ambulatory surgery procedures. Jama. 2014 Feb 19;311(7):709-16.
Zivanovic O, Chen LY, Vickers A, Straubhar A, Baser R, Veith M, Aiken N, Carter J, Curran K, Simon B, Mueller J, Jewell E, Chi DS, Sonoda Y, Abu-Rustum NR, Leitao MM Jr. Electronic patient-reported symptom monitoring in patients recovering from ambulatory minimally invasive gynecologic surgery: A prospective pilot study. Gynecol Oncol. 2020 Oct;159(1):187-194. doi: 10.1016/j.ygyno.2020.07.004. Epub 2020 Jul 24. PMID: 32718730; PMCID: PMC7380930. _________________________________________________________________________
Medicare Fee For Service administrative and claims data as well as CMS enrollment and demographic data.
Numerator
The measure defines the outcome as any (one or more), all-cause, acute, unplanned hospital visit within seven days of an outpatient general surgery performed at an ASC; a hospital visit includes any emergency department (ED) visit, observation stay, or unplanned inpatient admission.
Outcome Definition
The ASC General Surgery measure defines the outcome as any (one or more), all-cause, acute, unplanned hospital visit within seven days of an outpatient general surgery performed at an ASC; a hospital visit includes any emergency department (ED) visit, observation stay, or unplanned inpatient admission.
Time Period for Data
Numerator time window: Seven days after ASC procedures for unplanned hospital visits.
Planned Hospital Visits
The measure outcome includes hospital visits within the first seven days following the procedure, unless that inpatient admission is deemed a “planned” admission as defined by the measures Planned Admissions Algorithm (PAA). The algorithm is a set of criteria for classifying readmissions as planned or unplanned using Medicare claims. The Centers for Medicare & Medicaid Services (CMS) seeks to count only unplanned admissions in the measure outcome, because variation in “planned” admissions does not reflect quality differences. The measure does not consider observation stays or ED visits as planned.
In brief, the algorithm identifies admissions that are typically planned and may occur after the patient’s index event. The algorithm always considers a few specific, limited types of care planned (e.g., major organ transplant, rehabilitation, or maintenance chemotherapy). Otherwise, the algorithm defines a planned admission as a non-acute admission for a scheduled procedure (e.g., total hip replacement or cholecystectomy), and the algorithm never considers admissions for acute illness or for complications of planned care. For example, the algorithm considers hip replacement unplanned if hip fracture (an acute condition) is the discharge diagnosis but planned if osteoarthritis (a non-acute condition) is the discharge diagnosis. The algorithm considers admissions that include potentially planned procedures with acute diagnoses or that might represent complications of a procedure unplanned and thus counts these admissions in the measure outcome.
For more information about the PAA, please see the Facility 7-Day Risk-Standardized Visits After General Surgery Procedures Performed at Ambulatory Surgical Centers 2023 Measure Updates and Specifications Report.
Also see tabs PAA1 Always Planned Px, PAA2 Always Planned Dx, PAA3 Not Planned Px, and PAA4 Acute Dx in the attached Data Dictionary for the most current sets of codes in the algorithm for always planned procedures (PAA1), always planned diagnoses (PAA2), potentially planned procedures (PAA3), and acute diagnoses (PAA4)
Denominator
The target population for this measure is Medicare FFS patients aged 65 years and older, undergoing selected outpatient general surgery procedures in ASCs that are within the scope of general surgery training. Specifically, the cohort of procedures includes the following types of procedures: abdominal, alimentary tract, breast, skin/soft tissue, wound, and varicose vein.
Target Population
The target population for this measure is Medicare FFS patients aged 65 years or older undergoing selected outpatient general surgery procedures in ASCs that are within the scope of general surgery training. Specifically, the cohort of procedures includes the following types of procedures: abdominal, alimentary tract, breast, skin/soft tissue, wound, and varicose vein. The measure includes patients who are:
- Medicare FFS patients aged 65 years or older.
- Patients with continuous enrollment in Medicare FFS Parts A and B in the 12 months prior to the surgery.
Included Procedures
The measure includes procedures that (1) are routinely performed at ASCs, (2) involve increased risk of post-surgery hospital visits, and (3) are within the scope of general surgery training. The measure includes a subset of procedures performed at ASCs identified using Medicare’s list of covered ASC procedures. The measure includes “major” and “minor” procedures, as indicated by the Medicare Physician Fee Schedule global surgery indicator (GSI) values of 090 and 010, respectively, and certain cystoscopy procedures, as described below. The GSI code reflects the number of post-operative days that are included in a given procedure’s global surgical payment and identifies surgical procedures of greater complexity and follow-up care.
The measure does not include procedures that:
- Are on the ASC list of covered procedures that do not involve or require major or prolonged invasion of body cavities, extensive blood loss, major blood vessels, or care that is emergent or life-threatening.
- Are part of the Agency for Healthcare Research and Quality (AHRQ) Clinical Classifications Software’s (CCS’s) gastrointestinal endoscopy, endocrine, or vascular procedures, other than varicose vein procedures, as reasons for hospital visits are typically related to patients’ underlying comorbidities.
We use Medicare FFS claims to identify surgeries performed in the outpatient setting and subsequent hospital visits, as well as CMS enrollment and demographic data. Patient history is also assessed using claims data collected in the 12 months prior to the eligible same-day surgery. We identify outpatient surgeries using Medicare’s list of covered ASC procedures. CMS reviews and updates this list of surgeries annually. The process includes a transparent public comment submission and review process for addition and/or removal of procedures codes. The lists are posted at: https://www.cms.gov/Medicare/Medicare-Fee-for-ServicePayment/ASCPayment/11_Addenda_Updates.html (refer to Addendum AA of the respective link). Procedures listed on Medicare’s list of covered ASC procedures are defined using Healthcare Common Procedure Coding System (HCPCS) and Current Procedural Terminology® (CPT) codes.
The CPT and HCPCS procedure codes that define the cohort are in the attached Data Dictionary, tab ASC_Surg_Cohort.
Ambulatory procedures include a heterogeneous mix of non-surgical procedures, minor surgeries, and more substantive surgeries. The measure is not intended to include very low-risk (minor) surgeries or non-surgical procedures, which typically have a high volume and a very low outcome rate. Therefore, to focus the measure only on the subset of surgeries on Medicare’s list of covered ASC procedures that impose a meaningful risk of post-procedure hospital visits, the measure includes only “major” and “minor” procedures, as indicated by the Medicare Physician Fee Schedule GSI values of 090 and 010, respectively. The GSI code reflects the number of post-operative days that are included in a given procedure’s global surgical payment and identifies surgical procedures of greater complexity and follow-up care. This list of GSI values is publicly available at: http://www.cms.hhs.gov/Reports/downloads/pope_2000_2.pdf.
During initial measure development, to identify the subset of general surgery ASC procedures, we reviewed with consultants and Technical Expert Panel (TEP) members the CCS categories of procedures developed by AHRQ. We identified and included CCS categories within the scope of general surgery, and only included individual procedures within the CCS categories at the procedure (CPT code) level if they were within the scope of general surgery practice. We did not include in the measure gastrointestinal endoscopy, endocrine, or vascular procedures, other than varicose vein procedures, because reasons for hospital visits are typically related to patients’ underlying comorbidities.
See the attached Data Dictionary, tab ASC Surg Cohort, for a complete list of all CPT procedure codes included in the measure cohort.
Definition of ED Visits and Observation Stay
ED visits and observation stays are defined using revenue center codes identified in Medicare Part B outpatient hospital claims. The 2023 General Surgery Measure Data Dictionary tab ASC Surg Outcome ED Obs provides the specific codes used to identify ED visits and observation stays.
_________________
Exclusions
The measure excludes surgeries for patients without seven or more days of continuous enrollment in Medicare FFS Parts A and B after the surgery to ensure all patients have full data available for outcome assessment (see Figure 1 in attachment).
Lack of 7 or more days of continuous enrollment in Medicare FFS after the ASC surgery is determined by patient enrollment status in FFS Parts A and B using the Medicare enrollment file (unless lack of enrollment was due to death). The procedure must be 7 or more days from the end of the month or the enrollment indicators must be appropriately marked for the month that falls within 7 days of the procedure date (unless disenrollment is due to death), otherwise the procedure is excluded.
This exclusion is narrowly targeted and removes a small number of general surgery ASC procedures.
A diagram showing the inclusion and exclusion criteria for this measure is shown in Figure 1 in the attachment.
Measure Calculation
Ratio
The measure score uses a two-level hierarchical logistic regression model to estimate ASC-level risk-standardized hospital visit ratios (RSHVRs). This approach accounts for the clustering of patients within ASCs and variation in sample size across ASCs. The RSHVR is calculated as the ratio of the predicted-to-expected (P/E) number of post-surgical unplanned hospital visits among the ASC’s patients. For each ASC, the numerator of the ratio is the number of hospital visits predicted for the ASC’s patients, accounting for its observed rate, the number and complexity of general surgery procedures performed at the ASC, and the case mix. The denominator is the number of hospital visits expected nationally for the ASC’s case/procedure mix. To calculate an ASC’s P/E ratio, the measure uses a two-level hierarchical logistic regression model. The log-odds of the outcome for an index procedure is modeled as a function of the patient demographic, comorbidity, procedure characteristics, and a random ASC-specific intercept. A ratio greater than one indicates that the ASC’s patients have more visits than expected, compared to an average ASC with similar patient and procedural complexity. A ratio less than one indicates that the ASC’s patients have fewer post-surgical visits than expected, compared to an average ASC with similar patient and procedural complexity. This approach is analogous to an observed-to-expected ratio, but accounts for within-facility correlation of the observed outcome and sample size differences and accommodates the assumption that underlying differences in quality across ASCs lead to systematic differences in outcomes, and is tailored to and appropriate for a publicly reported outcome measure as articulated in published scientific guidelines (Krumholz et al., 2006; Normand et al., 2007).
Below we provide the individual steps to calculate the measure score:
- Identify surgeries meeting the inclusion criteria described in the denominator section above and in tab 1, ASC Surg Cohort, of the Data Dictionary.
- Exclude procedures meeting any of the criteria described in the exclusion section above.
- Identify a binary flag for an unplanned hospital visit within seven days of index procedures as described above.
- Use patients’ historical and index procedure claims data to create risk-adjustment variables.
- Fit a hierarchical generalized linear model (HGLM) and calculate the ratio of the number of “predicted” hospital visits to the number of “expected” hospital visits for each facility, given its case/procedure mix using the results. This is the RSHVR. The HGLM is adjusted for age, clinical risk factors, and procedure relative value units (RVUs) and body system that vary across patient populations, are unrelated to quality, and influence the outcome. Details about the risk-adjustment model can be found in the original measure development methodology report: Facility-Level 7-Day Hospital Visits after General Surgery Procedures Performed at Ambulatory Surgical Centers under archived resources at: https://qualitynet.cms.gov/asc/measures/surgery/resources#tab2.
- Use statistical bootstrapping to construct a 95% confidence interval estimate for each facility’s RSHVR. For more information about the measure methodology, please see Appendix A in the most recent 2023 Measure Updates and Specifications Report posted at: https://qualitynet.cms.gov/files/651b5f9970a30f001c388001?filename=2023_ASC_MeasureUpdatesRpt.pdf
The data used for measure calculation contains 100% of qualifying procedures at each facility and provides adequate sample size for a reliable measure score.
References
Normand S-LT, Shahian DM. Statistical and clinical aspects of hospital outcomes profiling. Statistical Science. 2007;22(2):206-226.
Krumholz HM, Brindis RG, Brush JE, et al. Standards for Statistical Models Used for Public Reporting of Health Outcomes An American Heart Association Scientific Statement From the Quality of Care and Outcomes Research Interdisciplinary Writing Group: Cosponsored by the Council on Epidemiology and Prevention and the Stroke Council Endorsed by the American College of Cardiology Foundation. Circulation. 2006;113(3):456-462.
Not applicable. This measure is not stratified.
Not applicable. This measure is not based on a sample.
Supplemental Attachment
Point of Contact
Not applicable
Marsha Hertzberg
Centers for Medicare & Medicaid Services (CMS)
7500 Security Boulevard
Windsor Mill, MD 21244
United States
Oscar Gonzalez
Acumen, LLC
500 Airport Blvd., Suite 100
Burlingame, CA 94010
United States
Importance
Evidence
The measurement of unplanned hospital visits following outpatient surgery remains a critical indicator of healthcare quality, reflecting essential aspects of patient care processes and outcomes. Understanding the factors contributing to these visits is vital for improving patient safety and optimizing healthcare delivery. Unplanned hospital visits after outpatient surgery are associated with adverse events and complications that can negatively impact patient health outcomes and satisfaction (Bongiovanni et al., 2021). Research continues to demonstrate variability in rates of unplanned hospital visits after outpatient surgery, influenced by factors such as surgical complexity, patient demographics, and postoperative care protocols (Desai et al., 2022).
Common reasons for unplanned hospital visits include surgical site infections, postoperative pain management issues, and medication-related complications, highlighting the importance of comprehensive perioperative care. Evidence-based interventions aimed at optimizing preoperative assessment, enhancing postoperative monitoring, and improving patient education have shown promise in reducing rates of unplanned hospital visits following outpatient surgery. Technical factors, such as surgical technique, anesthesia management, and operating room efficiency, play a significant role in determining the likelihood of postoperative complications and subsequent hospital visits. While much of the research has focused on hospital-based outpatient surgery, the principles underlying the prevention of unplanned hospital visits are equally relevant in ASC settings, emphasizing the importance of comprehensive perioperative care and quality improvement initiatives (Erhun et al., 2016).
Ongoing efforts to enhance patient safety and optimize outcomes in outpatient surgery include the development of standardized care pathways, implementation of enhanced recovery after surgery protocols, and integration of telemedicine and remote monitoring technologies (Cukierman et al., 2023; Jain et al., 2023). See section 6.2.1 for more details on improvements and interventions to enhance outcomes. Measuring unplanned hospital visits following outpatient surgery remains essential for identifying areas for quality improvement and optimizing patient outcomes. By addressing modifiable risk factors, implementing evidence-based interventions, and fostering a culture of continuous learning and improvement, healthcare providers can mitigate the occurrence of adverse events and enhance the overall quality of outpatient surgical care (Davis et al., 2019).
References
Bongiovanni T, Parzynski C, Ranasinghe I, Steinman MA, Ross JS. Unplanned hospital visits after ambulatory surgical care. PLoS One. 2021 Jul 20;16(7):e0254039. doi: 10.1371/journal.pone.0254039. PMID: 34283840; PMCID: PMC8291649.
Cukierman, D. S., Cata, J. P., & Gan, T. J. (2023). Enhanced recovery protocols for ambulatory surgery. Best practice & research. Clinical Anesthesiology. 37(3), 285–303. https://doi.org/10.1016/j.bpa.2023.04.007
Davis KK, Mahishi V, Singal R, Urman RD, Miller MA, Cooke M, Berry WR. Quality Improvement in Ambulatory Surgery Centers: A Major National Effort Aimed at Reducing Infections and Other Surgical Complications. J Clin Med Res. 2019 Jan;11(1):7-14. doi: 10.14740/jocmr3603w. Epub 2018 Dec 3. PMID: 30627272; PMCID: PMC6306128.
Desai MM, Zogg CK, Ranasinghe I, Parzynski CS, Lin Z, Gorbaty M, Merrill A, Krumholz HM, Drye EE. Variation in Risk-standardized Rates and Causes of Unplanned Hospital Visits Within 7 Days of Hospital Outpatient Surgery. Ann Surg. 2022 Dec 1;276(6):e714-e720. doi: 10.1097/SLA.0000000000004627. Epub 2020 Nov 17. PMID: 33214469.
Erhun F, Malcolm E, Kalani M, Brayton K, Nguyen C, Asch SM, Platchek T, Milstein A. Opportunities to improve the value of outpatient surgical care. Am J Manag Care. 2016 Sep 1;22(9):e329-35. PMID: 27662397.
Jain SN, Lamture Y, Krishna M. Enhanced Recovery After Surgery: Exploring the Advances and Strategies. Cureus. 2023 Oct 17;15(10):e47237. doi: 10.7759/cureus.47237. PMID: 38022245; PMCID: PMC10654132.
Measure Impact
Improving the quality of care provided at ASCs is a key priority in the context of growth in the number of ASCs and procedures performed in this setting. Ambulatory care now accounts for the majority of surgical care in the United States. From both the patient and provider perspective, several factors have fueled this growth. ASCs provide greater convenience for patients and families (due to advances in surgical technology), as well as lower financial costs for both patients and healthcare providers. As such, the wide range of surgeries and high-risk patient groups continue to increase in ASC settings warranting a need to improve quality of care at ASCs (Munich et al., 2014; Goldfarb et al., 2017).
A hospital visit following same-day surgery is an unexpected and potentially preventable outcome for patients scheduled for elective same-day surgeries that have a low anticipated risk. ASC providers may be unaware of their patients’ hospital visits after surgery because patients often present to the ED or to different hospitals, leading to understated adverse event rates and suggesting the need for better measurement to drive quality improvement. Therefore, both patients and providers benefit from outcome measures of hospital visits – a broad, patient-centered outcome that reflects the full range of reasons leading to hospitalization among patients undergoing same-day surgery.
The ASC General Surgery measure is part of the Ambulatory Surgical Center Quality Reporting (ASCQR) Program, a pay-for-reporting program. The measure went through a “dry run” in 2020 to help educate measure entities on data collection and interpretation, familiarize them with facility-specific reports (FSRs), and for CMS to collect feedback on the measure specifications. ASCs first saw their facility-specific measure scores in October of 2023. This measure recently went through its first public reporting in January of 2024. Currently, there are no other publicly available quality reports related to this procedure which underscores the measurement gap that would exist without this measure. Thus, this measure addresses an important quality measurement area and enhances the information available to patients choosing among ASCs that provide same-day outpatient surgery. Furthermore, providing outcome rates to ASCs makes visible to clinicians and hospitals meaningful quality differences and incentivizes improvement.
References
Goldfarb CA, Bansal A, Brophy RH. Ambulatory surgical centers: a review of complications and adverse events. Journal of the American Academy of Orthopaedic Surgeons. 2017 Jan 1;25(1):12-22.
Munnich EL, Parente ST. Procedures take less time at ambulatory surgery centers, keeping costs down and ability to meet demand up. Health Affairs. 2014 May 1;33(5):764-9.
Performance Gap
We found variation in measure scores among ASCs. Using updated data from the Endorsement and Maintenance 2024 dataset (January 1, 2021 – December 31, 2022) we found that among the 3,435 ASCs with at least one qualifying index general surgery procedure, RSHVRs ranged from 0.59 (or 40% better than expected) to 1.84 (or 86% worse than expected). Table 2 shows the measure score distribution by decile.
We additionally provide evidence of variation by calculating the median odds ratio and calculating measure score outliers in the form of 95% confidence intervals, which are described below.
Measure Score Distribution
Table 1 and Figure 3 in the attachment shows the distribution of measure scores, or RSHVRs, using the most recent testing data (2024 EM Dataset; January 1, 2022 – December 31, 2022). A ratio of less than one indicates performance that is better than expected (better quality); a ratio of more than one indicates performance that is worse than expected (lower quality). ASC General Surgery RSHVRs range from 0.59 (or about 40% better than expected) to 1.84 (84% worse than expected). The interquartile range (IQR; or the range between the 25th percentile and the 75th percentile) is 0.97 to 1.01.
Table 2 in the attachment shows the distribution of measure scores in deciles. Of note, we found that facilities in the top three deciles (worse performance) account for almost 90,000 procedures (36%) that took place in facilities with worse than expected performance.
Median Odds Ratio
We provide further evidence of variation by calculating and interpreting the median odds ratio (MOR) (Merlo et al., 2006). The MOR represents the median increase in odds of a hospital visit if a procedure on a single patient was performed at a higher risk ASC compared to a lower risk ASC. It is calculated by taking all possible combinations of ASCs, always comparing the higher risk ASC to the lower risk ASC. The MOR is interpreted as a traditional odds ratio would be.
The median odds ratio for this updated analysis is 1.35. The median odds ratio suggests a meaningful increase in the risk of a hospital visit if a procedure was performed at a higher risk ASC compared to a lower risk ASC. A value of 1.35 indicates that a patient has a 35% increase in the odds of a hospital visit if the same procedure was performed at higher risk ASC compared to a lower risk ASC indicating the impact of quality on the outcome rate is substantial.
References
Merlo J, Chaix B, Ohlsson H, Beckman A, Johnell K, Hjerpe P, Råstam L, Larsen K. (2006) A brief conceptual tutorial of multilevel analysis in social epidemiology: Using measures of clustering in multilevel logistic regression to investigate contextual phenomena. J Epidemiol Community Health. 60(4):290-7.
| Overall | Minimum | Decile_1 | Decile_2 | Decile_3 | Decile_4 | Decile_5 | Decile_6 | Decile_7 | Decile_8 | Decile_9 | Decile_10 | Maximum | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Mean Performance Score | 1.00 (0.09) | 0.59 | 0.86 | 0.95 | 0.97 | 0.98 | 0.99 | 1.00 | 1.00 | 1.02 | 1.07 | 1.20 | 1.84 |
| N of Entities | 3,485 | 1 | 348 | 349 | 348 | 349 | 348 | 349 | 349 | 348 | 349 | 348 | 1 |
| N of Persons / Encounters / Episodes | 255,719 | 1,892 | 106,851 | 24,044 | 12,868 | 11,268 | 4,615 | 4,569 | 1,088 | 27,123 | 22,425 | 40,868 | 74 |
Equity
Equity
The version of the ASC General Surgery measure that was recently publicly reported is not adjusted for social risk factors. We performed two analyses to explore the impact of adding either of two social risk factors (DE and ADI) to the model, on measure scores. We found that adding either social risk factor to the model resulted in little to no impact on measure scores, suggesting that the variables in the risk model account for most of the differences we see in unadjusted patient-level outcome rates. Therefore, in this pay-for-reporting program, providers will not be unfairly profiled when assessed by the ASC General Surgery measure. We describe the analyses and results below.
To examine the impact of social risk factors on measure scores, we first examined correlations (Pearsons) between measure scores with and without either social risk factor and found that correlations were near 1 (0.999, and 1.000, for DE and high ADI variables, respectively; Figure 5 and 6). Second, we examined the association (Spearman) between the facility proportion of patients with each social risk factor and measure scores for all facilities (Figures 7 and 8) and for facilities with at least 25 procedures, focusing on the quartile of facilities with the highest proportion of patients with social risk factors (not shown). We found that there is a very weak negative correlation (r=-0.047, p=.01) between the proportion of patients with DE and the measure score for the fourth quartile of facility-proportion of patients with DE for facilities for all facilities. There is also a weak but negative correlation for the high ADI variable (r=-0.042, p=.01). For facilities with at least 25 procedures, we found that there was no association (r=0.037, p=.16) between the proportion of patients with DE and the measure score for the for the fourth quartile of facility-level of patients with DE. There is a weak but significant correlation (r=0.064; p=.014) for the high ADI variable for facilities with at least 25 procedures. We concluded therefore, that there is little to no impact of adding social risk factors on measure scores for this ASC General Surgery measure.
Feasibility
Feasibility
This is a claims-based measure and as mentioned above, the measure score is calculated automatically from claims data which are routinely generated during the delivery of care. No data are collected by facilities; therefore, this measure imposes no burden on measured entities, and no implementation effort. CMS monitors feedback from the public and measured entities through CMS’ Question & Answer (Q&A) portal on QualityNet; there have been no concerns about burden related to this measure. There are also no concerns about patient confidentiality because the measure is based on claims data submitted by facilities to CMS, and CMS then uses that data for both payment and calculation of the measure score.
We did not perform an analysis of missing data for the measure because it is based on a 100% sample of paid, final action claims submitted by facilities for payment. To ensure complete claims, we allow at least three months between accessing the data and the end of the performance period.
Because this is a claims-based measure the is no burden on the facility; rates are automatically calculated by CMS based on claims data submitted by facilities for payment.
Proprietary Information
Scientific Acceptability
Testing Data
To identify the cohort and outcome for this measure, we use Medicare Fee-For-Service (FFS) claims and administrative data. See Section 4.1.2 Differences in Data for details.
The measure requires a data source that allows us to link patient data across care settings to identify appropriate surgical procedures for inclusion, comorbidities for risk adjustment, and the outcome of hospital visits. Therefore, we used claims data, as they support these linkages and were available for the population of interest.
The datasets, dates, number of measured entities, number of general surgery procedures, and demographic profile for the patients used in each type of testing are indicated below:
1. Medicare FFS Calendar Year (CY) 2015 Dataset
- Dates: January 1, 2015 – December 31, 2015
- Number of facilities: 3,251 ASCs
- Number of general surgery procedures: 149,468
- Demographic characteristics: average age of 76.3 years; 45.73% female
- Dataset used for: Original measure development; face validity testing; creating the development and validation samples for calibration testing (overfitting)
2. Development Sample and Validation Sample
The 2015 Development and Validation Samples were derived by selecting two random samples from the Medicare FFS CY 2015 Dataset. The Development Sample included 50% of the general surgery ASC procedures in the Medicare FFS CY 2015 Dataset, and the Validation Sample included 50% of the general surgery ASC procedures in the Medicare FFS CY 2015 Dataset.
Development Sample
- Dates: January 1, 2015 – December 31, 2015
- Number of facilities: 2,966 ASCs
- Number of general surgery procedures: 74,734
- Demographic characteristics: average age of 76.3 years; 45.83% female
- Dataset used for: Original measure development, including calibration testing (overfitting)
Validation Sample
- Dates: January 1, 2015 – December 31, 2015
- Number of facilities: 2,961 ASCs
- Number of general surgery procedures: 74,734
- Demographic characteristics: average age of 76.3 years; 45.62% female
- Dataset used for: Original measure development, including calibration (overfitting) testing
3. 2024 Endorsement Maintenance Dataset
- Dates: January 1, 2021 – December 31, 2022
- Number of facilities with at least one procedure: 3,485 ASCs
- Number of facilities with at least 25 procedures: 1,455 ASCs
- Number of procedures: 255,719
- Mean age (SD): 75.69 (6.94)
- Female: 46.52%
- Dual eligible: 5,510 (2.61%)
- High ADI: 15,874 (7.56%)
- Dataset used for: all updated testing provided in this submission (cohort size, risk variable frequencies and odds ratios, social risk factor testing, model performance, measure score distribution, validation analyses)
The number of measured entities differs by testing type and dataset. Please see details in the above, Section 4.1.2 Differences in Data.
The number of measured entities and patient characteristics differ by testing type and dataset. Please see details in the above, Section 4.1.2 Differences in Data.
Reliability
Measure Score Reliability
We provide facility-level measure score reliability using the signal-to-noise method, using the formula presented by Adams and colleagues (Yu et al., 2013; Adams et al., 2010). Specifically, for each facility we calculate the reliability as: Reliability=(σ_(facility-to-facility)^2)/(σ_(facility-to-facility)^2+ (σ_(facility error variance)^2)/n).
Where facility-to-facility variance is estimated from the hierarchical logistic regression model, n is equal to each facility’s observed case size, and the facility error variance is estimated using the variance of the logistic distribution (pi^2/3). The facility-level reliability testing is limited to facilities with at least 30 admissions for public reporting.
Signal-to-noise reliability scores can range from 0 to 1. A reliability of zero implies that all the variability in a measure is attributable to measurement error. A reliability of one implies that all the variability is attributable to real difference in performance.
We calculated the measure score reliability for all facilities, and for facilities with a volume cutoff of 25 procedures, the current public reporting cutoff, using the 2024 endorsement maintenance dataset. Our rationale for this is described below.
In general, CMS sets the volume cutoff for publicly reporting facility measures scores based on two considerations. CMS considers the empiric results of reliability testing conducted on the dataset used for public reporting. CMS also considers the volume cutoff for score reporting used for related measures. CMS has empirically determined that measure scores (RSHVRs) for facilities with 25 or more procedures are reliable. Regardless of the score reporting volume cutoff, all facilities and their cases are used in calculating the measure scores. In the dry run and in public reporting CMS typically reports scores for facilities with fewer procedures than the volume cutoff as having “too few cases” to support a reliable estimate. In summary, the measure specifications do not prejudge the ideal volume cutoff. The minimum sample size for public reporting is a policy choice that balances considerations such as the facility-level reliability testing results on the reporting data and consistency across measures for consumers.
References
Adams J, Mehrota, A, Thoman J, McGlynn, E. (2010). Physician cost profiling – reliability and risk of misclassification. New England Journal of Medicine. 362(11): 1014-1021.
Yu, H, Mehrota, A, Adams J. (2013). Reliability of utilization measures for primary care physician profiling. Healthcare. 1, 22-29.
Measure score reliability (signal-to-noise reliability) is shown in Table 3 of the attachment for all facilities and facilities with at least 25 cases, and in Table 4 of the attachment (Table 2 below) for facilities with at least 25 cases (the public reporting threshold) described previously. For facilities with at least 25 procedures, signal-to-noise reliability ranged from 0.432 – 0.990 with a median of 0.690.
Using two years of performance data, the median facility-level reliability score for ASCs is 0.690 (IQR, 0.555-0.824) for ASCs with at least 25 cases (the public reporting threshold) representing moderate reliability.
At the current public reporting threshold, most facilities (about 75% of facilities with at least 25 cases) fall above the 0.6 minimum threshold stated in Battelle’s current Endorsement & Maintenance Guidebook. If CMS were to increase the case volume minimum so that all facilities exceeded this threshold, it would remove publicly available information from about 350 facilities that are currently publicly reported.
We believe that median reliability of 0.6 (signal to noise) is sufficiently high for a facility-level publicly reported measure in a pay for reporting program. Increasing the minimum case volume has the tradeoff of removing about half of facilities from reporting to the public.
| | Overall | Minimum | Decile_1 | Decile_2 | Decile_3 | Decile_4 | Decile_5 | Decile_6 | Decile_7 | Decile_8 | Decile_9 | Decile_10 | Maximum |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Reliability | 0.690 | 0.432 | 0.452 | 0.502 | 0.553 | 0.600 | 0.658 | 0.717 | 0.775 | 0.826 | 0.879 | 0.947 | 0.990 |
| Mean Performance Score | 1.007 (0.135) | 1.008 | 1.001 | 1.023 | 1.006 | 1.005 | 1.033 | 1.012 | 1.034 | 1.011 | 0.933 | ||
| N of Entities | 1,455 | 163 | 137 | 131 | 147 | 143 | 153 | 146 | 145 | 145 | 145 | ||
| N of Persons / Encounters / Episodes | 240,050 | 4,426 | 4,554 | 5,326 | 7,258 | 9,068 | 12,783 | 16,648 | 22,800 | 35,501 | 121,686 |
Validity
For this endorsement maintenance submission, we provide evidence of measure validity using three approaches: (1) face validity during measure development, (2) validity through association with volume, and (3) validation of the outcome. We describe the methods below.
Face Validity
During measure development, we asked our TEP, made up of 15 members including patient representatives, expert clinicians, methodologist, researchers, and providers, to formally assess the measure’s face validity. We provided the TEP with background information on the previous consensus based entity’s (National Quality Forum) measure evaluation criteria and presented the measure specifications and testing and performance results for their evaluation.
List of TEP Members:
- Robin Blomberg, BA, MA – National Forum of End-Stage Renal Disease, Network 16 (Representative for Kidney Patient Advisory Council); Seattle, WA
- Kirk Campbell, MD – New York University Hospital for Joint Diseases (Clinical Assistant Professor of Orthopedic Surgery); New York, NY
- Gary Culbertson, MD, FACS – Iris Surgery Center (Surgeon; Medical Director); Sumter, SC
- Martha Deed, PhD – Consumers Union Safe Patient Project (Patient Safety Advocate); Austin, TX
- James Dupree, MD, MPH – University of Michigan (Urologist; Health Services Researcher); Ann Arbor, MI
- Nester Esnaola, MD, MPH, MBA – Fox Chase Cancer Center (Professor of Surgery; Associate Director for Cancer Health Disparities and Community Engagement); Philadelphia, PA
- John Gore, MD, MS – University of Washington (Associate Professor of Urology); Seattle, WA
- Lisa Ishii, MD, MHS – Johns Hopkins School of Medicine (Associate Professor); American Academy of Otolaryngology-Head and Neck Surgery (Coordinator for Research and Quality); Baltimore, MD; Alexandria, VA
- Atul Kamath, MD – Perelman School of Medicine, University of Pennsylvania (Assistant Professor and Clinical Educator Director of Orthopedic Surgery); Hospital of the University of Pennsylvania (Attending Surgeon); Philadelphia, PA
- Tricia Meyer, PharmD, MS, FASHP – Scott & White Medical Center (Regional Director of Pharmacy); Texas A&M University College of Medicine (Associate Professor of Anesthesiology); Temple, TX
- Linda Radach, BA – Consumers Union Safe Patient Project (Patient Safety Advocate); Austin, TX
- Amita Rastogi, MD, MHA, CHE, MS – Health Care Incentives Improvement Institute (Chief Medical Officer); Newtown, CT
- Donna Slosburg, RN, BSN, LHRM, CASC – ASC Quality Collaboration (Executive Director); St. Pete Beach, FL
We systematically assessed the face validity of the measure score as an indicator of quality by soliciting the TEP members’ agreement with the following statements:
“The risk-standardized hospital visit rates obtained from the Hospital Visits after General Surgery Ambulatory Surgical Center Procedures ASC measure, as specified, are valid and useful measures of ASC general surgical quality of care.- The risk-standardized hospital visit rates obtained from the Hospital Visits after General Surgery Ambulatory Surgical Center Procedures’ measure, as specified, will provide ASCs with information that can be used to improve their quality of care.
TEP members indicated their agreement with the face validity of the measure on a six-point scale:
1=Strongly disagree
2=Moderately disagree
3=Somewhat disagree
4=Somewhat agree
5=Moderately agree
6=Strongly agree
Validation of the Outcome
The outcome of an unplanned hospital visit following an outpatient procedure, is intended to capture adverse events that occur as part of the care received before, during, and after the procedure. To validate the outcome, we identified the most commonly occurring principal discharge diagnosis codes associated with the post-procedure hospital visit. (For any hospitalization, a claim for the hospital visit is submitted to CMS that indicates the main reason for the hospitalization; there is only one such main reason, called the “principal diagnosis code” that is used to capture this information.) Based on previous research to validate the outcome during measure development, we know that the most frequent reasons for a post-surgery hospital visit are complications from the procedure (Tevis et al., 2014). We updated this analysis for this endorsement maintenance submission.
External Empiric Validity
One approach for assessing the validity of a quality measure is to show that performance on the test measure is associated with another quality measure in the same causal pathway. To do this, we needed to identify a comparator measure, however, as summarized below, we did not identify a suitable measure with currently publicly available data.
We first considered CMS’ two related consensus-based entity-endorsed measures, the Hospital Visits after Orthopedic ASC Procedures (ASC Orthopedic) measure and the Hospital Visits after Urology ASC Procedures (ASC Urology) measure. The outcome of both measures is nearly identical to that of the ASC General Surgery measure; an unplanned hospital visit is defined as an ED visit, observation stay, or unplanned inpatient admission. Hence, the measures target the same quality domains. The patient cohort is also somewhat similar in that the measures target Medicare FFS patients aged 65 years or older. The cohorts (in terms or procedures included in the measure) for the ASC General Surgery and ASC Urology measures do not, however, overlap with the ASC Orthopedic measure. Furthermore, the clinicians performing the procedures across the different cohorts are unlikely to be the same individuals, and in addition, most ASCs (about 70% as of 2022) subspecialize (MedPAC, 2024).
Association with Volume
A recent study by Jain et al., found that patients undergoing general surgery procedures at a low volume ASC had a higher odds of a post-procedure hospital visit compared with patients who went to a high-volume ASC. However, there was no difference in outcome by volume when considering patients undergoing “general surgery” without multimorbidity (Jain et al., 2024). We therefore hypothesized that there would be a weak to moderate, negative relationship between facility procedural volume and ASC General Surgery measure scores, with higher volumes being associated with better (lower) measure scores. We calculated the Spearman correlation coefficient between procedural volume and the measure score.
References
Jain S, Rosenbaum PR, Reiter JG, Ramadan OI, Hill AS, Silber JH, Fleisher LA. Assessing the Ambulatory Surgery Center Volume-Outcome Association. JAMA Surgery. 2024 Apr 10.
Tevis SE, Kohlnhofer BM, Weber SM, Kennedy GD. Post discharge complications are an important predictor of postoperative readmissions. The American Journal of Surgery. 2014 Oct 1;208(4):505-10.
MedPAC’s Report to Congress, Chapter 5, Ambulatory Surgical Center Services, March 2024. https://www.medpac.gov/wpcontent/uploads/2024/03/Mar24_MedPAC_Report_To_Congress_SEC.pdf.
Below we provide the results of the validity testing approach.
Face Validity as Determined by the Technical Expert Panel (TEP)
Fourteen out of the 15 TEP members responded to the face validity survey. Of the 14 respondents, 12 respondents (86.7%) indicated that they somewhat, moderately, or strongly agreed; and two respondents moderately disagreed with the validity statements.
Empirical Validity
As noted above, we did not identify any measures that were suitable for comparison with the ASC General Surgery measure, therefore we examined the association of the measure score with facility volume. Consistent with our hypothesis, the correlation coefficient for the association between measure scores and facility procedural volume was -0.207 (p<0.0001) for ASCs with at least 25 procedures and -0.173 (p<0.0001) for ASCs with at least one procedure showing an overall negative correlation with the expected strength between measure score and procedural volume.
Validation Outcome
Table 5 in the attachment shows the most frequent (the top 25) principal diagnosis codes associated with the outcome (unplanned hospital visits within seven days of a qualifying surgery). Many of these codes indicate complications from the procedure (Bongiovanni et al., 2021). For example, the most frequent codes in Table 5 include urinary retention, hemorrhage, constipation, sepsis, and post-procedural pain.
The validity of the ASC General Surgery measure score is supported by three sources of data: (1) face validity of the measure score, (2) association with procedural volume in the expected strength and direction, and (3) validity of the outcome based on an analysis of diagnosis codes. Face validity assessed by the TEP shows high agreement (87%) that the ASC General Surgery measure is a valid indicator of ASC general surgery quality of care and that it will give ASCs the information they need to improve the quality of care. Second, in terms of the volume-outcome relationship, as expected, we see a significant negative association between ASC volume and measure scores. Finally, we see the validation of the outcome by analysis of ICD-10 codes associated with unplanned hospital visits within seven days of a qualifying surgery.
Risk Adjustment
Our approach to risk adjustment is tailored to, and appropriate for, a publicly reported outcome measure as articulated in published scientific guidelines (Krumholz et al., 2006; Normand & Shahian, 2007). For example, we only adjust for risk factors that are present at the start of care. We do not risk adjust for conditions that are possible adverse events of care and that are only recorded at the time of the surgery (see Data Dictionary, tab 4, ASC_GenSurg_CoC CCs). We do not adjust for factors related to the delivery of care that may reflect care quality.
The ASC General Surgery measure employs a hierarchical logistic regression model (a form of HGLM) to create an ASC-level 7-day RSHVR. This approach to modeling appropriately accounts for the structure of the data (patients clustered within facilities), the underlying risk due to patients’ procedures/comorbidities, and sample size at a given ASC when estimating hospital visit rates. In brief, the approach simultaneously models two levels (patient and facility) to account for the variance in patient outcomes within and between facilities (Normand & Shahian, 2007). At the patient level, the model adjusts the long-odds of hospital visits within seven days after the procedure for selected demographic, clinical, and procedure risk variables. The second level models the facility-specific intercepts as arising from a normal distribution. The facility intercept, or facility-specific effect, represents the ASC contribution to the risk of 7-day hospital visits, after accounting for patient risk and sample size, and can be inferred as a measure of quality. If there were no differences among ASCs, then after adjusting for patient risk, the facility intercepts would be identical across all ASCs.
Candidate Risk-Adjustment Variables
The measure adjusts for differences across facilities in patient demographic and clinical factors and in procedure-related risk. Potential candidate risk factors were identified from related quality measures and the literature; a preliminary list of risk factors was developed and then revised based on TEP and expert clinical input.
To define the candidate risk factors, during measure development we defined the clinical risk factors in claims data using Version 22 of the CCs from CMS’ Hierarchical Condition Categories (HCC) grouper, which classified over 15,000 ICD-9 diagnosis codes into 200 clinically coherent and mutually exclusive groups of codes, or condition categories (Pope et al., 2000). In some cases (for example, morbid obesity), individual codes were used to define the risk factor. The measure does not apply the hierarchical logic of the HCC. Based on prior validation work conducted for similar measures, we have confidence that model variables defined using the CCs are reasonable proxies for clinical conditions. CMS has validated similar risk-adjustment models that use the CCs against models that use chart-abstracted data for risk adjustment. Note that the measure was later re-specified for use in ICD-10.
To address surgical procedural complexity, we used the work RVUs of the procedure, an approach employed by the American College of Surgeons National Surgical Quality Improvement Program (Raval et al., 2010). We then reviewed the list of candidate risk variables with TEP members. None of the clinical experts suggested removing any of the candidate risk factors from the list. Several TEP members suggested additional variables: failure to thrive (poor nutritional status), history of falling, sleep apnea, and history of steroid use. We added all suggested candidate variables; the final list included 80 candidate risk variables.
Finally, to consolidate similar risk factors, we checked the bivariate direction and strength of association of the individual risk factors defined by CCs or ICD-9 codes and then combined risk factor diagnoses into clinically coherent comorbidity variables. For example, a “cancer” variable was created that combined several individual cancer diagnoses.
Variable Selection
To select the final set of variables to include in the risk-adjustment model, all risk variables were entered into logistic regression analyses predicting the outcome of hospital visits within seven days in the Development Sample. To develop a parsimonious risk model, non-significant variables (at the 0.05 level) were iteratively removed from the model using a stepwise purposeful selection approach described by Hosmer and Lemeshow (Hosmer & Lemeshow, 2000). Our goal was to minimize the number of variables in the model while preserving model performance (as measured by the c-statistic). All variables significant at p<0.05 were retained in the final model. In addition, we retained in the model two variables (tobacco use disorder and morbid obesity) because experts advised that these were important risk predictors and expressed a strong preference for including them in the model.
Social Risk Factors for Supplemental Disparities Analyses
We selected variables representing social risk factors based on a review of literature, conceptual pathways, and feasibility. Below, we describe the pathways by which social risk factors may influence risk of hospital visits following outpatient surgical procedures. Our conceptualization of the pathways by which social risk factors may affect the outcome is informed by the literature (Bhattacharyya, 2015; Jha, Orav, & Epstein, 2011; Menachemi et al., 2007; Reames et al., 2014; Skinner et al., 2005; Trivedi et al., 2014) and IMPACT Act–funded work by the National Academy of Science, Engineering and Medicine (NASEM) and the Department of Health and Human Services Assistant Secretary for Policy and Evaluation (ASPE) (ASPE, 2016; NASEM, 2016a; NASEM, 2016b).
Literature Review: Social Risk Variables and Ambulatory Surgery Post-Procedure Hospital Visits
During measure development we performed a literature search that examined the relationship between social risk factors and risk of hospital visits following outpatient surgical procedures. A total of 176 studies were reviewed by title and abstract, and all but two studies were excluded from full-text review based on the above criteria. The two studies indicated that Black and Hispanic patients and patients from lower-income households were at increased risk of post-procedure hospital visits in the ambulatory surgery setting (Bhattacharyya, 2015; Hosmer & Lemeshow, 2000). No studies were found that suggested that variation in patients’ social risk affected variation in outcome risk across facilities performing ambulatory surgical procedures.
Regarding the outcome of hospital visits following a screening colonoscopy, we performed an updated focused literature search and found additional evidence for the impact of social risk factors on outcomes for patients undergoing outpatient procedures, but no studies specific to general surgery. For example, a 2019 study found that while patients with low income undergoing colectomy had higher rates of surgical-site infections compared with higher-income patients, there was no difference in surgical-site infection rates based on income for patients undergoing hysterectomy (Qi et al., 2019). A 2023 study in cancer patients undergoing surgery found that patients with psychosocial risk factors were more likely to experience complications following surgery (Leeds et al., 2019). Finally, a 2021 study found that for some procedures, people living in counties with a high Social Vulnerability Index (SVI) were more likely to experience complications compared with patients who live in low SVI counties (Diaz et al., 2021).
We note that compared to the patient mix for elective outpatient procedures at (HOPDs, ASCs serve a low proportion of patients with social risk factors therefore there are disparities in access to care at ASCs, with Black patients and patients with public insurance being less likely to receive care at an ASC compared with others without those social risk factors (Janeway et al., 2020). A recent study found that there are disparities in the geographic distribution of ASCs, with counties with higher socioeconomic status having more ASCs per capita compared with counties with lower socioeconomic status (Chatterjee, Amen, & Khormaee, 2022).
Conceptual Pathways for Social Risk Factors
Although there is limited literature linking social risk factors and adverse outcomes for ambulatory surgery, potential pathways may include:
- Differential care within an ASC. One pathway by which social risk factors may contribute to hospital visit risk is that patients may not receive equivalent care within a facility (Trivedi et al., 2014; ASPE, 2016). Moreover, patients with social risk factors, such as lower education, may require differentiated care – e.g., provision of information at a lower health literacy level – to achieve outcomes comparable to those of patients without social risk factors. Facilities that do not identify the need for and provide such care could have worse outcome rates for their patients with social risk factors.
- Use of lower-quality facilities. Patients of lower income, lower education, or unstable housing may not have equitable access to high-quality facilities because such facilities are less likely to be found in geographic areas with large populations of poor patients; thus, patients with low income may be more likely to be seen in lower-quality facilities, which can contribute to increased risk of post-procedure adverse outcomes (Jha, Orav, & Epstein, 2011; Trivedi et al., 2014). Similarly, Black patients have been shown to have less access to high-quality facilities compared with White patients (Reames et al., 2014). As described above, patients with social risk factors are less likely to have access to care at ASCs, in general (Chatterjee, Amen, & Khormaee, 2022; Janeway et al., 2020).
- Influence of social risk factors on hospital visit risk outside of ASC quality. Some social risk factors, such as income or wealth, may affect the likelihood of post-procedure hospital visits without directly being associated with the quality of care received at the ASC. For instance, while an ASC may make appropriate care decisions and provide tailored care and education, a lower-income patient may have a worse outcome post-procedure due to competing economic priorities or a lack of access to care outside of the facility.
We developed and used the conceptual framework described below to identify potential social risk factors. We analyzed two well-studied social risk factors that could best be operationalized in data, outlined below. We note that this measure already adjusts for age.
- Medicare and Medicaid Dual Eligibility Status
Dual eligibility (DE) for Medicare and Medicaid is available at the patient level in the Medicare Master Beneficiary Summary File. The eligibility threshold for Medicare patients aged 65 years or older considers both income and assets. For the DE indicator, there is a body of literature demonstrating differential health care and health outcomes among beneficiaries, therefore the DE indicator allows us to examine some of the pathways of interest (ASPE, 2016). - Area Deprivation index (ADI)
The ADI, initially developed by the Health Resources & Services Administration, is based on 17 measures across four domains: income, education, employment, and housing quality (Kind et al., 2018; Singh, 2003).
The 17 components are listed below:
- Population aged ≥ 25 y with < 9 y of education, %
- Population aged ≥ 25 y with at least a high school diploma, %
- Employed persons aged ≥ 16 y in white-collar occupations, %
- Median family income, $
- Income disparity
- Median home value, $
- Median gross rent, $
- Median monthly mortgage, $
- Owner-occupied housing units, % (home ownership rate)
- Civilian labor force population aged ≥16 y unemployed, % (unemployment rate)
- Families below poverty level, %
- Population below 150% of the poverty threshold, %
- Single-parent households with children aged < 18 y, %
- Households without a motor vehicle, %
- Households without a telephone, %
- Occupied housing units without complete plumbing, % (log)
- Households with more than 1 person per room, % (crowding)
ADI scores were derived using beneficiary’s 9-digit ZIP Code of residence, which is obtained from the Master Beneficiary Summary File, and is linked to 2017-2021 US Census/American Community Survey data. In accordance with the ADI developers’ methodology, an ADI score is calculated for the census block group corresponding to the beneficiary’s 9-digit ZIP Code using 17 weighted Census indicators. Raw ADI scores were then transformed into a national percentile ranking ranging from 1 to 100, with lower scores indicating lower levels of disadvantage and higher scores indicating higher levels of disadvantage. Percentile thresholds established by the ADI developers were then applied to ADI percentiles to dichotomize neighborhoods into more disadvantaged (high ADI areas=ranking equal to or greater than 85) or less disadvantaged areas (Low ADI areas=ranking of less than 85).
References
Bhattacharyya, N. (2015). Healthcare disparities in revisits for complications after adult tonsillectomy. American Journal of Otolaryngology. 36(2), 249-253.
Chatterjee, A., Amen, T. B., & Khormaee, S. (2022). Trends in geographic disparities in access to ambulatory surgery centers in New York, 2010 to 2018. JAMA Health Forum. 3(10), Article e223608. https://doi.org/10.1001/jamahealthforum.2022.3608
Department of Health and Human Services Office of the Assistant Secretary of Planning and Evaluation. (2016). Report to Congress: Social risk factors and performance under Medicare’s value-based payment programs. Retrieved from https://aspe.hhs.gov/pdf-report/report-congress-social-risk-factors-and-performance-under-medicares-value-based-purchasing-programs.
Diaz, A., Hyer, J. M., Barmash, E., Azap, R., Paredes, A. Z., & Pawlik, T. M. (2021). County-level social vulnerability is associated with worse surgical outcomes especially among minority patients. Annals of Surgery. 274(6), 881–891. https://doi.org/10.1097/SLA.0000000000004691
Forefront Group. (2023). The Area Deprivation Index is the most scientifically validated social exposome tool available for policies advancing health equity. https://doi.org/10.1377/forefront.20230714.676093
Hosmer, D. W., & Lemeshow, S. (2000). Applied Logistic Regression. New York: Wiley.
Janeway, M. G., Sanchez, S. E., Chen, Q., Nofal, M. R., Wang, N., Rosen, A., & Dechert, T. A. (2020). Association of race, health insurance status, and household income with location and outcomes of ambulatory surgery among adult patients in 2 US states. JAMA Surgery. https://doi.org/10.1001/jamasurg.2020.3318
Jha, A. K., Orav, E. J., & Epstein, A. M. (2011). Low-quality, high-cost hospitals, mainly in South, care for sharply higher shares of elderly black, Hispanic, and Medicaid patients. Health Affairs. 30, 1904-1911.
Krumholz, H. M., Brindis, R. G., Brush, J. E., et al. (2006). Standards for statistical models used for public reporting of health outcomes: An American Heart Association scientific statement from the Quality of Care and Outcomes Research Interdisciplinary Writing Group: cosponsored by the Council on Epidemiology and Prevention and the Stroke Council endorsed by the American College of Cardiology Foundation. Circulation. 113(3), 456-462.
Leeds, I. L., Meyers, P. M., Zachary Obinna Enumah, He, J., Burkhart, R. A., Haut, E. R., Efron, J. E., & Johnston, F. M. (2019). Psychosocial risks are independently associated with cancer surgery outcomes in medically comorbid patients. Annals of Surgical Oncology. 26(4), 936–944. https://doi.org/10.1245/s10434-018-07136-3
National Academies of Sciences, Engineering, and Medicine; (2016a). Accounting for Social Risk Factors in Medicare Payment: Identifying Social Risk Factors. Washington DC: National Academies Press.
National Academies of Sciences, Engineering, and Medicine; (2016b). Accounting for Social Risk Factors in Medicare Payment: Data. Washington DC: National Academies Press.
Normand, S.-L. T., & Shahian, D. M. (2007). Statistical and clinical aspects of hospital outcomes profiling. Statistical Science. 22(2), 206-226.
Pope, G., et al. (2000). Principal inpatient diagnostic cost group models for Medicare risk adjustment. Health Care Financing Review. 21(3), 26.
Qi, A. C., Peacock, K., Luke, A. A., Barker, A., Olsen, M. A., & Joynt Maddox, K. E. (2019). Associations between social risk factors and surgical site infections after colectomy and abdominal hysterectomy. JAMA Network Open. 2(10), Article e1912339. https://doi.org/10.1001/jamanetworkopen.2019.12339
Raval, M. V., Cohen, M. E., Ingraham, A. M., et al. (2010). Improving American College of Surgeons National Surgical Quality Improvement Program risk adjustment: Incorporation of a novel procedure risk score. Journal of the American College of Surgeons. 211(6), 715-723.
Reames, B. N., Birkmeyer, N. J., Dimick, J. B., et al. (2014). Socioeconomic disparities in mortality after cancer surgery: Failure to rescue. JAMA Surgery. 149, 475-481.
Singh, G. K. (2003). Area deprivation and widening inequalities in US mortality, 1969–1998. American Journal of Public Health. 93(7), 1137–1143. https://doi.org/10.2105/ajph.93.7.1137
Trivedi, A. N., Nsa, W., Hausmann, L. R., et al. (2014). Quality and equity of care in U.S. hospitals. New England Journal of Medicine. 371, 2298-2308.
The risk-adjustment model includes 27 variables (age, 25 comorbidity variables, and one surgical complexity variable). Work RVUs are assigned to each CPT procedure code and approximate surgical procedural complexity by incorporating elements of physician time and effort. For patients with multiple concurrent CPT procedure codes, we risk adjust for the CPT code with the highest work RVU value.
Certain CCs are considered possible complications of care, and thus are not risk-adjusted for if they are coded only at the time of surgery. Please see the attached Data Dictionary for CCs that are considered possible complications of care and are not risk-adjusted for if they occur only at the surgery (tab 4, ASC Surgery CoC CCs)
Table 6 in the attachment includes the risk variable frequencies for each risk variable in the final risk model.
Table 7 in the attachment shows the distribution of social risk factors identified in the conceptual model. The facility median proportion of patients with the DE and high ADI variables is 0% for both; the median count of patients is 0 for each variable.
The final list of risk of clinical, procedural, and demographic variables was selected during development and is shown here, defined in the data dictionary in tab ASC_Surg_Risk_Factor_CCs, and the attachment (Tables 6 and 8) that include risk variable frequencies and odds ratios.
- Age
- Procedure type: Alimentary tract
- Procedure type: Breast
- Procedure type: Skin/soft tissue
- Procedure type: Wound
- Procedure type: Vascular
- Other benign tumors (CC 15, 16)
- Liver or biliary disease (CC 27, 28, 29, 30, 31, 32)
- Intestinal obstruction or perforation (CC 33)
- Dementia or senility (CC 51, 52, 53)
- Psychiatric disorders (CC 57, 58, 59, 60, 61, 62, 63)
- Other significant central nervous system (CNS) disease (CC 77, 78, 79, 80)
- Ischemic Heart Disease (CC 86, 87, 88, 89)
- Specified arrhythmias and other heart rhythm disorders (CC 96, 97)
- Stroke (CC 99, 100)
- Chronic lung disease (CC 110, 111, 112, 113)
- Pneumonia (CC 114, 115, 116)
- Dialysis or severe chronic kidney disease (CC 134, 136, 137)
- Benign prostatic hyperplasia
- Cellulitis, local skin infection (CC 164)
- Major traumatic fracture or internal injury (CC 169, 170, 171, 172, 173, 174)
- Complications of care (CC 176, 177)
- Long term (current) use of anticoagulants
- Opioid abuse
- Work RVU in abdomen and its contents procedure
- Work RVU in alimentary tract procedure
- Work RVU in breast procedure
- Work RVU in skin/soft tissue procedure
- Work RVU in wound procedure
- Work RVU in vascular procedure
Our measures undergo an annual measure reevaluation process, which ensures that the risk-standardized models are continually assessed and remain valid, given possible changes in clinical practice and coding standards over time. Modifications made to measure cohorts, risk models, and outcomes are informed by a review of the most recent literature related to measure conditions or outcomes, feedback from various stakeholders, and empirical analyses, including assessment of coding trends that reveal shifts in clinical practice or billing patterns. Input is solicited from a workgroup composed of up to 20 clinical and measure experts, inclusive of internal and external consultants and subcontractors.
To assess model performance, we computed three summary statistics: two discrimination statistics (the C-statistic, predictive ability) and one calibration statistic (overfitting) (Harrell et al., 2001). In addition, we provide risk-decile plots.
Discrimination statistics (c-statistic and predictive ability)
1. Area under the receiver operating characteristic (ROC) curve (the c-statistic) indicates the probability that predicting the outcome is better than chance, which is a measure of how accurately a statistical model can distinguish between a patient with and without an outcome.
C-statistic: .699
The c-statistic of 0.699 indicates continued good model discrimination.
2. Discrimination – Predictive ability. Discrimination in predictive ability measures the ability to distinguish high-risk subjects from low-risk subjects; therefore, we would hope to see a wide range between the lowest decile and highest decile.
Predictive Ability, % (lowest decile – highest decile): 0.51%-5.18%
The model continues to show a wide range between the lowest decile and highest decile, indicating the ability to distinguish high-risk subjects from low-risk subjects.
Calibration Statistics (Overfitting)
3. Overfitting indices (overfitting refers to the phenomenon in which a model accurately describes the relationship between predictive variables and outcome in the development dataset but fails to provide valid predictions in new patients).
Estimated calibration values of γ0 far from 0 and estimated values of γ1 far from 1 provide evidence of overfitting. We used Dataset #1 for this analysis. Our results show a calibration value of close to 0 at one end and close to 1 to the other end indicating good calibration of the model.
We note that after initial measure development we do not re-test our risk models for overfitting using a dataset that is external to the testing sample. In our risk models, coefficients are updated each time the measure is calculated; we refit the model with new data each time the measure is calculated. Therefore, random statistical fluctuations in model coefficients across repeated reporting cycles are part of the overall random error in the facility performance estimates.
2015- 2016 Development Sample results:
Calibration: (0, 1)
2015-2016 Validation Sample results:
Calibration: (-0.08, 0.98)
Risk Decile Plots
We provide updated risk decile plots for all patients below.
Higher deciles of the predicted outcomes are associated with higher observed outcomes, which continue to show good calibration of the model. The risk decile plot indicates continued good discrimination of the model and good predictive ability for all patients (Figure 4 in attachment).
We describe the approach to risk variable selection in Section 4.4.2, and analysis and rationale for not including social risk factors in the final model in Section 5.1. In this section we provide the interpretation of the risk model testing results described in section 4.4.5.
The following results demonstrate that the risk-adjustment model adequately controls for differences in patient characteristics:
Discrimination Statistics
The calculated c-statistic was 0.699, which indicates good model discrimination. The model also predicted a wide range between the lowest decile and highest decile, indicating the ability to distinguish high-risk subjects from low-risk subjects.
Calibration Statistics
The calibration value which was consistently close to 0 at one end and close to 1 to the other, indicating good calibration of the model.
Risk Decile Plot
Higher deciles of the predicted outcomes are associated with higher observed outcomes, which show a good calibration of the model.
Overall Interpretation
Interpreted together with information provided in the aforementioned sections, our diagnostic results demonstrate the risk-adjustment model adequately controls for differences in patient characteristics (case mix).
Use & Usability
Use
The level of analysis is facility level, and the care setting is ASCs.
Usability
While ASCs continue to show promising results of reduced complications post-surgery in comparison to hospital settings, the possibility of an unplanned hospital visit due to adverse events post-surgery still exists for patients (Munnich et. al., 2014; Schwartz, 2020). These complications can include surgical site infections, postoperative pain management issues, and other medical-related complications (Owens et. al., 2014; Erhun et. al., 2016; Rajput et. al., 2021). This measure provides the opportunity for improvement in reducing unplanned visits due to complications after surgery in the ASC setting.
In a 2021 qualitative study that used clinician interviews at the U.S. Department of Veterans Affairs to build a quality framework for complications in outpatient surgery, researchers found that clinicians identified pre- and post-operative processes of care as important in preventing adverse events (Mull et al., 2021). For example, clinicians noted the following important processes: pre- and post-operative patient instructions, preoperative assessment on day of surgery, and post-operative follow up. Intra-operative surgical processes (such as checklists) were also identified as important in preventing harm. Researchers additionally identified that standardization of processes, and communication and coordination (between clinicians and between clinicians and patients), were important factors in preventing adverse events.
Evidence-based interventions focusing on components of the framework above, including preoperative assessment, enhancing postoperative monitoring, and improving patient education, can reduce complications and rates of planned hospital visits (Erhun et. al., 2016; York et. al., 2019). For example, ASCs can opt to provide patient education pre- and post-procedure to facilitate the recovery process and reduce complications. Patient education improves transparency while allowing providers to clarify patient responsibilities and expectations. Continued and systemic monitoring of symptoms after surgery using validated assessments can lead to early detection of symptoms by providers post-surgery and help in planning a course of action before complications exacerbate (Girish, 2021). A 2020 pilot-study tested an electronic postoperative symptom-tracking platform to determine its clinical usefulness in the first week after minimally invasive ambulatory surgery. Responses above a defined threshold on the symptom instrument triggered an alert to the healthcare provider. The authors found that for a majority of the patients presenting symptoms, a simple consultation phone call and adjustment of medications was sufficient in controlling complications (Zivanovic et al., 2022).
Finally, the adoption of evidence-based quality improvement systems in ASC settings can overall advance patient care and sustain a culture of improvement among facilities. Employing a methodological system of quality improvement focused on improving certain performance domains of quality provides guidelines for ASCs to follow and offsets the burden from clinical staff to develop their own plans for quality control (Rakover et al., 2020). In addition, AHRQ developed a tool kit to support quality improvement (safety-related) that ASCs can use to reduce the incidence of adverse events (AHRQ, 2023).
Facilities can use resources provided by CMS to help improve the drivers of hospital visits after ambulatory surgery. To support quality improvement, CMS shares reports with measured entities that include measure results benchmarked against the state and nation (facility-specific reports [FSRs]), as well as reports that provide claim-level details for each claim that meets numerator and denominator criteria (claims-detail reports [CDRs]). These reports include, among other details, the principal diagnosis code associated with the post-procedure hospital visit, which allows facilities to tie their quality improvement efforts to the specific complications that are occurring.
References
Agency for Healthcare Research and Quality. Toolkit To Improve Safety in Ambulatory Surgery. (2023). Retrieved from https://www.ahrq.gov/hai/tools/ambulatory-surgery/index.html.
Erhun F, Malcolm E, Kalani M, Brayton K, Nguyen C, Asch SM, Platchek T, Milstein A. Opportunities to improve the value of outpatient surgical care. Am J Manag Care. 2016 Sep 1;22(9):e329-35. PMID: 27662397.
Joshi GP. Putting patients first: ambulatory surgery facilitates patient-centered care. Current Opinion in Anesthesiology. 2021 Dec 1;34(6):667-71.
Mull, H. J., Rosen, A. K., Charns, M. P., Itani, K. M. F., & Rivard, P. E. (2021). Identifying Risks and Opportunities in Outpatient Surgical Patient Safety: A Qualitative Analysis of Veterans Health Administration Staff Perceptions. Journal of Patient Safety. 17(3), e177–e185.
Munnich EL, Parente ST. Procedures take less time at ambulatory surgery centers, keeping costs down and ability to meet demand up. Health Affairs. 2014 May 1;33(5):764-9.
Owens PL, Barrett ML, Raetzman S, Maggard-Gibbons M, Steiner CA. Surgical site infections following ambulatory surgery procedures. JAMA. 2014 Feb 19;311(7):709-16.
Rakover J, Little K, Scoville R, Holder B. Implementing daily management systems to support sustained quality improvement in ambulatory surgery centers. AORN Journal. 2020 Apr;111(4):415-22.
Rajput K, Vadivelu N, Kaye AD, Shah RV, editors. Pain Control in Ambulatory Surgery Centers. Springer International Publishing. 2021 Mar.
Schwartz JC. Mastectomy and prepectoral reconstruction in an ambulatory surgery center reduces major infectious complication rates. Plastic and Reconstructive Surgery–Global Open. 2020 Jul 1;8(7):e2960.
York PJ, Gang CH, Qureshi SA. Patient education in an ambulatory surgical center setting. J Spine Surg. 2019 Sep;5(Suppl 2):S206-S211. doi: 10.21037/jss.2019.04.07. PMID: 31656877; PMCID: PMC6790809.
Zivanovic O, Chen LY, Vickers A, Straubhar A, Baser R, Veith M, Aiken N, Carter J, Curran K, Simon B, Mueller J, Jewell E, Chi DS, Sonoda Y, Abu-Rustum NR, Leitao MM Jr. Electronic patient-reported symptom monitoring in patients recovering from ambulatory minimally invasive gynecologic surgery: A prospective pilot study. Gynecol Oncol. 2020 Oct;159(1):187-194. doi: 10.1016/j.ygyno.2020.07.004. Epub 2020 Jul 24. PMID: 32718730; PMCID: PMC7380930.
1. Stakeholder feedback was obtained during a national dry run period between August 12, 2020 – September 9, 2020. The goal of the dry run was to educate ASCs about the measure and how to interpret results before the measure was implemented in the ASCQR Program. Additionally, it allowed ASCs to test production and reporting, review data used to calculate results for confidential reporting, and share feedback with CMS about the overall measure. ASCs were provided with their confidential measure results in a FSR for the July 1, 2017 – June 30, 2019 performance period as well as the FSR user guide.
CMS addressed concerns and inquiries through a Q&A email inbox before, during, and after the dry run. CMS received 40 questions via the email with 97% of the questions answered within five business days of receiving the inquiry. The most common types of questions were about FSR access (34.9%), followed by questions about the dry run process (30.2%), and requests to re-upload FSRs (11.6%).
2. CMS shares several reports with ASCs with their data to help improve during performance on the measure.
- FSRs: These reports include measure results benchmarked against the state and the nation. For the reporting period covered for this data submission, facilities received one FSR, released October 2023 on procedures performed January 1, 2021 – December 31, 2022.
- CDRs: Reports that provide claim-level details for each claim that meets the numerator and denominator criteria. CDRs provide facilities with an opportunity to improve the quality of care provided to patients receiving general surgery procedures prior to final measure calculation and public reporting of measure results. For this reporting period, covered for this data, facilities received one report released September 2023 on procedures performed January 1, 2022 – May 31, 2023.
3. Stakeholders can submit questions and issues to CMS through an online tool (Q&A tool) available to the public on QualityNet. CMS responds to each question submitted by stakeholders. During the time between the last maintenance review and re-endorsement of this measure (2020), we have received no major reports of issues with this measure. Through the Q&A tool, stakeholders have asked for assistance with:
- Interpreting their patient-level data
- Understanding measure specifications (inclusion, exclusion, risk adjustment)
- Interpretation or clarification of results.
Each year, we review and consider issues raised through the Q&A or in the literature related to this measure by measure and clinical experts. Any issues that warrant additional analytic work due to potential changes in the measure specifications are addressed as a part of annual measure reevaluation. If small changes are indicated after additional analytic work is complete, those changes are usually incorporated into the measure in the next measurement period. If the changes are substantial CMS may propose the changes through rulemaking and adopt the changes only after CMS received public comment on the changes and finalizes those changes in the Hospital Outpatient Prospective Payment System or other rule. Since its endorsement in 2018, there have not been substantive revisions to the specifications based on feedback.
Minor updates to the measure include:
- Reviewing and considering changes to HCPCS and ICD-10 codes that are then incorporated into the measure. Those code set files are made available to the public on QualityNet.
- Modifications to the PAA adapted from the CMS Planned Readmission Algorithm Version 4.0. The algorithm is a set of criteria for classifying admissions within seven days of a procedure as planned or unplanned using Medicare claims.
Finally, for this endorsement and maintenance submission, ASC General Surgery was a two-year measure which means we did not lose any cohort volume impacting reliability. However, the look-back period used for risk adjustment was shortened.
We did not assess improvement for this measure for two reasons. First, the measure captures an ever-changing list of procedures within the ASC covered procedure list. Therefore, year after year, different procedures are added to, and assessed by the measure. Second, this measure only began public reporting in January 2024, leaving insufficient time for facilities to consider their results, make adjustments to their processes, and then reassess their progress.
There have been no unexpected findings during implementation.
Comments
Staff Preliminary Assessment
CBE #3357 Staff Assessment
Importance
Strengths:
- The developer posits that this measure provides an opportunity for improvement in reducing unplanned hospital visits due to complications after surgery in the ambulatory surgical center setting.
The developer cites evidence from the primary literature interventions that ASCs can implements to reduce hospitalizations post-surgery, such as pre-operative assessments, enhancing post-operative monitoring, and improving patient education. The developer further suggests facilities can implement a system of quality improvement and use CMS benchmarking reports, which include the principal diagnosis on admission, to tie quality improvement efforts to complications that are occurring.
Using data from Jan 1, 2021 through Dec 31, 2022, the developer reported a distribution of scores with an overall score of 1.00 (national average) and a measure score range of 0.59 – 1.84. For this measure, a lower score is better. The distribution shows ~30% of ASCs are higher than the mean facility-level measure score.
Limitations:
- The developer does not provide direct evidence from patients about the meaningfulness of this measure focus but argues that due to the increase in procedures in ASCs there is a need for this measure to inform ASC providers and patients of the outcome of hospital visits. The developer further posits that this measure addresses an important quality measurement area and enhances the information available to patients choosing among ASCs that provide same-day outpatient surgery. Further, by providing reports of this measure to providers will incentivize quality improvement efforts.
Rationale:
- The developer posits that this measure provides an opportunity for improvement in reducing unplanned hospital visits due to complications after surgery in the ambulatory surgical center setting.
The developer cites evidence from the primary literature interventions that ASCs can implements to improve hospitalizations post-surgery, such as pre-operative assessments, enhancing post-operative monitoring, and improving patient education. The developer further suggests facilities can implement a system of quality improvement and use CMS benchmarking reports, which include the principal diagnosis on admission, to tie quality improvement efforts to complications that are occurring.
Using data from Jan 1, 2021 through Dec 31, 2022, the developer reported a distribution of scores with an overall score of 1.00 (national average) and a measure score range of 0.59 – 1.84. For this measure, a lower score is better. The distribution shows ~30% of ASCs are higher than the national average.
Although, the developer does not provide direct evidence from patients about the meaningfulness of this measure focus but argues that due to the increase in procedures in ASCs there is a need for this measure to inform ASC providers and patients of the outcome of hospital visits. The developer further posits that this measure addresses an important quality measurement area and enhances the information available to patients choosing among ASCs that provide same-day outpatient surgery. Further, by providing reports of this measure to providers will incentivize quality improvement efforts.
Feasibility Acceptance
Strengths:
- The developer did not conduct a feasibility assessment of an assessment of missing data for this measure, as it used a 100% sample of paid Medicare claims. The developer further states that since this is an claims-based measure, there is no added burden of reporting as all required data elements are routinely generated. The measure does not include any proprietary information.
Limitations:
- None identified.
Rationale:
- The developer did not conduct a feasibility assessment of an assessment of missing data for this measure, as it used a 100% sample of paid Medicare claims. The developer further states that since this is an claims-based measure, there is no added burden of reporting as all required data elements are routinely generated. The measure does not include any proprietary information.
Scientific Acceptability
Strengths:
- The measure is clear and well defined.
Entity level reliability (signal-to-noise) is calculated on 2 years of data (1/21-12/22) for the risk adjusted measure across 1455 entities and is >0.6 for about 75% of the entities. (The estimated lower quartile is 0.555.)
Limitations:
- About 25% of the entities have a reliability below 0.6.
Rationale:
- The measure is well defined. Reliability was assessed at the entity level. Reliability statistics are above the established thresholds for at least 70% of the entities.
Strengths:
- The developer provided three sources of evidence in support of measure validity (1) face validity of the measure score, (2) association with procedural volume, and (3) validity of the outcome based on an analysis of diagnosis codes. Face validity assessed by the TEP shows high agreement (87%) that the ASC General Surgery measure is a valid indicator of ASC general surgery quality of care and that it will give ASCs the information they need to improve the quality of care. In terms of the volume-outcome relationship, the developer reported significant negative association between ASC volume and measure scores, as hypothesized. However, some limitations should be considered (see limitations). Lastly, the developer provided supportive analysis of the outcome by providing ICD-10 codes associated with unplanned hospital visits within seven days of a qualifying surgery.
The developer also conducted statistical risk adjustment, based on their conceptual model, and selecting factors that had a significant correlation to the outcome. The developer also explored social risk factors, such as dual eligibility and the Area Deprivation Index. The developer did not include these in the final models due to the minimal impact these social risk factors have on the measure scores and to the model overall. The developer reported a c-statistic of 0.699, indicating good model discrimination.
Limitations:
- The developer conducted accountable entity-level validity testing by associating measure score to facility volume. The developer considered conducting a correlation analysis to another quality measure but did not find any that “was suitable for comparison.” With respect to the association by facility volume, the developer hypothesized a weak-to-moderate, negative relationship meaning that facilities with higher volume have lower (better) measure scores. The developer based this hypothesis on prior literature, which showed a volume-outcome relationship with higher facility volume associated with better outcomes. The developer reported a statistically significant correlation of -0.207, consistent with their hypothesis. However, it is unclear, based on the evidence provided, how facility procedural volume supports the validity of the measure, including the quality construct. The developer did not identify procedural volume within the measure's logic model. Volume may indicate higher number of surgeons or increased quality control and resources, but this is not clearly articulated in the submission. In addition, some facilities may not be able to control procedural volume, thus based on this analysis, facilities with low volumes will have worse outcomes. Overall, it is not clear how procedural volume is an appropriate covariate for validity analysis.
Rationale:
- The developer provided three sources of evidence in support of measure validity (1) face validity of the measure score, (2) association with procedural volume, and (3) validity of the outcome based on an analysis of diagnosis codes. Face validity results were strong. With respect to empirical testing, the developer considered conducting a correlation analysis to another quality measure but did not find a measure that “was suitable for comparison.” With respect to the association by facility volume, the developer hypothesized a weak-to-moderate, negative relationship meaning that facilities with higher volume have lower (better) measure scores. The developer based this hypothesis on prior literature, which showed a (negative) volume-outcome relationship with higher facility volume associated with better outcomes. The developer reported a statistically significant correlation of -0.207, consistent with their hypothesis. However, it is unclear, based on the evidence provided, how facility procedural volume supports the validity of the measure, including the quality construct. The devleoper did not identify procedural volume wihtin the measure's logic model. Volume may indicate higher number of surgeons or increased quality control and resources, but this is not clearly articulated in the submission. In addition, some facilities may not be able to control procedural volume, thus based on this analysis, facilities with low volumes will have worse outcomes. Overall, it is not clear how procedural volume is an appropriate covariate for validity analysis. The developer may be able to speak to this during the endorsement meeting.
The developer also convened a technical expert panel (TEP) for face validity testing during the measure’s initial development. 12/14 (86.7%) of TEP respondents agreed that the measure is valid and useful and can improve the quality of care.
The developer also conducted statistical risk adjustment, based on their conceptual model, and selecting factors that had a significant correlation to the outcome. The developer also explored social risk factors, such as dual eligibility and the Area Deprivation Index. The developer did not include these in the final models due to the minimal impact these social risk factors have on the measure scores and to the model overall. The developer reported a c-statistic of 0.699, indicating good model discrimination.
Equity
Strengths:
- None identified.
Limitations:
- The developer did not identify whether the measure rates are different across different patient populations. Rather, for this section, the developer provided a rationale for why dual eligibility and Area Deprivation Index were not included in the risk model.
Rationale:
- The developer did not identify whether the measure rates are different across different patient populations. Rather, for this section, the developer provided a rationale for why dual eligibility and Area Deprivation Index were not included in the risk model. For future submissions, the developer may consider whether the measure can identify differences in performance across different patient groups by their social and/or economic status.
Use and Usability
Strengths:
- The measure is currently used in the ASC Reporting Program
The developer cites evidence of interventions that can improve hospital visits, as noted previously. The developer further suggests facilities can implement a system of quality improvement and use CMS benchmarking reports, which include the principal diagnosis on admission, to tie quality improvement efforts to complications that are occurring
The developer described that when this measure was implemented, it underwent a “dry-run” period in which ASCs could ask questions and become acclimated to the measure. Part of the measure’s feedback includes providing measure reports on performance to ASCs and answering questions through an online tool. The developer summarized how questions and feedback were considered, some of which included minor updates to the measure (e.g., procedural codes, admission algorithm). The developer notes that since 2018 there have been no substantive updates to the measure. The developer reports no unexpected findings.
Limitations:
- The developer did not report any findings on the progress on improvement since the procedures are “ever-changing,” making it challenging to accurately assess year-over-year performance. In addition, since the measure was publicly reporting in Jan 2024, the developer argues that facilities have not had enough time to implement changes the impact of those changes on performance.
Rationale:
- The measure is currently used in the ASC Reporting Program
The developer cites evidence of interventions that can improve hospital visits, as noted previously. The developer further suggests facilities can implement a system of quality improvement and use CMS benchmarking reports, which include the principal diagnosis on admission, to tie quality improvement efforts to complications that are occurring
The developer described that when this measure was implemented, it underwent a “dry-run” period in which ASCs could ask questions and become acclimated to the measure. Part of the measure’s feedback includes providing measure reports on performance to ASCs and answering questions through an online tool. The developer summarized how questions and feedback were considered, some of which included minor updates to the measure (e.g., procedural codes, admission algorithm). The developer notes that since 2018 there have been no substantive updates to the measure.
The developer did not report any findings on the progress on improvement since the procedures are “ever-changing,” making it challenging to accurately assess year-over-year performance. In addition, since the measure was publicly reporting in Jan 2024, the developer argues that facilities have not had enough time to implement changes the impact of those changes on performance.
The developer reports no unexpected findings.
Committee Independent Review
Do not support
Importance
agree with staff
Feasibility Acceptance
Agree with staff
Scientific Acceptability
agree with staff
agree with staff
Equity
Patients and their caregivers might find this measure more respectful if more settings of follow-up were considered, such as home care, satellite clinics, portal communication, and telehealth, as well as transportation and appointment access.
Use and Usability
Patients and their caregivers might find this measure more meaningful if it considered the corporatization of healthcare with associated increased out-of-pocket expenses and the shift from solo to group practices; the increase in minimally invasive procedures, imaging advancements, telemedicine, the impact of the COVID-19 pandemic, and included data from Medicare Advantage Plans.
Summary
Patients and their caregivers might find this measure more meaningful if it considered the corporatization of healthcare with associated increased out-of-pocket expenses and the shift from solo to group practices; the increase in minimally invasive procedures, imaging advancements, telemedicine, the impact of the COVID-19 pandemic, and included data from Medicare Advantage Plans.
Patients and their caregivers might find this measure more respectful if more settings of follow-up were considered, such as home care, satellite clinics, portal communication, and telehealth, as well as transportation and appointment access.
Support
Importance
agree with staff assessment
Feasibility Acceptance
agree with staff assessment
Scientific Acceptability
agree with staff assessment
agree with staff assessment; would explore modifications that would allow validity to be confirmed, perhaps by narrowing the list of admission diagnoses.
Equity
agree with staff assessment; would explore the contribution of alternative social risk factors.
Use and Usability
inadequate evidence to date of the value of this measure.
Summary
Although there are outstanding concerns, the measure is important considering the growth of ASC procedures
There are opportunities for…
Importance
This measure provides an opportunity for improvement in reducing unplanned hospital visits due to complications after surgery in the ambulatory surgical center setting. The measure identified only 9 out of 3,500 ASCs as being either better or worse than the national average. Over half were classified as number of cases too small and the rest were classified as no different than the national average.
Feasibility Acceptance
Uses 100% Medicare claims and developer did not conduct feasibility.
Scientific Acceptability
The minimum reliability with at least 25 cases was 0.432 and approximately 25% had a reliability below 0.6.
Not clear on the procedural volumes and how used in validity analysis.
Equity
The developer reports that ASCs do not serve a high proportion of patients with social risk factors—the developer may want to consider analysis to look at differences across populations.
Use and Usability
The developer did not report any findings on the progress improvement to be able to accurately assess performance over time.
Summary
There are opportunities for further clarification as noted above.
no additional comments
Importance
agree with staff assessment
Feasibility Acceptance
agree with staff assessment
Scientific Acceptability
agree with staff assessment
agree with staff assessment
Equity
agree with staff assessment
Use and Usability
agree with staff assessment
Summary
n/a
No additional comments
Importance
the developer highlighted various literature and sources to support the importance criteria.
Feasibility Acceptance
Agree with staff assessment; no burden on the facility since this is a claims-based measure; also no feasibility assessment was conducted.
Scientific Acceptability
agree with staff assessment
agree with staff assessment
Equity
The developer did not explicitly analyze whether measure rates differ across different patient populations based on social or economic factors like dual eligibility.
Use and Usability
The measure demonstrates a structured approach to improving care quality in ASC settings through evidence-based practices, continuous monitoring, and CMS-supported quality reporting mechanisms.
Summary
No additional comments
Support with limitation
Importance
Agree w staff assessment
Feasibility Acceptance
Agree w staff assessment
Scientific Acceptability
Agree w staff assessment
Agree with staff assessment
Equity
Agree with Staff assessment
Use and Usability
Agree with staff assessment
Summary
Agree with staff assessment along with criteria not met but addressable. Support if addressed
Support
Importance
Agree with staff assessment.
Feasibility Acceptance
Agree with staff assessment.
Scientific Acceptability
Agree with staff assessment.
As per facilities with a reliability less than 0.60, we may consider the utility of publicly reporting data that does not meet minimum reliability thresholds.
Agree with staff assessment.
Equity
Agree with staff assessment.
Use and Usability
Agree with staff assessment.
The developer cites an "ever changing" list of covered ASC procedure preventing assessment of improvement over time. This will be a consistent issue for this measure and the developer should explore strategies to mitigate this issue.
Summary
As an increasing number of surgeries move from the inpatient to ASC setting, it is important to track this outcome. This is overall a robust measure but could be strengthened with methods to address performance assessment in the face of the evolving list of covered ASC procedures. Additional information is needed in regards to stratified equity analysis, validity assessment of the quality-volume relationship, and public reporting for ASCs with reliability lower than the minimum threshold.
Public Comments
Public Comment Shared During May 29 Listening Session
So my daughter recently had surgery about 6 months ago, and her after care, it felt like, okay, she was there to have surgery and the post op care was like very rocky. The discussions weren’t there. Like, okay, you got the surgery, kid. She was 17, and now move on, and like visiting psychiatry and the physical therapist after the surgery was not a smooth transition, but I believe this is a very important discussion to have and it's well needed, but needs to be fine-tuned, the post-op care.
Public Comment Shared During May 29 Listening Session
So, these [CBE 3470, 3357, 2539, and 3366] are all ambulatory, which means you were not admitted to the hospital to have the surgery. I had to go into a hospital after a general surgery, but it was orthopedic, right. Mine was within that 7 days. But really any hospital visit after any kind of surgery is important. I totally recognize these and think it's very important, particularly when there's sort of a push to get people in and out of the hospital. So, thank you.
Yeah, I mean, they need to watch. There's a risk standardized hospital visit. They all are about some hospital visit, right? And that does not mean that actually means hospital. Either a stay or observational. I'm assuming. So it's important. And these are all very important. Because sometimes it's just check the box, but it’s important to know why the person ends up back in the hospital after some sort of procedure. Thank you.
Public Comment Shared During May 29 Listening Session
Well, Hello! So my question would just be regarding the age, the 65 years and older for the Medicare patients Fee-for-Service. I don't know if the previous maintenance [review] was still in the same age category, but if the age could be expanded for Medicare Fee-for-Service patients that are younger as well, too. Just widening the age range for these Medicare patients. I know they're looking at older patients, but I don't know if they're looking at older patients being able to bounce back after the surgery, but not because they have to be re-admitted or have an ED visit or an unplanned inpatient admission. So, to just kind of like understand whether it's the age-related readmission, or if it's the procedural readmission, or if they're having complications from the actual surgical event that they experienced. So, I just have a question on why do they [the developer] have the age at 65, and not maybe in the fifties, or just to kind of exclude whether it's age related or procedure related.
Endorsement Should Be Removed From This Measure
CBE ID: 3357
Facility-Level 7-Day Hospital Visits After General Surgery Procedures
Performed at Ambulatory Surgical Centers
Endorsement should be removed from this measure. It is not a valid or reliable measure of the quality or efficiency of outpatient surgery due to problems with the numerator and the risk adjustment methodology. Continued public reporting of the results could mislead patients about where they should receive surgery.
Problems with the Numerator
The stated goal of the measure is to improve the quality of care delivered to patients undergoing general surgery procedures in an ambulatory surgery center (ASC). However, the numerator includes events that have nothing to do with the surgery, while excluding events that represent undesirable outcomes and avoidable costs.
The Measure Information Form provides a table showing the diagnosis codes associated with post-surgical visits, but there is no analysis as to whether these diagnoses are likely to represent complications of the type of surgery the patient received. Some of the most frequent diagnoses were “COVID-19,” “benign prostatic hyperplasia,” and “primary hypertension,” none of which are complications of outpatient surgeries. Others, including the most frequent diagnosis (“retention of urine”), could well have represented chronic conditions that existed prior to the surgery.
Other studies have found that a large fraction of post-surgical visits are not related to the index surgery:
The inclusion of so many visits unrelated to the surgery creates two different problems.
Problems with the Risk Adjustment Methodology
The measure does not report the actual rate of post-surgical visits at an ASC; it reports a ratio of the “predicted” number of visits to the “expected” number of visits. Both of these parameters are calculated using a hierarchical logistical regression model that is intended to adjust for differences in the characteristics of the patients receiving surgery that affect outcomes. However, the model fails to adjust for differences in the propensity of patients to visit a hospital for other health problems.
For example, a patient may come to the ED for an acute condition that is unrelated to the surgery (e.g., a viral infection, such as flu or COVID-19). An ASC in a community experiencing high rates of such illnesses could have higher rates of post-surgical visits, and this measure would make it appear that the ASC delivers lower quality surgical care. The risk adjustment used in the measure controls for the presence of chronic conditions, but it does not control for the frequency of new acute conditions.
In addition, patients with more severe chronic conditions will have higher rates of ED visits for exacerbations of their conditions, and the rates of ED visits for chronic disease exacerbations will be higher in communities where chronic disease management services are less available and for patients who have lower incomes or lack of family support that make it more difficult for them to access the services that do exist. In these types of communities and for these types of patients, there will likely be more post-surgical ED visits for chronic disease exacerbations that are unrelated to outpatient surgery. The risk adjustment methodology only controls for the presence of a chronic condition; it does not control for either the severity of the condition or for differences in the care patients receive for the chronic condition. Care for the chronic condition is not the responsibility of the physicians or ASC staff who delivered the outpatient surgery, but under this measure, a higher rate of ED visits for problems related to chronic conditions will be interpreted as poor quality care for the outpatient surgery.
One of the earliest studies of outcomes following outpatient surgery adjusted for this by including a factor measuring a patient’s hospital utilization prior to surgery. (Fleisher LA, et al. “Inpatient Hospital Admission and Death After Outpatient Surgery in Elderly Patients,” Archives of Surgery 139:67-72.) However, no such adjustment is included in the risk adjustment model for this measure. As a result, an ASC that has more patients with poorly managed chronic conditions will inappropriately appear to be delivering lower-quality care. Other patient characteristics that could affect the likelihood of post-surgery hospital visits include whether the patient is in a Skilled Nursing Facility or is receiving home health care, but no adjustment is made for these factors and there is no indication in the Measure Information Form that they were even examined.
Poor Measure Reliability and an Inappropriately Low Minimum Case Threshold
The measure is inappropriately being used for ASCs that perform as few as 25 surgeries. With that few cases, a single post-discharge visit results in a 4% post-discharge visit rate. The Measure Information Form never says what the average visit rate is for ASCs, but when this measure was originally created, the developers reported an average visit rate of 2.2%. As a result, for small facilities, a single visit could make the difference between having a “low” or “high” visit rate. Moreover, because of the wide range of surgeries that are included in the measure, variation in outcomes could easily occur simply because of differences in the specific procedures performed and patients treated during a particular year.
Because of the low volumes and high variability of post-discharge visits, this measure is not sufficiently reliable to use for public reporting. The developers report that the signal-to-noise reliability was below 0.6 for 30% of ASCs, below 0.7 for half of ASCs, and below 0.8 for most ASCs. It does not appear that any effort was made to assess the reliability of the measure over time.
Lack of Business Case for Using the Measure and Undesirable Effects of Doing So
The measure identified only 9 out of 3,500 ASCs as being either better or worse than the national average. Over half were classified as “number of cases too small” and the rest were classified as “no different than the national average.” Moreover, it does not appear that the developers made any effort to examine the 3 facilities identified as “better than the national rate” or the 6 facilities identified as “worse than the national rate” to assess whether this was really a valid classification based on the number of cases, the reasons for post-discharge visits, and the consistency of the classification over time.
Since only a very small number of ASCs are being classified as anything other than “no different than expected,” this measure is unlikely to encourage improvements in the quality of care delivery. On the other hand, since the weaknesses in the measure methodology could cause some ASCs to be inappropriately labeled as “worse than expected” because of the kinds of patients they treat, use of the measure creates an undesirable incentive for ASCs to avoid treating patients with characteristics that are likely to result in higher numbers of visits, such as patients with multiple comorbidities and patients who do not have good access to primary care and chronic disease management services. It could also cause patients to inappropriately avoid obtaining needed surgery, or to try and obtain surgery in a facility that is not adequately equipped to address their needs.
3357: Facility Level 7-day Hospital Visits after General Surgery
The American Medical Association (AMA) questions whether this measure produces scores that can be considered sufficiently reliable. Specifically, the minimum reliability with at least 25 cases was 0.432, which falls well below what the AMA considers to be acceptable. We believe that measures must achieve a minimum reliability of at least 0.7 or higher in order to be achieve endorsement.