When the measure returns for maintenance, the committee would like to see:
- Explore methods to enable the evaluation of improvement over time; and
- Consider additional approaches for the reliability assessment that inform the reliability-validity (e.g. shrinkage) and reliability-usability (e.g. stability) tradeoffs
This measure was developed to improve the quality of care delivered to patients undergoing urology procedures in an ambulatory surgical center (ASC). The measure estimates a facility-level rate of risk-standardized, all-cause, unplanned hospital visits within seven days of a urology surgery at an ASC among Medicare fee-for-service (FFS) patients aged 65 years and older. An unplanned hospital visit is defined as an emergency department (ED) visit, observation stay, or unplanned inpatient admission.
Measure Specs
- General Information(active tab)
- Numerator
- Denominator
- Exclusions
- Measure Calculation
- Supplemental Attachment
- Point of Contact
General Information
The Hospital Visits after Urology Procedures Performed at Ambulatory Surgical Centers (hereafter “ASC Urology Surgery”) measure captures complications arising from elective urology procedures performed at ASCs by measuring unplanned hospital visits within 7 days (ED visits, observation stays, inpatient admissions). The goal of this measure is to reduce adverse patient outcomes associated with ASC urology surgeries and improve follow-up care by capturing and making more visible to providers and patients unplanned hospital visits following orthopedic procedures. The measure score, publicly reported since 2018, holds ASCs accountable for their performance, and informs facility-based quality improvement.
There is a national trend toward increased use of ambulatory settings for several common outpatient urological procedures (Suskind et al., 2014). In general, the patient population served at ASCs has increased not only in volume but also in age and complexity, which can be partially attributed to improvements in anesthetic care, innovations in minimally invasive surgical techniques, and lower costs associated with care (Munnich et. al., 2014). For example, a study by Gul et al. found large variation in the cost for urological procedures based on both hospitals and insurance. ASC settings for such procedures appeal to patients when compared to HOPDs because of associated lower costs and recovery time (MedPac, 2024).
A hospital visit after outpatient surgery is unexpected, and many of the reasons for such hospital visits are preventable. Hospital visits following an ambulatory surgery vary from 0.5% to 9.0%, based on the type of surgery, outcome measured, and timeframe for measurement after surgery (Bongiovanni et al., 2021). Hospital visits can occur due to a range of potentially preventable adverse events including uncontrolled pain, urinary retention, surgical site infection, bleeding, septicemia, and venous thromboembolism. Patients also frequently report minor adverse events -- for example, uncontrolled pain, nausea, and vomiting -- that may result in unplanned acute care visits following surgery (Owens et al., 2014; Bongiovanni et al., 2021).
Several factors make unanticipated hospital visits a priority quality indicator. Because ASC providers may not be aware of all post-surgical hospital visits that occur among their patients, reporting this outcome will help to illuminate problems that may not be currently visible (Zivanovic et al., 2020). In addition, the outcome of hospital visits is a broad, patient-centered outcome that reflects the full range of reasons leading to hospital use among patients undergoing same-day surgery. Public reporting of the ASC Urology measure will provide ASCs with critical information and incentives to implement strategies to reduce unplanned hospital visits.
References
Bongiovanni T, Parzynski C, Ranasinghe I, Steinman MA, Ross JS. Unplanned hospital visits after ambulatory surgical care. PloS one. 2021 Jul 20;16(7):e0254039.
Gul, Z. G., Sharbaugh, D. R., Guercio, C. J., Pelzman, D. L., Jones, C. A., Hacker, E. C., ... & Davies, B. J. (2023). Large variations in the prices of urologic procedures at academic medical centers 1 year after implementation of the Price Transparency Final Rule. JAMA Network Open, 6(1), e2249581-e2249581.
Medicare Payment Advisory Commission (MedPAC). Report to Congress: Medicare Payment Policy. March 2024; https://www.medpac.gov/wp-content/uploads/2024/03/Mar24_MedPAC_Report_To_Congress_SEC.pdf
Owens PL, Barrett ML, Raetzman S, Maggard-Gibbons M, Steiner CA. Surgical site infections following ambulatory surgery procedures. Jama. 2014 Feb 19;311(7):709-16.
Suskind, A. M., Dunn, R. L., Zhang, Y., Hollingsworth, J. M., & Hollenbeck, B. K. (2014). Ambulatory surgery centers and outpatient urologic surgery among Medicare beneficiaries. Urology, 84(1), 57-61.
Zivanovic O, Chen LY, Vickers A, Straubhar A, Baser R, Veith M, Aiken N, Carter J, Curran K, Simon B, Mueller J, Jewell E, Chi DS, Sonoda Y, Abu-Rustum NR, Leitao MM Jr. Electronic patient-reported symptom monitoring in patients recovering from ambulatory minimally invasive gynecologic surgery: A prospective pilot study. Gynecol Oncol. 2020 Oct;159(1):187-194. doi: 10.1016/j.ygyno.2020.07.004. Epub 2020 Jul 24. PMID: 32718730; PMCID: PMC7380930.
Medicare Fee-for-Service (FFS) Part A and Part B claims, and administrative data.
Numerator
The measure defines the outcome as any (one or more), all-cause, unplanned hospital visit within seven days of an outpatient urology surgery; a hospital visit includes any emergency department (ED) visit, observation stay, or unplanned inpatient admission occurring within seven days after the ASC procedure.
Unplanned Hospital Visits
The measure defines the outcome as any unplanned hospital visit within seven days of an outpatient urologic procedure; a hospital visit includes any emergency department (ED) visit, observation stay, or unplanned inpatient admission.
The measure focuses on the outcome of unplanned hospital visits because this is a broad, patient-centered outcome that captures the full range of hospital visits resulting from adverse events or poor care coordination following outpatient surgery. This measure’s goal is to support quality improvement through the assessment and illumination of the variation in risk-adjusted hospital visits following surgery.
ED visits and observation stays are defined using revenue center codes identified in Medicare Part B outpatient hospital claims. The 2023 Urology Measure Data Dictionary tab “ASC Uro_Outcome ED Obs” provides the specific codes used to identify ED visits and observation stays.
Seven Day Time Frame
The measure limits the outcome to seven days, as existing literature suggests that most adverse events after outpatient surgery occur within the first seven days following the surgery and our empirical analyses during measure development indicated that the highest rates of hospital visits were within seven days of an outpatient urologic procedure. Thus, based on existing literature and empirical findings, as well as input from the Technical Expert Panel (TEP) and urology consultants, the measure development team concluded that unplanned hospital visits within seven days is the optimal outcome to ensure capture of surgery-related adverse events and to minimize capture of hospital visits unrelated to the surgery.
Planned Hospital Visits
For inpatient admissions, the measure outcome includes hospital visits within the first 7 days following the procedure, unless that inpatient admission is deemed a “planned” admission as defined by the measures Planned Admissions Algorithm (PAA). The algorithm is a set of criteria for classifying readmissions as planned or unplanned using Medicare claims. The Centers for Medicare & Medicaid Services (CMS) seeks to count only unplanned admissions in the measure outcome, because variation in “planned” admissions does not reflect quality differences. The measure does not consider observation stays or ED visits as planned.
In brief, the algorithm identifies admissions that are typically planned and may occur after the patient’s index event. The algorithm always considers a few specific, limited types of care planned (e.g., major organ transplant, rehabilitation, or maintenance chemotherapy). Otherwise, the algorithm defines a planned admission as a non-acute admission for a scheduled procedure (e.g., total hip replacement or cholecystectomy), and the algorithm never considers admissions for acute illness or for complications of care planned. For example, the algorithm considers hip replacement unplanned if hip fracture (an acute condition) is the discharge diagnosis but planned if osteoarthritis (a non-acute condition) is the discharge diagnosis. The algorithm considers admissions that include potentially planned procedures with acute diagnoses or that might represent complications of a procedure unplanned and thus counts these admissions in the measure outcome.
For more information about the PAA, please see the 2023 Urology Measure Updates and Specification Report.
Also see sheets ‘PAA1 Always Planned Px’, ‘PAA2 Always Planned Dx’, ‘PAA3 Post Planned Px’, and ‘PAA4 Acute Dx’ in the attached Data Dictionary for the most up-to-date sets of codes in the algorithm for ‘always planned procedures,’ ‘always planned diagnoses,’ ‘potentially planned procedures, and ‘acute’ diagnoses.
Denominator
The target population for this measure is Medicare FFS patients aged 65 years and older undergoing outpatient urology surgeries, typically performed by a urologist, at ASCs.
See Figure 1 (in attachment) for a flow chart that outlines the cohort construction.
The target population for this measure is Medicare FFS patients aged 65 years and older, undergoing outpatient urologic procedures. We limit the measure to patients who have been enrolled in Medicare FFS Parts A and B for the 12 months prior to the date of surgery to ensure that we have adequate data for identifying comorbidities for risk adjustment. See the attached Data Dictionary, “ASC Uro Cohort” for a complete list of all CPT procedure codes included in the measure cohort.
The measure includes procedures that (1) are routinely performed at ASCs, (2) involve increased risk of post-procedure hospital visits, and (3) are routinely performed by urologists. The measure includes a subset of procedures performed at ASCs identified using Medicare’s list of covered ASC procedures. The measure includes “major” and “minor” procedures, as indicated by the Medicare Physician Fee Schedule GSI values of 090 and 010, respectively, and certain cystoscopy procedures, as described below. The GSI code reflects the number of post-operative days that are included in given procedure’s global surgical payment and identifies surgical procedures of greater complexity and follow-up care. This list of GSI values is publicly available at: http://www.cms.hhs.gov/Reports/downloads/pope_2000_2.pdf
We identify eligible outpatient urologic surgeries using Medicare’s ASC list of covered procedures, which includes procedures for which ASCs can be reimbursed under the ASC payment system. The January 2023 list of surgeries is publicly available at: https://www.cms.gov/Medicare/Medicare-Fee-for-Service-Payment/ASCPayment/11_Addenda_Updates.html (refer to Addendum AA). Surgeries on the ASC list of covered procedures do not involve or require major or prolonged invasion of body cavities, extensive blood loss, major blood vessels, or care that is either emergent or life-threatening. The ASC list is publicly available, is annually reviewed and updated by Medicare, and includes a transparent public comment submission and review process for addition and/or removal of procedure codes. Using an existing, defined list of surgeries, rather than defining surgeries de novo, is useful for long-term measure maintenance. Procedures listed in Medicare’s list of covered ASC procedures are defined using Healthcare Common Procedure Coding System (HCPCS) and Common Procedural Terminology (CPT) codes.
Finally, to define the urologic procedures cohort, we use the Clinical Classifications Software (CCS) developed by the Agency for Healthcare Research and Quality (AHRQ). The CCS is a tool for clustering procedures into clinically meaningful categories using CPT codes by operation site.
The measure does not, however, include procedures that are part of the AHRQ CCS’s “operations on the urinary system” and “operations on the male genital organs” groups of procedures. The measure only includes cystoscopy with therapeutic intervention and does not include cystoscopy alone or cystoscopy with biopsy alone in the measure cohort. We do not include other endoscopy procedures in the measure cohort. These excluded endoscopy procedures are lower-risk procedures often with a high volume and a low outcome rate (much like minor surgeries), are not typically performed by surgical teams, and do not require an operating room. Nephrotomy and nephrostomy procedures (defined by AHRQ clinical category CCS 103) were removed because our experts indicated that these procedures are typically performed by interventional radiologists.
Exclusions
The measure excludes surgeries for patients without 7 or more days of continuous enrollment in Medicare FFS Parts A and B after the surgery. The measure excludes these patients to ensure all patients have full data available for outcome assessment. Please see Figure 1 (in attachment) for a flow chart that diagrams the cohort construction (inclusion and exclusion).
Lack of 7 or more days of continuous enrollment in Medicare FFS after the ASC surgery is determined by patient enrollment status in FFS Parts A and B using the Medicare Enrollment Database (unless lack of enrollment was due to death). The procedure must be 7 or more days from the end of the month, or the enrollment indicators must be appropriately marked for the month that falls within 7 days of the procedure date (unless disenrollment is due to death); otherwise, the procedure is excluded.
Measure Calculation
The measure uses a two-level hierarchical logistic regression model to estimate ASC-level risk-standardized hospital visit rates (RSHVRs). This approach accounts for the clustering of patients within ASCs and variation in sample size across ASCs The RSHVR is calculated as the ratio of the predicted to the expected number of post-surgical unplanned hospital visits among an ASC’s patients, multiplied by the national observed rate of unplanned hospital visits. For each ASC, the numerator of the ratio is the number of hospital visits predicted for the ASC’s patients, accounting for its observed rate, the number and complexity of urologic procedures performed at the ASC, and the case mix. The denominator is the number of hospital visits expected nationally for the ASC’s case/procedure mix. As noted above, to calculate an ASC’s predicted-to-expected (P/E) ratio, the measure uses a two-level hierarchical logistic regression model. The log-odds of the outcome for an index procedure is modeled as a function of the patient demographics, comorbidities, procedure characteristics, and a random ASC-specific intercept. A ratio greater than one indicates that the ASC’s patients have more post-surgical hospital visits than expected, compared to an average ASC with similar patient and procedural complexity. A ratio less than one indicates that the ASC’s patients have fewer post-surgical hospital visits than expected, compared to an average ASC with similar patient and procedural complexity. An ASC’s P/E ratio is then multiplied by the overall national rate of unplanned hospital visits to calculate the ASC-level RSHVR. This approach is analogous to an observed-to-expected ratio, but accounts for within-facility correlation of the observed outcome and sample size differences, accommodates the assumption that underlying differences in quality across ASCs lead to systematic differences in outcomes, and is tailored to and appropriate for a publicly reported outcome measure as articulated in published scientific guidelines (Normand and Shahian, 2007; Krumholtz et al., 2007).
Please see the measure updates document for a detailed description of the hierarchical model and measure score calculation: https://qualitynet.cms.gov/files/651b5f9970a30f001c388001?filename=2023_ASC_MeasureUpdatesRpt.pdf
Below we provide the individual steps to calculate the measure score:
- Identify surgeries meeting the inclusion criteria described in the denominator section above, and in Tab 2, “ASC_Uro_Cohort,” of the data dictionary.
- Exclude procedures meeting any of the exclusion criteria described in the exclusion section above.
- Identify a binary flag for an unplanned hospital visit within 7 days of index procedures.
- Use patients’ historical and index procedure claims data to create risk-adjustment variables.
- Fit a hierarchical generalized linear model (HGLM) and calculate the ratio of the number of “predicted” hospital visits to the number of “expected” hospital visits for each facility, given its case/procedure mix using the results. This is the risk-standardized hospital visit ratio. The HGLM is adjusted for age, clinical risk factors, and procedure RVU and body system that vary across patient populations, are unrelated to quality, and influence the outcome. Details about the risk-adjustment model can be found in the original measure development methodology report: Facility-Level 7-Day Hospital Visits after General Surgery Procedures Performed at Ambulatory Surgical Centers under archived resources at https://qualitynet.cms.gov/asc/measures/surgery/resources#tab2
- Multiply the hospital visit ratio by the observed national 7-day hospital visit rate to obtain a risk-standardized hospital visit rate (RSHVR) for each facility.
- Use statistical bootstrapping to construct a 95% confidence interval estimate for each facility’s RSHV rate. For more information about the measure methodology, please see Appendix A in the most recent 2023 Measure Updates and Specifications Report posted here: https://qualitynet.cms.gov/files/651b5f9970a30f001c388001?filename=2023_ASC_MeasureUpdatesRpt.pdf
References
Krumholz HM, Brindis RG, Brush JE, et al. Standards for Statistical Models Used for Public Reporting of Health Outcomes An American Heart Association Scientific Statement From the Quality of Care and Outcomes Research Interdisciplinary Writing Group: Cosponsored by the Council on Epidemiology and Prevention and the Stroke Council Endorsed by the American College of Cardiology Foundation. Circulation. 2006;113(3):456-462.
Normand S-LT, Shahian DM. Statistical and clinical aspects of hospital outcomes profiling. Statistical Science. 2007;22(2):206-226.
This measure is not stratified.
Not applicable. This measure is not based on a sample.
Supplemental Attachment
Point of Contact
Not applicable
Marsha Hertzberg
Centers for Medicare & Medicaid Services (CMS)
7500 Security Boulevard
Windsor Mill , MD 21244
United States
Oscar Gonzalez
Acumen LLC
500 Airport Blvd., Suite 100
Burlingame, CA 94010
United States
Importance
Evidence
The number of Ambulatory Surgical Centers (ASCs), as well as the volume of, and payment for services has grown in recent years. According to MedPAC, in 2022, about 6,100 ASCs treated 3.3 million fee-for-service (FFS) Medicare beneficiaries (MedPAC, 2024). Among urologic surgical procedures, the endoscopic management of ureteral and kidney stones on outpatient basis is well established and widely practiced worldwide (Salciccia et al. 2021). Endoscopic management of benign prostatic hyperplasia (BPH) remains one of the most performed surgeries in urology both in an inpatient and outpatient setting. Due to advancement in technology, many endourologic procedures developed for BPH have been widely practiced as outpatient operations (Salciccia et al. 2021).
A hospital visit after outpatient surgery is unexpected, and many of the reasons for such hospital visits are preventable. In the literature, hospital visit rates following outpatient surgery vary widely, based on the type of surgery, outcome measured (admissions alone or admissions and emergency department [ED] visits), and timeframe for measurement after surgery. For example, in one study 2.5% of patients who had outpatient surgery experienced an inpatient admission within 30 days (Foley et al., 2021); in another study, ED visit rates (within 90 days) were 14.7% for outpatient surgery overall, and 47.5% for patients who had urologic surgery (Witherspoon et al., 2021); a study in Veterans Health Administration (VA) patients reported an ED visit rate within seven days of 4.4% (Mull et al., 2019). These hospital visits can occur due to a range of adverse events, including major adverse events, such as infection, post-operative bleeding, and urinary retention (Mull et al., 2018). Patients also frequently experience other less severe adverse events – for example, uncontrolled pain, nausea, and vomiting (Mull et al., 2018) – that may result in unplanned acute care visits following surgery
Interventions, before, during, and after a procedure, can improve patient outcomes after outpatient surgery. For example, pre-procedural factors, such as patient selection, and patient education/preparation, can also improve post-procedural outcomes. During the procedure, various technical aspects of the surgery influence the risk of unplanned hospital visits, including anesthetic technique (Romero et al. 2014; Whippey et al. 2013) and length of surgery (Whippey et al. 2013). Other interventions to improve post-discharge complications and utilization include pre- and post-operative patient education, protocols for prevention of complications such as nausea and vomiting, infection, and venous thromboembolism, medication management, pain management, and availability of a responsible caregiver (Cukierman et al., 2023; Becker et al., 2021). Please also see the section on “Usability” for more information about post-procedural interventions.
Additionally, there have been efforts to enable ambulatory surgery center providers to systematically address issues of complications of surgical care and communication between providers of adverse events when they occur (Agency for Healthcare Research and Quality 2013; The Health Research & Educational Trust of the American Hospital Association 2017). For example, the Agency for Healthcare Research and Quality (AHRQ) developed a quality improvement collaborative for the 65 ambulatory surgery facilities in 47 states to reduce healthcare-associated infections and surgical harms in ambulatory surgery centers through 1) the use of a surgical safety checklist curriculum, and 2) improved safety culture through teamwork and communication [The Health Research & Educational Trust of the American Hospital Association 2017). Ambulatory surgery center providers involved in the collaborative concluded that efforts to increase the availability of meaningful data would be beneficial to the accurate assessment of outcomes in the ambulatory surgery center setting, reduce unplanned hospital admissions, and would facilitate ability to follow patients after discharge.
References
Agency for Healthcare Research and Quality. Proactive Risk Assessment of Surgical Site Infection in Ambulatory Surgery Centers: Final Contract Report. Chapter 3: Risk-Informed Interventions. April 2013.
Becker, C., Zumbrunn, S., Beck, K., Vincent, A., Loretz, N., Müller, J., Amacher, S. A., Schaefert, R., & Hunziker, S. (2021). Interventions to Improve Communication at Hospital Discharge and Rates of Readmission: A Systematic Review and Meta-analysis. JAMA network open, 4(8), e2119346. https://doi.org/10.1001/jamanetworkopen.2021.19346
Cukierman, D. S., Cata, J. P., & Gan, T. J. (2023). Enhanced recovery protocols for ambulatory surgery. Best practice & research. Clinical anaesthesiology, 37(3), 285–303. https://doi.org/10.1016/j.bpa.2023.04.007
Foley, C., Kendall, M. C., Apruzzese, P., & De Oliveira, G. S. (2021). American Society of Anesthesiologists Physical Status Classification as a reliable predictor of postoperative medical complications and mortality following ambulatory surgery: an analysis of 2,089,830 ACS-NSQIP outpatient cases. BMC surgery, 21(1), 253. https://doi.org/10.1186/s12893-021-01256-6
Medicare Payment Advisory Commission (MedPAC). Report to Congress: Medicare Payment Policy. March 2024; https://www.medpac.gov/wp-content/uploads/2024/03/Mar24_MedPAC_Report_To_Congress_SEC.pdf
Mull, H. J., Itani, K. M. F., Charns, M. P., Pizer, S. D., Rivard, P. E., Hawn, M. T., & Rosen, A. K. (2018). The Nature and Severity of Adverse Events in Select Outpatient Surgical Procedures in the Veterans Health Administration. Quality management in health care, 27(3), 136–144. https://doi.org/10.1097/QMH.0000000000000177
Mull, H. J., Rosen, A. K., Charns, M. P., OʼBrien, W. J., Hawn, M. T., Itani, K. M. F., & Pizer, S. D. (2019). Emergency Department Use After Outpatient Surgery Among Dually Enrolled VA and Medicare Patients. Quality management in health care, 28(4), 191–199. https://doi.org/10.1097/QMH.0000000000000225
Romero A, Joshi GP. Adult Patient for Ambulatory Surgery: Are There Any Limits? ASA Newsletter. 2014; 78(9): 18-20.
Salciccia S, Del Giudice F, Maggi M, Eisenberg ML, Chung BI, Conti SL, Kasman AM, Vilson FL, Ferro M, Lucarelli G, Viscuso P, Di Pierro G, Busetto GM, Luzi M, Sperduti I, Ricciuti GP, De Berardinis E, Sciarra A. Safety and Feasibility of Outpatient Surgery in Benign Prostatic Hyperplasia: a Systematic Review and Meta-Analysis. J Endourol. 2021 Apr; 35(4): 395-408. doi: 10.1089/end.2020.0538. Epub 2021 Mar 4. PMID: 33081521.
The Health Research & Educational Trust of the American Hospital Association. AHRQ Safety Program for Ambulatory Surgery, May 2017. Available at: https://www.ahrq.gov/sites/default/files/wysiwyg/professionals/quality-patient-safety/hais/tools/ambulatory-surgery/sections/ambulatory-surgery-report.pdf. Accessed June 23, 2017.
Whippey A, Kostandoff G, Paul J, Ma J, Thabane L, Ma HK. Predictors of unanticipated admission following ambulatory surgery: a retrospective case-control study. Can J Anaesth. 2013; 60(7): 675-683.
Witherspoon, L., Breau, R. H., Langley, C., Mallick, R., Cagiannos, I., Morash, C., Oake, J. S., & Lavallée, L. T. (2021). Returning to the emergency room: An analysis of emergency encounters following urological outpatient surgery. Canadian Urological Association journal = Journal de l'Association des urologues du Canada, 15(10), 333–338. https://doi.org/10.5489/cuaj.7063
Measure Impact
Improving the quality of care provided at ASCs is a key priority in the context of growth in the number of ASCs and procedures performed in this setting. Ambulatory care now accounts for the majority of elective surgical care in the United States. From both the patient and provider perspective, several factors have fueled this growth. ASCs provide greater convenience for patients and families (due to advances in surgical technology), as well as lower financial costs for both patients and healthcare providers. As such, as more complex procedures migrate to the ASC setting, and ASCs serve and higher-risk patients there is an increased need to monitor the quality of care at ASCs (Munich et al., 2014; Goldfarb et al., 2017).
A hospital visit following same-day surgery is an unexpected and potentially preventable outcome for patients scheduled for elective same-day surgeries that have a low anticipated risk. As ASC providers may be unaware of their patients’ hospital visits after surgery, they may underestimate adverse event rates, suggesting the need for better measurement to drive quality improvement. Therefore, both patients and providers benefit from outcome measures of hospital visits – a broad, patient-centered outcome that reflects the full range of reasons leading to hospitalization among patients undergoing same-day surgery.
Patients express concern about post-operative complications, in particular, pain, and say that they want to be informed about potential complications prior to the procedure. For example, in comments submitted to researchers following a survey of 800 patients on the topic of post-operative complications (Rendell et al., 2020), patients stated: “Make sure patients know all possible complications before their surgery,” and patients said they wanted to know what could any patients wanted happen to them after an operation; for example one patient wrote, “I wish someone had warned me of this [complication] before the surgery as it is quite common I now know.” Public reporting of post-operative hospital visit rates following urology procedures (described below) is an act of transparency that can help patients better understand those post-operative risks.
The ASC Urology measure is part of the Ambulatory Surgical Center Quality Reporting (ASCQR) Program, a pay-for-reporting program. The measure went through a “dry run” in 2018 to help educate measure entities on data collection and interpretation, familiarize them with facility-specific reports (FSRs), and for CMS to collect feedback on the measure specifications. This measure was first publicly reported January 2022. Currently, there are no other publicly available quality reports related to these elective procedures, which underscores the measurement gap that would exist without this measure. Thus, this measure addresses an important quality measurement area and enhances the information available to patients choosing among ASCs that provide same-day outpatient surgery. Furthermore, providing outcome rates to ASCs makes visible to clinicians and hospitals meaningful quality differences and incentivizes improvement.
References
Goldfarb CA, Bansal A, Brophy RH. Ambulatory surgical centers: a review of complications and adverse events. JAAOS-Journal of the American Academy of Orthopaedic Surgeons. 2017 Jan 1;25(1):12-22.
Munnich EL, Parente ST. Procedures take less time at ambulatory surgery centers, keeping costs down and ability to meet demand up. Health Affairs. 2014 May 1;33(5):764-9.
Rendell, V. R., Siy, A. B., Stafford, L. M. C., Schmocker, R. K., Leverson, G. E., & Winslow, E. R. (2020). Severity of postoperative complications from the perspective of the patient. Journal of Patient Experience, 7(6), 1568–1576. https://doi.org/10.1177/2374373519893199
Performance Gap
There remains variation in measure scores among facilities. Below we provide the distribution of measures scores, and evidence of variation through the median odds ratio.
Measure score distribution
Table 1 and Table 2 (see attachment) show the distribution of measure scores, the risk-standardized hospital visit rate, or RSHVR, using the most recent testing data (2024 EM Dataset; January 1, 2021-December 31, 2022). We note that the table provided by Battelle is not editable; the last row for this measure is the number of procedures, not the number of patients.
RSHVRs range from 2.94% to 8.85%; the median is 5.10%; the 25th percentile is 4.91% and the 75th percentile is 5.35%. The best performer (2.94%) is performing about 42 percent better than the median, and the worst performer (8.85%) is performing about 74 percent worse than the median.
Median Odds Ratio
We provide further evidence of variation by calculating and interpreting the median odds ratio (MOR) (Merlo, et al., 2006). The MOR represents the median increase in odds of a hospital visit if a procedure on a single patient was performed at a higher risk facility compared to a lower risk facility. It is calculated by taking all possible combinations of facilities, always comparing the higher risk facility to the lower risk facility. The MOR is interpreted as a traditional odds ratio would be.
The median odds ratio for the ASC Urology median odds ratio suggests a meaningful increase in the risk of a hospital visit if a urology procedure was performed at a higher risk facility compared to a lower risk facility. A value of 1.27 indicates that a patient has a 27 percent increase in the odds of a hospital visit if the procedure was performed at higher risk facility compared to a lower risk facility indicating the impact of quality on the outcome rate is meaningful.
References
Merlo J, Chaix B, Ohlsson H, Beckman A, Johnell K, Hjerpe P, Råstam L, Larsen K. (2006) A brief conceptual tutorial of multilevel analysis in social epidemiology: Using measures of clustering in multilevel logistic regression to investigate contextual phenomena. J Epidemiol Community Health, 60(4):290-7.
| Overall | Minimum | Decile_1 | Decile_2 | Decile_3 | Decile_4 | Decile_5 | Decile_6 | Decile_7 | Decile_8 | Decile_9 | Decile_10 | Maximum | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Mean Performance Score | 5.16 | 2.94 | 4.26 | 4.72 | 4.91 | 5.03 | 5.08 | 5.11 | 5.16 | 5.35 | 5.60 | 6.35 | 8.85 |
| N of Entities | 1,086 | 1 | 108 | 109 | 109 | 108 | 109 | 109 | 108 | 108 | 109 | 108 | 1 |
| N of Persons / Encounters / Episodes | 157,758 | 844 | 44,683 | 20,969 | 10,025 | 7,386 | 3,897 | 947 | 10,593 | 12,135 | 14,985 | 32,138 | 163 |
Equity
Equity
As shown above, and described in the conceptual model, ASCs do not serve a high proportion of patients with social risk factors. The median facility proportion of patients with the DE variable for the ASC Urology measure is 0% (count of 0 patients); and for high ADI, 1% (count of 1 patient). Therefore, we would expect, and show below, that adjusting this measure for social risk factors would have minimal impact on measure scores. Furthermore, when CMS attempted to stratify ASC measures for patients with DE, there were insufficient numbers of patients to support the stratification approach.
Nonetheless, we performed two analyses to explore the impact of adding either of two social risk factors (DE, and ADI) to the risk model, on measure scores. We found that adding either social risk factor to the model did not result in major impacts on measure scores, even for providers with the highest proportion of patients with either social risk factor (the top quartile, in our analyses). Therefore, in this pay-for-reporting program, providers will not be unfairly profiled when assessed this measure. We describe the analyses and results below.
To examine the impact of social risk factors on measure scores, we first examined correlations (Pearsons) between measure scores calculated with and without either social risk factor in the risk model, and found that correlations were 1.00, and 0.999, for the DE and high ADI variables, respectively (see Figure 5 and 6 in attachment). Second, we examined the association (Spearman) between the facility proportion of patients with each social risk factor and measure scores for all facilities (see Figures 7 and 8 in attachment) focusing on the quartile of facilities with the highest proportion of patients with social risk factors (top quartile). We found that there is no significant correlation between the measures score and the proportion of patients with the DE or ADI variables. Taken together this suggests there is no reason to adjust for social risk factors for the ASC Urology measure.
Feasibility
Feasibility
This is a claims-based measure, and the measure score is calculated automatically from claims data which are routinely generated during the delivery of care. No data are collected by facilities; therefore, this measure imposes no burden on measured entities, and no implementation effort. CMS monitors feedback from the public and measured entities through CMS’s Q&A portal on QualityNet; there have been no concerns about burden related to this measure. There are no concerns about patient confidentiality because the measure is based on claims data submitted by facilities to CMS, and CMS then uses that data for both payment and calculation of the measure score.
We did not perform an analysis of missing data for the measure because it is based on a 100% sample of paid, final action claims submitted by facilities for payment. To ensure complete claims, we allow at least 3 months of time between accessing the data and the end of the performance period.
Because this is a claims-based measure there is no burden on the facility; rates are automatically calculated by CMS based on claims data submitted by facilities for payment.
Proprietary Information
Scientific Acceptability
Testing Data
We used Medicare Fee-For-Service claims data and administrative data. Dates of data vary by testing. See Section 4.1.2 Differences in Data for details.
The measure requires a data source that allows us to link patient data across care settings in order to identify appropriate surgical procedures for inclusion, comorbidities for risk adjustment, and the outcome of hospital visits. Therefore, we used claims data, as they support these linkages and were available for the population of interest.
The datasets, dates, number of measured entities, number of urology procedures, and demographic profile for the patients used in each type of testing are indicated below:
1. 2015 Medicare Dataset: Full Dataset, and Development Sample and Validation Sample (Measure Development)
The full 2015 dataset was used for measure development, including the selection of risk variables, and for the measure score results on which the face validity results were based.
The Development and Validation Samples were derived by selecting two random samples from the Medicare FFS FY 2015 Dataset. The Development Sample included 70% of the urology ASC procedures in the Medicare FFS FY 2015 Dataset, and the Validation Sample included 30% of the urology ASC procedures in the Medicare FFS FY 2015 Dataset
- Development Sample
- Dates: October 1, 2014 – September 30, 2015
- Number of facilities: 1,017 ASCs
- Number of urology procedures: 45,619
- Demographic characteristics: average age of 75.52 years; 31% female
- Dataset used for: Used during initial measure development; for this submission this data was used for model assessment of overfitting.
- Validation Sample
- Dates: October 1, 2014 – September 30, 2015
- Number of facilities: 905 ASCs
- Number of urology procedures: 19,550
- Demographic characteristics: average age of 75.53 years; 31% female
- Dataset used for: Used during initial measure development; for this submission this data was used for model assessment of overfitting.
2. 2024 Endorsement Maintenance Dataset
- Total number of facilities (with at least one procedure): 1,086 (100%)
- Total number of patients 157,758 (100%)
- Male 101,282 (64.2%)
- Female L 56,476 (35.8%)
- Dual eligible: 3,321 (2.1%)
- High ADI: 11.465 (7.3%)
- Number of facilities with at least 35 procedures: 581 (56.3%)
- Number of patients (within facilities >=35 procedures): 152,863 (97.5%)
See Section 4.1.2 Differences in Data for details., above, for information on measured entities, which differ based on dates of data.
See Section 4.1.2 Differences in Data for details., above, for information for characteristics of the eligible population, which vary based on dates of data.
Reliability
We tested facility-level measure score reliability using the signal-to-noise method, using the formula presented by Adams and colleagues (Yu et a., 2013; Adams et al., 2010). Specifically, for each facility we calculate the reliability as:
Reliability=(σ_(facility-to-facility)^2)/(σ_(facility-to-facility)^2+ (σ_(facility error variance)^2)/n)
Where facility-to-facility variance is estimated from the hierarchical logistic regression model, n is equal to each facility’s observed case size, and the facility error variance is estimated using the variance of the logistic distribution (pi^2/3).
Signal-to-noise reliability scores can range from 0 to 1. A reliability of zero implies that all the variability in a measure is attributable to measurement error. A reliability of one implies that all the variability is attributable to real difference in performance.
We calculated the measure score reliability for all facilities, and for facilities with a volume cutoff of 35 procedures. Our rationale for this is described below.
In general, CMS sets the volume cutoff for publicly reporting facility measures scores based on two considerations. CMS considers the empiric results of reliability testing conducted on the dataset used for public reporting. CMS also considers the volume cutoff for score reporting used for related measures (for example, Facility 7-Day Risk-Standardized Hospital Visit Rate after Outpatient Colonoscopy, or CBE #3366, Hospital Visits after Orthopedic Ambulatory Surgical Center Procedures, also under CBE review in this cycle). Regardless of the score reporting volume cutoff, all facilities and their cases are used in calculating the measure scores. The minimum sample size for public reporting is a policy choice that balances considerations such as the facility-level reliability testing results on the reporting data and consistency across measures for consumers.
References
Adams J, Mehrota, A, Thoman J, McGlynn, E. (2010). Physician cost profiling – reliability and risk of misclassification. NEJM, 362(11): 1014-1021.
Yu, H, Mehrota, A, Adams J. (2013). Reliability of utilization measures for primary care physician profiling. Healthcare, 1, 22-29.
Measure score reliability (signal-to-noise reliability) is shown in Table 3 (see attachment) for all facilities and facilities with at least 35 cases, and in Table 4 (see attachment) for facilities with at least 35 cases, the public reporting threshold described previously. For facilities with at least 35 procedures, signal-to-noise reliability ranged from 0.402-0.984 with a median of 0.720.
Using two years of performance data, the median facility-level reliability score is 0.720 (IQR, 0.573 - 0.849) for ASCs with at least 35 procedures (the public reporting threshold) representing moderate reliability.
At the current public reporting threshold, most facilities (about 75 percent of facilities with at least 35 cases) fall above the 0.6 minimum threshold stated in Battelle’s current CBE guidebook. If CMS were to increase the case volume minimum so that all facilities exceeded this threshold, it would remove publicly available information from about 25 percent of facilities that are currently publicly reported.
We believe that median reliability of 0.6 (signal to noise) is sufficiently high for a facility-level publicly reported measure in a pay for reporting program. Increasing the minimum case volume has the tradeoff of removing more than half of facilities from reporting to the public.
| | Overall | Minimum | Decile_1 | Decile_2 | Decile_3 | Decile_4 | Decile_5 | Decile_6 | Decile_7 | Decile_8 | Decile_9 | Decile_10 | Maximum |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Reliability | 0.706 | 0.402 | 0.422 | 0.486 | 0.568 | 0.637 | 0.698 | 0.749 | 0.798 | 0.846 | 0.903 | 0.949 | 0.984 |
| Mean Performance Score | 5.19 | 5.11 | 5.31 | 5.26 | 5.27 | 5.15 | 5.08 | 5.22 | 5.29 | 5.08 | 5.18 | 5.04 | 6.81 |
| N of Entities | 581 | 6 | 56 | 58 | 60 | 58 | 59 | 57 | 59 | 58 | 58 | 58 | 1 |
| N of Persons / Encounters / Episodes | 152,863 | 210 | 2,134 | 2,874 | 4,128 | 5,335 | 7,113 | 8,892 | 12,223 | 16,823 | 28,853 | 64,488 | 3,107 |
Validity
For this endorsement maintenance submission, we provide evidence of validity through three approaches: face validity conducted during measure development, validation of the outcome, and empiric validity through associations with volume. Below we describe the methodology; the results are provided in the subsequent section.
Face validity
As part of the measure development, we systematically assessed the face validity of the measure score as an indicator of quality by confidentially soliciting the TEP members’ agreement with the following statement via an online survey following the final TEP meeting: “The risk-standardized hospital visit rates obtained from the urology ASC measure as specified can be used to distinguish between better and worse quality facilities.” The survey offered participants six response options ranging from “strongly disagree” to “strongly agree.”
List of TEP Members:
- Robin Blomberg, BA, MA – National Forum of End-Stage Renal Disease, Network 16 (Representative for Kidney Patient Advisory Council); Seattle, WA
- Kirk Campbell, MD – New York University Hospital for Joint Diseases (Clinical Assistant Professor of Orthopedic Surgery); New York, NY
- Gary Culbertson, MD, FACS – Iris Surgery Center (Surgeon; Medical Director); Sumter, SC
- Martha Deed, PhD – Consumers Union Safe Patient Project (Patient Safety Advocate); Austin, TX
- James Dupree, MD, MPH – University of Michigan (Urologist; Health Services Researcher); Ann Arbor, MI
- Nester Esnaola, MD, MPH, MBA – Fox Chase Cancer Center (Professor of Surgery; Associate Director for Cancer Health Disparities and Community Engagement); Philadelphia, PA
- John Gore, MD, MS – University of Washington (Associate Professor of Urology); Seattle, WA
- Lisa Ishii, MD, MHS – Johns Hopkins School of Medicine (Associate Professor); American Academy of Otolaryngology-Head and Neck Surgery (Coordinator for Research and Quality); Baltimore, MD; Alexandria, VA
- Atul Kamath, MD – Perelman School of Medicine, University of Pennsylvania (Assistant Professor and Clinical Educator Director of Orthopedic Surgery); Hospital of the University of Pennsylvania (Attending Surgeon); Philadelphia, PA
- Tricia Meyer, PharmD, MS, FASHP – Scott & White Medical Center (Regional Director of Pharmacy); Texas A&M University College of Medicine (Associate Professor of Anesthesiology); Temple, TX
- Linda Radach, BA – Consumers Union Safe Patient Project (Patient Safety Advocate); Austin, TX
- Amita Rastogi, MD, MHA, CHE, MS – Health Care Incentives Improvement Institute (Chief Medical Officer); Newtown, CT
- Donna Slosburg, RN, BSN, LHRM, CASC – ASC Quality Collaboration (Executive Director); St. Pete Beach, FL
- Thomas Tsai, MD, MPH – Brigham and Women’s Hospital (General Surgeon); Harvard School of Public Health (Research Associate); Boston, MA
- Katherine Wilson, RN, BA, MHA – AMSURG Corp (Vice President of Quality); Nashville, TN – She was not polled for face validity as she participated in early measure development.
Validation of the Outcome
The outcome of an unplanned hospital visit following an outpatient elective urologic surgery is intended to capture adverse events that occur as part of the care received before, during, and after the procedure. To validate the outcome, we identified the most commonly occurring principal discharge diagnosis codes associated with the post-procedure hospital visit. (For any hospitalization captured as part of the outcome, a claim for the hospital visit is submitted to CMS that indicates the main reason for the hospitalization; there is only one such main reason, called the “principal diagnosis code” that is used to capture this information.) Based on previous research to validate the outcome during measure development, we know that the most frequent reasons for a hospital visit following elective urology procedures are complications from the procedure. We updated this analysis for this endorsement maintenance submission.
External Empiric Validity
One approach for assessing the validity of a quality measure is to show that performance on the test measure is associated with another quality measure in the same causal pathway. To do this, we needed to identify a comparator measure, however, as summarized below, we did not identify a suitable measure with currently publicly available data.
We first considered CMS’s two related CBE-endorsed measures, Facility-Level 7-Day Hospital Visits after General Surgery Procedures Performed at ASCs (ASC General Surgery), and the ASC Orthopedic Surgery measure. The outcome of both measures is nearly identical to that of the ASC Urology measure; an unplanned hospital visit is defined as an emergency department (ED) visit, observation stay, or unplanned inpatient admission. Hence, the measures target the same quality domains. The patient cohort is also somewhat similar in that the measures target Medicare Fee-For-Service (FFS) patients aged 65 years and older. The cohorts (in terms or procedures included in the measure) for the ASC General Surgery and ASC Orthopedic Surgery measures do not, however, overlap with the ASC Urology measure. Furthermore, the clinicians performing the procedures across the different cohorts are unlikely to be the same individuals, and in addition, most ASCs (about 70% as of 2022) subspecialize (MedPAC, 2024).
Association with volume
Studies have shown that there is a volume outcome relationship between volume and some outcomes for some, but not all, surgical procedures (Levaillant et al., 2021); a recent study examining surgeries specifically performed at ASCs found a volume-outcome relationship in this setting as well but while study authors did compare orthopedic procedures with all procedures, they did not separately assess urologic procedures (Jain et al., 2024). However, based on the overall observation of a volume/outcome relationship we assessed if higher facility volume was associated with the ASC Urology measure score. We hypothesized that there would be a weak to moderate, negative relationship between facility procedural volume and ASC Urology measure scores, with higher procedural volumes being associated with better (lower) measure scores. We limited this analysis to facilities with at least 200 procedures based on literature suggesting this relationship would be evident at higher volumes (Kugler et al., 2022).
References
Kugler, C. M., Goossen, K., Rombey, T., De Santis, K. K., Mathes, T., Breuing, J., Hess, S., Burchard, R., & Pieper, D. (2022). Hospital volume–outcome relationship in total knee arthroplasty: A systematic review and dose–response meta-analysis. Knee Surgery, Sports Traumatology, Arthroscopy, 30(8), 2862–2877. https://doi.org/10.1007/s00167-021-06692-8
Jain, S., Rosenbaum, P. R., Reiter, J. G., Ramadan, O. I., Hill, A. S., Silber, J. H., & Fleisher, L. A. (2024). Assessing the Ambulatory Surgery Center Volume-Outcome Association. JAMA surgery, 159(4), 397–403. https://doi.org/10.1001/jamasurg.2023.7161
Levaillant, M., Marcilly, R., Levaillant, L., et al. (2021). Assessing the hospital volume-outcome relationship in surgery: A scoping review. BMC Medical Research Methodology, 21(204). https://doi.org/10.1186/s12874-021-01396-6
MedPAC’s Report to Congress, Chapter 5, Ambulatory Surgical Center Services, March 2024. https://www.medpac.gov/wpcontent/uploads/2024/03/Mar24_MedPAC_Report_To_Congress_SEC.pdf
Face Validity as Determined by the Technical Expert Panel (TEP)
Validity was assessed by the TEP. The TEP provided input on the cohort, risk model, and outcome to strengthen the measure; the TEP supported the final measure specifications with high agreement. A total of 14 TEP members completed the face validity survey.
Of the 14 respondents, 12 respondents (86%) indicated that they somewhat, moderately, or strongly agreed; and 2 respondents moderately disagreed.
Among respondents who “strongly agreed” with the validity statement even though there were relatively few outliers, one noted that there is room for improvement among average performing facilities. Another TEP member said that the measure demonstrated an appropriate rate of effectiveness/variation to identify performance across ASCs.
Validation of the Outcome
Table 5 (see attachment) shows the most frequent (the top 25) principal diagnosis codes associated with the outcome (unplanned hospital visits within 7 days of a qualifying procedure); many of these codes indicate a complication from the procedure. For example, the top four principal diagnoses are “retention of urine, unspecified,” “urinary tract infection, site not specified,” “sepsis, unspecified organism,” and “Hydronephrosis with renal and ureteral calculous obstruction.” Many of the complications in Table 5 are preventable, and/or could be managed in a non-acute care setting.
External Empiric Validity
As noted above, we did not identify any measures that were suitable for comparison with the ASC Urology measure, therefore we examined the association of the measure score with facility volume. We found no association between facility volume and measure scores when examining all facilities with a measure score; however when we limited the analysis to facilities that specialize in urologic procedures (those that had a measure score for the Urology hospital visit measure, but not the Orthopedic, General Surgery, or Colonoscopy measures) we found a moderate, significant association (r=-.301, p<.05, n=78).
Our results, shown above, demonstrate validity of the ASC Urology measure at several levels: face validity as assessed by experts (with 86 percent agreement); validity of the outcome, shown by analysis of ICD-10 codes associated with the outcome, and validity of the measure score as shown by association (in the expected direction) with procedural volume, however this was limited to ASCs that specialized in urology.
Risk Adjustment
Our approach to risk adjustment is tailored to, and appropriate for, a publicly reported outcome measure as articulated in published scientific guidelines (Krumholz et al., 2006; Normand & Shahian, 2007). For example, we only adjust for risk factors that are present at the start of care. We do not risk adjust for conditions that are possible adverse events of care and that are only recorded at the time of the surgery (see Data Dictionary, Tab 5, “ASC_Uro_CoC CCs”). We do not adjust for factors related to the delivery of care that may reflect care quality.
The measure employs a hierarchical logistic regression model (a form of hierarchical generalized linear model [HGLM]) to create an ASC-level 7-day RSHVR. This approach to modeling appropriately accounts for the structure of the data (patients clustered within facilities), the underlying risk due to patients’ procedures/comorbidities, and sample size at a given ASC when estimating hospital visit rates. In brief, the approach simultaneously models two levels (patient and facility) to account for the variance in patient outcomes within and between facilities (Normand & Shahian, 2007). At the patient level, the model adjusts the log-odds of hospital visits within 7 days after the procedure for selected demographic, clinical, and procedure risk variables. The second level models the facility-specific intercepts as arising from a normal distribution. The facility intercept, or facility-specific effect, represents the ASC contribution to the risk of 7-day hospital visits, after accounting for patient risk and sample size, and can be inferred as a measure of quality. If there were no differences among ASCs, then after adjusting for patient risk, the facility intercepts would be identical across all ASCs.
Candidate Risk-Adjustment Variables:
The measure adjusts for differences in patient comorbidities, demographics, and in procedure-related differences in risk across ASCs. We identified potential candidate risk factors through: 1) prior work on related quality measures (including the related orthopedic ASC measure); 2) a focused literature review; and 3) TEP and expert input.
To define the candidate risk factors, we defined the clinical risk factors in claims data using Version 22 of the CCs from CMS’s Hierarchical Condition Categories (HCC) grouper, which classifies thousands of ICD-10 diagnosis codes into clinically coherent and mutually exclusive groups of codes, or condition categories. In some cases (for example, morbid obesity), individual codes were used to define the risk factor. The measure does not apply the hierarchical logic of the HCC. Based on prior validation work conducted for similar measures, we have confidence that model variables defined using the CCs are reasonable proxies for clinical conditions. Specifically, as discussed in the response to the next question, CMS has validated similar risk-adjustment models that use the CCs against models that use chart-abstracted data for risk adjustment.
To address surgical procedural complexity, we used the work RVUs of the procedure, an approach employed by the American College of Surgeons National Surgical Quality Improvement Program.
We reviewed the candidate risk factors with TEP members and clinical consultants. None of the clinical experts suggested removing any of the candidate risk factors from the list. Several TEP members suggested that we consider additional risk adjustment for procedural complexity, beyond work RVU. One TEP member suggested we consider risk adjusting for benign prostatic hyperplasia, nocturia, urinary frequency, use of alpha blockers, and anesthetic type. We reviewed the suggested risk factors and added benign prostatic hyperplasia, nocturia, and urinary frequency. We were not able to include alpha blockers because we do not have data on patient-level medication use, and we did not include anesthetic type because we do not risk adjust for discretionary procedure differences (such as approach to anesthesia or surgical techniques).
Finally, to consolidate similar risk factors, we checked the bivariate direction and strength of association of the individual risk factors defined by CCs or ICD-10 codes and then combined risk factor diagnoses into clinically coherent comorbidity variables. For example, a “cancer” variable was created that combined several individual cancer diagnoses.
Variable Selection
To select the final set of variables to include in the risk-adjustment model, we performed a bootstrap selection method. Briefly, 1,000 samples were selected with replacement from the Development Sample dataset. For each of the 1,000 samples, a parsimonious logistic regression model was selected by iteratively removing non-significant candidate variables from the model using a stepwise purposeful selection approach described by Hosmer and Lemeshow [5]. Our goal was to minimize the number of variables in the model while preserving model performance (as measured by the c-statistic). All variables significant at p<0.05 were retained in the final model. This approach led to 1,000 models from which we then selected all variables that entered the model at least 70% of the time for our final model. This allowed us to select variables that reliably and consistently enter the model across the 1,000 bootstrap samples, and avoid spurious relationships that may occur due to low volume and event rate.
Social Risk Factors for Supplemental Disparities Analyses
We selected variables representing social risk factors based on a review of literature, conceptual pathways, and feasibility. Below, we describe the pathways by which social risk factors may influence risk of hospital visits following outpatient surgical procedures. Our conceptualization of the pathways by which social risk factors may affect the outcome is informed by the literature and IMPACT Act–funded work by the National Academy of Science, Engineering and Medicine (NASEM) and the Department of Health and Human Services Assistant Secretary for Policy and Evaluation (ASPE) (Department of Health and Human Services, 2016; National Academies of Sciences, Engineering, and Medicine, 2016a; National Academies of Sciences, Engineering, and Medicine, 2016b).
Literature Review: Social Risk Variables and Ambulatory Surgery Post-Procedure Hospital Visits
To examine the relationship between social risk factors and risk of hospital visits following outpatient surgical procedures, a literature search was performed during measure development A total of 176 studies were reviewed by title and abstract, and all but two studies were excluded from full-text review based on the above criteria. The two studies indicated that Black and Hispanic patients and patients from lower-income households were at increased risk of post-procedure hospital visits in the ambulatory surgery setting (Bhattacharyya, 2015; Menachemi et al., 2007). No studies were found that suggested that variation in patients’ social risk affected variation in outcome risk across facilities performing ambulatory surgical procedures. A 2024 update to the literature search (going back 5 years) did not identify any new studies that focused on ASCs or urology procedures, but we find additional evidence for the impact of social risk factors on outcomes for patients undergoing surgery in general. For example, a 2019 study found that while patients with low income undergoing colectomy had higher rates of surgical-site infections compared with higher-income patients, there was no difference in surgical-site infection rates based on income for patients undergoing hysterectomy (Qi et al., 2019). A 2023 study in cancer patients undergoing surgery found that patients with psychosocial risk factors were more likely to experience complications following surgery (Leeds et al., 2019). Finally, a 2021 study found that for some procedures, people living in counties with high social vulnerability (SVI) were more likely to experience complications compared with patients who live in low SVI counties (Diaz et al., 2021).
We note that compared to the patient mix for elective outpatient procedures at hospital outpatient departments (HOPDs), ASCs serve a low proportion of patients with social risk factors therefore there are disparities in access to care at ASCs, with Black patients and patients with public insurance being less likely to receive care at an ASC compared with others without those social risk factors (Janeway et al., 2020).. A recent study found that there are disparities in the geographic distribution of ASCs, with counties with higher socioeconomic status having more ASCs per capita compared with counties with lower socioeconomic status (Chatterjee, Amen, & Khormaee, 2022).
Conceptual Pathways for Social Risk Factors
Although there is limited literature linking social risk factors and adverse outcomes for ambulatory surgery, potential pathways may include:
- Differential care within an ASC. One pathway by which social risk factors may contribute to hospital visit risk is that patients may not receive equivalent care within a facility. Alternatively, patients with social risk factors, such as lower education, may require differentiated care – e.g., provision of information at a lower health literacy level – that they do not receive.
- Use of lower-quality facilities. Patients of lower income, lower education, or unstable housing may not have equitable access to high-quality facilities because such facilities are less likely to be found in geographic areas with large populations of poor patients; thus, patients with low income may be more likely to be seen in lower-quality facilities, which can contribute to increased risk of post-procedure adverse outcomes (Jha, Orav, & Epstein, 2011; Trivedi et al., 2014). Similarly, Black patients have been shown to have less access to high-quality facilities compared with white patients (Reames et al., 2014). As described above, patients with social risk factors are less likely to have access to care at ASCs, in general (Chatterjee, Amen, & Khormaee, 2022; Janeway et al., 2020).
- Influence of social risk factors on hospital visit risk outside of ASC quality. Some social risk factors, such as income or wealth, may affect the likelihood of post-procedure hospital visits without directly being associated with the quality of care received at the ASC. For instance, while an ASC may make appropriate care decisions and provide tailored care and education, a lower-income patient may have a worse outcome post-procedure due to competing economic priorities or a lack of access to care outside of the facility.
We developed and used the conceptual framework described below to identify potential social risk factors. We analyzed two well-studied social risk factors that could best be operationalized in data, outlined below. We note that this measure already adjusts for age.
- Medicare-Medicaid dual-eligibility status
Dual eligibility (DE) for Medicare and Medicaid is available at the patient level in the Medicare Master Beneficiary Summary File. The eligibility threshold for over 65-year-old Medicare patients considers both income and assets. For the dual-eligible indicator, there is a body of literature demonstrating differential health care and health outcomes among beneficiaries, therefore the DE indicator allows us to examine some of the pathways of interest (Department of Health and Human Services, 2016).
- Area Deprivation index (ADI)
The ADI, initially developed by Health Resources & Services Administration (HRSA), is based 17 measures across four domains: income, education, employment, and housing quality (Kind et al., 2018; Singh, 2003).
The 17 components are listed below:
- Population aged ≥ 25 y with < 9 y of education, %
- Population aged ≥ 25 y with at least a high school diploma, %
- Employed persons aged ≥ 16 y in white-collar occupations, %
- Median family income, $
- Income disparity
- Median home value, $
- Median gross rent, $
- Median monthly mortgage, $
- Owner-occupied housing units, % (home ownership rate)
- Civilian labor force population aged ≥16 y unemployed, % (unemployment rate)
- Families below poverty level, %
- Population below 150% of the poverty threshold, %
- Single-parent households with children aged < 18 y, %
- Households without a motor vehicle, %
- Households without a telephone, %
- Occupied housing units without complete plumbing, % (log)
- Households with more than 1 person per room, % (crowding)
ADI scores were derived using beneficiary’s 9-digit ZIP Code of residence, which is obtained from the Master Beneficiary Summary File, and is linked to 2017-2021 US Census/American Community Survey (ACS) data. In accordance with the ADI developers’ methodology, an ADI score is calculated for the census block group corresponding to the beneficiary’s 9-digit ZIP Code using 17 weighted Census indicators. Raw ADI scores were then transformed into a national percentile ranking ranging from 1 to 100, with lower scores indicating lower levels of disadvantage and higher scores indicating higher levels of disadvantage. Percentile thresholds established by the ADI developers were then applied to ADI percentile to dichotomize neighborhoods into more disadvantaged (high ADI areas=ranking equal to or greater than 85) or less disadvantaged areas (Low ADI areas= ranking of less than 85).
References
Bhattacharyya, N. (2015). Healthcare disparities in revisits for complications after adult tonsillectomy. American Journal of Otolaryngology, 36(2), 249-253.
Chatterjee, A., Amen, T. B., & Khormaee, S. (2022). Trends in geographic disparities in access to ambulatory surgery centers in New York, 2010 to 2018. JAMA Health Forum, 3(10), Article e223608. https://doi.org/10.1001/jamahealthforum.2022.3608
Department of Health and Human Services, Office of the Assistant Secretary of Planning and Evaluation. (2016). Report to Congress: Social risk factors and performance under Medicare’s value-based payment programs. Retrieved from https://aspe.hhs.gov/pdf-report/report-congress-social-risk-factors-and-performance-under-medicares-value-based-purchasing-programs
Diaz, A., Hyer, J. M., Barmash, E., Azap, R., Paredes, A. Z., & Pawlik, T. M. (2021). County-level social vulnerability is associated with worse surgical outcomes especially among minority patients. Annals of Surgery, 274(6), 881–891. https://doi.org/10.1097/SLA.0000000000004691
Forefront Group. (2023). The Area Deprivation Index is the most scientifically validated social exposome tool available for policies advancing health equity. https://doi.org/10.1377/forefront.20230714.676093
Hosmer, D. W., & Lemeshow, S. (2000). Applied Logistic Regression. New York: Wiley.
Janeway, M. G., Sanchez, S. E., Chen, Q., Nofal, M. R., Wang, N., Rosen, A., & Dechert, T. A. (2020). Association of race, health insurance status, and household income with location and outcomes of ambulatory surgery among adult patients in 2 US states. JAMA Surgery. https://doi.org/10.1001/jamasurg.2020.3318
Jha, A. K., Orav, E. J., & Epstein, A. M. (2011). Low-quality, high-cost hospitals, mainly in South, care for sharply higher shares of elderly black, Hispanic, and Medicaid patients. Health Affairs, 30, 1904-1911.
Krumholz, H. M., Brindis, R. G., Brush, J. E., et al. (2006). Standards for statistical models used for public reporting of health outcomes: An American Heart Association scientific statement from the Quality of Care and Outcomes Research Interdisciplinary Writing Group: cosponsored by the Council on Epidemiology and Prevention and the Stroke Council endorsed by the American College of Cardiology Foundation. Circulation, 113(3), 456-462.
Leeds, I. L., Meyers, P. M., Zachary Obinna Enumah, He, J., Burkhart, R. A., Haut, E. R., Efron, J. E., & Johnston, F. M. (2019). Psychosocial risks are independently associated with cancer surgery outcomes in medically comorbid patients. Annals of Surgical Oncology, 26(4), 936–944. https://doi.org/10.1245/s10434-018-07136-3
Menachemi, N., Chukmaitov, A., Brown, L. S., et al. (2007). Quality of care differs by patient characteristics: outcome disparities after ambulatory surgical procedures. American Journal of Medical Quality, 22(6), 395-401.
National Academies of Sciences, Engineering, and Medicine (NASEM); (2016a). Accounting for Social Risk Factors in Medicare Payment: Identifying Social Risk Factors. Washington DC: National Academies Press.
National Academies of Sciences, Engineering, and Medicine (NASEM); (2016b). Accounting for Social Risk Factors in Medicare Payment: Data. Washington DC: National Academies Press.
Normand, S.-L. T., & Shahian, D. M. (2007). Statistical and clinical aspects of hospital outcomes profiling. Statistical Science, 22(2), 206-226.
Pope, G., et al. (2000). Principal inpatient diagnostic cost group models for Medicare risk adjustment. Health Care Financing Review, 21(3), 26.
Qi, A. C., Peacock, K., Luke, A. A., Barker, A., Olsen, M. A., & Joynt Maddox, K. E. (2019). Associations between social risk factors and surgical site infections after colectomy and abdominal hysterectomy. JAMA Network Open, 2(10), Article e1912339. https://doi.org/10.1001/jamanetworkopen.2019.12339
Singh, G. K. (2003). Area deprivation and widening inequalities in US mortality, 1969–1998. American Journal of Public Health, 93(7), 1137–1143. https://doi.org/10.2105/ajph.93.7.1137
Trivedi, A. N., Nsa, W., Hausmann, L. R., et al. (2014). Quality and equity of care in U.S. hospitals. New England Journal of Medicine, 371, 2298-2308.
The measure uses a two-level hierarchical logistic regression model to estimate facility-level RSHVRs. This approach accounts for the clustering of patients within ASCs and variation in sample size across ASCs.
The risk-adjustment model includes 9 variables (age, 7 comorbidity variables, and one surgical complexity variable). Work RVUs are assigned to each Current Procedural Terminology (CPT®) procedure code and approximate surgical procedural complexity by incorporating elements of physician time and effort. For patients with multiple concurrent CPT procedure codes, we risk adjust for the CPT code with the highest work RVU value. We define comorbidity variables using CMS Condition Categories (CCs), which are clinically meaningful groupings of thousands of ICD-10 diagnosis codes.
Certain CCs are considered possible complications of care, and thus are not risk-adjusted for if they are coded only at the time of surgery. Please see the attached Data Dictionary for CCs that are considered possible complications of care and are not risk-adjusted for if they occur only at the surgery (Tab 5: “ASC_Uro_CoC CCs”)
Table 6 (see attachment) includes the risk variable frequencies for each risk variable in the final risk model.
Table 7 (see attachment) shows the distribution of social risk factors identified in the conceptual model. The facility median proportion of patients with the DE and high ADI variables is 0% and 1%, respectively; the median count of patients is 1 for each variable.
The final risk variables in the model are listed below, and are also shown in the data dictionary, and the attachment (see Tables 6 and 8 in attachment) that include risk variable frequencies and odds ratios.
Final Model Variables:
From 1,000 bootstrapped models, the following candidate variables were selected greater than 70% of the time, and thus we reselected as the final risk-adjustment variables:
- Age (years > 65)
- Work Relative Value Units (work RVUs)
- Benign prostatic hyperplasia with obstruction
- Complications of specified implanted device or graft (CC 176)
- Number of qualifying procedures: 1, 2, 3 or more
- Poisonings and allergic and inflammatory reactions (CC 175)
- Major symptoms, abnormalities (CC 178)
- Parkinson’s and Huntington’s diseases; seizure disorders and convulsions (CC 78, 79)
- Ischemic heart disease (CC 86, 87, 88, 89)
Risk Model Calibration and Discrimination
CORE’s measures undergo an annual measure reevaluation process, which ensures that the risk-standardized models are continually assessed and remain valid, given possible changes in clinical practice and coding standards over time. Modifications made to measure cohorts, risk models, and outcomes are informed by a review of the most recent literature related to measure conditions or outcomes, feedback from stakeholders, and empirical analyses, including assessment of coding trends that reveal shifts in clinical practice or billing patterns. Input is solicited from a workgroup composed of up to 20 clinical and measure5/1/2024 experts, inclusive of internal and external consultants and subcontractors.
To assess model performance, we computed three summary statistics for the ASC Urology measure: two discrimination statistics (the C-statistic, predictive ability) and one calibration statistic (overfitting) (Harrell et al, 2001). In addition, we provide risk-decile plots.
Discrimination Statistics
1. Area under the receiver operating characteristic (ROC) curve (c-statistic)
The c-statistic is the probability that predicting the outcome is better than chance, which is a measure of how accurately a statistical model is able to distinguish between a patient with and without an outcome.
C-statistic (2024 EM dataset): 0.615
C-statistic results indicate good model discrimination.
2. Predictive ability
Discrimination in predictive ability measures the ability to distinguish high-risk subjects from low-risk subjects; therefore, for a model with good predictive ability we would expect to see a wide range in hospital visit ratios between the lowest decile and highest decile. To calculate the predictive ability, we calculated the range of observed hospital visit ratios between the lowest and highest predicted deciles.
Predictive Ability, % (lowest decile - highest decile): 2.71 – 9.81
The model continues to show wide range between the lowest decile and highest decile, indicating the ability to distinguish high-risk subjects from low-risk subjects.
Calibration Statistics (from original measure development)
3. Over-fitting indices
Over-fitting refers to the phenomenon in which a model accurately describes the relationship between predictive variables and outcomes in the development dataset but fails to provide valid predictions in new patients. Estimated calibration values of γ0 far from 0 and estimated values of γ1 far from 1 provide evidence of over-fitting. We used Dataset #1 for this analysis. Our results, shown below, show a calibration value of close to 0 at one end and close to 1 to the other end indicating good calibration of the model.
Development Sample results:
Calibration: (0, 1)
FY 2015 Validation Sample results:
Calibration: (-0.05, 0.98)
CORE notes that after initial measure development we do not re-test our risk models for overfitting using a dataset that is external to the testing sample. In our risk models, coefficients are updated each time the measure is calculated; we refit the model with new data each time the measure is calculated. Therefore, random statistical fluctuations in model coefficients across repeated reporting cycles are part of the overall random error in the facility performance estimates.
Risk Decile Plots
We provide updated risk decile plots for all patients (see Figures 4 in attachment).
Higher deciles of the predicted outcomes are associated with higher observed outcomes, which continue to show good calibration of the model. The risk decile plot indicates continued good discrimination of the model and good predictive ability, for all patients, and for patients with DE and high ADI, separately.
We describe the approach to risk variable selection in Section 4.4.2, and analysis and rationale not including social risk factors in the final model in Section 5.1. In this section we provide the interpretation of the risk model testing results described in section 4.4.5.
The following results demonstrate that the risk-adjustment model adequately controls for differences in patient characteristics:
Discrimination Statistics
The calculated c-statistic was 0.615, which indicates good model discrimination. The model also predicted a wide range between the lowest decile and highest decile, indicating the ability to distinguish high-risk subjects from low-risk subjects.
Calibration Statistics
The calibration value which was consistently close to 0 at one end and close to 1 to the other, indicating good calibration of the model.
Risk Decile Plot
Higher deciles of the predicted outcomes are associated with higher observed outcomes, which show a good calibration of the model.
Overall Interpretation
Interpreted together with information provided in the aforementioned sections, our diagnostic results demonstrate the risk-adjustment model adequately controls for differences in patient characteristics (case mix).
Use & Usability
Use
The level of analysis is facility, and the care setting is ambulatory surgery center.
Usability
While ASCs continue to show promising results of reduced complications post-surgery in comparison to hospital settings, the possibility of an unplanned hospital visit due to adverse events post-surgery still exists for patients (Munnich et. al., 2014; Schwartz, 2020). These complications can include surgical site infections, postoperative pain management issues, and other complications, as shown in the literature and in the empiric results shared with this submission (Owens et. al., 2014; Erhun et. al., 2016; Rajput et. al., 2021). This measure provides the opportunity for improvement in reducing unplanned visits due to complications after surgery in the ASC setting.
Evidence-based interventions focusing on preoperative assessment, enhancing postoperative monitoring, and improving patient education can further reduce complications and rates of unplanned hospital visits (Erhun et. al., 2016; York et. al., 2019). For example, ASCs can opt to provide patient education pre- and post-procedure to facilitate the recovery process and reduce complications. Patient education improves transparency while allowing providers to clarify patient responsibilities and expectations. Continued and systemic monitoring of symptoms after surgery using validated assessments can lead to early detection of symptoms by providers post-surgery and can help in planning a course of action before complications exacerbate (Girish, 2021). A 2020 pilot study tested an electronic postoperative symptom-tracking platform to determine its clinical usefulness in the first week after minimally invasive ambulatory surgery. Responses above a defined threshold on the symptom instrument triggered an alert to the healthcare provider. The authors found that for majority of the patients presenting symptoms, a simple consultation phone call and adjustment of medications was sufficient in controlling complications (Zivanovic et. al., 2022). The adoption of evidence-based quality improvement systems in ASC settings can overall advance patient care and sustain a culture of improvement among facilities. Employing a methodological system of quality improvement focused on improving certain performance domains of quality provides guidelines for ASCs to follow and offsets the burden from clinical staff to develop their own plans for quality control (Rakover 2020, et. al).
Finally, specific to this measure, CMS shares several reports with ASCs to support quality improvement, including:
- Facility- Specific Reports (FSRs): These reports include measure results benchmarked against the state and the nation. For the reporting period covered for this data submission, facilities received one FSR, released in fall of 2023 on procedures performed January 1, 2021, through December 31, 2022. (Click here to view a mock FSR.)
- Claims-Detail Reports (CDRs): These reports provide claim-level details (including the principal diagnosis associated with the post-procedure visit) for each claim that meets numerator and denominator criteria. CDRs provide facilities with an opportunity to improve the quality of care provided to patients undergoing procedures prior to final measure calculation and public reporting of measure results. For this reporting period, covered for this data, facilities received two reports (one in September 2022, and one in March 2023). (Click here to view a mock CDR.)
References
Munnich EL, Parente ST. Procedures take less time at ambulatory surgery centers, keeping costs down and ability to meet demand up. Health Affairs. 2014 May 1;33(5):764-9.
Schwartz JC. Mastectomy and prepectoral reconstruction in an ambulatory surgery center reduces major infectious complication rates. Plastic and Reconstructive Surgery–Global Open. 2020 Jul 1;8(7):e2960.
Owens PL, Barrett ML, Raetzman S, Maggard-Gibbons M, Steiner CA. Surgical site infections following ambulatory surgery procedures. Jama. 2014 Feb 19;311(7):709-16.
Erhun F, Malcolm E, Kalani M, Brayton K, Nguyen C, Asch SM, Platchek T, Milstein A. Opportunities to improve the value of outpatient surgical care. Am J Manag Care. 2016 Sep 1;22(9):e329-35. PMID: 27662397.
Rajput K, Vadivelu N, Kaye AD, Shah RV, editors. Pain Control in Ambulatory Surgery Centers. Springer International Publishing; 2021 Mar.
York PJ, Gang CH, Qureshi SA. Patient education in an ambulatory surgical center setting. J Spine Surg. 2019 Sep;5(Suppl 2):S206-S211. doi: 10.21037/jss.2019.04.07. PMID: 31656877; PMCID: PMC6790809.
Joshi GP. Putting patients first: ambulatory surgery facilitates patient-centered care. Current Opinion in Anesthesiology. 2021 Dec 1;34(6):667-71.
Zivanovic O, Chen LY, Vickers A, Straubhar A, Baser R, Veith M, Aiken N, Carter J, Curran K, Simon B, Mueller J, Jewell E, Chi DS, Sonoda Y, Abu-Rustum NR, Leitao MM Jr. Electronic patient-reported symptom monitoring in patients recovering from ambulatory minimally invasive gynecologic surgery: A prospective pilot study. Gynecol Oncol. 2020 Oct;159(1):187-194. doi: 10.1016/j.ygyno.2020.07.004. Epub 2020 Jul 24. PMID: 32718730; PMCID: PMC7380930.
Rakover J, Little K, Scoville R, Holder B. Implementing daily management systems to support sustained quality improvement in ambulatory surgery centers. AORN journal. 2020 Apr;111(4):415-22.
Stakeholders can submit questions and issues to CMS through an online tool (Q&A tool) available to the public on QualityNet. CMS responds to each question submitted by stakeholders. During the time between the last maintenance review and re-endorsement of this measure (2020), we have received no major reports of issues with this measure. Through the Q&A tool, stakeholders have asked for assistance with their questions including: interpreting their facility’s patient-level data, understanding measure specifications (inclusion, exclusion, risk adjustment), interpretation/clarification of results.
Each year, we review and consider issues raised through the Q&A or in the literature related to this measure, and those issues are considered by measure and clinical experts. Any issues that warrant additional analytic work due to potential changes in the measure specifications are addressed as a part of annual measure reevaluation. If small changes are indicated after additional analytic work is complete, those changes are usually incorporated into the measure in the next measurement period. If the changes are substantial CMS may propose the changes through rulemaking and adopt the changes only after CMS received public comment on the changes and finalizes those changes in rulemaking.
Each year we also review and consider changes to HCPCS and ICD-10 codes that are then incorporated into the measure. Those code set files are made available to the public on QualityNet.
During the time between the last maintenance submission and the measure’s re-endorsement in 2020 and this current submission we have made no major changes to the ASC Urology measure based on stakeholder feedback.
This measure captures an ever-changing mix of procedures that have been added (or removed) from the ASC-covered procedure list, which is the basis for inclusion of procedures for this measure. Therefore it may not be meaningful to compare performance year over year as procedures shift from one setting to another.
There have been no unexpected findings during implementation.
Comments
Staff Preliminary Assessment
CBE #3366 Staff Assessment
Importance
Strengths:
- The developer posits that this measure provides an opportunity for improvement in reducing unplanned hospital visits due to complications after urology surgery in the ambulatory surgical center setting.
The developer cites evidence from the primary literature interventions that ASCs can implement to improve hospitalizations post-surgery, such as pre-operative assessments, enhancing post-operative monitoring, and improving patient education. The developer further suggests facilities can implement a system of quality improvement and use CMS facility-based and claims-based reports for quality improvement efforts.
The developer reported a distribution of facility-level scores with an overall score of 5.16 (mean facility-level score) and a decile range of 4.26 – 6.35. For this measure, a lower score is better. The distribution shows ~30% of ASCs are higher than the mean facility-level score.
The developer cites evidence, in which patients expressed concern over post-operative complications. The developer argues that due to the increase in procedures in ASCs there is a need for this measure to inform ASC providers and providers of the outcome of hospital visits. Further, by providing reports of this measure to providers will incentivize quality improvement efforts.
Limitations:
- None identified.
Rationale:
- The developer posits that this measure provides an opportunity for improvement in reducing unplanned hospital visits due to complications after orthopedic surgery in the ambulatory surgical center setting.
The developer cites evidence from the primary literature interventions that ASCs can implement to improve hospitalizations post-surgery, such as pre-operative assessments, enhancing post-operative monitoring, and improving patient education. The developer further suggests facilities can implement a system of quality improvement and use CMS facility-based and claims-based reports for quality improvement efforts.
The developer reported a distribution of scores with an overall score of 5.16 (mean facility-level measure score) and a decile range of 4.26 – 6.35. For this measure, a lower score is better. The distribution shows ~30% of ASCs are higher than the mean facility-level measure score. The developer reported the dates of data as January 1, 2021-December 31, 2022.
The developer cites evidence, in which patients expressed concern over post-operative complications. The developer argues that due to the increase in procedures in ASCs there is a need for this measure to inform ASC providers and providers of the outcome of hospital visits. Further, by providing reports of this measure to providers will incentivize quality improvement efforts.
Feasibility Acceptance
Strengths:
- The developer did not conduct a feasibility assessment of an assessment of missing data for this measure, as it used a 100% sample of paid Medicare claims. The developer further states that since this is an claims-based measure, there is no added burden of reporting as all required data elements are routinely generated. The measure does not include any proprietary information.
Limitations:
- None identified.
Rationale:
- The developer did not conduct a feasibility assessment of an assessment of missing data for this measure, as it used a 100% sample of paid Medicare claims. The developer further states that since this is an claims-based measure, there is no added burden of reporting as all required data elements are routinely generated. The measure does not include any proprietary information.
Scientific Acceptability
Strengths:
- The measure is well-defined and precisely specified.
This is a claims-based measure, which historically have been evaluated for data element reliability and validity. The developer conducted signal-to-noise of the measure score with a minimum procedure volume of 35 (which is currently the public reporting cutoff). Of note, the measure is not specified to have a minimum procedure volume.
The developer reports an overall reliability estimate of 0.706 with a decile range of 0.422 – 0.949. Roughly 20% of ASCs have a signal-to-noise estimate less than 0.6.
Limitations:
- None identified.
Rationale:
- The measure is well-defined and precisely specified.
This is a claims-based measure, which historically have been evaluated for data element reliability and validity. The developer conducted signal-to-noise of the measure score with a minimum procedure volume of 35 (which is currently the public reporting cutoff). Of note, the measure is not specified to have a minimum procedure volume.
The developer reports an overall reliability estimate of 0.706 with a decile range of 0.422 – 0.949. Roughly 20% of ASCs have a signal-to-noise estimate less than 0.6.
Strengths:
- The developer provided three sources of validity: (1) face validity; (2) validity of the outcome, shown by analysis of ICD-10 codes associated with the outcome; and (3) validity of the measure score as shown by association (in the expected direction) with procedural volume. However, this was limited to ASCs that specialized in urology. For face validity, the developer convened a technical expert panel (TEP) for face validity testing during the measure’s initial development. 12/14 (86%) of TEP respondents agreed that the measure is valid and useful and can improve the quality of care. Second, in a claims-based analysis to validate the outcome, the developer shows the most frequent (the top 25) principal diagnosis codes associated with the outcome (unplanned hospital visits within 7 days of a qualifying procedure) indicate a complication from the procedure. For example, the top four principal diagnoses are “retention of urine, unspecified,” “urinary tract infection, site not specified,” “sepsis, unspecified organism,” and “Hydronephrosis with renal and ureteral calculous obstruction.” Lastly, the developer conducted accountable entity-level validity testing by associating measure score to facility volume. The developer considered conducting a correlation analysis to another quality measure but did not find any that “was suitable for comparison.” With respect to the association by facility volume, the developer hypothesized a weak-to-moderate, negative relationship meaning that facilities with higher volume would be associated with lower (better) measure scores. The developer based this hypothesis on prior literature, which found a volume-outcome relationship in “some (but not all) outcomes” for surgical procedures when considering procedural volume (see limitations).
The developer also conducted statistical risk adjustment, based on their conceptual model, and selecting factors that had a significant correlation to the outcome. The developer also explored social risk factors, such as dual eligibility and the Area Deprivation Index. The developer did not include these in the final models due to the minimal impact these social risk factors have on the measure scores and to the model overall. Further, the developer reported that when stratifying the measure by dual eligibility, there were insufficient numbers of patients to support the stratification approach. The developer reported a c-statistic of 0.615, indicating good model discrimination.
Limitations:
- With respect to the empirical testing of procedural volume, the association was only significant when analysis was limited to facilities with 200 procedures, indicating the measure may not be valid for facilities with fewer procedures. Study referenced does not examine urologic procedures. Results of association analysis do not demonstrate the measure can distinguish between better or worse quality facilities. Lastly, it is unclear, based on the evidence provided, how facility procedural volume supports the validity of the measure, including the quality construct. The developer did not identify procedural volume within the measure's logic model. Volume may indicate higher number of surgeons or increased quality control and resources, but this is not clearly articulated in the submission. In addition, some facilities may not be able to control procedural volume, thus based on this analysis, facilities with low volumes will have worse outcomes. Overall, it is not clear how procedural volume is an appropriate covariate for validity analysis.
Rationale:
- The developer conducted accountable entity-level validity testing by associating measure score to facility volume. The developer considered conducting a correlation analysis to another quality measure but did not find any that “was suitable for comparison.” With respect to the association by facility volume, the developer hypothesized a weak-to-moderate, negative relationship indicating that lower volume facilities have a slight improvement on the measure score. The developer based this hypothesis on prior literature, which found a volume-outcome relationship in “some (but not all) outcomes” for surgical procedures when considering procedural volume. However, the association was only significant when analysis was limited to facilities with 200 procedures, indicating the measure may not be valid for facilities with fewer procedures. Study referenced does not examine urologic procedures. Results of association analysis do not demonstrate the measure can distinguish between better or worse quality facilities. Lastly, it is unclear, based on the evidence provided, how facility procedural volume supports the validity of the measure, including the quality construct. The developer did not identify procedural volume within the measure's logic model. Volume may indicate higher number of surgeons or increased quality control and resources, but this is not clearly articulated in the submission. In addition, some facilities may not be able to control procedural volume, thus based on this analysis, facilities with low volumes will have worse outcomes. Overall, it is not clear how procedural volume is an appropriate covariate for validity analysis. The developer may be able to speak to this during the endorsement meeting.
The developer also convened a technical expert panel (TEP) for face validity testing during the measure’s initial development. 12/14 (86%) of TEP respondents agreed that the measure is valid and useful and can improve the quality of care.
The developer also conducted statistical risk adjustment, based on their conceptual model, and selecting factors that had a significant correlation to the outcome. The developer also explored social risk factors, such as dual eligibility and the Area Deprivation Index. The developer did not include these in the final models due to the minimal impact these social risk factors have on the measure scores and to the model overall. Further, the developer reported that when stratifying the measure by dual eligibility, there were insufficient numbers of patients to support the stratification approach. The developer reported a c-statistic of 0.615, indicating good model discrimination.
Equity
Strengths:
- The developer reports that ASCs do not serve a high proportion of patients with social risk factors with the median facility proportion of patients with the dual eligibility variable being 0% (count of 0 patients) and for the high ADI, 1% (count of 1 patient).
Limitations:
- The developer did not identify whether the measure rates are different across other patient populations or how this measure may help to reduce disparities. Rather, for this section, the developer provided a rationale for why dual eligibility and Area Deprivation Index were not included in the risk model. For future submissions, the developer may consider whether the measure can identify differences in performance across different patient groups by their social and/or economic status.
Rationale:
- The developer reports that ASCs do not serve a high proportion of patients with social risk factors with the median facility proportion of patients with the dual eligibility variable being 0% (count of 0 patients) and for the high ADI, 1% (count of 1 patient). However, the developer did not identify whether the measure rates are different across other patient populations or how this measure may help to reduce disparities. Rather, for this section, the developer provided a rationale for why dual eligibility and Area Deprivation Index were not included in the risk model. For future submissions, the developer may consider whether the measure can identify differences in performance across different patient groups by their social and/or economic status.
Use and Usability
Strengths:
- The measure is currently used in the ASC Reporting Program
The developer cites evidence of interventions that can improve hospital visits, as noted previously. The developer further suggests facilities can implement a system of quality improvement and use CMS benchmarking reports, which include the principal diagnosis on admission, to tie quality improvement efforts to complications that are occurring
The developer summarized that stakeholders may submit questions about the measure to CMS via an online tool. CMS then responds to those questions. The developer notes that since re-endorsement in 2020 there have been no major reports or issues with the measure. In addition, no major changes to the measure have been conducted as a result of literature reviews and feedback received via the online Q&A tool.
The developer reports no unexpected findings.
Limitations:
- The developer did not report any findings on the progress on improvement since the procedures are “ever-changing,” making it challenging to accurately assess year-over-year performance.
Rationale:
- The measure is currently used in the ASC Reporting Program
The developer cites evidence of interventions that can improve hospital visits, as noted previously. The developer further suggests facilities can implement a system of quality improvement and use CMS benchmarking reports, which include the principal diagnosis on admission, to tie quality improvement efforts to complications that are occurring
The developer summarized that stakeholders may submit questions about the measure to CMS via an online tool. CMS then responds to those questions. The developer notes that since re-endorsement in 2020 there have been no major reports or issues with the measure. In addition, no major changes to the measure have been conducted as a result of literature reviews and feedback received via the online Q&A tool.
The developer did not report any findings on the progress on improvement since the procedures are “ever-changing,” making it challenging to accurately assess year-over-year performance.
The developer reports no unexpected findings.
Committee Independent Review
Do not support
Importance
Agree with staff
Feasibility Acceptance
Agree with staff
Scientific Acceptability
Agree with staff
Does not include Medicare Advantage
Equity
Patients and their caregivers might find this measure more respectful if more settings of follow-up were considered, such as home care, satellite clinics, portal communication, and telehealth, as well as transportation and appointment access.
Use and Usability
Patients and their caregivers might find this measure more meaningful if it considered the corporatization of healthcare with associated increased out-of-pocket expenses and the shift from solo to group practices; the increase in minimally invasive procedures, imaging advancements, telemedicine, the impact of the COVID-19 pandemic, and included data from Medicare Advantage Plans.
Summary
Patients and their caregivers might find this measure more meaningful if it considered the corporatization of healthcare with associated increased out-of-pocket expenses and the shift from solo to group practices; the increase in minimally invasive procedures, imaging advancements, telemedicine, the impact of the COVID-19 pandemic, and included data from Medicare Advantage Plans.
Patients and their caregivers might find this measure more respectful if more settings of follow-up were considered, such as home care, satellite clinics, portal communication, and telehealth, as well as transportation and appointment access.
Support
Importance
agree with staff assessment
Feasibility Acceptance
agree with staff assessment
Scientific Acceptability
Noted comments from the Center for Healthcare Quality and Payment Reform regarding the variation in reliability, and that half of the ASC's had signal to noise reliability ratios below 0.7. How does that compare with standards for acceptable reliability?
agree with staff assessment
Equity
agree with staff assessment
Use and Usability
agree with staff assessment
Summary
The measure is part of an important category to monitor ASC performance with efforts to ensure patient safety. Would explore whether minor adjustments to the measure can improve reliability without removing a large number of ASCs from the measure. Would also explore how to better report performance according to socioeconomic factors.
Hospital Visits after Hospital Outpatient Surgery
Importance
This is an important measure to address any complications from an ASC after urologic surgery
Feasibility Acceptance
The developer did not address feasibility as it used 100% of the claims information.
Scientific Acceptability
There is reliability and validity to the measure. Of note, ASCs with lower volume were not included (only ASCs with over 200 procedures).
This is a limitation as those ASCs with lower volume are not accounted for. The TEP did find the validity acceptable
Equity
The developer reports that ASCs do not serve a high proportion of patients with social risk factors. Would like the developer to further discuss why not included in the model-would like to know why data was not stratified by health equity
Use and Usability
The measure is currently used in the ASC Reporting Program. The developer reported no unexpected findings but would like to have information on progress improvement with the measure over time.
Summary
There are some areas that need further clarification from the developer in the areas of scientific acceptability especially for lower volume ASCs, equity-and why not used to stratify data, and Use and Useability in how the measures performs over time.
no additional comments
Importance
agree with staff assessment
Feasibility Acceptance
agree with staff assessment
Scientific Acceptability
agree with staff assessment
agree with staff assessment
Equity
agree with staff assessment
Use and Usability
agree with staff assessment
Summary
agree with staff assessment
No additional comments
Importance
The developer highlighted various literature and sources to support the importance criteria.
Feasibility Acceptance
agree with staff assessment; no burden on the facility since this is a claims-based measure; also no feasibility assessment was conducted.
Scientific Acceptability
agree with staff assessment
agree with staff assessment
Equity
The developer did not identify whether the measure rates are different across other patient populations or how this measure may help to reduce disparities.
Use and Usability
agree with staff assessment
Summary
No additional comments
Support with limitation
Importance
Agree w Staff assessment which was very well done
Feasibility Acceptance
Agree with staff assessment
Scientific Acceptability
Agree w Staff assessment
Agree with staff assessment but having problems with the numerator regarding capture of all causes for ER admit...would like to see procedure specific complications as the metric purports
Equity
Agree w Staff assessment
Use and Usability
Agree w Staff assessment
Summary
I can support this but would like to hear from developers how a trauma admission/.cardiac/renal etc have anything to do with the urological procedures
Support
Importance
Agree with staff assessment.
Feasibility Acceptance
Agree with staff assessment.
Scientific Acceptability
Agree with staff assessment.
Agree with staff assessment.
Equity
Agree with staff assessment.
Use and Usability
Agree with staff assessment.
Summary
Some additional information is needed regarding validity testing of the volume-outcome relationship, stratified data analysis to determine health disparities, and use and usability in assessing performance over time.
Public Comments
Public Comment Shared During May 29 Listening Session
So, these [CBE 3470, 3357, 2539, and 3366] are all ambulatory, which means you were not admitted to the hospital to have the surgery. I had to go into a hospital after a general surgery, but it was orthopedic, right. Mine was within that 7 days. But really any hospital visit after any kind of surgery is important. I totally recognize these and think it's very important, particularly when there's sort of a push to get people in and out of the hospital. So, thank you.
Yeah, I mean, they need to watch. There's a risk standardized hospital visit. They all are about some hospital visit, right? And that does not mean that actually means hospital. Either a stay or observational. I'm assuming. So it's important. And these are all very important. Because sometimes it's just check the box, but it’s important to know why the person ends up back in the hospital after some sort of procedure. Thank you.
Public Comment Shared During May 29 Listening Session
Well, Hello! So my question would just be regarding the age, the 65 years and older for the Medicare patients Fee-for-Service. I don't know if the previous maintenance [review] was still in the same age category, but if the age could be expanded for Medicare Fee-for-Service patients that are younger as well, too. Just widening the age range for these Medicare patients. I know they're looking at older patients, but I don't know if they're looking at older patients being able to bounce back after the surgery, but not because they have to be re-admitted or have an ED visit or an unplanned inpatient admission. So, to just kind of like understand whether it's the age-related readmission, or if it's the procedural readmission, or if they're having complications from the actual surgical event that they experienced. So, I just have a question on why do they [the developer] have the age at 65, and not maybe in the fifties, or just to kind of exclude whether it's age related or procedure related. My [other] question is regarding the urology procedure. I was unclear about whether they [the developer] are they looking at certain [urology] procedures or is it across the board with any procedure? Because this is a specific area, and it's kind of broad. And I didn't know if it was a specialized procedure that they're referencing or is just the broad area that they were specifying because it's for hospital visits, ED, or unplanned visits. So, I don't know if it was for procedures where this may be more frequent that a patient may be readmitted.
Endorsement Should Be Removed From This Measure
CBE ID: 3366
Hospital Visits After Urology Ambulatory Surgical Center Procedures
Endorsement should be removed from this measure. It is not a valid or reliable measure of the quality or efficiency of outpatient urology procedures due to problems with the numerator and the risk adjustment methodology. Continued public reporting of the results could mislead patients about where they should receive such procedures.
Problems with the Numerator
The stated goal of the measure is to improve the quality of care delivered to patients undergoing urology procedures in an ambulatory surgery center (ASC). However, the numerator includes events that have nothing to do with the procedure, while excluding events that represent undesirable outcomes and avoidable costs.
Studies have found that a large fraction of post-surgical visits are not related to the index surgery. For example, in one study of unplanned hospital admissions after surgery, more than half of the admissions for elderly patients were not related to the procedure. Older seniors had a higher proportion of admissions that were unrelated to the procedure. (De Oliveira Jr. GS, et al. “Older Adults and Unanticipated Hospital Admission Within 30 Days of Ambulatory Surgery: An Analysis of 53,667 Ambulatory Surgical Procedures,” Journal of the American Geriatric Society 63:1679-1685.)
The inclusion of so many visits unrelated to the procedure creates two different problems.
Problems with the Risk Adjustment Methodology
The actual visit rate is modified using a hierarchical logistical regression model that is intended to adjust for differences in the characteristics of the patients receiving procedures that affect outcomes. However, the model fails to adjust for differences in the propensity of patients to visit a hospital for other health problems.
For example, a patient may come to the ED for an acute condition that is unrelated to the urologic procedure (e.g., a viral infection, such as flu or COVID-19). An ASC in a community experiencing high rates of such illnesses could have higher rates of post-procedure visits, and this measure would make it appear that the ASC delivers lower quality care. The risk adjustment used in the measure controls for the presence of chronic conditions, but it does not control for the frequency of new acute conditions.
In addition, patients with more severe chronic conditions will have higher rates of ED visits for exacerbations of their conditions, and the rates of ED visits for chronic disease exacerbations will be higher in communities where chronic disease management services are less available and for patients who have lower incomes or lack of family support that make it more difficult for them to access the services that do exist. In these types of communities and for these types of patients, there will likely be more post-procedure ED visits for chronic disease exacerbations that are unrelated to outpatient procedures. The risk adjustment methodology only controls for the presence of a chronic condition; it does not control for either the severity of the condition or for differences in the care patients receive for the chronic condition. Care for the chronic condition is not the responsibility of the physicians or ASC staff who delivered the outpatient procedure, but under this measure, a higher rate of ED visits for problems related to chronic conditions will be interpreted as poor quality care for the outpatient procedure.
One of the earliest studies of outcomes following outpatient surgery adjusted for this by including a factor measuring a patient’s hospital utilization prior to surgery. (Fleisher LA, et al. “Inpatient Hospital Admission and Death After Outpatient Surgery in Elderly Patients,” Archives of Surgery 139:67-72.) However, no such adjustment is included in the risk adjustment model for this measure. As a result, an ASC that has more patients with poorly managed chronic conditions will inappropriately appear to be delivering lower-quality care. Other patient characteristics that could affect the likelihood of post-procedure hospital visits include whether the patient is in a Skilled Nursing Facility or is receiving home health care, but no adjustment is made for these factors and there is no indication in the Measure Information Form that they were even examined.
Poor Measure Reliability and an Inappropriately Low Minimum Case Threshold
The measure is inappropriately being used for ASCs that perform as few as 35 procedures. With that few cases, one post-discharge visit results in a 3% visit rate and two post-discharge visits results in a 6% post-discharge visit rate. Since the average national rate is 5.2%, a single visit at a small ASC could make the difference between having a “low” or “high” visit rate. Moreover, because of the wide range of procedures that are included in the measure, variation in outcomes could easily occur simply because of differences in the specific procedures performed and patients treated during a particular year.
Because of the low volumes and high variability of post-discharge visits, this measure is not sufficiently reliable to use for public reporting. The developers report that the signal-to-noise reliability was below 0.6 for 30% of ASCs, below 0.7 for half of ASCs, and below 0.8 for most ASCs. It does not appear that any effort was made to assess the reliability of the measure over time.
Lack of Business Case for Using the Measure and Undesirable Effects of Doing So
The measure identified only 20 out of over 1,000 ASCs as being either better or worse than the national average. Over one-third were classified as “number of cases too small” and more than half were classified as “no different than the national average.” Moreover, it does not appear that the developers made any effort to examine the 6 facilities identified as “better than the national rate” or the 14 facilities identified as “worse than the national rate” to assess whether this was really a valid classification based on the number of cases, the reasons for post-discharge visits, and the consistency of the classification over time.
Since only a very small number of ASCs are being classified as anything other than “no different than expected,” this measure is unlikely to encourage improvements in the quality of care delivery. On the other hand, since the weaknesses in the measure methodology could cause some ASCs to be inappropriately labeled as “worse than expected” because of the kinds of patients they treat, use of the measure creates an undesirable incentive for ASCs to avoid treating patients with characteristics that are likely to result in higher numbers of visits, such as patients with multiple comorbidities and patients who do not have good access to primary care and chronic disease management services. It could also cause patients to inappropriately avoid obtaining a needed procedure, or to try and obtain the procedure in a facility that is not adequately equipped to address their needs.
3366 Hospital Visits after Urology Ambulatory Surgical Center Pr
The American Medical Association (AMA) questions whether this measure produces scores that can be considered sufficiently reliable. Specifically, the minimum reliability with at least 35 cases was 0.402, which falls well below what the AMA considers to be acceptable. We believe that measures must achieve a minimum reliability of at least 0.7 or higher in order to be achieve endorsement.