When the measure comes back for maintenance in 5 years, the developer will have: Explored other risk factors that may impact post-discharge VTE (e.g., social determinants of health).
The proportion of inpatient hospitalizations for patients age 18 and older, who have at least one surgical procedure performed inside the operating room during the encounter, and who suffer the harm of a postoperative venous thromboembolism (VTE) during the encounter or within 30 days after the first surgical procedure. This measure is adjusted by patient-level risk factors (bleeding disorders, cancer, respiratory operations, central venous catheter insertion, vascular surgeries, obesity, stroke, and history of VTE).
Measure Specs
General Information
In-hospital postoperative venous thromboembolism (VTE) encompasses both deep vein thrombosis (DVTs) and pulmonary embolisms (PE) that are associated with poor clinical outcomes, including post-thrombotic syndrome in the leg, anticoagulation-related bleeding, and death (Bysshe et al., 2017). Furthermore, VTE represents the leading cause of preventable mortality in hospitalized patients and is an established predictor of hospital readmissions (The Centers for Disease Control and Prevention [CDC], 2024a; Secemsky et al., 2018).
Despite a reported 17 percent reduction in the incidence of postoperative VTE between 2014 and 2017, the rate of postoperative VTE in hospitals remains high in the United States (US), leaving an opportunity to further reduce the occurrence of these events (Agency for Healthcare Research and Quality [AHRQ], 2020a). The Centers for Disease Control and Prevention estimates that up to 900,000 people in the US are affected by VTE each year (CDC, 2024b). Supporting this finding, one study found that hospital-associated VTE occurs in 1.2 percent of admissions (Neeman et al. 2022). Another study estimated that the annual incidence of VTE in the US following a surgery could range from 70,000 to 600,000, incurring an additional cost of $12,000 per case (Bartlett et al., 2020). These findings underscore the clinical and economic burden of postoperative VTE events and the need for continued quality improvement.
Evidence shows that hospital-level changes can significantly reduce and prevent hospital-acquired VTE events. Kakkos et al. (2022) found that combining hospital interventions, such as mechanical and pharmacological prophylaxis, can reduce the incidence of DVT among patients undergoing surgery or admitted with trauma. Another study found that interruptions or delays in the prescription of VTE prophylaxis in surgical patients are associated with a 2- to 3-fold increased risk of VTE (Henke et al., 2020). These findings suggest that there is opportunity to lower postoperative VTE rates through hospital intervention. The Hospital Harm – Postoperative Venous Thromboembolism measure is designed to capture the occurrence of postoperative VTE events, either during the inpatient encounter or within 30 days of the initial surgical procedure. Adoption of this measure has the potential to improve the quality of care for surgical patients and, therefore, advance patient safety, which is a priority area for the Centers for Medicare & Medicaid Services (CMS) as documented in its National Quality Strategy (AHRQ, 2020b; CMS, 2024).
References:
Agency for Healthcare Research and Quality (2020a). AHRQ National Scorecard on Hospital-Acquired Conditions. https://www.ahrq.gov/hai/pfp/index.html
Agency for Healthcare Research and Quality (2020b). About the National Quality Strategy. https://www.ahrq.gov/workingforquality/about/nqs-fact-sheets/nqs-fact-s…
Bartlett, M. A., Mauck, K. F., Stephenson, C. R., Ganesh, R., & Daniels, P. R. (2020). Perioperative venous thromboembolism prophylaxis. Mayo Clinic Proceedings, 95(12), 2775–2798. https://doi.org/10.1016/j.mayocp.2020.06.015
Bysshe, T., Gao, Y., Heaney-Huls, K., Hockenberry, J., Hovey, L., Laffan, A. M., Lee, S., Murphy, D. J. &, Watts, E. (2017). Estimating the additional hospital inpatient cost and mortality associated with selected Hospital-Acquired conditions. Agency for Healthcare Research and Quality. https://www.norc.org/content/dam/norc-org/documents/standard-projects-p…;
Centers for Medicare & Medicaid Services. (2024). Quality in Motion: Acting on the CMS National Quality Strategy. https://www.cms.gov/files/document/quality-motion-cms-national-quality-…;
The Centers for Disease Control and Prevention. Data and Statistics on Venous Thromboembolism. (2024a, May 15). Venous Thromboembolism (Blood Clots). https://www.cdc.gov/blood-clots/data-research/facts-stats/index.html
The Centers for Disease Control and Prevention. Understanding your risk for Healthcare-Associated VTE (Blood Clots). (2024b, May 15). Venous Thromboembolism (Blood Clots). https://www.cdc.gov/blood-clots/risk-factors/ha-vte.html
Henke, P. K., Kahn, S. R., Pannucci, C. J., Secemksy, E. A., Evans, N. S., Khorana, A. A., Creager, M. A., & Pradhan, A. D. (2020). Call to action to prevent venous thromboembolism in hospitalized patients: A policy statement from the American Heart Association. Circulation, 141(24). https://doi.org/10.1161/cir.0000000000000769
Kakkos, S., Kirkilesis, G., Caprini, J. A., Geroulakos, G., Nicolaides, A., Stansby, G., & Reddy, D. J. (2022). Combined intermittent pneumatic leg compression and pharmacological prophylaxis for prevention of venous thromboembolism. The Cochrane database of systematic reviews, 1(1), CD005258. https://doi.org/10.1002/14651858.CD005258.pub4
Neeman E, Liu V, Mishra P, et al. Trends and Risk Factors for Venous Thromboembolism Among Hospitalized Medical Patients. JAMA Net Open. 2022;5(11):e2240373. doi:10.1001/jamanetworkopen.2022.40373
Secemsky, E. A., Rosenfield, K., Kennedy, K. F., Jaff, M., & Yeh, R. W. (2018). High Burden of 30-Day Readmissions After Acute Venous Thromboembolism in the United States. Journal of the American Heart Association, 7(13), e009047. https://doi.org/10.1161/JAHA.118.009047
Hospitals collect electronic health record (EHR) data using certified electronic health record technology (CEHRT). The measure specification is available as a package via MADiE export. The package includes a human-readable HTML file and XML, clinical quality language (CQL), and ELM files for machine processing. No additional tools are used for data collection for eCQMs.
Numerator
Inpatient encounters for patients with a postoperative venous thromboembolism (VTE) within 30 days of the first surgical procedure.
Inpatient encounters for patients with a postoperative venous thromboembolism (VTE) within 30 days of the first surgical procedure.
Evidence of a postoperative VTE is determined by either Criterion A, B or C:
Criterion A: A qualifying inpatient encounter, excluding encounters where the first surgical procedure was an intracranial or spinal surgery procedure, with a diagnostic imaging study performed during the qualifying inpatient encounter and within 30 days or less after the end of the first surgical procedure performed and at least one of the following:
- An anticoagulation medication order within 24 hours after the end of the diagnostic imaging study during the qualifying inpatient encounter. An anticoagulation medication order is evidenced by:
- Enoxaparin (Lovenox) >80 mg per day
- Apixaban (Eliquis) >=10 mg per day
- Rivaroxaban (Xarelto) >=20 mg per day
- Fondaparinux (Arixtra) >= 5 mg per day
- Dalteparin sodium (Fragmin)>= 10,000 IU per day; or
- A heparin intravenous administration within 24 hours after the end of the diagnostic imaging study, with at least 2 activated partial thromboplastin time (aPTT) heparin therapy monitoring tests or at least 2 anti-factor Xa assays within 35 hours of the start of heparin intravenous therapy administration all which occur during the qualifying inpatient encounter; or
- Placement of an inferior vena cava (IVC) filter within 24 hours after the end of the diagnostic imaging study and during the qualifying inpatient encounter; or
- A diagnosis of VTE which was not present on admission during the qualifying inpatient encounter.
OR
Criterion B: A qualifying inpatient encounter, where the first surgical procedure was for an intracranial or spinal surgery, and a diagnostic imaging study was performed during the qualifying inpatient encounter and the diagnostic imagining study occurred between 5 days and up to 30 days after the end of the first surgical procedure and at least one of the following:
- An anticoagulation medication order within 24 hours after the end of the diagnostic imaging study during the qualifying inpatient encounter. An anticoagulation medication order is evidenced by:
- Enoxaparin (Lovenox) >80 mg per day
- Apixaban (Eliquis) >=10 mg per day
- Rivaroxaban (Xarelto) >=20 mg per day
- Fondaparinux (Arixtra) >= 5 mg per day
- Dalteparin sodium (Fragmin)>= 10,000 IU per day; or
- A heparin intravenous administration within 24 hours after the diagnostic imaging study, with at least 2 aPTT heparin therapy monitoring tests or at least 2 anti-factor Xa assays within 35 hours of the start of heparin intravenous therapy administration, all of which occur during the qualifying inpatient encounter; or
- Placement of an IVC filter within 24 hours after the end of the diagnostic imaging study and during the qualifying inpatient encounter; or
- A diagnosis of VTE which was not present on admission during the qualifying inpatient encounter.
OR
Criterion C: A VTE that occurs during a subsequent inpatient encounter and within 30 days or less after the end of the first surgical procedure that occurred during the qualifying inpatient encounter as evidenced by:
- A diagnosis of VTE during the subsequent encounter;
and
- Anticoagulation therapy ordered or prescribed during the subsequent encounter.
The value sets for the numerator data elements are provided in the attached data dictionary.
To reduce sensitivity to overdiagnosis bias, diagnosis codes for distal (calf) vein thromboses, solitary subsegmental pulmonary emboli, and unspecified VTE diagnosis codes are not considered as a qualifying VTE diagnosis.
Denominator
Inpatient encounters for patients age 18 and older where a surgical procedure was performed inside the operating room during the encounter.
Inpatient encounters for patients age 18 and older where a surgical procedure was performed inside the operating room during the encounter.
Inpatient encounters: Includes time in the emergency department, observation, and/or outpatient surgery service when the transition between discharge from these encounters and admission to the inpatient encounter is one hour or less.
Surgical procedures: The combined concepts of anesthesia data AND operating room location are used to determine if a surgical procedure occurred. Patients are admitted to the operating room location for any surgical procedure. Using these combined concepts identifies patients at higher risk of developing a postoperative venous thromboembolism (VTE).
Qualifying inpatient encounter: The inpatient encounter that qualifies a patient for the measure denominator. The measure identifies whether a VTE occurs within 30 days of the first surgical procedure that occurs during this qualifying inpatient encounter. Criteria A and B identify instances when that post-operative VTE occurs during the qualifying inpatient encounter, and Criterion C identifies instances when that post-operative VTE occurs during a subsequent encounter, but all of these criteria limit their scope to VTEs occurring within the 30-day window following the first surgical procedure of the qualifying inpatient encounter.
Subsequent encounter: An inpatient encounter occurring within 30 days of the first surgical procedure of the qualifying inpatient encounter. If a patient experiences a VTE during this encounter as defined by numerator Criterion C, the qualifying inpatient encounter is included in the measure numerator. If a subsequent encounter qualifies the preceding qualifying inpatient encounter for inclusion in the measure numerator, then the subsequent encounter cannot qualify as a qualifying inpatient encounter, even if it otherwise satisfies the measure denominator criteria.
The value sets for the denominator data elements are provided in the attached data dictionary.
Exclusions
Inpatient encounters for
-Patients with an obstetric-related diagnosis during the inpatient encounter
-Patients with a venous thromboembolism (VTE) diagnosis present on admission
-Patients who had extracorporeal membrane oxygenation (ECMO) during the inpatient encounter
-Patients with acute brain or spinal injury or hemorrhage present on admission
-Patients who had a thrombectomy procedure which occurred before or on the same day as the first surgical procedure during the inpatient encounter
-Patients who had intracranial or spinal surgery during the inpatient encounter who were discharged less than 5 days after the end of the surgery
-Patients who have a duration of stay less than 2 calendar days
All data elements necessary to calculate the denominator exclusions are defined within value sets available in the Value Set Authority Center and listed below. Data elements required to calculate denominator exclusions are collected during the qualifying inpatient encounter, including time in the emergency department, observation, and/or outpatient surgery service when the transition between discharge from these encounters and admission to the inpatient encounter is one hour or less.
Present on Admission or Clinically Undetermined (2.16.840.1.113762.1.4.1147.197)
Venous Thromboembolism (2.16.840.1.113883.3.117.1.7.1.279)
Extracorporeal Membrane Oxygenation (2.16.840.1.113762.1.4.1248.81)
Acute Brain or Spinal Injury or Hemorrhage (2.16.840.1.113762.1.4.1248.79)
Pulmonary Arterial Thrombectomy (2.16.840.1.113762.1.4.1248.80)
Intracranial Neurosurgery (2.16.840.1.113883.3.117.1.7.1.260)
Spinal Surgery (2.16.840.1.113762.1.4.1248.185)
Obstetrics and VTE Obstetrics (2.16.840.1.113762.1.4.1248.33)
To access the value sets for the measure, please visit the Value Set Authority Center, sponsored by the National Library of Medicine, at https://vsac.nlm.nih.gov/ or reference the attached data dictionary referenced in field 1.13.
Measure Calculation
Data elements required to calculate the measure score are collected during the qualifying inpatient encounter, including time spent in the emergency department, observation unit, and outpatient surgery services—provided that transitions between these settings and the inpatient admission occur within one hour or less. The measurement period spans from January 1, XXXX to December 31, XXXX.
To create the denominator:
1. If the inpatient encounter ended during the measurement period, go to step 2. If no, do not include in the denominator.
2. Determine the patient’s age in years. The patient’s age is equal to the encounter date minus the birth date. If the patient is 18 years or older, go to step 3. If less than 18 years old, do not include in the denominator.
3. Determine if the inpatient hospitalization had an anesthesia procedure (General and Neuraxial Anesthesia 2.16.840.1.113762.1.4.1248.208) in the operating room (HSLOC code 1096-7). If no, do not include in the denominator. If yes, move to step 4.
4. Inpatient admissions that remain after step 3 are included in the denominator.
To implement the denominator exclusions:
1. If the inpatient encounter had an obstetric-related diagnosis, exclude from the denominator. If not, move to step 2.
2. If the inpatient encounter had any of the following diagnoses present on admission – VTE, acute brain injury, spinal injury, hemorrhage - exclude the encounter from the denominator. If not, then move to step 3.
3. Determine the length of stay. The length of stay is the discharge date minus the beginning of the inpatient encounter date (including time spent in the emergency department, observation unit, and outpatient surgery services if the transitions between these settings and the inpatient admission occur within one hour or less) . If the length of stay is less than 2 calendar days, then exclude the encounter from the denominator. If the length of stay is greater than 2 calendar days, then move to step 4.
4. If the inpatient encounter had extracorporeal membrane oxygenation (ECMO) any time during the hospitalization, then exclude the encounter from the denominator. If not, then move to step 5.
5. If the inpatient encounter had a thrombectomy procedure before or on the same day as the first surgical procedure during the hospitalization, then exclude the encounter from the denominator. If not, then move to step 6.
6. If the inpatient encounter includes an intracranial or spinal surgery, determine the number of days the patient was discharged after the end of that intracranial or spinal surgery by subtracting the discharge date from the end of the surgery date. If the patient was discharged less than 5 days after the end of intracranial or spinal surgery, then exclude the encounter from the denominator. If not, then move to step 7.
7. Inpatient encounters that remain after step 6 are not excluded and continue to be included in the measure’s denominator.
To create the numerator, for each encounter included in the denominator use the following steps:
1. Determine if the first surgical procedure is an intracranial or spinal surgery. If yes, proceed to step 2. If no, proceed to step 3.
2. Determine if a diagnostic imaging study was performed during the encounter and between 5 days and up to 30 days after the end of the first surgical procedure performed during the encounter. To do this, subtract the date of the diagnostic imaging study from the end date of the first surgery performed during the encounter. If there are multiple imaging studies, conduct this subtraction for each imaging study. If any imaging study occurs between 5 and 30 days after the first surgical procedure, then proceed to step 4. If no, proceed to step 5.
3. Determine if a diagnostic imaging study was performed during the encounter and within 30 days or less after the end of the first surgical procedure performed during the encounter. To do this, subtract the date of the diagnostic imaging study from the end date of the first surgery performed during the encounter. If there are multiple imaging studies, conduct this subtraction for each imaging study. If any imaging study occurs between 0 and 30 days after the first surgery, then proceed to step 4. If no, proceed to step 5.
4. If at least one of the following is observed in the patient record, include in the numerator. If none are observed, proceed to step 5.
4a. An anticoagulation medication order within 24 hours after the end of the imaging study during the same encounter.
4b. A heparin intravenous administration within 24 hours after the imaging study, with at least 2 activated partial thromboplastin time (aPTT) heparin therapy monitoring tests or at least 2 anti factor Xa assays within 35 hours of the start of heparin intravenous therapy administration.
4c. Placement of an inferior vena cava (IVC) filter within 24 hours after the end of the imaging study.
4d. A diagnosis of VTE which was not present on admission.
5. Determine whether the patient had a subsequent encounter in which anticoagulation therapy was ordered or prescribed, along with a diagnosis of VTE occurring within 30 days or less after the end of the first surgical procedure of the denominator eligible inpatient encounter. Two steps are required to assess if a patient has a qualifying subsequent encounter.
5a. First, review the patient’s subsequent encounters to determine if any occurred within 30 days or less from the end of the first surgical procedure that occurred during the denominator eligible inpatient encounter. To do this subtract the admission date of the subsequent encounter from the end of first surgical procedure date from the denominator eligible encounter. A subsequent encounter can be an inpatient encounter, emergency department visit, or observation services. If there are no encounters within 30 days of the first surgical procedure that occurred within the denominator eligible inpatient encounter, then the subsequent encounter is not included in the numerator.
5b. Second, determine if the subsequent encounter has both a diagnosis of VTE (which can be either present on admission or not present on admission) and anticoagulant therapy ordered or prescribed during the encounter. If yes, then the subsequent encounter is included in the numerator. If no, do not include in the numerator.
Note: Only one harm (qualifying postoperative VTE) is counted per encounter and within the 30-day evaluation period.
To calculate the risk-adjusted measure score use the following steps:
1. Calculate the observed rate as the number of inpatient hospitalizations found in the numerator count divided by the number of inpatient hospitalizations found in the denominator count.
2. Use the risk adjustment model to estimate the predicted probabilities of denominator-eligible inpatient hospitalization to experience the VTE event. Incorporate the following patient-level factors into the risk-adjustment model: age, sex, bleeding disorders, obesity, cancer, central venous catheter insertion, history of VTE, respiratory operations, vascular surgeries, and stroke. Sum up these discharge-level predicted probabilities per hospital and divide by the denominator count per hospital to obtain the hospital’s expected rate. The expected rate reflects the rate of adverse events that a hospital would anticipate based on the hospital’s case-mix.
3. Calculate the risk-adjusted rate first by dividing the observed rate by the expected rate per hospital, and then by multiplying the resulting observed/expected ratio by the national average observed rate observed among hospitals.
Note: The risk-adjusted rate reflects the performance of a hospital treating its patients relative to the hypothetical average hospital treating patients with the same characteristics.
The measure is not stratified.
CMS, in the future, may use this measure within CMS’ Inpatient Quality Reporting (IQR) Program. Hospitals are eligible to submit quarterly data for IQR program reporting of measures if there are at least five cases in the initial patient population and at least one patient in the denominator (Centers for Medicare & Medicaid Services, 2024).
Reference
Centers for Medicare & Medicaid Services. (2024). Fiscal Year 2026 Hospital Inpatient Quality Reporting (IQR) Program Guide. Washington, DC; CMS: 2024.
Supplemental Attachment
Measure Record
Point of Contact
Not applicable
Donta Henson
Baltimore, MD
United States
Ethan Jacobs
Mathematica
Boston, MA
United States
Importance
Evidence
Deep venous thrombosis (DVT) of the legs or pelvis and pulmonary embolism (PE) are complications that comprise acute venous thromboembolism (VTE) (Henke et al., 2020). Hospital-acquired VTE is a major public health concern and a leading cause of preventable death in the US (CDC, 2024b). The Centers for Disease Control and Prevention estimates that up to 900,000 people in the United States (US) are affected by VTE each year (The Centers for Disease Control and Prevention [CDC], 2024b). Supporting this finding, another study found that hospital-associated VTE occurs in 1.2 percent of admissions and is associated with increased readmissions (hazard ratio [HR], 3.33; 95% CI, 3.25-3.41) and mortality (HR, 1.63; 95% CI, 1.57-1.70) (Neeman et al. 2022). There are processes and structures of care that hospitals can implement to reduce the likelihood that patients experience a VTE event and the associated costs and downstream harms.
Association of VTE with Processes of Care
Hospitals can develop processes of care that aid clinician staff with performing guideline-supported processes of care. For example, to improve adherence to safety practices on a hospital-level, evidence indicates that the adaption of evidence-based guidelines into local protocols significantly increases the proportion of at-risk patients who receive appropriate thromboprophylaxis (Geerts, 2009). Ensuring patients receive appropriate prophylaxis is important because studies suggest that interruptions or delays in the prescription of VTE prophylaxis in surgical patients are associated with a 2- to 3- fold increased risk of VTE, and 34 percent to 56 percent of VTE events occur after discharge (Henke et al., 2020).
Adequate hospital interventions can contribute to the reduction of patient risk of hospital-acquired VTE. One randomized clinical trial of 2,500 at-risk hospital patients found that among patients who were not initially ordered prophylaxis the use of VTE-risk assessment and physician alerts improves prophylaxis use from 14.5 percent to 33.5 percent and reduces thromboembolic complications from 8.2 percent to 4.9 percent (Geerts, 2009). Another study found evidence that combining hospital interventions, such as mechanical and pharmacological prophylaxis, can reduce the incidence of DVT among patients undergoing surgery or admitted with trauma (Kakkos et al., 2022). These findings highlight the importance of the measure to incentivize hospitals to assess their harm reduction efforts and monitor VTE events occurring after surgery.
Additionally, clinical practice guidelines are available for supporting clinician decision-making regarding prevention of VTEs. We provide a non-exhaustive list of published guidance documents in the zip file “5325e-Supplemental-Fall-2025” and the attachment "5325e-Importance-Fall 2025.docx".
Associations of VTE with cost
Researchers estimated that annually VTE events cost the US healthcare system between $7 billion and $10 billion each year (Grosse et al., 2016). These costs include incremental direct medical costs of $12,000 to $15,000 (2014 US dollars) among first-year survivors, controlling for risk factors, and $6,000 to $8,000 of additional costs to care for subsequent complications (such as VTE reoccurrence, post-thrombotic syndrome [PTS], and chronic thromboembolic pulmonary hypertension [CTEPH]) (Grosse et al., 2016). Another set of researchers examining over 1 million hospitalizations found the median cost of a readmission after a hospitalization with acute VTE was $9,782 (Secemsky et al., 2018).
Association of VTE with downstream harms
VTE events can contribute to several adverse consequences, including additional clinical complications and mortality. While symptoms of a DVT are manageable, a PE can be fatal and can cause difficulty breathing, chest pain, coughing up blood, and very low blood pressure (CDC, 2024b). In addition to direct symptoms, VTE events can lead to adverse clinical complications, such as post-thrombotic syndrome (CDC, 2024b). Researchers estimate that PTS occurs in 20 to 50 percent of patients with DVT (one condition included within VTE) and that CTEPH occurs in 1 to 2 percent of patients who experience a PE (another condition included within VTE) (Secemsky et al., 2018). Overall, it is estimated that there are 50,000 VTE-associated postoperative deaths occurring annually in the US (Matthay et al., 2021).
References:
Anderson, D. R., Morgano, G. P., Bennett, C., Dentali, F., Francis, C. W., Garcia, D. A., Kahn, S. R., Rahman, M., Rajasekhar, A., Rogers, F. B., Smythe, M. A., Tikkinen, K. a. O., Yates, A. J., Baldeh, T., Balduzzi, S., Brożek, J. L., Ikobaltzeta, I. E., Johal, H., Neumann, I., . . . Dahm, P. (2019). American Society of Hematology 2019 guidelines for management of venous thromboembolism: prevention of venous thromboembolism in surgical hospitalized patients. Blood Advances, 3(23), 3898–3944. https://doi.org/10.1182/bloodadvances.2019000975
The Centers for Disease Control and Prevention. Data and Statistics on Venous Thromboembolism. (2024a, May 15). Venous Thromboembolism (Blood Clots). https://www.cdc.gov/blood-clots/data-research/facts-stats/index.html&nb…;
The Centers for Disease Control and Prevention. Understanding your risk for Healthcare-Associated VTE (Blood Clots). (2024b, May 15). Venous Thromboembolism (Blood Clots). https://www.cdc.gov/blood-clots/risk-factors/ha-vte.html
Geerts, W. (2009). Prevention of venous thromboembolism: a key patient safety priority. Journal of Thrombosis and Haemostasis, 7, 1–8. https://www.sciencedirect.com/science/article/pii/S1538783622174040
Gould, M. K., Garcia, D. A., Wren, S. M., Karanicolas, P. J., Arcelus, J. I., Heit, J. A., & Samama, C. M. (2012). Prevention of VTE in nonorthopedic surgical patients. CHEST Journal, 141(2), e227S-e277S. https://doi.org/10.1378/chest.11-2297
Grosse, S. D., Nelson, R. E., Nyarko, K. A., Richardson, L. C., & Raskob, G. E. (2016). The economic burden of incident venous thromboembolism in the United States: A review of estimated attributable healthcare costs. Thrombosis research, 137, 3–10. https://doi.org/10.1016/j.thromres.2015.11.033
Henke, P. K., Kahn, S. R., Pannucci, C. J., Secemksy, E. A., Evans, N. S., Khorana, A. A., Creager, M. A., & Pradhan, A. D. (2020). Call to action to prevent venous thromboembolism in hospitalized patients: A policy statement from the American Heart Association. Circulation, 141(24). https://doi.org/10.1161/cir.0000000000000769
Kakkos, S., Kirkilesis, G., Caprini, J. A., Geroulakos, G., Nicolaides, A., Stansby, G., & Reddy, D. J. (2022). Combined intermittent pneumatic leg compression and pharmacological prophylaxis for prevention of venous thromboembolism. The Cochrane database of systematic reviews, 1(1), CD005258. https://doi.org/10.1002/14651858.CD005258.pub4
Key, N. S., Khorana, A. A., Kuderer, N. M., Bohlke, K., Lee, A. Y. Y., Arcelus, J. I., Wong, S. L., Balaban, E. P., Flowers, C. R., Gates, L. E., Kakkar, A. K., Tempero, M. A., Gupta, S., Lyman, G. H., & Falanga, A. (2023). Venous Thromboembolism Prophylaxis and Treatment in Patients With Cancer: ASCO Guideline Update. Journal of clinical oncology : official journal of the American Society of Clinical Oncology, 41(16), 3063–3071. https://doi.org/10.1200/JCO.23.00294
Konstantinides, S. V., Meyer, G., Becattini, C., Bueno, H., Geersing, G., Harjola, V., Huisman, M. V., Humbert, M., Jennings, C. S., Jiménez, D., Kucher, N., Lang, I. M., Lankeit, M., Lorusso, R., Mazzolai, L., Meneveau, N., Áinle, F. N., Prandoni, P., Pruszczyk, P., . . . Zamorano, J. L. (2019). 2019 ESC Guidelines for the diagnosis and management of acute pulmonary embolism developed in collaboration with the European Respiratory Society (ERS). European Respiratory Journal, 54(3), 1901647. https://doi.org/10.1183/13993003.01647-2019
Matthay, Z. A., Flanagan, C. P., Sanders, K., Smith, E. J., Lancaster, E. M., Gasper, W. J., Kornblith, L. Z., Hiramoto, J. S., Conte, M. S., & Iannuzzi, J. C. (2022). Risk factors for venous thromboembolism after vascular surgery and implications for chemoprophylaxis strategies. Journal of vascular surgery. Venous and lymphatic disorders, 10(3), 585–593.e2. https://doi.org/10.1016/j.jvsv.2021.10.001
National Institute for Health and Care Excellence Venous thromboembolic diseases: diagnosis, management and thrombophilia testing NG158. 2020 https://www.nice.org.uk/guidance/ng158 (accessed 2 Jul 2021).
Neeman E, Liu V, Mishra P, et al. Trends and Risk Factors for Venous Thromboembolism Among Hospitalized Medical Patients. JAMA Net Open. 2022;5(11):e2240373. doi:10.1001/jamanetworkopen.2022.40373
Secemsky, E. A., Rosenfield, K., Kennedy, K. F., Jaff, M., & Yeh, R. W. (2018). High Burden of 30-Day Readmissions After Acute Venous Thromboembolism in the United States. Journal of the American Heart Association, 7(13), e009047. https://doi.org/10.1161/JAHA.118.009047
Stevens, S. M., Woller, S. C., Baumann Kreuziger, L., Doerschug, K., Geersing, G. J., Klok, F. A., King, C. S., Murin, S., Vintch, J. R. E., Wells, P. S., Wasan, S., & Moores, L. K. (2024). Antithrombotic Therapy for VTE Disease: Compendium and Review of CHEST Guidelines 2012-2021. Chest, 166(2), 388–404. https://doi.org/10.1016/j.chest.2024.03.003
Measure Impact
The goal of the Hospital Harm – Postoperative Venous Thromboembolism measure is to improve patient safety by incentivizing hospitals to reduce the incidence of postoperative VTE events. The Centers for Disease Control and Prevention (CDC) states that up to 70 percent of hospital-acquired VTE cases are preventable through appropriate anticoagulant treatment or compression stockings; however, fewer than half of hospital patients received these preventive interventions (CDC, 2024).
Postoperative VTEs include both deep vein thrombosis (DVT) and pulmonary embolisms (PE). A DVT is a blood clot that develops in a deep vein and a PE is when the blood clot travels to the lung. While symptoms of a DVT are manageable, a PE can be fatal and cause difficulty breathing, chest pain, coughing up blood, and very low blood pressure (CDC, 2024). In addition to direct symptoms, VTE events can lead to adverse clinical complications, such as post-thrombotic syndrome (CDC, 2024). One-third of patients who experience VTE events will have a recurrence within ten years (CDC, 2024), and VTE events increase a patient’s likelihood of mortality within 90 days of the development of the VTE event (Matthay et al., 2022).
In addition to improvements in morbidity and mortality, reducing instances of postoperative VTE is expected to reduce associated medical costs. Henke and colleagues reported that VTEs resulting from surgical procedures contribute to higher medical costs due to the increased length of hospital stays and likelihood of readmission (Henke et al., 2020). Grosse and colleagues (2016) found that treatment of an acute VTE on average appears to be associated with incremental direct medical costs of $12,000 to $15,000 (2014 US dollars) among first-year survivors.
A potential unintended consequence of this measure is that hospital performance would be affected by surveillance bias. The concern being that hospitals that have aggressive VTE surveillance programs may be unfairly penalized as they detect and treat more VTEs (Bilimoria et al. 2013). This concern is not new and has been raised in relation to the related claims-based measure, Perioperative Pulmonary Embolism or Deep Vein Thrombosis Rate (PSI - 12). The stewards of PSI 12 accounted for surveillance bias by updating the measure’s specifications to only capture (1) proximal (groin/thigh), not distal (calf) vein thromboses diagnosis; and (2) not include solitary subsegmental pulmonary emboli diagnosis. These changes resulted in a decreasing trend in measure rates with no changes in case fatality, which indicates improved specificity to capture VTEs which impact patient mortality. We made these same modifications to the Hospital Harm – Postoperative VTE measure. Capturing VTEs that require medical intervention and impact patient outcomes is the goal of this measure regardless of whether these VTEs are identified through a hospital implemented VTE surveillance program or due to individual patient care protocols.
Another potential unintended consequence of this measure is a more aggressive use of anticoagulants which could result in subsequent major bleeding events. To mitigate this, CMS has developed a complementary measure titled Hospital Harm – Anticoagulant-Related Major Bleeding. This measure captures major bleeding events subsequent to hospital anticoagulant administration.
We will continue monitoring for potential unintended consequences, but we expect the benefits of implementation of the Hospital Harm – Postoperative Venous Thromboembolism measure to outweigh potential unintentional consequences. The measure has the potential to incentivize hospitals to monitor VTE prevention protocols for patients undergoing surgical procedures, track the incidence of postoperative VTE events, and assess the quality of postoperative VTE prophylaxis administered. These actions have the potential to subsequently reduce the incidence of a VTE event, associated direct symptoms, and downstream clinical complications. In addition, the reduction of VTE events can lead to reduced hospital costs, subsequent VTE-related admissions, and mortality due to VTE events.
References:
Bilimoria, K. Y., Chung, J., Ju, M. H., Haut, E. R., Bentrem, D. J., Ko, C. Y., & Baker, D. W. (2013). Evaluation of surveillance bias and the validity of the venous thromboembolism quality Measure. JAMA, 310(14), 1482. https://doi.org/10.1001/jama.2013.280048
The Centers for Disease Control and Prevention. (2024, May 15). Data and Statistics on Venous Thromboembolism. Venous Thromboembolism (Blood Clots). https://www.cdc.gov/blood-clots/data-research/facts-stats/index.html&nb…;
Grosse, S. D., Nelson, R. E., Nyarko, K. A., Richardson, L. C., & Raskob, G. E. (2016). The economic burden of incident venous thromboembolism in the United States: A review of estimated attributable healthcare costs. Thrombosis research, 137, 3–10. https://doi.org/10.1016/j.thromres.2015.11.033
Henke, P. K., Kahn, S. R., Pannucci, C. J., Secemksy, E. A., Evans, N. S., Khorana, A. A., Creager, M. A., & Pradhan, A. D. (2020). Call to action to prevent venous thromboembolism in hospitalized patients: A policy statement from the American Heart Association. Circulation, 141(24). https://doi.org/10.1161/cir.0000000000000769
Matthay, Z. A., Flanagan, C. P., Sanders, K., Smith, E. J., Lancaster, E. M., Gasper, W. J., Kornblith, L. Z., Hiramoto, J. S., Conte, M. S., & Iannuzzi, J. C. (2022). Risk factors for venous thromboembolism after vascular surgery and implications for chemoprophylaxis strategies. Journal of vascular surgery. Venous and lymphatic disorders, 10(3), 585–593.e2. https://doi.org/10.1016/j.jvsv.2021.10.001
Existing VTE measures
Perioperative Pulmonary Embolism or Deep Vein Thrombosis Rate (PSI - 12). PSI 12 is a claims-based measure currently included in the PSI 90 composite measure which is publicly reported on Care Compare and is used by CMS in its Hospital Acquired Conditions Reduction Program. The Hospital Harm – Postoperative VTE measure offers advantages over PSI-12 by leveraging electronic health record (EHR) data, which enables more accurate and timely identification of postoperative VTE events. The EHR data offers greater accuracy, detail, and timeliness compared to claims data.
Venous Thromboembolism Prophylaxis (VTE-1) and Intensive Care Unit Venous Thromboembolism Prophylaxis (VTE-2). CMS currently uses VTE-1 and VTE-2 in the hospital Inpatient Quality Reporting program. These process measures incentivize the proper administration of VTE prophylaxis in the hospital setting. The Hospital Harm-Postoperative VTE outcome measure builds on these process measures and evaluates the incidence of the patient safety event itself. Together, these measures can be used to advance quality improvement efforts around VTE.
Reference:
Matthay, Z. A., Flanagan, C. P., Sanders, K., Smith, E. J., Lancaster, E. M., Gasper, W. J., Kornblith, L. Z., Hiramoto, J. S., Conte, M. S., & Iannuzzi, J. C. (2022). Risk factors for venous thromboembolism after vascular surgery and implications for chemoprophylaxis strategies. Journal of vascular surgery. Venous and lymphatic disorders, 10(3), 585–593.e2. https://doi.org/10.1016/j.jvsv.2021.10.001
We asked five patient and caregiver representatives of the Hospital Harm Technical Expert Panel (TEP) about the measure’s meaningfulness. Specifically, we asked whether the HH- Postoperative VTE measure is meaningful and whether it produces information that is valuable in making care decisions. Response options were strongly agree/agree/disagree/strongly disagree. 5 out of 5 patients indicated “agree.”
Performance Gap
Using data from 34 hospitals in the Truveta dataset, we examined 50,203 encounters in calendar year (CY) 2022 and 50,708 encounters in CY 2023. We list below the risk-adjusted performance scores by deciles for CY 2022-2023. Unadjusted performance scores for CY2022-2023 are found in the "5325e-Performance-Fall 2025. xlsx" attachment.
| Overall | Minimum | Decile_1 | Decile_2 | Decile_3 | Decile_4 | Decile_5 | Decile_6 | Decile_7 | Decile_8 | Decile_9 | Decile_10 | Maximum | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Mean Performance Score | 1.12% | 0% | 0.21% | 0.45% | 0.56% | 0.67% | 0.78% | 0.91% | 0.99% | 1.31% | 1.94% | 4.21% | 5.71% |
| N of Entities | 34 | 1 | 4 | 4 | 4 | 4 | 3 | 3 | 3 | 3 | 3 | 3 | 1 |
| N of Persons / Encounters / Episodes | 100911 | 263 | 4391 | 8401 | 21371 | 16284 | 12470 | 10167 | 12051 | 6324 | 3194 | 6258 | 3203 |
Care Gaps
Closing Care Gaps
Venous thromboembolism (VTE), including deep vein thrombosis (DVT) and pulmonary embolism (PE), is a leading cause of preventable postoperative morbidity and mortality (Nicholson et al., 2020). Despite the availability of effective evidence-based strategies, significant risks are associated across surgical populations and subpopulations (Nicholson et al., 2020).
Evidence of Known Gaps in Care
- Underutilization of prophylaxis. Grant et al. (2022) found a 22 percent underuse rate of VTE prophylaxis for high-risk patients, emphasizing that hospitals should engage in efforts in improvement.
- Age. Individuals over the age of 70 account for the majority of VTE events, likely due to age-related increases in procoagulant factors and the prevalence of comorbid conditions (Nicholson et al., 2020). During a technical expert panel (TEP) meeting convened to discuss this measure, a member also acknowledged age as a risk factor for post-surgical VTE events and recommended the project team to consider including age as part of the risk model. Age and risk of VTE is also reflected in the Caprini Score for VTE, which assigns increasing risk for older age groups as a predictor of VTE risk (Caprini, 2010).
- Sex. Sex also plays a role in VTE risk. During the childbearing years, women experience a higher incidence of VTE. Outside of childbearing years, however, the incidence of VTE is greater in men (Nicholson et al., 2020).
Methodology
To understand the gap in VTE performance among hospitals, we examined the rates of postoperative VTE events in 34 hospitals from 2022 to 2023 using Truveta data. We first used indirect standardization with logistic regression to evaluate hospital performance, incorporating differences by age and sex. Risk predictions from the full patient population were applied to each hospital’s case mix to calculate expected outcome rates. The ratio of observed to expected outcomes (O/E) was then used to compare hospital performance, with values below 1 indicating better-than-expected quality and values above 1 indicating worse-than-expected quality. To validate the risk-adjustment model, we applied a bootstrap resampling method with 300 samples, which allowed us to assess predictive accuracy and reduce bias in identifying significant risk factors. We evaluated both univariate and multivariate models for their association with VTE outcomes. To assess differences in postoperative VTE risk across patient subgroups, we relied on the odds ratios for age and sex generated from the multivariate model to isolate the differences by subgroup.
Results/Interpretation
Age is a strong and consistent predictor of postoperative VTE risk, with increasing odds across all older age groups compared to the reference group (18-29 years old). Odds ratios rose steadily across age groups, ranging from 1.30 (p = 0.032) for ages 30–39, to 1.51 (p < 0.001) for ages 40–49, and further to 1.80 (p < 0.001) for ages 50–59. Patients ages 60–69 and 70–79 had nearly 1.9 times the odds of VTE compared with the youngest group (OR = 1.92 and 1.94, both p < 0.001), while those 80 years and older had 1.67 times the odds (p < 0.001).
Sex was also significantly associated with postoperative VTE events: males had higher odds than females (OR = 1.14, p < 0.001).
Anticipated Impact
These findings show that there are substantial differences in risk of post-operative VTE by age and sex that hospitals can account for in developing strategies for preventing and managing risk of VTE.
Transparency and accountability: Hospitals can use this measure to compare performance across departments and peer institutions, identifying areas for improvement.
Targeted quality improvement: By internally stratifying data by age and sex, health systems can implement focused interventions to address differences in postoperative VTE events.
Patient safety:
Implementing VTE clinical practice guidelines and evidence-based practices in hospitalized medical patients is associated with fewer hospital-acquired VTE events, reduced complications, shorter hospital stays, and improved recovery; therefore, adopting this measure can drive higher quality of care and patient safety (Abboud et al., 2020).
References:
Abboud, J., Abdel-Rahman, A., Kahale, L., Dempster, M., & Adair, P. (2020). Prevention of health care associated venous thromboembolism through implementing VTE prevention clinical practice guidelines in hospitalized medical patients: A systematic review and meta-analysis. Implementation Science, 15(1), 49. https://doi.org/10.1186/s13012-020-01008-9
Caprini, J. A. (2010). Risk assessment as a guide for the prevention of the many faces of venous thromboembolism. The American Journal of Surgery, 199(1 Suppl), S3–S10. https://doi.org/10.1016/j.amjsurg.2009.10.006
Grant, P. J., Conlon, A., Chopra, V., & Flanders, S. A. (2018). Use of Venous Thromboembolism Prophylaxis in Hospitalized Patients. JAMA internal medicine, 178(8), 1122–1124. https://doi.org/10.1001/jamainternmed.2018.2022
Henke, P. K., Kahn, S. R., Pannucci, C. J., Secemsky, E. A., Evans, N. S., Khorana, A. A., Creager, M. A., & Pradhan, A. D. (2020). Call to action to prevent venous thromboembolism in hospitalized patients: A policy statement from the American Heart Association. Circulation, 141(24), e914–e931. https://doi.org/10.1161/CIR.0000000000000769[1]
Kakkos, S., Kirkilesis, G., Caprini, J. A., Geroulakos, G., Nicolaides, A., Stansby, G., & Reddy, D. J. (2022). Combined intermittent pneumatic leg compression and pharmacological prophylaxis for prevention of venous thromboembolism. The Cochrane database of systematic reviews, 1(1), CD005258. https://doi.org/10.1002/14651858.CD005258.pub4
Nicholson, M., Chan, N., Bhagirath, V., & Ginsberg, J. (2020). Prevention of Venous Thromboembolism in 2020 and Beyond. Journal of clinical medicine, 9(8), 2467. https://doi.org/10.3390/jcm9082467
Feasibility
Feasibility
We collaborated with a total of 15 hospitals to complete the eCQM feasibility scorecard, which assesses whether the data required for hospital-level calculation are available in structured fields, are collected through routine workflows, are documented using standard terminology, and are accurate. Of these, 13 hospitals evaluated the feasibility of the data elements necessary to calculate the observed measure score, while 2 hospitals focused on assessing the feasibility of the risk adjustment data elements.
Data used in the Hospital Harm – Postoperative Venous Thromboembolism measure are generated or collected by and used by health care personnel during provision of care and may be coded by someone other than the person obtaining original information (e.g., Diagnosis-Related Group, International Classification of Diseases, 10th Revision, Clinical Modification/Procedure Coding System codes). All data elements are defined in structured fields within the electronic health records (EHR) system.
To assess missing data’s impact on measure scores, we conducted a sensitivity analysis using chart abstracted information for 40 encounters (20 per test site) as the gold standard and upon which the electronically extracted data was evaluated from two different electronic health record systems. Sensitivity assesses whether patients who should be grouped into the denominator, exclusion, and numerator categories are accurately identified.
Missing data in the EHR did not prevent cases from being correctly assigned to the denominator or numerator. However, missing data in the EHR on denominator exclusion criteria did result in some denominator cases (5 of 31) not being properly excluded. For example, at one test site, for a limited number of cases the hospital captured VTE diagnoses that were present on admission in a clinical note, but not in a structured field of the EHR, resulting in the EHR undercounting exclusions (and correspondingly, overcounting denominator and potentially numerator cases). If hospitals fail to consistently document exclusion-related criteria in structured fields, the measure may not exclude all patients who should be removed from the denominator. However, these results may be driven by site-specific factors, given that feasibility results from another cohort of 13 sites indicated that these exclusion-related data elements are available, accurate, collected through normal workflow, and coded to terminology standards.
Because the Hospital Harm – Postoperative Venous Thromboembolism (VTE) measure relies on electronic health record (EHR) data, organizations must ensure that their systems are capable of capturing relevant clinical information accurately. This may require updates to documentation workflows, integration of structured data fields, and alignment of coding practices with measure specifications.
Clinician workflow may also be affected, particularly if documentation practices need to be modified to support measure implementation, both with respect to the data needed to calculate the measure and to apply the risk adjustment model. For example, clinicians may be required to enter prophylaxis decisions, VTE risk assessments, or VTE diagnoses in structured formats rather than free-text notes.
Patient–physician interactions are unlikely to be directly impacted by the measure. Institutions should monitor for unintended consequences and consider workflow redesigns that preserve the quality of patient care while supporting data capture.
Some barriers may arise during implementation. These include variability in EHR capabilities across hospitals within a system, inconsistent coding practices, and challenges in linking data across encounters (e.g., outpatient surgery followed by inpatient admission).
Despite these challenges, the long-term benefits of implementing the measure—including improved patient safety, reduced VTE-related complications, and enhanced accountability—justify the adoption of this measure.
The Hospital Harm – Postoperative Venous Thromboembolism (VTE) measure is designed to be implemented using structured data from electronic health records (EHRs), which supports secure and confidential data collection. The measure does not rely on patient surveys.
The Feasibility Scorecard assessing the measure’s data elements indicated that all data elements were feasible. Therefore, the measure team did not change any measure specifications.
The missing ECMO data from test sites who do not conduct ECMOs would not impact those hospitals’ measure scores. Given the ECMO is an exclusion criteria, the missing ECMO data would result in no patients being excluded due to ECMO at those sites which is appropriate given ECMO is not offered at those sites.
In addition to testing the feasibility of the measure’s data elements, we also tested the feasibility of the risk adjustment data elements, collecting data from two hospitals. For the eight data elements that we included in the final version of the tested risk model, both hospitals confirmed that their EHRs captured most of the data elements—such as bleeding disorders, cancer, respiratory operations, central venous catheter placement, vascular surgeries, and obesity—in structured fields. The exception is that one hospital did not consistently document history of venous thromboembolism (VTE) or stroke in structured fields, and the hospital noted these data elements may instead be documented in free-text fields.
Proprietary Information
Scientific Acceptability
Testing Data
We used two data sources to test the Hospital Harm- Postoperative VTE measure, one for developing the risk-adjustment model, obtaining performance results, and calculating reliability of adjusted and non-adjusted measure scores; and one for testing the data element validity, face validity, and usability of the measure.
Truveta data. We used two consecutive years of EHR data from Truveta (2022-2023) to develop the risk adjustment model, calculate measure scores, and compute and obtain importance/performance distributions and reliability results. The Truveta test sites spanned six states—Texas, Oregon, New Mexico, Alaska, California, and Washington—and were Epic-based.
Society of Critical Care Medicine’s (SCCM) Discovery, The Critical Care Research. Society of Critical Care Medicine’s (SCCM) Discovery, The Critical Care Research Network was used as a coordinating center. SCCM’s Discovery enrolled two hospitals within the measure that provided two consecutive years of EHR de-identified limited datasets (2022-2023) to test the measures’ data element validity. SCCM’s Discovery enrolled four sites within the network to participate in interviews with clinicians for the purpose of obtaining face validity and usability results.
Truveta data: January 2022-December 2023
Hospital data: January 2022-December 2023
We used two data sources to test the Hospital Harm- Postoperative VTE measure, one for developing the risk-adjustment model and assessing the performance gap and reliability of adjusted and non-adjusted measure scores, and one for testing the data element validity, face validity, and usability of the measure.
Truveta data. To generate the risk adjustment model, we used EHR data from 444,518 unique denominator-eligible patients. This sample was ideal for developing the risk-adjustment model since it allowed us to test for the association between patient characteristics and postoperative VTE in a large national sample to improve generalizability.
To calculate performance scores and assess measure reliability, we needed to attribute stays to hospitals. The Truveta platform provides anonymized hospital IDs for a subset of their collaborating systems. As such, we relied on a smaller sample consisting of EHR data from 100,911 denominator encounters using 34 hospitals.
Society of Critical Care Medicine’s (SCCM) Discovery, The Critical Care Research. Society of Critical Care Medicine’s (SCCM) Discovery, The Critical Care Research Network was used as a coordinating center. SCCM’s Discovery enrolled two hospitals that provided two consecutive years of EHR de-identified limited datasets (2022-2023) to test the measures’ data element validity. SCCM’s Discovery enrolled four hospitals within the network to also conduct interviews with clinicians and obtain face validity and usability results.
We used two data sources – Truveta and hospitals recruited by the Society of Critical Care Medicine’s (SCCM) Discovery, The Critical Care Research Network for the scientific acceptability and risk adjustment testing.
Truveta. The Truveta platform provides anonymized hospital IDs for a subset of their collaborating systems. As such, we relied on a sample consisting of EHR data from 100,911 denominator encounters from 34 hospitals. The Truveta hospitals spanned six states: Texas, Oregon, New Mexico, Alaska, California, and Washington. All of the hospitals used Epic as their EHR. Although we confirmed that all hospitals were inpatient, Truveta’s use of anonymized hospital IDs prevents access to hospital-level characteristics, including bed size. We were able to obtain denominator encounters per hospital. In 2022, the number of denominator encounters per hospital ranged from 62 to 6,746, and in 2023, the range was 56 to 6,716.
Society of Critical Care Medicine’s (SCCM) Discovery, The Critical Care Research Network was used as a coordinating center. SCCM’s Discovery enrolled inpatient hospitals from diverse geographic regions, primarily within academic-based systems. Of the hospitals that participated in the data element validity assessment—one from North Carolina with more than 850 beds that uses Epic, and another from Puerto Rico with more than 200 beds that uses Meditech—we were able to obtain 20 randomly selected cases to support analysis of data element validity. The two additional hospitals that participated in face validity testing were located in Virginia and New York.
Please refer to the zip file “5325e-Supplemental-Fall-2025” and the attachment “5325e-Population-Fall-2025.xlsx.
Reliability
Measure score reliability:
To assess the reliability of the measure’s scores, we used the signal-to-noise (SNR) method, revealing the precision of entity-level scores. We used the empirical Bayesian estimation scheme since the measure is a risk-adjusted measure. The empirical Bayesian approach assumes that the distribution of hospital measure scores is continuous and normally distributed, and no hospital characteristics, other than quality, influence the true hospital measure rates. In the context of risk-adjusted measures, the patient-level score represents a probability of an event occurring, rather than a simple binary outcome. The empirical Bayesian approach is based on continuous distributions for the outcome and is therefore suitable for testing the reliability of the risk adjusted Hospital Harm - Postoperative VTE measure. The total number of encounters was 100,911, (50,203 encounters in 2022 and 50,708 encounters in 2023).
Data element reliability testing:
We also compared electronically extracted test site data to manually abstracted data from the patient record to test for data element reliability.
We tested data element reliability by assessing the agreement for a set of randomly selected patients (N=20 per site for a total of 40 across sites) between the measure’s data elements from two sources: EHR data extracted by test sites from the structured fields and data manually abstracted by test sites from the patient’s medical record. To aid test sites in the manual abstraction, we provided test sites with a template showing each of the data elements to abstract for each record.
We randomly selected a subset of 20 extracted records per test site from the test site EHR data extract for data element validity testing. The patient records represented cases meeting exclusion, denominator, and numerator criteria. Abstractors used the data extraction template to identify whether data elements found in the EHR were also found through manual chart review—to calculate positive predictive value (PPV)—as well as to identify whether relevant data found through manual chart review was missing from the EHR extract—to calculate negative predictive value (NPV). We then used the data from each of the completed templates to calculate the percentage agreement as well as the Gwet’s AC1 statistic (inter-rater agreement), sensitivity, specificity, PPV, and NPV for the measure data elements.
Measure score reliability:
Because the measure is intended to be risk-adjusted, we evaluated the reliability of its risk-adjusted version by calculating percentiles based on the number of encounters across measured entities. Using 2022 data, we observed that the signal-to-noise reliability estimates were consistent across hospitals with a reliability score among the 10th percentile of 0.9998 and a reliability score among the 90th percentile of 0.9999. The minimum reliability score was 0.9998 and the maximum reliability score was 0.9999. Using 2023 data, we observed that the signal-to-noise reliability estimates did not change from the 2022 reported results. Additional results are found in the "5325e-Reliability-Fall-2025.xlsx" attachment.
Data element reliability testing:
Please note that the results for data element reliability are also found in the validity section of the submission (Section 5.3.4). We calculated percent agreement for critical data elements included in the measure, using the chart abstracted data as the ‘gold-standard’. Percent agreement results can be found in the attachment: 5325e-Validity-Fall-2025. xlsx, which is saved in field 5.3.4a.
We calculated four additional statistics to evaluate data element reliability: positive predictive value (PPV), sensitivity, negative predictive value (NPV), and specificity. We report these statistics separately by each test site below. Additional statistics can be found in the attachment: 5325e-Validity-Fall-2025. xlsx, which is saved in field 5.3.4a.
Measure score reliability:
The measure score reliability results from calendar years 2022 and 2023 (the latter results are reported in the attachment "5325e-Reliability-Fall-2025") both exceed the 0.6 reliability testing threshold published by the Partnership for Quality Measurement (2023) and are approximately 1, indicating high reliability. Large sample size and low measure rates reduce the "noise" or random variability within hospitals, making it easier to discern true differences in performance between hospitals, thus resulting in very high reliability of the measure at the hospital level. High signal-to-noise ratio, in turn, indicates that a larger proportion of the observed variance in the measure is attributable to real differences between hospitals relative to the random variation in performance (noise) within hospitals.
Endorsement and Maintenance (E&M) Guidebook. (2023). https://p4qm.org/sites/default/files/Del-3-6-Endorsement-and-Maintenanc…
Data element reliability:
Percentage agreement represents the proportion of instances where two sources, electronic data and manual abstraction, agree on the classification of a data element.
We provide interpretation of the sensitivity and specificity results as those two assessments provide information regarding whether the eCQM accurately captures the true classification of patients into the denominator, exclusion, and numerator groupings. Sensitivity and specificity is considered high for >90%, moderate for results between 80% and 90%, and limited for results below 80%.
Sensitivity and specificity
Sensitivity assesses whether patients who should be grouped into the denominator, exclusion, and numerator categories are accurately identified. One site’s electronic data showed moderate to high sensitivity for classifying cases into the denominator (82%) and numerator (100%) and limited sensitivity for classifying cases as exclusions (40%). The limited sensitivity for the exclusion was due to the manual extraction identifying VTE diagnoses that were present on admission, that were not available in the electronic extraction. The site clarified that these are cases in which the patient had been diagnosed with VTE shortly prior to the inpatient stay but in which VTE was not the primary reason for the inpatient stay, and in those instances that earlier VTE was documented in the non-structured history and physical section of the record; the site noted these are edge cases that would not be prevalent in their full data set. The second site’s electronic data showed high sensitivity for classifying cases into the numerator (100%) and moderate sensitivity for classifying cases into the denominator (82%), and sensitivity just under the moderate threshold (75%) for classifying cases into the denominator exclusion group. The lower sensitivity results for exclusion cases for this second site was due to two instances in which the electronic data did not include data on exclusion factors identified through manual review. In one instance, the electronic data did not include procedure codes for intracranial surgery, and in another instance the electronic data included diagnosis codes for VTE that are not part of the measure’s value set. We will consider whether to expand the value set to include these additional VTE diagnosis codes in future measure updates.
Specificity assesses whether patients who should not be grouped into the denominator, exclusion, and numerator are appropriately not included within these categories. One site’s electronic data showed moderate to high specificity for not incorrectly classifying cases into any of these groups, with results ranging from 81% to 93%. The second site’s electronic data showed excellent specificity for not incorrectly classifying cases as exclusions (100%) and moderate specificity for not incorrectly classifying cases as denominators (89%) or numerators (84%).
Percent agreement
Among all the data elements we tested, the data element representing transfer encounters has the lowest percentage agreement score of 0.30, and this score is based on data from one of the two test sites. (The other site’s transfer data was unable to be assessed due to processing differences between the manual and electronic extraction.) The transfer data element is used to identify whether there are encounters for emergency department, observation, and outpatient surgery services that should be included in the measure calculation because they occur within an hour or less of the start of the inpatient encounter. As such, if this data element is inaccurate the measure may inappropriately include or exclude information from emergency department, observation, and outpatient surgery services. However, this data element and the timing relationships it captures are included in other hospital harm eCQMs (such as the Hospital Harm (HH) – Severe Hyperglycemia measure that has been in a CMS program since 2023) and as such, we expect this low agreement to likely be a site-specific issue.
Other data elements used within the measure had stronger agreement:
- Denominator: All other denominator data elements with the exception of transfers, for both test sites, had 100% percentage agreement rates.
- Denominator exclusion: Data elements representing the denominator exclusions, for both test sites, had agreement rates of between 95% and 100%.
- Numerator: Data elements representing the numerator, for both test sites, had between 80% - 100% agreement.
- Risk adjustment: Data elements used for risk adjustment for both test sites, had between 80% - 100% agreement, with the exception of obesity risk which had an agreement of 60% at one site and 65% at a second site.
Data element reliability: Summary
In summary, the agreement rates of the individual data elements were strong. The sensitivity and specificity results ranged from moderate to excellent for all groups and sites with two exceptions, both related to the sensitivity of classing cases as denominator exclusions. At the first site, the sensitivity of classifying cases as exclusions fell slightly under the threshold for moderate sensitivity, and at the second site sensitivity of classifying cases as exclusions was limited; the limited sensitivity at this latter site was due to diagnoses of VTE POA that were not captured in structured fields. This second site noted that these particular cases with the VTE POA diagnoses were edge cases likely not indicative of how data are captured across all of their patients. Moreover, as eCQMs move to Fast Healthcare Interoperability Resources
(FHIR) we expect data elements related to diagnoses that are POA to become even more reliable as data on their POA status will be pulled from claims. We believe that these two instances of limited sensitivity are likely due to workflow issues specific to that site rather than indicative of broader reliability limitations, given that feasibility results from 13 sites indicated that these data elements are available, accurate, collected through normal workflow, and coded to terminology standards.
| | Overall | Minimum | Decile_1 | Decile_2 | Decile_3 | Decile_4 | Decile_5 | Decile_6 | Decile_7 | Decile_8 | Decile_9 | Decile_10 | Maximum |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Reliability | 1.0000 | 0.9998 | 0.9999 | 0.9999 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 |
| Mean Performance Score | 1.0139% | 1.6835% | 1.4223% | 1.2129% | 0.9141% | 0.3865% | 1.0237% | 1.1458% | 0.5982% | 0.8200% | 1.8656% | 0.7903% | 0.6378% |
| N of Entities | 34 | 1 | 4 | 4 | 4 | 4 | 3 | 3 | 3 | 3 | 3 | 3 | 1 |
| N of Persons / Encounters / Episodes | 50203 | 62 | 344 | 667 | 1658 | 2465 | 3063 | 3609 | 4105 | 6226 | 9424 | 18642 | 6746 |
Validity
We compared electronically extracted test site data to manually abstracted data from the patient record to test for data element validity and we interviewed clinicians and the Patient Safety Hospital Harm TEP members about the measure’s face validity to test the measure score validity.
Data element validity testing
We tested data element validity by assessing the agreement for a set of randomly selected patients (N=20 per site for a total of 40 across sites) between the measure’s data elements from two sources: EHR data extracted by test sites from the structured fields and data manually abstracted by test sites from the patient’s medical record. To aid test sites in the manual abstraction, we provided test sites with a template showing each of the data elements to abstract for each record.
We randomly selected a subset of 20 extracted records per test site from the test site EHR data extract for data element validity testing. The patient records represented cases meeting exclusion, denominator, and numerator criteria. Abstractors used the data extraction template to identify whether data elements found in the EHR were also found through manual chart review—to calculate positive predictive value (PPV)—as well as to identify whether relevant data found through manual chart review was missing from the EHR extract—to calculate negative predictive value (NPV). We then used the data from each of the completed templates to calculate the percentage agreement as well as the Gwet’s AC1 statistic (inter-rater agreement), sensitivity, specificity, PPV, and NPV for the measure data elements.
Face validity
We conducted hour-long interviews with six clinical staff from the three test sites to assess the measure’s face validity, and we augmented these findings by collecting similar feedback from Patient Safety project’s Hospital Harm TEP members. We asked both sets of respondents to respond to the following statements, based on the measure specifications provided (scored on a 4-point Likert scale):
- The measure score is an accurate reflection of quality (4 = strongly agree, 3 = agree, 2 = disagree, 1 = strongly disagree).
- The measure score can be used to distinguish between good and poor quality of care (4 = strongly agree, 3 = agree, 2 = disagree, 1 = strongly disagree).
Additionally, we asked clinician interview subjects to explain their responses if they disagreed or strongly disagreed with either of the two questions above, and we also asked for recommendations that would help strengthen the face validity of the measure.
Face Validity
We polled the Hospital Harm TEP members on the face validity of the HH – Postoperative VTE eCQM as currently specified. About 89 percent of TEP members (17 of 19 voting) agreed or strongly agreed that the measure score accurately reflects quality of care, and 79 percent (15 of 19 voting) agreed or strongly agreed that the measure score can be used to distinguish between good and poor quality of care.
We interviewed 7 hospital clinicians on the measure’s ability to (a) distinguish between hospitals with good quality of care and poor quality of care and (b) accurately reflect hospital quality. We used a scale of strongly disagree/disagree/agree/ strongly agree. The specialties of the clinicians interviewed were internal medicine, trauma, and neurocritical care. 4 out of 7 clinicians agreed that the risk-adjusted version of the measure can be used to distinguish between hospitals with good quality of care and hospitals with poor quality of care. The three clinicians that disagree provided the following rationales.
- One clinician stated that the measure needs to include a baseline surveillance rate in the risk-adjusted model because some hospitals conduct more imaging studies than others and therefore will have an increased number of VTE events. This concern is similar to concerns raised by implementers about surveillance bias in PSI-12, which had led to changes to PSI-12’s value sets to address those concerns. In response to the concerns about surveillance bias raised by the clinician we interviewed about the VTE measure, the measure team aligned the VTE value set with the PSI-12 VTE diagnosis codes, which account for surveillance bias. Specifically, the team refined the value set by removing diagnosis codes for distal (calf) vein thromboses, solitary subsegmental pulmonary emboli, and unspecified VTEs.
- The second clinician stated that the measure should be paired with a related process measure to distinguish quality. In response to this feedback, we note that two process measures, Venous Thromboembolism Prophylaxis and Intensive Care Unit Venous Thromboembolism Prophylaxis, are currently implemented in the Hospital Inpatient Quality Reporting (IQR) Program.
- The remaining clinician stated that hospital outcomes cannot accurately distinguish between the quality of care delivered by hospitals, as those outcomes are influenced by hospital location and patient population. We expect that, while it did not sway the opinion of this clinician, the inclusion of the risk adjustment model within the measure will mitigate concerns among most implementers regarding variability in patient populations.
We also asked these 7 clinicians whether the measure score provides an accurate reflection of hospital quality using a scale of strongly disagree/disagree/agree/ strongly agree. 5 out of 7 clinicians agreed that the risk-adjusted version of the measure provides an accurate reflection of hospital quality. The two clinicians who disagree provided the following feedback:
- One clinician re-stated that the measure needs to include a baseline surveillance rate in the risk-adjusted model.
- The other clinician stated that postoperative VTE outcomes are not necessarily indicative of care quality, and emphasized that adherence to evidence-based protocols was a more accurate reflection of hospital quality.
Data element validity
We calculated percent agreement for critical data elements included in the measure, using the chart abstracted data as the ‘gold-standard’. Percent agreement results can be found in the attachment: 5325e-Validity-Fall-2025. xlsx
We calculated four additional statistics to evaluate data element validity: positive predictive value (PPV), sensitivity, negative predictive value (NPV), and specificity. We report these statistics separately by each test site below. Additional statistics can be found in the attachment: 5325e-Validity-Fall-2025. xlsx
Face validity:
The majority of the respondents indicated that the risk-adjusted version of the measure as specified can be used to distinguish between hospitals with good quality of care and hospitals with poor quality of care.
Data element validity:
Percentage agreement represents the proportion of instances where two sources, electronic data and manual abstraction, agree on the classification of a data element.
We provide interpretation of the sensitivity and specificity results as those two assessments provide information regarding whether the eCQM accurately captures the true classification of patients into the denominator, exclusion, and numerator groupings. Sensitivity and specificity is considered high for >90%, moderate for results between 80% and 90%, and limited for results below 80%.
Sensitivity and specificity
Sensitivity assesses whether patients who should be grouped into the denominator, exclusion, and numerator categories are accurately identified. One site’s electronic data showed moderate to high sensitivity for classifying cases into the denominator (82%) and numerator (100%) and limited sensitivity for classifying cases as exclusions (40%). The limited sensitivity for the exclusion was due to the manual extraction identifying VTE diagnoses that were present on admission, that were not available in the electronic extraction. The site clarified that these are cases in which the patient had been diagnosed with VTE shortly prior to the inpatient stay but in which VTE was not the primary reason for the inpatient stay, and in those instances that earlier VTE was documented in the non-structured history and physical section of the record; the site noted these are edge cases that would not be prevalent in their full data set. The second site’s electronic data showed high sensitivity for classifying cases into the numerator (100%) and moderate sensitivity for classifying cases into the denominator (82%), and sensitivity just under the moderate threshold (75%) for classifying cases into the denominator exclusion group. The lower sensitivity results for exclusion cases for this second site was due to two instances in which the electronic data did not include data on exclusion factors identified through manual review. In one instance, the electronic data did not include procedure codes for intracranial surgery, and in another instance the electronic data included diagnosis codes for VTE that are not part of the measure’s value set. We will consider whether to expand the value set to include these additional VTE diagnosis codes in future measure updates.
Specificity assesses whether patients who should not be grouped into the denominator, exclusion, and numerator are appropriately not included within these categories. One site’s electronic data showed moderate to high specificity for not incorrectly classifying cases into any of these groups, with results ranging from 81% to 93%. The second site’s electronic data showed excellent specificity for not incorrectly classifying cases as exclusions (100%) and moderate specificity for not incorrectly classifying cases as denominators (89%) or numerators (84%).
Percent agreement
Among all the data elements we tested, the data element representing transfer encounters has the lowest percentage agreement score of 0.30, and this score is based on data from one of the two test sites. (The other site’s transfer data was unable to be assessed due to processing differences between the manual and electronic extraction.) The transfer data element is used to identify whether there are encounters for emergency department, observation, and outpatient surgery services that should be included in the measure calculation because they occur within an hour or less of the start of the inpatient encounter. As such, if this data element is inaccurate the measure may inappropriately include or exclude information from emergency department, observation, and outpatient surgery services. However, this data element and the timing relationships it captures are included in other hospital harm eCQMs (such as the Hospital Harm (HH) – Severe Hyperglycemia measure that has been in a CMS program since 2023) and as such, we expect this low agreement to likely be a site-specific issue.
Other data elements used within the measure had stronger agreement:
- Denominator: All other denominator data elements with the exception of transfers, for both test sites, had 100% percentage agreement rates.
- Denominator exclusion: Data elements representing the denominator exclusions, for both test sites, had agreement rates of between 95% and 100%.
- Numerator: Data elements representing the numerator, for both test sites, had between 80% - 100% agreement.
- Risk adjustment: Data elements used for risk adjustment for both test sites, had between 80% - 100% agreement, with the exception of obesity risk which had an agreement of 60% at one site and 65% at a second site.
Data element validity: Summary
In summary, the agreement rates of the individual data elements were strong. The sensitivity and specificity results ranged from moderate to excellent for all groups and sites with two exceptions, both related to the sensitivity of classing cases as denominator exclusions. At the first site, the sensitivity of classifying cases as exclusions fell slightly under the threshold for moderate sensitivity, and at the second site sensitivity of classifying cases as exclusions was limited; the limited sensitivity at this latter site was due to diagnoses of VTE POA that were not captured in structured fields. This second site noted that these particular cases with the VTE POA diagnoses were edge cases likely not indicative of how data are captured across all of their patients. Moreover, as eCQMs move to Fast Healthcare Interoperability Resources
(FHIR) we expect data elements related to diagnoses that are POA to become even more valid as data on their POA status will be pulled from claims. We believe that these two instances of limited sensitivity are likely due to workflow issues specific to that site rather than indicative of broader validity limitations, given that feasibility results from 13 sites indicated that these data elements are available, accurate, collected through normal workflow, and coded to terminology standards.
Risk Adjustment
The Hospital Harm – Postoperative Venous Thromboembolism (VTE) measure is risk-adjusted because there are some factors that are independent of quality that may produce worse outcomes for the hospital, thus, penalizing hospitals for a sicker hospital case mix. To identify risk factors for postoperative VTE events we engaged in multiple steps.
Step 1. We gathered an initial set of candidate risk factors by reviewing published literature and through discussions with the Patient Safety Hospital Harm technical expert panel (TEP). As part of this first step, we identified candidate risk factors using scoring indices such as the Rogers Score, the Caprini Score for Venous Thromboembolism, and an intensive care unit – VTE risk assessment. We included the variables within these indices within our set of candidate risk factors.
Step 2. We reviewed the candidate risk factors with two clinical subject matter experts (SMEs) on the project team and added or removed risk factors based on their perspective of the factor’s clinical relevance to postoperative VTE events and a hospital’s ability to mitigate the factor’s risk by using best practices.
Step 3. Building on the feedback received from TEP members and clinical SMEs, we developed a list of risk factors for potential inclusion in the measure’s risk model. We then asked the TEP members to provide their feedback on each of these risk factors through a survey. We used this feedback to finalize the clinical conceptual risk factor model which we then tested in patient level data.
In addition to clinical risk factors, we tested the association between age and sex and risk of postoperative VTE events.
To develop the risk-adjustment model, we used data obtained from Truveta. First, we developed the risk-adjustment model on the original patient cohort. We implemented a stepwise logistic regression process with backward elimination of variables, using 300 bootstrap samples derived from the entire measure population via random selection with replacement. The bootstrap resampling method effectively allowed us to obtain 300 samples of the same size as the original measure cohort but with a different patient case mix. For each bootstrap sample, we estimated a logistic regression model, including all the candidate risk factors. We retained all variables that showed an association with postoperative VTE events at p < 0.15 in 70 percent of the bootstrap samples (that is, in 210 out of 300 samples).
We then fitted the following models:
1. A set of univariate models measuring the association between each candidate risk factor and the outcome unadjusted for the other candidate risk factors.
2. A multivariate logistic regression model that captures the effect of all candidate risk factors on the outcome simultaneously, and in which the effect of each candidate risk factor is adjusted for the other variables in the model. We fit separate multivariate models with and without social risk factors.
We evaluated the univariate and multivariate associations between each candidate risk factor and the outcome and determined whether the effects (odds ratios representing one unit change on a continuous variable or a difference between patient groups on the categorical variables) are in the expected direction.
We selected risk factors for inclusion into the final risk-adjustment model by evaluating the multivariate associations and bootstrap resampling results. We selected clinical risk factors for the final risk-adjustment model if the following criteria were met:
1. The odds ratio (OR) from the multivariate analysis indicated statistically significant adverse risk, not a protective factor (OR > 1.0, p-value <0.05), and
2. There was p value < 0.15 in 70 percent of the bootstrap samples generated during the bootstrap resampling.
Attachment "5325e-Conceptual-Fall-2025" displays the conceptual model for risk adjustment considers patients’ key demographic characteristics and clinical/comorbidities.
Attachment "5325e-Supplemental-Fall-2025" in the zip file “5325e-Supplemental-Fall-2025” presents a matrix of clinical risk factors, along with their identification in the literature and scoring indices.
References:
Caprini, J. A., Arcelus, J. I., Hasty, J. H., Tamhane, A. C., & Fabrega, F. (1991). Clinical assessment of venous thromboembolic risk in surgical patients. Seminars in thrombosis and hemostasis, 17 Suppl 3, 304–312.
Rogers, S. O., Kilaru, R. K., Hosokawa, P., Henderson, W. G., Zinner, M. J., & Khuri, S. F. (2007). Multivariable Predictors of Postoperative Venous Thromboembolic Events after General and Vascular Surgery: Results from the Patient Safety in Surgery Study. Journal of the American College of Surgeons, 204(6), 1211–1221.https://doi.org/10.1016/j.jamcollsurg.2007.02.072
Viarasilpa, T., Panyavachiraporn, N., Marashi, S. M., Van Harn, M., Kowalski, R. G., & Mayer, S. A. (2020). Prediction of Symptomatic Venous Thromboembolism in Critically Ill Patients: The ICU-Venous Thromboembolism Score. Critical care medicine, 48(6), e470–e479. https://doi.org/10.1097/CCM.0000000000004306
Please refer to the zip file “5325e-Supplemental-Fall-2025” and the attachment “5325e-Population-Fall-2025.xlsx" for statistics on the risk/case-mix factors for the hospitals included in the Truveta sample (used to create the risk-adjustment model) and for the two SCCM hospitals (used to assess data element validity).
From the candidate list of risk factors, variables were selected for inclusion in the final risk adjustment model using a bootstrap sampling method and multivariate analysis.
See attachment "5325e-Risk-Fall-2025" for the set of risk adjustment models capturing the effects of patients’ demographic characteristics (Model 1) and demographic and clinical characteristics (Model 2).
Model 1
In Model 1 (adjusted for age and sex only), a statistically significantly higher rate of VTE events were observed in all age groups compared to the referenced group (18-29 years old). The male sex compared to the female sex group experienced a small but statistically significant higher rate of VTE events (odds ratio [OR] 1.14, p-value <0.0001). The C-statistic (concordance statistic) for Model 1 was 0.540, indicating a moderate improvement relative to a model that is no better at predicting an outcome other than random chance (C-statistic = 0.5).
Model 2
In Model 2 (adjusted for age, sex, and clinical characteristics), we observed a statistically significant effect for patients aged 40-49, 50-59, 70-79, and 80 plus years old compared to the reference group of 18-29 years old. We also observed a statistically significant effect for eight clinical risk factors: bleeding disorders, cancer, catheter insertion, history of VTE, obesity, respiratory operations, stroke, and vascular surgeries. Diabetes and immobility had a statistically significant effect on the outcome; however, their respective odds ratios of 0.840 and 0.720 suggest an associated reduction in the likelihood of the outcome. The goal of developing the risk model is to identify risk factors that increase the risk for VTE, and therefore diabetes was not included in the final risk-adjustment model. The C-statistic for Model 2 was 0.692, indicating an improvement relative to Model 1 (C-statistic = 0.540).
Overall, in Model 2, 14 clinical risk factors were included. Of these, eight (described in the previous paragraph) were significantly associated with the outcome. Two factors—diabetes and immobility—demonstrated a protective effect, while the remaining four (fractures, hip/knee replacements, hormone therapy, and tobacco use) were not significantly associated with the outcome.
Final Model
We conducted a multivariate analysis to examine the relationships between demographic characteristics and the finalized set of clinical factors. The clinical factors retained in the final model were statistically significant, with bootstrap confidence levels exceeding 95 percent, indicating that these factors serve as important determinants of VTE within the dataset.
The final model had a modest C-statistic of 0.692.
Calibration and discrimination of the final risk-adjustment model:
We evaluated the final risk-adjustment model and its performance using a variety of metrics such as C-statistics, Hosmer-Lemeshow test, and calibration plots.
The model’s ability to discriminate between high-risk and low-risk postoperative VTE events, as measured by the C-statistic, was modest (0.692). If a c-statistic is closer to one, this indicates that the model can classify outcomes correctly. Although the model suggested a moderate predictive discrimination, statistical findings of excellent calibration are confirmed when comparing observed to predicted probabilities by risk deciles.
The Hosmer-Lemeshow test—a statistical test for goodness of fit and calibration for logistic regression models—divides patients into deciles (i.e., ten groups with equal number of patients) based on the expected rate of postoperative VTE events, from lowest to highest. In decile assessment, we should see similar numbers of observations in each decile group and increasing observed rates when we move from low to high deciles.
The results indicate an increasing trend in the sum of predicted rates by decile, indicating that the model is adequately capturing the relationship between the predictors and probability of the outcome. The observed predicted rates by decile demonstrate the same pattern. An observed-to-predicted rate of one indicates that the model accurately predicts the outcome for every observation. In this sample, the observed-to-predicted rate is approximately one, particularly for the higher deciles, indicating that the model is more effective at predicting the outcome for patients at higher risk of VTE. While this test allows us to obtain a general understanding of how the model predicts the outcome for patients with different risks, there are limitations to this analysis. These limitations are due to the limited number of risk factors included in the final model, all of which are categorical, and the rarity of the outcome. This results in numerous ties and a skewed sample proportion. As such, we interpret these findings with caution, and we supplement them with additional analyses of the model performance, such as the calibration plot described below.
The calibration plot, also known as the calibration curve, visualizes the relationship between the observed and expected values for each decile of patient risk. If a model is calibrated well, there will not be a substantial deviation from the 45° line of perfect fit or bisector.
The predicted risk in our sample ranged from 0.54 percent in the lowest decile to 3.69 percent in the highest decile. Consistent with the results of the Hosmer-Lemeshow test, the plot indicates a higher agreement between the observed and expected rates of postoperative VTE events for the higher deciles.
Our final risk model was conceptually developed using input from the literature, our clinician consultants, and our TEP. We tested the conceptual model using Truveta data identifying eight clinical factors that met our requirements of indication of a statistically significant increased likelihood of a patient experiencing a postoperative VTE event. We then tested a final model that included these eight factors and two demographic factors. The final model had modest C-statistic (0.6923) and showed fair discrimination, particularly for patients with higher risk of the outcome.
Please see attachment "5325e-Calibration-Fall-2025" for calibration results.
From our analysis of the multivariate output and bootstrap sampling, we interpret the risk factors included in the final risk-adjustment model to be the grouping of risk factors that have a strong association with the outcome. The final risk adjustment model included the following clinical risk factors: bleeding disorders, cancer, central venous catheter insertion, history of VTE, obesity, respiratory operations, stroke, and vascular surgeries. The final risk adjustment also included the demographic factors age and sex. Each of the clinical factors were associated with higher likelihood of VTE using a threshold of p<0.15 in at least 70 percent of the 300 bootstrap samples. Clinical factors that were not included in the final risk-adjustment model were not statistically significantly associated with the outcome and were only associated with the outcome in less than 50 percent of bootstrap samples. Furthermore, the model’s ability to discriminate between high-risk and low-risk postoperative VTE events, as measured by the C-statistic, was 0.692, which is considered sufficient for model discrimination power.
These findings, with the exception of obesity, were supported by scoring indices that indicate that the clinical factors are associated with either moderate or high risk of VTE (Caprini et al. 1991; Viasrasilpa, 2020). Obesity is not considered a high or moderate risk in scoring indices; however, sources such as the Surgeon General report indicate that obesity, especially coupled with other risk factors, are associated with a greater risk of VTE (Office of the Surgeon General, 2008).
References:
Caprini, J. A., Arcelus, J. I., Hasty, J. H., Tamhane, A. C., & Fabrega, F. (1991). Clinical assessment of venous thromboembolic risk in surgical patients. Seminars in thrombosis and hemostasis, 17 Suppl 3, 304–312.
Office of the Surgeon General (US); National Heart, Lung, and Blood Institute (US). The Surgeon General's Call to Action to Prevent Deep Vein Thrombosis and Pulmonary Embolism. Rockville (MD): Office of the Surgeon General (US); 2008. Available from: https://www.ncbi.nlm.nih.gov/books/NBK44178/
Viarasilpa, T., Panyavachiraporn, N., Marashi, S. M., Van Harn, M., Kowalski, R. G., & Mayer, S. A. (2020). Prediction of Symptomatic Venous Thromboembolism in Critically Ill Patients: The ICU-Venous Thromboembolism Score. Critical care medicine, 48(6), e470–e479. https://doi.org/10.1097/CCM.0000000000004306
Use & Usability
Use
Usability
There are several strategies to achieve meaningful venous thromboembolism (VTE) prevention in a hospital setting and subsequently reduce the incidence of VTE after a surgical encounter. As discussed in Section 2 of this form, there are extensive clinical guidelines and recommendations to support processes of care that impact the likelihood of a post-operative VTE. These primarily focus on hospital selection and timing of appropriate VTE prophylaxis for patients undergoing surgery. Clinicians can use these recommendations to guide their clinical care.
In addition to these processes of care, there are structures of care that can also lead to reduction in postoperative VTE events. These structures include:
* Adapting of evidence-based guidelines into local protocols. A randomized clinical trial of 2,500 at-risk hospital patients indicated that the use of VTE-risk assessment and physician alerts improves prophylaxis use from 14.5 percent to 33.5 percent and reduces thromboembolic complications from 8.2 percent to 4.9 percent (Geerts, 2009).
* Optimizing order set design and integration. By optimizing order sets, hospitals can overcome common obstacles, such as a lack of standardized protocols for VTE prevention and inadequate guidance from order sets, to administer VTE prophylaxis properly and reduce the incidence of VTE after surgery (AHRQ, 2016).
* Establishing and empowering an institutional supported, interdisciplinary team to implement structures needed to reduce VTEs. This team may standardize processes, monitor and measure VTE processes and outcomes, implement institutional policies, and educate providers and patients (AHRQ, 2016).
* Implement patient and nurse education and engagement programs to improve reliable administration of mechanical prophylaxis (AHRQ 2016).
* Implement active surveillance techniques to monitor and improve reliable administration of mechanical prophylaxis (AHRQ, 2016).
*Create and implement activity and mobility order sets that encourage progressive mobility, reducing sedatives, restraints, and inappropriate Foley catheters, and using sequential compression devices only when appropriate. These order sets aim to overcome patient and hospital factors that can be barriers to early ambulation programs (AHRQ, 2016).
Reference:
Chapter 7. Layering Interventions and Moving Toward Excellence. Content last reviewed May 2016. Agency for Healthcare Research and Quality, Rockville, MD. https://www.ahrq.gov/patientsafety/settings/hospital/vtguide/guide7.htm…;
The goal of the Hospital Harm – Postoperative Venous Thromboembolism measure is to improve patient safety by incentivizing hospitals to reduce the incidence of postoperative VTE events.
A potential unintended consequence of this measure is that hospital performance would be affected by surveillance bias. The concern being that hospitals that have aggressive VTE surveillance programs may be unfairly penalized as they detect and treat more VTEs. This concern is not new and has been raised in relation to the related claims-based measure, Perioperative Pulmonary Embolism or Deep Vein Thrombosis Rate (PSI - 12). The stewards of PSI 12 accounted for surveillance bias by updating the measure’s specifications to only capture (1) proximal (groin/thigh), not distal (calf) vein thromboses diagnosis; and (2) not include solitary subsegmental pulmonary emboli diagnosis. These changes resulted in a decreasing trend in measure rates with no changes in case fatality, which indicates improved specificity to capture VTEs which impact patient morality. We made these same modifications to the Hospital Harm – Postoperative VTE measure. Capturing VTEs that require medical intervention and impact patient outcomes is the goal of this measure regardless of whether these VTEs are identified through a hospital implemented VTE surveillance program or due to individual patient care protocols.
Another potential unintended consequence of this measure is a more aggressive use of anticoagulants which could result in subsequent major bleeding events. To mitigate this, CMS has developed a complementary measure titled Hospital Harm – Anticoagulant-Related Major Bleeding. This measure captures major bleeding events subsequent to hospital anticoagulant administration.
We will continue monitoring for potential unintended consequences, but we expect the benefits of implementation of the Hospital Harm – Postoperative Venous Thromboembolism measure to significantly outweigh any potential unintended consequences. The measure has the potential to incentivize hospitals to monitor VTE prevention protocols for patients undergoing surgical procedures, track the incidence of postoperative VTE events, and assess the quality of postoperative VTE prophylaxis administered. These actions have the potential to subsequently reduce the incidence of an VTE event, associated direct symptoms, and downstream clinical complications. The reduction of VTE events can lead to reduced hospital costs, subsequent VTE-related admissions, and mortality due to VTE events.
Comments
Staff Preliminary Assessment
CBE #5325e Staff Preliminary Assessment
Importance
Strengths
- A clear logic model is provided, depicting the relationships between inputs (e.g., electronic health record (EHR) and associated data; hospital staff), activities (e.g., pharmacological prophylaxis, mechanical prophylaxis, and use of VTE risk-assessment or screening tools and physician alerts), and desired outcomes (e.g., improvements in hospital practices for monitoring and detecting post-operative venous thromboembolism (VTE) events, improvements in patient satisfaction, and improvements in patient safety). This model demonstrates how the measure’s implementation will lead to the anticipated outcomes.
The problem this measure addresses presents a significant burden for patients and cost impact as 70% of VTE events are preventable and VTE events are estimated to cost the U.S. health care system $7-10 billion annually. If implemented, the developer argued the measure’s anticipated impact on important outcomes, such as patient safety, will improve based on published evidence. Additionally, improvements in morbidity/mortality, and reduced instances of postoperative VTE are expected to reduce associated medical costs.
The measure is supported by a comprehensive literature review, including systematic reviews with high evidence quality and clinical practice guidelines with evidence grading of strong/high, and high-quality empirical studies, including a randomized clinical trial, demonstrating improved outcomes and reduced associated medical costs for patients who experience VTE within 30 days of the first surgical procedure. The systematic review addressed the prevalence and incidence of VTE, the additional costs associated with each case, and the health impacts of VTE, including morbidity and mortality, underscoring the need for continued quality improvement. In addition, the systematic review addressed effective interventions or actions that can reduce and prevent hospital-acquired VTE events, including mechanical and pharmacological prophylaxis.
The proposed measure addresses a health care need not sufficiently covered by existing measures, offering advantages in terms of leveraging electronic health record (EHR) data, which offers greater accuracy, detail, and timeliness in comparison to the related claims-based measure, Perioperative Pulmonary Embolism or Deep Vein Thrombosis Rate (PSI-12). 5325e builds off of Venous Thromboembolism Prophylaxis (VTE-1) and Intensive Care Unit Venous Thromboembolism Prophylaxis (VTE-2) and evaluates the incidence of the patient safety event itself, creating a benefit to using these measures together.
While not required for initial endorsement, data from 34 hospitals in the Truveta data set from calendar year (CY) 2022 (50,203 encounters) and CY 2023 (50,708 encounters) show a performance gap, with decile ranges from 0.21% to 4.21% indicating variation in measure performance.
Description of patient input supports the conclusion that the measured outcome is meaningful with at least moderate certainty. Five patient and caregiver representatives of the Hospital Harm Technical Expert Panel (TEP) were asked about the measure’s meaningfulness and 5/5 patients indicated “agree”, based on a response scale of strongly agree/agree/disagree/strongly disagree.
Limitations
- Some of the evidence (specifically, a few studies and statistics) are getting to be outdated (e.g., 2016, 2009) so finding some more recent literature could strengthen this submission, if available.
Rationale
- This new measure meets all criteria for ‘Met’ for importance due to the significance of the problem it addresses and its significant anticipated impact, its robust evidence base, justifiable advantages over existing measures, and well-articulated logic model, making it essential for addressing the harm of VTE post-surgery. There is at least moderate confidence that the business case is adequate, i.e., the anticipated impacts of the measure on patient outcomes and cost justify the use of the measure.
Closing Care Gaps
Strengths
- The developer provided evidence of gaps in care related to the measures focus for subgroups, including a literature review, and their claim that the measure will help close care gaps through targeted quality improvement and increasing transparency, accountability, and patient safety, is credible.
The measure’s performance was empirically tested across age and sex; the developer’s rationale for selecting these subgroups is based on existing evidence. Data from the analyses were from 34 hospitals from 2022 to 2023 using Truveta data. The analysis employed indirect standardization with logistic regression, bootstrap resampling, and univariate and multivariate models.
The analysis revealed significant differences in performance scores by sex and age. For example, odds increased across all older age groups compared to the reference group of 18-29 years old. Patients aged 60-69 and 70-79 had nearly 1.9 times the odds of VTE compared with the youngest group. The developer provided a clear interpretation of results, mapping back to the current evidence.
Based on the findings, the developer noted recommended actions entities can take to close care gaps, including internally stratifying results so entities can focus quality improvement on areas of need, and implementing a plan for implementing guidelines and evidence-based practices.
Limitations
- None identified.
Rationale
- This measure meets all criteria for ‘Met’ due to sufficient assessment of gaps in care with respect to age and sex, providing crucial insights into how accountable entities can use this measure to improve differences in care for these subgroups. This includes targeted quality improvement, and increased transparency, accountability, and patient safety.
Feasibility Assessment
Strengths
- All required data elements are routinely generated during care delivery, and required elements are available from digital or electronic sources.
The developer completed an eCQM Scorecard showing the measure was tested in at least two EHRs (Epic, Allscripts, Cerner, Meditech). All data required for this measure are readily available in a structured format, residing in fixed fields within the EHR systems tested. The data elements are accurate and coded using nationally accepted terminology, facilitating interoperability and standardization across different systems.
The developer stated that no feasibility issues were found requiring adjustment of the final measure specifications.
The developer described the costs and burden associated with data collected and data entry, validation, and analysis. They discussed potential barriers that could be encountered in implementing the measure which includes variability in EHR capabilities across hospitals within a system, inconsistent coding practices, challenges linking data across encounters. They also noted mitigation approaches, such as updating documentation workflows, integration of structured data fields, and alignment of coding practices with specifications.
The developer described how all required data elements can be collected without risk to patient confidentiality, including how the measure is designed to be implemented using structured data from EHRs, which supports secure and confidential data collection. The measure does not rely on patient surveys.
There are no fees, licensing, or other requirements to use any aspect of the measure (e.g., value/code set, risk model, programming code, algorithm).
Limitations
- None identified.
Rationale
- The new measure meets all criteria for 'Met' for feasibility due to its well-documented feasibility assessment, clear and implementable data collection strategy, and transparent handling of patient confidentiality, burden, licensing, and fees. These factors collectively ensure that the measure can be implemented effectively and sustainably in a real-world health care setting.
Scientific Acceptability
Strengths
- The developer performed the required reliability testing for this new measure, namely, they conducted person/encounter-level (“data element”) reliability testing for all critical data elements. Data sources used for reliability analysis are adequately described and include EHR data from Truveta. A comparison of EHR and manual data extraction for 40 randomly selected patients from two sites.
The developer calculated percent agreement and Gwet's AC1 at the person- and/or encounter-level for 39 critical data elements. The developer reported that all data elements (with the exception of "transfer encounters") show a percent agreement and Gwet's AC1 greater than 0.4, which meets the expected threshold of 0.4 for inter-rater agreement.
Although not required for this new measure, the developer performed reliability testing, namely, they conducted accountable entity-level (“measure score”) reliability testing at the level for which the measure is specified. Data sources used for reliability analysis are adequately described and include EHR data from Truveta across six states during the two-year period of 2022-2023. The entities included in the analysis were characterized by over 50K patients across 34 entities. The developer reports an average reliability of 1.0.
Limitations
- None identified.
Rationale
- This new measure is rated as ‘Met’ for reliability because the developer performed the required reliability testing for this measure and results demonstrate sufficient reliability at the patient- or encounter-level.
Strengths
- The developer performed the required validity testing for this new measure, namely, they conducted encounter-level (“data element”) validity testing for all critical data elements and accountable entity-level (“measure score”) in the form of face validity testing. Data element validity testing used EHR data over the period January 2022 to December 2023 from two hospitals coordinated through the Society of Critical Care Medicine’s (SCCM) Discovery, The Critical Care Research Network. The two hospitals were located in North Carolina (850+ beds, using Epic as the EHR vendor) and Puerto Rico (200+ beds, using Meditech).
The developer performed manual chart abstraction of critical data elements for a sample of 20 patients at each hospital (total n = 40) and compared them with EHR data, using the abstracted data as the gold standard. They conducted sensitivity and specificity testing with positive and negative predictive values to evaluate classification of cases into the numerator, denominator, denominator exclusions, and percent agreement and inter-rater agreement for each data element at the encounter level. Sensitivity scores for classifying cases into the numerator and denominator at both sites were moderate to high (82% to 100%). Specificity scores for correctly excluding cases from the numerator, denominator, and denominator exclusions were moderate to high at both sites (81% to 100%). Most data elements had between 80% and 100% agreement between manual abstraction and EHR.
The developer tested the measure’s face validity by collecting feedback in interviews with 7 clinical staff from 3 test sites (specialties internal medicine, trauma, and neurocritical care) and from the Patient Safety project’s Hospital Harm TEP members (n=19). TEP members were asked to rate on a Likert scale their agreement with the statements: 1) "the measure score is an accurate reflection of quality" (17 of 19 [89%] agreed or strongly agreed), and 2) "the measure score can be used to distinguish good from poor quality of care" (15 of 19 [79%] agreed or strongly agreed).
The developer conducted a statistical risk adjustment, based on a conceptual model, selecting risk factors that are present at the start of care and have a significant association with the outcome. The developer reported a c-statistic of 0.692, indicating moderate model discrimination.
Limitations
- Limitations in the data element validity testing were lower sensitivity for denominator exclusions (40% and 75%), which the developer explained was due to site specific factors and in one case identified diagnosis codes for VTE that are not part of the measure’s current value set, and lower percent agreement for transfer encounters (30% at one site; unable to be assessed at the other site) and a risk factor, obesity risk (60% to 65%). In addition, positive predictive value (PPV) for the numerator is low at both sites (25% and 57%), while negative predictive value (NPV) is perfect (100% at both sites). This indicates a substantial number of false positives (i.e., patients included in the numerator when they should not be) and no false negatives (patients who should not be in the numerator when they are), potentially resulting in biased measure scores. These results suggest that a few data elements used in the quality measure have moderate accuracy but require improvement to meet the desired standards.
Face validity testing among the 7 clinicians was less conclusive than for TEP members. Clinicians were asked to rate on a Likert scale their agreement with the statements: 1) "the measure score can distinguish between hospitals with good quality of care and poor quality of care" (4 out of 7 [57%] agreed, and 2) "the measure score accurately reflects hospital quality" (5 out of 7 [71%] agreed). Those in disagreement cited the need to include a baseline surveillance rate in the risk adjustment model, the need for a paired process measure, and belief that care quality does not influence VTE outcomes.
Rationale
- This new measure is rated as ‘Not Met But Addressable’ for validity because the validity testing results partially support an inference of validity for the measure, suggesting that the measure somewhat accurately reflects performance on quality and can distinguish good from poor performance to a limited extent.
The risk adjustment methods used are appropriate and demonstrate that risk factors contribute to unique variation in the outcome. The model performance is acceptable.
Use and Usability
Strengths
- The measure is not currently in use, but the developer described a plan for use in public reporting and payment program.
The developer provided a summary of how accountable entities can use the measure results to improve performance through processes and structures of care. Specifically, entities can include developing processes of care that aid clinician staff with performing guideline-supported processes of care and adequate hospital interventions; adapt evidence-based guidelines into local protocols; optimize order set design and integration; establish and empower and institutional supported, interdisciplinary team; implement patient and nurse education and engagement programs; and implement active surveillance techniques.
The developer described potential unintended consequences, such as hospital performance being affected by surveillance bias and more aggressive use of anticoagulants which could result in subsequent major bleeding events. The developer stated that these were addressed though modifications that the developers of PSI 12 made to address surveillance bias including: only capturing proximal, not distal, vein thrombosis diagnosis; and not include solitary subsegmental pulmonary emboli diagnosis. To mitigate anticoagulant-related major bleeding, CMS has developed a complementary measure titled Hospital Harm – Anticoagulant-Related Major Bleeding, which captures major bleeding events subsequent to hospital anticoagulant administration. The developer plausibly argued that the measure’s benefits outweigh the potential unintended consequences they identified.
Limitations
- The developers included “structures of care” that can lead to reduction in postoperative VTE events; however, the listed references are outside of the 5-year window (e.g., 2016, 2009) so the submission could be strengthened with more recent research, if available.
Rationale
- This new measure is rated ‘Met’ for use and usability because there is a clear plan for use in at least one accountability application, and the measure provides actionable information for improvement. The developer described potential unintended consequences such as surveillance bias and more aggressive use of anticoagulants. The developer stated they have updated the specifications to limit diagnoses, consistent with the approach taken by the developers of PSI-12, to address surveillance bias. The developer plausibly argued that the measure’s benefits outweigh the potential unintended consequences they identified.
Committee Independent Review
Support-Hospital Harm–VTE-5325e
Importance
As this is a new measure, the impact would address the CDC (Center for Disease Control and Prevention) data that up to 70% percent of hospital-acquired VTE (Venous Thromboembolism) cases are preventable. VTEs increase hospital stays, readmission, and costs. An unintended consequence could be more use of anticoagulants resulting in major bleeding events.
As far as importance, it is noted that despite 17% percent reduction in postoperative VTEs (2014-7), the rate of postoperative VTE in hospitals remains high. Up to 900,000 are affected annually per the CDC. Data shows that "hospital interventions, such as mechanical and pharmacological prophylaxis, can reduce the incidence" in trauma patients.
One concern with the denominator is consideration of outpatient surgeries due to increased use of ASCs (ambulatory surgical centers).
Closing Care Gaps
VTE is a leading cause of preventable postoperative morbidity and mortality. Risk factors such as age/sex need to be considered. Besides women of childbearing age, people over age 70 account for most VTEs.
Feasibility Assessment
It is noted that EHR missing data didn't "prevent cases from being correctly assigned to the denominator or numerator". But data missing on "denominator exclusion criteria did result in some denominator cases...not being properly excluded". On the patient perspective summary it was shown that most "hospitals found that the data needed to calculate the measure is available, accurate, and collected during normal patient care". It is further noted that the "Feasibility Scorecard assessing the measure’s data elements indicated that all data elements were feasible. Therefore, the measure team did not change any measure specifications".
Scientific Acceptability
For reliability it is noted that because "the measure is intended to be risk-adjusted, we evaluated the reliability of its risk-adjusted version by calculating percentiles based on the number of encounters across measured entities".
For validity, face validity was considered. Regarding risk adjustment, most respondents "indicated that the risk-adjusted version of the measure as specified can be used to distinguish between hospitals with good quality of care and hospitals with poor quality of care". This measure was risk-adjusted because "there are some factors that are independent of quality that may produce worse outcomes for the hospital".
Use and Usability
This measure will be used for public reporting and the payment program.
Summary
This is an important measure, particularly for older patients. VTEs are mostly preventable with appropriate interventions and this will impact 900,000 people per year. Risk factors such as pregnancy and type of surgery are considered. Would still suggest inclusion of ASCs.
Hospital Harm – Postoperative Venous Thromboembolism
Importance
Importance is adequately addressed including the consequences of VTE for health outcomes as well as cost of subsequent care for patients.
Closing Care Gaps
- Addressed gaps and potential remedies to prevent VTE. However, what additional gap will this measure close over the AHRQ measure PSI 12: Perioperative pulmonary embolism (PE/DVT) which is already in place. How much of ad additional gap will this fill in?
- The fitted logistic regression model to justify the affected population by sex and age falls short on its conclusion. The differences by age and sex could simply be resulting from other uncontrolled health risk factors that affect males and females differently as well as by age. Unless we are not told what else was in the model, I would caution the conclusion which is based on these two indicators.
Feasibility Assessment
I am concerned by the suggestion that those hospitals "... missing ECMO data from test sites who do not conduct ECMOs would not impact those hospitals’ measure scores." Wouldn't potentially affect comparability of results with hospitals for which ECMO patients are excluded?
Scientific Acceptability
I am not clear about the scientific merit/acceptability and reliability based on the justification provided.
Is an simple interview of 7 clinicians (that did not include surgeons) enough to determine face validity?
Use and Usability
In my opinion, the cost-benefit of implementing this measure is not clearly articulated. Despite the importance of this process measure, adopting this measure for public reporting as well for payment purposes may be too optimistic.
Summary
The measure addresses an important health condition that can be avoided/prevented. However, it is not clear whether the adoption of this measure complements or duplicates the AHRQ-based measure PSI 12 (Perioperative Pulmonary Embolism or Deep Vein Thrombosis Rate).
Support
Importance
Based on my understanding the information seems reasonable
Closing Care Gaps
Based on my understanding the information seems reasonable
Feasibility Assessment
Based on my understanding the information seems reasonable
Scientific Acceptability
Based on my understanding the information seems reasonable
Based on my understanding the information seems reasonable
Use and Usability
Based on my understanding the information seems reasonable
Summary
I am in support of the measure based on my understanding of the information provided.
Concerns about reliability and metric defintion
Importance
VTE is leading cause of preventable death in US, associated with increased re-admissions and mortality. Studies show that the occurrence of VTE can be reduced by training clinicians to perform guideline-supported processes of care. 5 out of 5 patients indicated “agree” that the measure is meaningful and provides information that is helpful in making care decisions.
Closing Care Gaps
Since this measure includes within 30 days after the first surgical procedure it will be important to include social risk factors in the risk adjustment models that would likely contribute to differences in postoperative VTE. This could include factors related to access to follow-up care measured through insurance status.
Feasibility Assessment
The feasibility testing identified challenges in capturing exclusion-related criteria in the facilities that do not use structured data fields. While the developers attributed these challenges to site-specific limitations, the pilot was conducted at only two facilities, raising concerns that these data capture challenges could be more widespread and worse as the measure is implemented across a broader range of sites.
Scientific Acceptability
While agreement rates of the individual data elements were strong, there are still concerns about the sensitivity and specificity results related to classifying cases as denominator exclusions. As noted in the feasibility section, as this measure becomes more widespread across different types of facilities, it is likely to amplify the challenges. Another notable limitation of the testing data is the lack of information on the hospital-level characteristics. Thus, we are unable to understand the types of sites most likely to be impacted by these data capture challenges.
About 89 percent of TEP members (17 of 19 voting) agreed or strongly agreed that the measure score accurately reflects quality of care, and 79 percent (15 of 19 voting) agreed or strongly agreed that the measure score can be used to distinguish between good and poor quality of care. 4 out of 7 clinicians agreed that the risk-adjusted version of the measure can be used to distinguish between hospitals with good quality of care and hospitals with poor quality of care. Notably, there was concern about including postoperative VTE outcomes as part of the metric. This aligns with comments in the closing care gap section that social factors should be considered in the risk adjustment models if data in the 30 day post operative period is being included in the metric.
Use and Usability
Not currently in use. The developers included ways that facilities could use the measure to improve performance including supporting implementation of guideline-concordant processes of care, optimizing ordersets, use of evidence-based interventions, patient and nurse education, and team care models. The developers also listed potential unintended consequences of the measure and how they may be mitigated.
Summary
Two main concerns about this measure: (1) the inclusion of the 30 day postoperative period without consideration of social risk factors in the risk adjustment models and (2) the feasibility testing being limited to two sites, which identified data capture challenges that will likely be amplified when more facilities use these measures (and the lack of an implementation guidance plan in the usability section).
(No subject)
Importance
Closing Care Gaps
Feasibility Assessment
Use and Usability
There should be additional discussion regarding appropriate inclusion and exclusion criteria which are not reflected in the current measure. For example, there are many times when providing anticoagulation is not safe for a patient in the perioperative setting beyond what is currently reflected in the measure.
Overall evaluation
Importance
Clearly a very important topic.
Closing Care Gaps
Authors do a nice job of address this issue.
Feasibility Assessment
Seems feasible in this current form, though that is not the main concern with this measure.
Scientific Acceptability
Agree with staff.
Agree with staff.
Use and Usability
I share the concerns raised by the AMA regarding potential issues with data interpretation.
Summary
While this measure addresses an important topic, there are concerns regading interpretation of the results due to outcomes of interest included.
support endorsement
Importance
no comments
Closing Care Gaps
no comments
Feasibility Assessment
no comments
Scientific Acceptability
agree with staff that as new measure, meets reliability threshold but for entity-level reliability testing, I find it deeply implausible that the average entity-level reliability is 1.0 for a risk adjusted outcome measure/eCQM
face validity satisfactory
Use and Usability
no comments
Summary
I support endorsement but encourage developer to review entity-level reliability testing as the reported results seem unusually high and unusually narrow in range of values for a risk adjusted outcome measure
The risk adjustment model…
Importance
-
Closing Care Gaps
-
Feasibility Assessment
-
Scientific Acceptability
-
The risk adjustment model only includes age, bleeding disorder, cancer, catheter insertion, history of VTE, obesity, stroke, vascular surgery, and respiratory operations. The methodology used to select risk factors in the risk adjustment model is not described. However, the list of included risk factors is unlikely to adequately risk adjust for case mix. There is significant amount of heterogeneity of surgical procedures, which is not well specified using only two surgery types. It is also likely that other patient characcteristics are also associated with the risk of VTE, including surgical urgency (elective vs non-elective), surgical complexity (could use work RVU as a poxy for this, in addition to body region; orthopedic surery [TKA, THR], other comorbidities (heart failure, diabetes, CKD).
Use and Usability
-
Summary
The risk adjustment model only includes age, bleeding disorder, cancer, catheter insertion, history of VTE, obesity, stroke, vascular surgery, and respiratory operations. The methodology used to select risk factors in the risk adjustment model is not described. However, the list of included risk factors is unlikely to adequately risk adjust for case mix. There is significant amount of heterogeneity of surgical procedures, which is not well specified using only two surgery types. It is also likely that other patient characcteristics are also associated with the risk of VTE, including surgical urgency (elective vs non-elective), surgical complexity (could use work RVU as a poxy for this, in addition to body region; orthopedic surery [TKA, THR], other comorbidities (heart failure, diabetes, CKD).
Evaluation
Importance
This measure fills an important gap by focusing on VTE‑related outcomes and associated costs. Its development is supported by a strong evidence base, including a comprehensive literature review and alignment with current clinical practice guidelines.
Closing Care Gaps
The developer provided stratified information identifying care gaps by age and sex, along with guidance on how these data can be applied in practice. These insights offer organizations actionable, population‑specific information that supports improved patient safety and informs targeted quality improvement activities.
Feasibility Assessment
All required data elements are routinely generated during clinical care and are available from digital or electronic sources. The measure was also tested in two EHR systems, confirming that the necessary data are captured in structured formats suitable for use. The data elements are accurate and rely on nationally standardized coded terminology, supporting interoperability and consistency across systems. The developer reported no feasibility concerns with data availability or collection.
Scientific Acceptability
Developer conducted reliability testing and the results are sufficiently reliable at the patient and encounter level.
Validity results do not completely support the inference of results which may suggest that performance can not be adequately assessed.
Use and Usability
The developer has put forth a plan for the measure to be incorporated into public reporting and payment programs, however, it is not in programs currently. Further, the measure provides actionable insights for QI.
Summary
This is an important measure but should be refined for endorsement related to validity to ensure reliable and accurate performance comparisons.
Support with addressed concern
Importance
this is met
Closing Care Gaps
this is met.
Feasibility Assessment
met.
Scientific Acceptability
met.
support the staff assessment on validity concerns and resolution
Use and Usability
this is met.
Summary
support overall
Overview
Importance
Agree laregely with staff preliminary assessment. Add: I am not seeing strength of evidence stated in literature review not stated although staff indicates high quality. Given that anticoagulation may be offered post-operatively for supraventricular arrhythmias or cardiac ischemia, it is not clear that the numerator reflects only thromboembolic events.
Closing Care Gaps
Agree with staff preliminary assessment.
Feasibility Assessment
I agree with staff preliminary assessment. I would like confirmation that the triggering event within 30 days is captured only if within the same inpatient encounter.
Scientific Acceptability
I agree with the staff preliminary assessment
I agree with the staff preliminary assessment but am concerned regarding the limited testing (two hospital sites).
Use and Usability
I agree with the staff preliminary assessment
Summary
This is an important area of concern. More data on validity would lend assurance that detected differences reflect variations in quality of care.
Great measure with some explanation needed
Importance
--
Closing Care Gaps
--
Feasibility Assessment
--
Scientific Acceptability
--
--
Use and Usability
I would like to understand the reasoning behind some of the exclusions as well as why the measure includes the 30 days after discharged, with no adjustment for what the patient is able or willing to do outside of the facility's control.
Summary
For the most part, I understand and agree with the measure. Some details are missing to be able to fully endorse the benefit with low risk of this measure.
Public Comments
Hospital Harm-Postoperative Venous Thromboembolism
The American Medical Association (AMA) has several concerns regarding the measure, including the lack of evidence to support the inclusion of 30 days after discharge, the limited data element validity testing provided, and lack of social risk factors in the risk adjustment model testing. Regarding the evidence, neither the logic model nor evidence summary discuss this timeframe specifically. In addition, the data element validity testing was limited to two vendor systems, and it was difficult to determine if it assessed the ability to accurately and completely capture a postoperative venous thromboembolism (VTE) after discharge. Based on previous experience with data derived from EHRs, we would assume that there could be a higher degree of missing data for these events since the sole data source for the measure is electronic health record systems (EHRs) and it is difficult for hospitals to track events after discharge.
Because the measure includes the 30 days after discharged, we expected to see testing of social risk factors in the risk adjustment model, but this information was not provided nor was there any discussion of these variables’ potential effect in the conceptual model. Developers must continue to conduct these analyses to understand the degree to which any of one factor may sufficiently impact a hospital’s performance score and may justify its inclusion in the model.
We believe that there is significant potential for misrepresentation of hospitals’ performance scores given the concerns we outlined. As a result, we do not support endorsement of this measure.