Patient Safety Indicator (PSI) 90: Patient Safety and Adverse Events Composite

CBE ID

0531

1.5 Project

Management of Acute Events, Chronic Disease, Surgery, and Behavioral Health

Endorsement Status

Endorsed

1.0 New or Maintenance

Maintenance

1.1 Measure Structure

Single Measure

Previous Endorsement Cycle

Fall 2020

Is Under Review

Yes

Next Maintenance Cycle

Fall 2024

1.6 Measure Description

PSI 90 is a composite of ten adverse event indicators that summarizes hospitals’ performance on patient safety for the CMS Medicare fee-for-service population. The timeframe used in the CMS Hospital Acquired Conditions Reduction Program (HACRP) and CareCompare public reporting are set within the Inpatient Prospective Payment Systems (IPPS) Final Rule annually. Typically, the performance periods use multiple months of claims data.

Measure Specs

General Information

1.7 Measure Type

Outcome

1.7 Composite Measure

Yes

1.3 Electronic Clinical Quality Measure (eCQM)

1.8 Level of Analysis

Facility

1.9 Care Setting

Hospital: Acute Care Facility

Hospital: Inpatient

1.10 Measure Rationale

The Patient Safety Indicator (PSI) 90 composite measure captures hospital performance related to the occurrence of 10 preventable adverse events among hospitalized patients. The ten risk-adjusted PSIs that comprise PSI 90 are listed below.

PSI 03 Pressure Ulcer Rate
PSI 06 Iatrogenic Pneumothorax Rate
PSI 08 In-Hospital Fall-Associated Fracture Rate
PSI 09 Postoperative Hemorrhage or Hematoma Rate
PSI 10 Postoperative Acute Kidney Injury Requiring Dialysis Rate
PSI 11 Postoperative Respiratory Failure Rate
PSI 12 Perioperative Pulmonary Embolism or Deep Vein Thrombosis Rate
PSI 13 Postoperative Sepsis Rate
PSI 14 Postoperative Wound Dehiscence Rate
PSI 15 Abdominopelvic Accidental Puncture or Laceration Rate.

The rationale for reporting this measure is three-fold: (1) to encourage hospitals to implement best practices to minimize adverse events, (2) to help patients understand hospitals’ quality of care and make selection decisions, and (3) to allow payers to track hospital performance. PSI 90 is implemented in CMS’s Hospital Acquired Condition Reduction Program (HACRP). This program evaluates hospitals’ performance on patient safety measures and reduces Medicare payments for hospitals that perform poorly. Given the measure’s results impact payment, hospitals are motivated to implement safe hospital practices that minimize risks of harmful patient outcomes and improve care quality.

PSI 90 results are also publicly posted on https://www.medicare.gov/care-compare/?providerType=Hospital for CMS and Veteran Affair (VA) hospitals. Prospective patients can use these scores to evaluate hospitals’ quality performance as it relates to patient safety and use this information to make informed decisions regarding where they seek care.

1.11 Measure Webpage

https://qualitynet.cms.gov/inpatient/measures/psi/resources

1.20 Types of Data Sources

Claims Data

1.25 Data Source Details

CMS calculates PSI 90 using claims data from Medicare fee-for-service (FFS) discharges, using final action claims for inpatient stays paid under the Inpatient Prospective Payment System (IPPS).

Numerator

1.14 Numerator

PSI 03: Discharges, among cases meeting the inclusion and exclusion rules for the denominator, with any secondary ICD-10-CM diagnosis code not present on admission for stage 3 or 4 (or unstageable) pressure ulcer, in the absence of a secondary ICD-10-CM diagnosis code present on admission for deep tissue injury or unstageable pressure injury at the same anatomic site.

PSI 06: Discharges, among cases meeting the inclusion and exclusion rules for the denominator, with a secondary ICD-10-CM diagnosis code for iatrogenic pneumothorax.

PSI 08: Discharges, among cases meeting the inclusion and exclusion rules for the denominator, with any secondary ICD-10-CM diagnosis code for fracture.

PSI 09: Discharges, among cases meeting the inclusion and exclusion rules for the denominator, with any secondary ICD-10-CM diagnosis code for postoperative hemorrhage or hematoma and any listed ICD-10-PCS procedure code for treatment of hemorrhage or hematoma. The ICD-10-CM specification is limited to postoperative hemorrhage or hematoma.

PSI 10: Discharges, among cases meeting the inclusion and exclusion rules for the denominator, with any secondary ICD-10-CM diagnosis code for acute kidney failure and any listed ICD-10-PCS procedure code for dialysis.

PSI 11: Discharges, among cases meeting the inclusion and exclusion rules for the denominator, with either:

any secondary ICD-10-CM diagnosis code of acute postprocedural respiratory failure
the last date of an ICD-10-PCS procedure code for a mechanical ventilation for greater than 96 consecutive hours is zero or more days after the first major operating room procedure, if the dates of both procedures are available
the last date of an ICD-10-PCS procedure code for a mechanical ventilation for 24 - 96 consecutive hours is two or more days after the first major operating room procedure, if the dates of both procedures are available
the last date of any ICD-10-PCS procedure code for an intubation is one or more days after the first major operating room procedure, if the dates of both procedures are available

PSI 12: Discharges, among cases meeting the inclusion and exclusion rules for the denominator, with a secondary ICD-10-CM diagnosis code for proximal deep vein thrombosis or a secondary ICD-10-CM diagnosis code for pulmonary embolism.

PSI 13: Discharges, among cases meeting the inclusion and exclusion rules for the denominator, with any secondary ICD-10-CM diagnosis code for sepsis.

PSI 14: Discharges, among cases meeting the inclusion and exclusion rules for the denominator, with any listed ICD10-PCS procedure code for repair of abdominal wall and with an ICD-10-CM diagnosis code for disruption of internal surgical wound.

PSI 15: Discharges among cases meeting the inclusion and exclusion rules for the denominator that include all of the following:

any secondary ICD-10-CM diagnosis code for accidental puncture or laceration of a specific organ or structure during an abdominopelvic procedure; and
a potentially related procedure for evaluation or treatment performed on an organ or structure related to where the reported accidental puncture or laceration occurred, and performed one to 30 days after the index abdominopelvic procedure; and
without a principal ICD-10-CM diagnosis code (or secondary diagnosis present on admission) for accidental puncture or laceration of a specific organ or structure that matches the organ or structure of the potentially related subsequent procedure described above

1.14a Numerator Details

Numerator details are outlined in the individual component's measure specifications, which are attached as Excel files in section 1.13. In each measure's specification attachment, the measure description tab shows the name of code sets included in the numerator, and the following tabs contain the lists of codes within each code set. Other code sets that are referenced as Appendix A-G are included in separate Excel files.

A complete list of the CMS PSI 90 and its component measure specifications can also be found on https://qualitynet.cms.gov/inpatient/measures/psi/resources.

The time periods are based on the particular year and use of the specifications. When CMS uses this measure in their programs, the measure is generally calculated using a two-year period. However, there have been exceptions to this timeframe which CMS has publicly shared.

Denominator

1.15 Denominator

PSI 03: Surgical or medical discharges for patients ages 18 years and older. Surgical and
medical discharges are defined by specific MS-DRG codes.
Exclude discharges:

with length of stay of less than 3 days
with a principal ICD-10-CM diagnosis code for site-specific pressure ulcer stage 3 or 4 (or unstageable) or deep tissue injury at the same anatomic site
with any ICD-10-CM diagnosis code for severe burns (≥20% body surface area)
with any ICD-10-CM diagnosis code for exfoliative disorders of the skin (≥20% body surface area)
with a principal ICD-10-CM diagnosis code assigned to MDC 14 Pregnancy, Childbirth & the Puerperium
with a principal ICD-10-CM diagnosis code assigned to MDC 15 Newborns & Other Neonates with Conditions Originating in Perinatal Period
with an ungroupable DRG (DRG=999)
with missing gender (SEX=missing), age (AGE=missing), quarter (DQTR=missing), year (YEAR=missing), or principal diagnosis (DGNS_CD1=missing)
with missing MDC (MDC=missing) when the user indicates that MDC is provided

PSI 06: Surgical or medical discharges for patients ages 18 years and older. Surgical and medical discharges are defined by specific MS-DRG codes.
Exclude discharges:

with a principal ICD-10-CM diagnosis (or secondary diagnosis present on admission) of non-traumatic pneumothorax
with any listed ICD-10-CM diagnosis code for specified chest trauma (rib fractures, traumatic pneumothorax and related chest wall injuries)
with any listed ICD-10-CM diagnosis code for pleural effusion
with any listed ICD-10-PCS procedure code for thoracic surgery, including lung or pleural biopsy and diaphragmatic repair
with any listed ICD-10-PCS procedure code for potentially trans-pleural cardiac procedure
with a principal ICD-10-CM diagnosis code assigned to MDC 14 Pregnancy, Childbirth & the Puerperium
with a principal ICD-10-CM diagnosis code assigned to MDC 15 Newborns & Other Neonates with Conditions Originating in Perinatal Period
with an ungroupable DRG (DRG=999)
with missing gender (SEX=missing), age (AGE=missing), quarter (DQTR=missing), year (YEAR=missing), or principal diagnosis (DGNS_CD1=missing)
with missing MDC (MDC=missing) when the user indicates that MDC is provided

PSI 08: Surgical or medical discharges for patients ages 18 years and older. Surgical and medical discharges are defined by specific MS-DRG codes.
Exclude discharges:

with a principal ICD-10-CM diagnosis code (or secondary diagnosis present on admission) for fracture
with any listed ICD-10-CM diagnosis code for joint prosthesis-associated fracture
with a principal ICD-10-CM diagnosis code assigned to MDC 14 Pregnancy, Childbirth & the Puerperium
with a principal ICD-10-CM diagnosis code assigned to MDC 15 Newborns & Other Neonates with Conditions Originating in Perinatal Period
with an ungroupable DRG (DRG=999)\
with missing gender (SEX=missing), age (AGE=missing), quarter (DQTR=missing), year (YEAR=missing), or principal diagnosis (DGNS_CD1=missing)
with missing MDC (MDC=missing) when the user indicates that MDC is provided

PSI 09: Surgical discharges for patients ages 18 years and older, with any listed ICD-10-PCS procedure code for an operating room procedure. Surgical discharges are defined by specific MS-DRG codes.
Exclude discharges:

with a principal ICD-10-CM diagnosis code (or secondary diagnosis present on admission) for postoperative hemorrhage or postoperative hematoma
where the only operating room procedure is for treatment of postoperative hemorrhage or hematoma
where the treatment of postoperative hemorrhage or hematoma occurs before the first operating room procedure, if the dates of both procedures are available
with any listed ICD-10-CM diagnosis code for coagulation disorder
with a principal ICD-10-CM diagnosis code assigned to MDC 14 Pregnancy, Childbirth & the Puerperium
with a principal ICD-10-CM diagnosis code assigned to MDC 15 Newborns & Other Neonates with Conditions Originating in Perinatal Period
with an ungroupable DRG (DRG=999)
with missing gender (SEX=missing), age (AGE=missing), quarter (DQTR=missing), year (YEAR=missing), or principal diagnosis (DGNS_CD1=missing)
with missing MDC (MDC=missing) when the user indicates that MDC is provided

PSI 10: Elective surgical discharges for patients ages 18 years and older with any listed ICD-10-PCS procedure code for an operating room procedure. Elective surgical discharges are defined by specific MS-DRG codes with admission type recorded as elective (TYPE_ADM=3).
Exclude discharges:

with a principal ICD-10-CM diagnosis code (or secondary diagnosis present on admission) for acute kidney failure
with any dialysis procedure that occurs before or on the same day as the first operating room procedure
with any dialysis access procedure that occurs before or on the same day as the first operating room procedure
with a principal ICD-10-CM diagnosis code (or secondary diagnosis present on admission) for cardiac arrest
with a principal ICD-10-CM diagnosis code (or secondary diagnosis present on admission) for severe cardiac dysrhythmia
with a principal ICD-10-CM diagnosis code (or secondary diagnosis present on admission) for shock
with a principal ICD-10-CM diagnosis code (or secondary diagnosis present on admission) for chronic kidney disease stage 5 or end stage renal disease
with a principal ICD-10-CM diagnosis code for urinary tract obstruction
with any ICD-10-CM diagnosis present on admission of solitary kidney and any ICD-10-PCS procedure code for partial or total nephrectomy
with a principal ICD-10-CM diagnosis code assigned to MDC 14 Pregnancy, Childbirth & the Puerperium
with a principal ICD-10-CM diagnosis code assigned to MDC 15 Newborns & Other Neonates with Conditions Originating in Perinatal Period
with an ungroupable DRG (DRG=999)
with missing gender (SEX=missing), age (AGE=missing), quarter (DQTR=missing), year (YEAR=missing), or principal diagnosis (DGNS_CD1=missing)

PSI 11: Elective surgical discharges for patients ages 18 years and older with any listed ICD-10-PCS procedure code for an operating room procedure. Elective surgical discharges are defined by specific MS-DRG codes with admission type recorded as elective (TYPE_ADM=3).
Exclude discharges:

with a principal ICD-10-CM diagnosis code (or secondary diagnosis present on admission) of acute respiratory failure
with any listed ICD-10-CM diagnosis code present on admission for tracheostomy
where the only operating room procedure is tracheostomy
where a procedure for tracheostomy occurs before the first operating room procedure, if the dates of both procedures are available
with any listed ICD-10-CM diagnosis code for malignant hyperthermia
with any listed ICD-10-CM diagnosis code present on admission for neuromuscular disorder
with any listed ICD-10-CM diagnosis code present on admission for degenerative neurological disorder
with any listed ICD-10-PCS procedure code for laryngeal, pharyngeal, nose, mouth, or facial surgery involving significant risk of airway compromise
with any listed ICD-10-PCS procedure code for esophageal surgery
with any listed ICD-10-PCS procedure code for lung cancer
with any listed ICD-10-PCS procedure code for lung or heart transplant
with a principal ICD-10-CM diagnosis code assigned to MDC 4 Diseases & Disorders of the Respiratory System
with a principal ICD-10-CM diagnosis code assigned to MDC 14 Pregnancy, Childbirth & the Puerperium
with a principal ICD-10-CM diagnosis code assigned to MDC 15 Newborns & Other Neonates with Conditions Originating in Perinatal Period
with an ungroupable DRG (DRG=999)
with missing gender (SEX=missing), age (AGE=missing), quarter (DQTR=missing), year (YEAR=missing)
with missing MDC (MDC=missing) when the user indicates that MDC is provided

PSI 12: Surgical discharges for patients ages 18 years and older with any listed ICD-10- PCS procedure code for an operating room procedure. Surgical discharges are defined by specific MS-DRG codes.
Exclude discharges:

with a principal ICD-10-CM diagnosis code (or secondary diagnosis present on admission) for proximal deep vein thrombosis
with a principal ICD-10-CM diagnosis code (or secondary diagnosis present on admission) for pulmonary embolism
with any listed ICD-10-CM diagnosis code for heparin-induced thrombocytopenia
where a procedure for interruption of vena cava occurs before or on the same day as the first operating room procedure
where a procedure for pulmonary arterial or dialysis access thrombectomy occurs before or on the same day as the first operating room procedure
where the only operating room procedure(s) is/are for interruption of vena cava and/or pulmonary arterial or dialysis access thrombectomy
with any listed ICD-10-CM diagnosis code present on admission for acute brain or spinal injury
with any listed ICD-10-PCS procedure code for extracorporeal membrane oxygenation (ECMO)
with a principal ICD-10-CM diagnosis code assigned to MDC 14 Pregnancy, Childbirth & the Puerperium
with a principal ICD-10-CM diagnosis code assigned to MDC 15 Newborns & Other Neonates with Conditions Originating in Perinatal Period
with an ungroupable DRG (DRG=999)
with missing gender (SEX=missing), age (AGE=missing), quarter (DQTR=missing), year (YEAR=missing), or principal diagnosis (DGNS_CD1=missing)
with missing MDC (MDC=missing) when the user indicates that MDC is provided

PSI 13: Elective surgical discharges for patients ages 18 years and older with any listed ICD- 10-PCS procedure code for an operating room procedure. Elective surgical discharges are defined by specific MS-DRG codes with admission type recorded as elective (TYPE_ADM=3).
Exclude discharges:

with a principal ICD-10-CM diagnosis code (or secondary diagnosis present on admission) or sepsis
with a principal ICD-10-CM diagnosis code (or secondary diagnosis present on admission) for infection
with a principal ICD-10-CM diagnosis code assigned to MDC 14 Pregnancy, Childbirth & the Puerperium
with a principal ICD-10-CM diagnosis code assigned to MDC 15 Newborns & Other Neonates with Conditions Originating in Perinatal Period
with an ungroupable DRG (DRG=999)
with missing gender (SEX=missing), age (AGE=missing), quarter (DQTR=missing), year (YEAR=missing), or principal diagnosis (DGNS_CD1=missing)
with missing MDC (MDC=missing) when the user indicates that MDC is provided

PSI 14: Discharges, for patients ages 18 years and older with any listed ICD-10-PCS procedure code for abdominopelvic surgery, open approach, or with any listed ICD-10-PCS procedure code for abdominopelvic surgery, other than open approach.
Exclude discharges:

the last date of a procedure for abdominal wall reclosure occurs on or before the date of the first open abdominopelvic surgery procedure, if any, and on or before the date of the first abdominopelvic surgery, other than open approach, if any
with an ICD-10-CM principal or secondary diagnosis code present on admission for disruption of internal operation (surgical) wound
with length of stay less than two (2) days
with a principal ICD-10-CM diagnosis code assigned to MDC 14 Pregnancy, Childbirth & the Puerperium
with a principal ICD-10-CM diagnosis code assigned to MDC 15 Newborns & Other Neonates with Conditions Originating in Perinatal Period
with an ungroupable DRG (DRG=999)

PSI 15: Surgical or medical discharges, for patients ages 18 years and older, with any ICD-10-PCS procedure code for an abdominopelvic procedure.
Exclude discharges:

with a missing index abdominopelvic procedure date and/or missing all subsequent abdominopelvic procedure dates
with a principal ICD-10-CM diagnosis code assigned to MDC 14 Pregnancy, Childbirth & the Puerperium
with a principal ICD-10-CM diagnosis code assigned to MDC 15 Newborns & Other Neonates with Conditions Originating in Perinatal Period
with an ungroupable DRG (DRG=999)
with missing gender (SEX=missing), age (AGE=missing), quarter (DQTR=missing), year (YEAR=missing), or principal diagnosis (DGNS_CD1=missing)
with missing MDC (MDC=missing) when the user indicates that MDC is provided

1.15a Denominator Details

Denominator details are outlined in the individual component's measure specifications, which are attached as Excel files in section 1.13. In each measure's specification attachment, the measure description tab shows the name of code sets included in the denominator, and the following tabs contain the lists of codes within each code set. Other code sets that are referenced as Appendix A-G are included in separate Excel files.

A complete list of the CMS PSI 90 and its component measure specifications can also be found on https://qualitynet.cms.gov/inpatient/measures/psi/resources.

1.15d Age Group

Adults (18-64 years)

Older Adults (65 years and older)

Exclusions

1.15b Denominator Exclusions

PSI 03:

Length of stay of less than 3 days
Principal ICD-10-CM diagnosis code for site-specific pressure ulcer stage 3 or 4 (or unstageable) or deep tissue injury at the same anatomic site
Any ICD-10-CM diagnosis code for severe burns (≥20% body surface area)
Any ICD-10-CM diagnosis code for exfoliative disorders of the skin (≥20% body surface area)
Principal ICD-10-CM diagnosis code assigned to MDC 14 Pregnancy, Childbirth & the Puerperium
Principal ICD-10-CM diagnosis code assigned to MDC 15 Newborns & Other Neonates with conditions originating in perinatal period
Ungroupable DRG (DRG=999)
Missing gender (SEX=missing), age (AGE=missing), quarter (DQTR=missing), year (YEAR=missing), principal diagnosis (DGNS_CD1=missing), or MDC (MDC=missing) when the user indicates that MDC is provided

PSI 06:

Principal ICD-10-CM diagnosis (or secondary diagnosis present on admission) of non-traumatic pneumothorax
Any listed ICD-10-CM diagnosis code for specified chest trauma (rib fractures, traumatic pneumothorax and related chest wall injuries)
Any listed ICD-10-CM diagnosis code for pleural effusion
Any listed ICD-10-PCS procedure code for thoracic surgery, including lung or pleural biopsy and diaphragmatic repair
Any listed ICD-10-PCS procedure code for potentially trans-pleural cardiac procedure
Principal ICD-10-CM diagnosis code assigned to MDC 14 Pregnancy, Childbirth & the Puerperium
Principal ICD-10-CM diagnosis code assigned to MDC 15 Newborns & Other Neonates with Conditions Originating in Perinatal Period
Ungroupable DRG (DRG=999)
Missing gender (SEX=missing), age (AGE=missing), quarter (DQTR=missing), year (YEAR=missing), principal diagnosis (DGNS_CD1=missing), or MDC (MDC=missing) when the user indicates that MDC is provided

PSI 08

Principal ICD-10-CM diagnosis code (or secondary diagnosis present on admission) for fracture
Any listed ICD-10-CM diagnosis code for joint prosthesis-associated fracture
Principal ICD-10-CM diagnosis code assigned to MDC 14 Pregnancy, Childbirth & the Puerperium
Principal ICD-10-CM diagnosis code assigned to MDC 15 Newborns & Other Neonates with Conditions Originating in Perinatal Period
Ungroupable DRG (DRG=999)
Missing gender (SEX=missing), age (AGE=missing), quarter (DQTR=missing), year (YEAR=missing), principal diagnosis (DGNS_CD1=missing), or MDC (MDC=missing) when the user indicates that MDC is provided

PSI 09

Principal ICD-10-CM diagnosis code (or secondary diagnosis present on admission) for postoperative hemorrhage or postoperative hematoma
The only operating room procedure is for treatment of postoperative hemorrhage or hematoma
Treatment of postoperative hemorrhage or hematoma occurs before the first operating room procedure, if the dates of both procedures are available
Any listed ICD-10-CM diagnosis code for coagulation disorder
Principal ICD-10-CM diagnosis code assigned to MDC 14 Pregnancy, Childbirth & the Puerperium
Principal ICD-10-CM diagnosis code assigned to MDC 15 Newborns & Other Neonates with Conditions Originating in Perinatal Period
Ungroupable DRG (DRG=999)
Missing gender (SEX=missing), age (AGE=missing), quarter (DQTR=missing), year (YEAR=missing), principal diagnosis (DGNS_CD1=missing), or MDC (MDC=missing) when the user indicates that MDC is provided

PSI 10

Principal ICD-10-CM diagnosis code (or secondary diagnosis present on admission) for acute kidney failure
Dialysis procedure that occurs before or on the same day as the first operating room procedure
Dialysis access procedure that occurs before or on the same day as the first operating
room procedure
Principal ICD-10-CM diagnosis code (or secondary diagnosis present on admission) for cardiac arrest, severe cardiac dysrhythmia, shock, chronic kidney disease stage 5, end stage renal disease, or urinary tract obstruction
Any ICD-10-CM diagnosis present on admission of solitary kidney and any ICD-10-PCS procedure code for partial or total nephrectomy
Principal ICD-10-CM diagnosis code assigned to MDC 14 Pregnancy, Childbirth & the Puerperium
Principal ICD-10-CM diagnosis code assigned to MDC 15 Newborns & Other Neonates with Conditions Originating in Perinatal Period
Ungroupable DRG (DRG=999)
Missing gender (SEX=missing), age (AGE=missing), quarter (DQTR=missing), year (YEAR=missing), or principal diagnosis (DGNS_CD1=missing)

PSI 11

Principal ICD-10-CM diagnosis code (or secondary diagnosis present on admission) of acute respiratory failure (ACURF3D*)
Any listed ICD-10-CM diagnosis code present on admission for tracheostomy (TRACHID*)
The only operating room procedure is tracheostomy (TRACHIP*)
Procedure for tracheostomy (TRACHIP*) occurs before the first operating room procedure (Appendix A: ORPROC), if the dates of both procedures are available
Any listed ICD-10-CM diagnosis code for malignant hyperthermia (MALHYPD*)
Any listed ICD-10-CM diagnosis code present on admission for neuromuscular disorder (NEUROMD*)
Any listed ICD-10-CM diagnosis code present on admission for degenerative neurological disorder (DGNEUID*)
Any listed ICD-10-PCS procedure code for laryngeal, pharyngeal, nose, mouth, or facial surgery involving significant risk of airway compromise (NUCRANP*)
Any listed ICD-10-PCS procedure code for esophageal surgery (PRESOPP*)
Any listed ICD-10-PCS procedure code for lung cancer (LUNGCIP*)
Any listed ICD-10-PCS procedure code for lung or heart transplant (LUNGTRANSP*)
Principal ICD-10-CM diagnosis code assigned to MDC 4 Diseases & Disorders of the Respiratory System
Principal ICD-10-CM diagnosis code assigned to MDC 14 Pregnancy, Childbirth & the Puerperium
Principal ICD-10-CM diagnosis code assigned to MDC 15 Newborns & Other Neonates with Conditions Originating in Perinatal Period
Ungroupable DRG (DRG=999)
Missing gender (SEX=missing), age (AGE=missing), quarter (DQTR=missing), year (YEAR=missing), or MDC (MDC=missing) when the user indicates that MDC is provided

PSI 12

Principal ICD-10-CM diagnosis code (or secondary diagnosis present on admission) for proximal deep vein thrombosis (DEEPVIB*)
Principal ICD-10-CM diagnosis code (or secondary diagnosis present on admission) for pulmonary embolism (PULMOID*)
Any listed ICD-10-CM diagnosis code for heparin-induced thrombocytopenia (HITD*)
Procedure for interruption of vena cava (VENACIP*) occurs before or on the same day as the first operating room procedure
Procedure for pulmonary arterial or dialysis access thrombectomy (THROMP*) occurs before or on the same day as the first operating room procedure
Operating room procedure(s) is/are for interruption of vena cava (VENACIP*) and/or pulmonary arterial or dialysis access thrombectomy (THROMP*)
Any listed ICD-10-CM diagnosis code present on admission for acute brain or spinal injury
(NEURTRAD*)
Any listed ICD-10-PCS procedure code for extracorporeal membrane oxygenation (ECMO) (ECMOP*)
Principal ICD-10-CM diagnosis code assigned to MDC 14 Pregnancy, Childbirth & the Puerperium
Principal ICD-10-CM diagnosis code assigned to MDC 15 Newborns & Other Neonates with Conditions Originating in Perinatal Period
Ungroupable DRG (DRG=999)
Missing gender (SEX=missing), age (AGE=missing), quarter (DQTR=missing), year (YEAR=missing), principal diagnosis (DGNS_CD1=missing), or MDC (MDC=missing) when the user indicates that MDC is provided

PSI 13

Principal ICD-10-CM diagnosis code (or secondary diagnosis present on admission) for sepsis (SEPTI2D*)
Principal ICD-10-CM diagnosis code (or secondary diagnosis present on admission) for infection
Principal ICD-10-CM diagnosis code assigned to MDC 14 Pregnancy, Childbirth & the Puerperium
Principal ICD-10-CM diagnosis code assigned to MDC 15 Newborns & Other Neonates with Conditions Originating in Perinatal Period
Ungroupable DRG (DRG=999)
Missing gender (SEX=missing), age (AGE=missing), quarter (DQTR=missing), year (YEAR=missing), principal diagnosis (DGNS_CD1=missing), or MDC (MDC=missing) when the user indicates that MDC is provided

PSI 14

Last date of a procedure for abdominal wall reclosure (RECLOIP*) occurs on or before the date of the first open abdominopelvic surgery procedure (ABDOMIPOPEN*), if any, and on or before the date of the first abdominopelvic surgery, other than open approach (ABDOMIPOTHER*), if any
ICD-10-CM principal or secondary diagnosis code present on admission for disruption of internal operation (surgical) wound (ABWALLCD*)
Length of stay less than two (2) days
Principal ICD-10-CM diagnosis code assigned to MDC 14 Pregnancy, Childbirth & the Puerperium
with a principal ICD-10-CM diagnosis code assigned to MDC 15 Newborns & Other Neonates with Conditions Originating in Perinatal Period
Ungroupable DRG (DRG=999)

PSI 15

Missing index abdominopelvic procedure date and/or missing all subsequent abdominopelvic procedure dates
Principal ICD-10-CM diagnosis code assigned to MDC 14 Pregnancy, Childbirth & the Puerperium
Principal ICD-10-CM diagnosis code assigned to MDC 15 Newborns & Other Neonates with Conditions Originating in Perinatal Period
Ungroupable DRG (DRG=999)
with missing gender (SEX=missing), age (AGE=missing), quarter (DQTR=missing), year (YEAR=missing), principal diagnosis (DGNS_CD1=missing, or MDC (MDC=missing) when the user indicates that MDC is provided

1.15c Denominator Exclusions Details

Denominator exclusion details are outlined in the individual component's measure specifications, which are attached as Excel files in section 1.13. In each measure's specification attachment, the measure description tab shows the name of code sets included in the denominator exclusion, and the following tabs contain the lists of codes within each code set. Other code sets that are referenced as Appendix A-G are included in separate Excel files.

A complete list of the CMS PSI 90 and its component measure specifications can also be found on https://qualitynet.cms.gov/inpatient/measures/psi/resources.

Measure Calculation

1.13a Attach Data Dictionary

1.13 CMS PSI v14 Measure Specifications - Excel.zip

1.16 Type of Score

Composite scale

1.17 Measure Score Interpretation

Better quality = Lower score

1.18 Calculation of Measure Score

For each PSI component, calculate the observed rate as the number of discharges where the patient experienced the component event (numerator) divided by the number of discharges where the patient is at risk or “eligible” for the component event (denominator) per hospital.
For each PSI component, use the component’s risk adjustment model (built using an external reference population and available in the CMS PSI software) to estimate the predicted probabilities of patients from each denominator-eligible discharge to experience the component event. Sum up these discharge-level predicted probabilities per hospital and divide by the denominator count per hospital to obtain the hospital’s expected rate. The expected rate reflects the rate of adverse events that a hospital would anticipate based on the hospital’s case-mix.
For each PSI component, calculate the risk-adjusted rate as the observed rate divided by the expected rate per hospital, multiplied by the reference population rate (available in the CMS PSI Software). The risk-adjusted rate reflects the performance of a hospital treating its patients relative to the hypothetical average hospital treating patients with the same characteristics.
For each PSI component, calculate the smoothed rate as the weighted average of the risk-adjusted rate and the reference population rate per hospital, where the weight is the hospital’s signal-to-noise reliability ratio. Smoothing adjusts risk-adjusted rates according to each hospital’s level of measure reliability, such that for hospitals with unstable estimates based on noisy data (e.g., due to small sample sizes), smoothing pulls the hospital’s risk-adjusted rate closer to the reference population rate.
Calculate the PSI 90 composite score as the weighted average of the ten PSI components’ smoothed observed-to-expected ratios per hospital, which are the smoothed rates calculated in step 4 divided by the reference population rate. The weights for each component reflect the component’s volume (frequency of event) in the reference population and harm (excess risk of harms generated by experiencing the event and perceived severity of those harms). Information on harm and volume weights are included in Table 1 of the 7.1 Supplemental Attachment.

1.19 Measure Stratification Details

The measure is not stratified.

1.26 Minimum Sample Size

PSI 90 scores are calculated only if hospitals have (1) at least 25 eligible discharges on at least 1 PSI component and (2) at least 3 eligible discharges on at least 7 PSI components.

Importance

Evidence

2.1 Attach Logic Model

ImportanceAttachmentPSI90.pdf

2.2 Evidence of Measure Importance

See 2.1 – 2.2 Evidence of Measure Importance Attachment

Performance Gap

2.4 Performance Gap

We assessed for performance gaps in the PSI 90 composite score using claims data from Medicare fee-for-service (FFS) discharges from the universe of Inpatient Prospective Payment System (IPPS) hospitals from July 1, 2021 to June 30, 2023. We used these same data in our testing activities which we describe in the scientific acceptability sections of this form. These data included a total of 3,156 hospitals and 13,702,080 discharges. Out of the 3,156 hospitals, 2,922 hospitals had data that meet the minimum sample size, whose distribution of PSI 90 scores is described in Table 1 (embedded in this submission form) below. The row “N of Persons / Encounters / Episodes” contains the number of unique denominator-eligible discharges across the 10 PSI components. Measure scores were calculated using the CMS PSI software version 14. Although many hospitals’ performance meet or exceed the national average (median PSI 90 score = 0.964), some hospitals fall behind, suggesting there is room for improvement. It is also crucial to continue monitoring all hospitals to ensure patient safety remains a priority.

Table 1. Performance Scores by Decile

Performance Gap
	Overall	Minimum	Decile_1	Decile_2	Decile_3	Decile_4	Decile_5	Decile_6	Decile_7	Decile_8	Decile_9	Decile_10	Maximum
Mean Performance Score	0.999	0.534	0.758	0.856	0.897	0.929	0.953	0.976	1.008	1.064	1.148	1.396	3.098
N of Entities	2,922	1	292	293	291	292	293	293	291	292	293	292	1
N of Persons / Encounters / Episodes	13,638,662	8,781	2,300,343	1,306,200	1,208,054	978,132	782,805	723,791	1,357,922	1,495,171	1,578,020	1,908,224	12,406

Equity

Equity

Equity

3.1 Contributions Toward Advancing Health Equity

PSI 90 is a composite measure focusing on patient safety related outcomes. To understand disparities in patient safety, we share the PSI 90 disparity analysis from the Agency for Healthcare Research and Quality's (AHRQ) 2023 National Healthcare Quality and Disparities Report (NHQDR). Specifically, information contained in Appendix B (available at https://www.ahrq.gov/sites/default/files/wysiwyg/research/findings/nhqrdr/2023-qdr-appendixb-measure-category-5.pdf) addresses this issue. We examined disparities by gender, race/ethnicity, income, and metropolitan status.

Given we are utilizing AHRQ’s report for this analysis, we first share the bulleted information about the analysis prior to sharing the results.

Component measures. The subgroup results below are reported at the individual component measure level. We report component measure information in this section because PSI 90 is not calculated at the patient or subgroup levels. Additionally, the composite scores are smoothed to the national average, which is not calculated at the subgroup level.
Adjusted for gender. All 10 component measures are risk-adjusted for gender (as well as a range of other demographic and clinical risk factors; see section 4.4.4 for more details). Therefore, the component measure rates used to calculate gender disparities in the AHRQ report used a modified risk-adjustment model that excluded gender as a risk factor.
Use of all payer data. The data (Healthcare Cost and Utilization Project [HCUP]) used within the AHRQ report captures all payers, which is appropriate for the disparity analysis to more broadly capture relevant patient subgroups. The rest of this submission form focuses on the Medicare Fee For Service (FFS) population given this is the population used to calculate hospitals’ pay for performance results in CMS programs.
AHRQ software. The disparity analysis used the AHRQ version of the PSI software, whereas the rest of the submission form uses the CMS version. The two software sets are substantively aligned, however the AHRQ version is structured to run on the HCUP data, and the CMS version is structured to run on Medicare FFS data.
Software version. The disparity analysis reports results using v13 of the software, whereas the rest of the submission form uses v14. While there are substantial differences between the software, the general results for patient safety should be aligned across v13 and v14.

The disparity results bulleted below indicate that care disparities exist for all but two (PSI 06 and PSI 08) component measures. This means that there are statistically significant differences (with a relative difference of at least 10%) across levels of at least one characteristic (gender, race/ethnicity, income quartiles, metropolitan status). However, the nature of disparities varied from component to component.

Gender. The analysis reports better component measure rates for females compared to males for PSI 03, PSI 09, PSI 13, and PSI 14, but worse measure rates for females for PSI 15.
Race. The analysis reports worse component measure rates for black patients compared to white patients for PSI 03, PSI 09, PSI 10, PSI 12, PSI 13, and PSI 15. Similarly, Asian and Pacific Islander (API) patients had worse component measure rates compared to white patients for PSI 09, PSI 11, and PSI 13, but API patients had better measure rates than white patients for PSI 03 and PSI 12.
Ethnicity. The analysis reports better component measure rates for non-Hispanic white patients compared to Hispanic (all races) patients for PSI 13 and PSI 14, but Hispanic (all races) patients had better measure rates for PSI 09.
Income. The analysis reports worse component measure rates for patients in lower quartiles of income compared to those in high quartiles for PSI 03, PSI 10, PSI 11, PSI 13, and PSI 14.
Metropolitan status. The analysis reports worse component measure rates for less populated areas (e.g., small and medium metropolitan, micropolitan, noncore) compared to more populated areas (e.g., large fringe metro, large central metro) for some measures, such as PSI 03. However, other components, such as PSI 12, had the opposite pattern.

Even though there is no one particular subgroup with consistently poor component measure rates reported across all 10 PSI components, we will continue to monitor disparities particularly for black patients and patients in lower quartiles of income. We will focus on these groups given that among PSI components with significant disparities based on race and income, the directionality of the disparity was consistently worse for these subgroups.

Feasibility

Feasibility
Proprietary Information

Feasibility

4.1 Feasibility Assessment

This measure has been in the Centers for Medicare & Medicaid Services' (CMS) quality programs for nine years, and reporters have encountered minimal barriers when implementing the measure specifications, data abstraction, measure calculation, or performance reporting.

Administrative burden and cost associated with the PSI 90 data collection and entry is low. Hospitals generate the required data during routine patient care delivery processes and submit this information within their Medicare claim. Burden with generating a hospital’s score is low as it can be calculated with minimal resources using the publicly available CMS PSI software. Additionally, PSI 90 scores are calculated using only final claims data; thus missing data and the measure's susceptibility to inaccuracies are minimized. Patient confidentiality is also not a concern as PSI 90 scores are reported in aggregate at the hospital level.

Patient and caregivers from Mathematica’s Patient Family Advisory Board (PFAB) were asked to share what they believed were unintended impacts/consequences of PSI 90 on patients. PSI 90’s background, purpose, measure definitions, and patient use cases/example scenarios were reviewed prior to requesting feedback from the representatives. There was a total of four patient and caregiver representatives that provided feedback. No unintended consequence related to feasibility of implementing PSI 90 were expressed during the PFAB meeting.

4.3 Feasibility Informed Final Measure

No feasibility assessment was conducted as this is an established measure in CMS quality programs and final (Medicare) claims data generated during patient care delivery are used for measure score calculation. Therefore, no adjustments were made to the measure based on feasibility.

Scientific Acceptability

Testing Data

5.1.1 Data Used for Testing

Testing analyses were conducted using claims data from Medicare fee-for-service (FFS) discharges from the universe of Inpatient Prospective Payment System (IPPS) hospitals from July 1, 2021 to June 30, 2023. Only discharge records from inpatient final action claims were included. Records were excluded if the total length of stay exceeded 365 days, if the patient was enrolled in Medicare Advantage at time of discharge, or if key data elements required to determine inclusion were missing or inconsistent. Measure scores were calculated using the CMS PSI software version 14.

To obtain hospital-level information (e.g., hospital characteristics), we supplemented the discharge-level claims data with the publicly available CMS FY 2025 IPPS Proposed Rule Impact File.

5.1.2 Differences in Data

All testing results reported in this section are based on the data described in section 4.1.1 above, with the exception of the risk adjustment model coefficients reported in section 4.4.1a.

The risk adjustment models implemented in the version 14 CMS PSI software were fitted using claims data from Medicare FFS discharges from IPPS hospitals from July 1, 2020 to June 30, 2022 (software reference population). Therefore, the coefficient estimates provided in section 4.4.1a are based on this time period of data.

5.1.3 Characteristics of Measured Entities

The testing data included 3,156 hospitals (universe of IPPS hospitals), comprised of small (<100 beds; 37.6%), medium (100-250 beds; 33.9%), and large (>250 beds; 26.7%) hospitals. The majority of the hospitals were located in urban geographical areas (75.4%; comprised of 35.7% “large” urban hospitals and 39.7% “other” urban hospitals), with a smaller subset of hospitals being located in rural areas (22.7%). Nearly all hospitals (98.5%) were acute care hospitals, with the remaining hospitals having missing hospital types.

See Table 2 in the 7.1 Supplemental Attachment for a table summarizing the distribution of hospital characteristics.

5.1.4 Characteristics of Units of the Eligible Population

The testing data included 13,702,080 unique discharges that were denominator-eligible for at least one PSI component. The mean patient age among these discharges was 74.2 years (SD = 12.4), with a 53% female composition. 82% of these discharges were associated with white patients, 11% black patients, 2.2% Hispanic patients, 1.7% Asian patients, 2.5% North American Native patients, and 1.4% categorized as “other”.

Table 3 in the 7.1 Supplemental Attachment, additionally describes the demographic composition of the denominator-eligible discharges for each PSI component separately, as well as for the set of all 13,702,080 discharges.

Reliability

5.2.1 Level(s) of Reliability Testing Conducted

Accountable entity level (i.e., measure score) (e.g., signal-to-noise analysis)

5.2.2 Method(s) of Reliability Testing

Component measure reliability:

We used a signal-to-noise approach to assess the precision of PSI component rates in distinguishing between hospitals according to their care quality. Signal-to-noise reliability assesses the extent to which variation in measure scores are attributed to true differences in quality across hospitals (signal) as opposed to error, or differences within hospitals (noise). Risk-adjusted PSI component rates were used in this analysis, as the signal-to-noise approach is not appropriate for smoothed rates, which are already corrected for each hospital’s signal-to-noise reliability. All references to PSI component rates in this section below assume risk-adjusted rates.

For each PSI component c and hospital h, we estimated a signal-to-noise reliability as:

reliability = signal variance / (signal variance + noise variance). The reliability estimate can range from 0 to 1, with 0 indicating no reliability where all variation in component rates is due to noise, and 1 indicating perfect reliability where all variation in component rates is due to between-hospital differences. Noise variance (one estimate per component per hospital) was estimated using the sampling variance of each hospital’s PSI component rate. Signal variance (one estimate per component) was computed iteratively using the empirical Bayes method (Morris, 1983) to estimate the between-hospital variance in each PSI component rate.

See Section 4.2.2 of the 4.2.3a Reliability Testing attachment for a full description of the computation, including formulas.

Morris, C. N. (1983). Parametric empirical Bayes inference: theory and applications. Journal of the American statistical Association, 78(381), 47-55.

Composite measure reliability:

The signal-to-noise approach is not appropriate for assessing the composite reliability, because the PSI 90 composite is constructed from smoothed PSI component rates, which already account for each hospital’s signal-to-noise reliability (see section 1.18, step #4 for an explanation of the smoothing process). Therefore, we used a split-half reliability approach and calculated an intraclass correlation coefficient (ICC) for each hospital using an estimated between-hospital variance component, error variance component, and the number of unique denominator-eligible discharges across all PSI components for each hospital. Similar to signal-to-noise, the ICC quantifies the amount of variation in measure scores due to between-hospital differences rather than within.

We obtained estimates of the between-hospital and error variance components from a simple, intercept-only random-effects model with no predictors. This random-effects model was fit to data made multilevel by creating random split-half samples, without replacement, of discharges (denominator-eligible for at least one PSI component) within each hospital and thereby producing two PSI 90 composite scores per hospital. ICCs range from 0 to 1, where 0 indicates no agreement or reliability, and 1 indicates perfect agreement or reliability. ICCs were additionally corrected using the Spearman-Brown formula to account for the fact that composite scores used in the random-effects model were computed using only half of the discharges from each hospital.

See Section 4.2.2 of the 4.2.3a Reliability Testing attachment for a full description of the method, including formulas. Note that these methods for assessing the component and composite measure reliability are consistent with methods used in the prior CBE maintenance review.

5.2.3 Reliability Testing Results

Component measure reliability: See Table 4 of the 4.2.3a Reliability Testing attachment for the distribution of signal-to-noise reliability estimates across hospitals for each PSI component. Note that component reliability results are only reported for hospitals with at least three denominator-eligible discharges for that component.

Composite measure reliability: See Table 5 of the 4.2.3a Reliability Testing attachment for the distribution of ICC estimates across hospitals for PSI 90. Note that composite reliability results are only reported for hospitals with a non-missing PSI 90 composite score (i.e., hospitals that met the minimum sample size requirements described in section 1.26; N = 2,922).

Table 2 (embedded in this submission form) below contains the distribution of ICC estimates across hospitals for PSI 90, where hospitals are grouped into deciles based on N, or the number of unique denominator-eligible discharges across all PSI components. For example, the “reliability” row of this table contains the mean ICC estimate among hospitals in the first decile of N, second decile of N, and so on. Similarly, the “mean performance score” row of this table contains the mean PSI 90 score among hospitals in the first decile of N, and so on. The row “N of Persons / Encounters / Episodes” of this table contains the sum of N among hospitals in each decile group. The minimum and maximum columns of this table contain the mean reliability, mean ICC, etc. of the hospital(s) with the minimum and maximum N, respectively.

5.2.3a Attach Additional Reliability Testing Results

4.2.3a-Reliability-Attachment.pdf

5.2.4 Interpretation of Reliability Results

The component reliability results show that signal-to-noise reliability estimates vary across the PSI components, although across the board, we observe that component reliability is low. This is aligned with testing results from the prior CBE maintenance review, which also found low reliability at the component-level. We observed the highest reliability in PSI 03, with a median estimate of 0.57 and 47% of hospitals having reliability estimates greater than or equal to the PQM threshold of 0.6. We observed the lowest reliability in PSI 14, with a median estimate of 0.04 and 0.03% of hospitals having reliability estimates greater than or equal to 0.6.

The composite reliability results, on the other hand, show that PSI 90 as a composite has acceptable reliability. This is also consistent with testing results from the prior CBE maintenance review. The median ICC was 0.82, with 78% of hospitals having ICCs greater than or equal to 0.6. This demonstrates the importance of the construction of PSI 90 as a composite measure that borrows strength from the collection of PSI components.

Table 2. Accountable Entity Level Reliability Testing Results by Denominator, Target Population Size

Accountable Entity-Level Reliability Testing Results
	Overall	Minimum	Decile_1	Decile_2	Decile_3	Decile_4	Decile_5	Decile_6	Decile_7	Decile_8	Decile_9	Decile_10	Maximum
Reliability	0.747	0.040	0.303	0.519	0.636	0.724	0.790	0.836	0.874	0.902	0.928	0.955	0.989
Mean Performance Score	0.999	0.993	0.979	0.976	0.988	0.979	0.987	1.033	1.014	0.997	1.016	1.016	1.053
N of Entities	2,992	2	293	292	293	292	292	292	292	292	292	292	1
N of Persons / Encounters / Episodes	13,638,662	58	94,969	221,590	360,087	538,059	769,468	1,041,782	1,417,480	1,876,743	2,640,780	4,677,704	59,941

Validity

5.3.1 Level(s) of Validity Testing Conducted

Accountable entity level (i.e., measure score) (e.g., criterion validity)

5.3.3 Method(s) of Validity Testing

We assessed the construct validity of the PSI components and PSI 90 composite to investigate the extent to which these measures assess the construct of patient safety that they are intended to measure. Construct validity can be demonstrated using multiple forms of empirical evidence, such as correlations, predictive relationships, and group differences, that support a priori hypotheses about the construct.

We conducted correlational analyses to demonstrate evidence of validity. Specifically, we calculated Spearman’s correlations between risk-adjusted PSI component rates and other relevant but independently collected hospital-level measures of patient safety or harm events. We also calculated correlations between the PSI 90 composite scores and these same criterion measures. We used risk-adjusted rather than smoothed rates in the component validity analysis to maximize variability in component rates. The criterion measures were obtained from the public Hospital Care Compare data (available at https://data.cms.gov/provider-data/topics/hospitals) and included healthcare-associated infection rates (central line-associated bloodstream infections, catheter-associated urinary tract infections, surgical site infection from colon surgery, surgical site infection from abdominal hysterectomy, Methicillin-resistant Staphylococcus aureus bloodstream infections, and clostridium difficile infections), complication rate for hip/knee replacement patients, and 30-day readmission rates (for acute myocardial infarction patients, coronary artery bypass graft patients, chronic obstructive pulmonary disease patients, heart failure patients, after hip/knee surgery, pneumonia patients, and after discharge from hospital).

While we expect positive correlations between the PSI components and PSI 90 and these criterion measures given that they all capture patient safety and adverse events, we anticipated weak positive correlations across the board. PSI 90 is a complex, formative composite measure that accounts for various aspects of patient safety and therefore may not correlate strongly with any one criterion measure. We also expected the strength of correlations to differ according to the closeness in content to the PSI components in the specific aspect of patient safety assessed by the criterion measure.

5.3.4 Validity Testing Results

See Table 6 of the 4.3.4a Validity Testing attachment for a correlation matrix of the PSI measures (components and the composite) and the criterion measures.

5.3.4a Attach Additional Validity Testing Results

4.3.4a-Validity-Attachment.pdf

5.3.5 Interpretation of Validity Results

The validity analysis shows that as expected, PSI 90 is positively, although weakly, correlated with the criterion measures. This is also consistent with testing results from the prior CBE maintenance review, which also found weak, positive correlations between PSI 90 and these criterion measures. PSI 90 had the strongest correlation with the hip/knee replacement complication rate, Methicillin-resistant Staphylococcus aureus (MRSA) bloodstream infection rate, and clostridium difficile (C. diff) infection rate (all correlated at r = 0.12, p<.001). Overall, we observed stronger correlations between PSI 90 and the infection rates than between PSI 90 and the readmission rates. This is expected, given that infection rates are more directly reflective of in-hospital safety events, as is PSI 90, whereas readmission rates can further depend on events occurring outside of the hospital after a patient has been discharged.

Among the PSI component correlations, we observed the strongest correlation (r = 0.12) between PSI 10 and the surgical site infection rate from colon surgery and between PSI 10 and the surgical site infection rate from abdominal hysterectomy. Similar to PSI 90, the PSI components generally had stronger and more positive correlations with the infection rates than the readmission rates. We also note that correlations with the criterion measures were generally stronger among PSI 90 rather than the PSI components.

Overall, the analysis provides support for the construct validity for the PSI 90. The weak, positive correlations observed among PSI 90 and the criterion measures demonstrate that a broad, common construct of patient safety underlies these measures, but PSI 90 does not entirely overlap with any one measure of patient safety. Given that we observed stronger correlations between PSI 90 and the criterion measures than between the PSI components and the criterion measures, we emphasize the importance of the construction of PSI 90 as a composite.

5.3.2 Type of Accountable Entity Level Validity Testing Conducted (derived)

Empirical validity testing at the accountable entity-level (e.g., criterion validity, construct validity, known groups analysis)

Risk Adjustment

5.4.1 Methods Used to Address Risk Factors

Statistical risk adjustment model with risk factors

5.4.2 Conceptual Model Rationale

PSI components are risk-adjusted because some patients have higher risks of experiencing in-hospital adverse events, regardless of the hospital's quality of care (e.g., due to patients' preexisting conditions). Each PSI component is risk-adjusted (via separate risk adjustment models) in order to account for these differences in patient case-mix across hospitals. The risk adjustment models are used to generate component event rates that reflect what would be observed in a hospital if its observed level of care were applied to a case-mix of patients similar to the reference population, allowing for more standardized comparisons in event rates across hospitals.

The conceptual model for risk adjustment considers patients’ key demographic characteristics, severity of illness, clinical/comorbidities, and discharge-specific information (e.g., transfer-in as an indicator of recent health service use at a different facility as risk factors). The key criterion for risk factors include:

Observability and validity using administrative data across hospitals
Clinical relevance to the adverse event (i.e., outcome)
Adequate prevalence to obtain reasonably precise estimates of risk
Systematic variation by hospital

Risk adjustment models are developed annually as part of the measure production process. The approach to risk adjustment is to consider as many risk factors as theoretically justified and computationally practical, and to apply a feature selection technique to select an optimal set of risk factors among the candidate variables before fitting a final risk adjustment model for each PSI component (see section 4.4.4 for more details).

5.4.2a Attach Conceptual Model

4.4.2aConceptualRiskModel.pdf

5.4.3 Variable Distribution Across Measured Entities

See Table 7 in 7.1 Supplemental Attachment for distributions of risk factor characteristics across hospitals. For example, for the risk factor Age_30_34 (indicator for whether a patient is 30-34 years old), the mean would indicate the average (across hospitals) percentage composition of discharges where the patient is 30-34 years old.

Table 7 additionally indicates whether each risk factor is included in the final risk adjustment model for each PSI component for version 14 of the specifications.

5.4.4 Risk/Case-Mix Adjustment Modeling and/or Stratification Results

There is a risk adjustment model for each PSI component. Each risk adjustment model begins with a candidate list of risk factors, including sex, age category, age-sex interactions, Medicare Severity-Diagnosis Related Groups (MS-DRGs) aggregated across levels of comorbidities or complications, Major Diagnostic Categories (MDCs), 38 Elixhauser comorbidity variables, categories of unweighted counts of comorbidities, whether the patient was transferred from another acute care hospital, COVID-19 diagnosis, COVID-19-quarter, and COVID-19 diagnosis-quarter interactions. Several PSI components had additional candidate risk factors specific to those components – for example, PSI 03, 06, 08, and 15 additionally considered a flag indicating medical (as opposed to surgical) discharges.

From the candidate list of risk factors, variables were selected for inclusion in the final risk adjustment models using least absolute shrinkage and selection operator (LASSO) regression models. Using the selected risk factors, final risk adjustment models were fitted using generalized estimating equations (GEE) to account for nesting of patients within hospitals.

See attachment in 4.4.4a for the resulting list of risk factors selected for inclusion in each of the final risk adjustment models of the version 14 CMS PSI software, coefficient estimates from the GEEs, and their 95% confidence intervals.

5.4.4a Attach Risk/Case-mix Adjustment Modeling and/or Stratification Specifications

4.4.4a Risk-Adjustment.xlsx

5.4.5 Calibration and Discrimination

Each component’s risk adjustment model was assessed for discriminative ability using the C-statistic, which is the probability that a randomly selected discharge with the PSI component event has a higher model-predicted probability of experiencing the event than a randomly selected discharge without the PSI component event. The C-statistic generally ranges from 0.5 to 1, with 0.5 indicating that the model classifies discharges with and without the event no better than random chance, and 1 indicating perfect classification. C-statistics between 0.7 and 0.8 are generally considered to be acceptable discrimination, and those between 0.8 and 0.9 are considered excellent discrimination (Hosmer & Lemeshow, 1995). We observed acceptable or excellent discrimination across all ten components, with the lowest C-statistic at 0.73 for PSI 12 and the highest at 0.90 for PSI 10. These results demonstrate that the risk-adjustment models are able to distinguish discharges with and without the respective PSI component event with adequate accuracy.

See Table 8 of the 4.4.5a Calibration and Discrimination attachment for all C-statistics.

Note that there is no risk-adjustment model for the PSI 90 composite. The individual PSI components are risk-adjusted, and the PSI 90 is a composite of those risk-adjusted (and smoothed) components. Therefore, we cannot report on the discriminative ability or calibration of the PSI 90 composite.

Calibration was assessed using a Hosmer-Lemeshow decile plot per risk-adjustment model. These plots are created by first grouping the discharges into deciles of model-predicted probabilities of experiencing the PSI component event. Then, within each decile, we calculated (1) the proportion of discharges in the decile group with the PSI component (observed risk) and (2) the mean model-predicted probability of experiencing the PSI component in that decile group (expected risk). These observed and expected risks are then plotted on the y and x axis, respectively, for each decile. Our resulting Hosmer-Lemeshow plots for all components show data points approximately lining up along the 45-degree diagonal, indicating that each decile group has a roughly equal observed and expected risk. This demonstrates that the risk-adjustment models are well-calibrated.

See the 4.4.5a Calibration and Discrimination attachment for all Hosmer-Lemeshow decile plots.

Hosmer DW, Lemeshow S. Confidence interval estimates of an index of quality performance model based on logistic regression. Statistics in Med. 1995;14(19):2161-72.

5.4.5a Attach Calibration and Discrimination Testing Results

4.4.5a-Testing-Attachment.pdf

5.4.6 Interpretation of Risk/Case-mix Factor Findings

We can interpret the risk factors included in each final risk adjustment model (i.e., the results of feature selection using LASSO regressions) to be the set of risk factors that provide an optimal balance between model complexity and performance. These final risk adjustment models were evaluated on our testing data and shown to have satisfactory discriminative ability (C-statistics above 0.7 in all ten PSI components) and calibration (all ten PSI components’ decile plots showing approximately equal observed to expected risks across most deciles). These results suggest that the risk adjustment models are adequately able to predict patients’ risk of PSI component events using the patient or discharge characteristics (i.e., risk factors) included in the risk adjustment models.

5.4.7 Final Approach to Address Risk Factors

Statistical risk adjustment model with risk factors

Use & Usability

Use
Usability

Use

6.1.1 Current Status

In use

6.1.3 Current Use(s)

Public Reporting

Payment Program

6.1.4 Program Details

Name of the program and sponsor

Centers for Medicare & Medicaid Services (CMS) - Hospital-Acquired Condition Reduction Program (HACRP)

Geographic area and percentage of accountable entities and patients included

In recent years, CMS has used Medicare FFS claims data from ~3,100 subsection (d) hospitals from all US states including DC and excluding MD which includes ~14 million inpatient stays for ~8 million patients to calculate PSI 90 results

Applicable level of analysis and care setting

The program reports measure scores at the hospital-level and applies to the inpatient care setting.

Name of the program and sponsor

Centers for Medicare & Medicaid Services (CMS) - Hospital Care Compare

Geographic area and percentage of accountable entities and patients included

In recent years, CMS has used Medicare FFS claims data from ~3,100 subsection (d) hospitals from all US states including DC plus 130 Veterans Affairs hospitals which includes ~14 million inpatient stays for ~8 million patients to calculate PSI 90 results

Applicable level of analysis and care setting

The program reports measure scores at the hospital-level and applies to the inpatient care setting.

Usability

6.2.1 Actions of Measured Entities to Improve Performance

Measured entities can improve the PSI 90 scores in a variety of ways. Annually, measured entities receive Hospital Specific Reports (HSR) from CMS with their PSI 90 score, detailing how their PSI 90 score compares to the national average, and patient level data related to the PSI 90 component events. Measured entities also have access to a guide to support the interpretation of the results listed in the HSR. These results can be used help measured entities understand which PSI 90 components are impacting their score, run further analyses, and compare their performance to other organizations. Additional instructions on how to use the HSR can be found on https://qualitynet.cms.gov/inpatient/measures/psi/reports.

The "Toolkit for Using the AHRQ Quality Indicators” provides resources for measured entities to use to improve their performance on the PSI 90 component measures. Improving clinical documentation and coding and implementing targeted clinical intervention were a few examples cited in the toolkit. More information about how hospital can use the "Toolkit for Using the AHRQ Quality Indicators " to improve their performance on the PSI 90 components can be found on https://www.ahrq.gov/patient-safety/settings/hospital/resource/qitool/index.html.

Additional processes and structures of care that have been shown to reduce the safety events included in the composite are documented in the supplemental importance attachment (question 2.2 Evidence of Measure Importance).

6.2.2 Feedback on Measure Performance

The public, including measured entities, can send questions or comments regarding PSI 90 and its PSI components to CMS via the QualityNet Question and Answer Tool. Input can also be submitted to AHRQ via Qlsupport@ahrq.hhs.gov, which is considered for inclusion in both the CMS and AHRQ versions of the PSI component measure specifications.

The CMS PSI support team triages, troubleshoots, and responds to technical inquiries related to methodology and rationale behind the indicators and general questions related to the use of the software. During a calendar year, CMS typically provides technical support to over 100 inquiries.

These inquiries commonly involve clarification regarding the technical specifications of the component indicators, as well as clarification about the population subject to inclusion in the PSI 90 composite, eligible admission types, the number of diagnosis fields used to calculate the component measures, and the Medicare fee-for-service date ranges used to calculate PSI rates.

In addition, CMS’s PSI support team gathers input on PSI implementation from technical expert panels convened to support PSI development and maintenance and offers recommended changes for CMS’s consideration. Specific suggestions for refining or enhancing the PSI specifications are addressed by CMS in consultation with AHRQ as needed, in its capacity as the original developer of PSI 90.

6.2.3 Consideration of Measure Feedback

The CMS PSIs are updated annually, including updating indicator technical specifications in accordance with the latest coding guidance; suggestions from users and other stakeholders obtained through technical assistance, or workgroups; and the latest clinical and scientific research. CMS and AHRQ regularly reviews these sources, identifies possible indicator updates, and prioritizes updates for each indicator and software update based on expected impact on users.

6.2.4 Progress on Improvement

As CMS reports in their 2024 National Impact Assessment of the Centers for Medicare & Medicaid Services (CMS) Quality Measures Report, from 2013 to 2021 PSI 90 scores have remained relatively steady (see table 9 in the 7.1 Supplemental Attachment). The best performing hospitals’ (10th percentile) rates ranged from 0.6 to 0.9 with most recent years at 0.8. The worst performing hospitals’ (90th percentile) rates ranged from 1.0 to 1.2 with most recent years at 1.2. The number of hospitals reporting each year slightly changed with the 3,106 being the lowest number to report and 3,422 being the highest number to report.

Given the composite is a summary score of risk and reliability adjusted component measure scores, large changes in the composite measure scores are not expected.

Improvement is more likely to be observed at the component measure level. The CMS report indicates improvement in three of the PSI component scores between 2017 and 2021 (PSI-15. Accidental Puncture or Laceration rate, PSI 06. Iatrogenic Pneumothorax Rate, PSI 12. Perioperative Pulmonary Embolism or Deep Vein Thrombosis Rate). Two additional PSIs had positive score trends until 2020 and 2021 (PSI 13. Postoperative sepsis rate and PSI 11. Postoperative respiratory failure rate).

Centers for Medicare & Medicaid Services. (2024) 2024 National Impact Assessment of the Centers for Medicare & Medicaid Services (CMS) Quality Measures Report. https://www.cms.gov/medicare/quality/measures/national-impact-assessment

6.2.5 Unexpected Findings

Patient and caregivers from Mathematica’s Patient Family Advisory Board (PFAB) were asked to share what they believed were unintended impacts of PSI 90 on patients. PSI 90’s background, purpose, measure definitions, and patient use cases/example scenarios were reviewed prior to requesting feedback from the representatives. There was a total of four patient and caregiver representatives that were asked to provide feedback. No unexpected findings or unintended impacts on patients were identified.

Additionally, we reviewed implementers submission to ServiceNow and did not identify any comments regarding unintended impacts on patients.

Comments

Public Comments

531 Patient Safety Indicator (PSI) 90: Patient Safety and Advers

The American Medical Association (AMA) notes that testing demonstrated that reliability is very poor for the individual measures in this composite and the minimum intraclass correlation coefficient (ICC) was 0.04 for the composite. It is not clear if applying a case minimum would sufficiently improve the reliability of this composite but ask that the committee discuss whether it should be required. In addition, because this measure is based solely on administrative claims, hospitals struggle to be able to drive improvements at the point of care from these data given the delayed distribution of measure scores. As a result, the AMA questions whether this measure is truly useful for accountability and improvement purposes.

Organization

American Medical Association

PSI-90 public comment reply - American Medical Association

Thank you for your feedback.

As safety events, Patient Safety Indicators (PSI) are typically rare events that naturally show lower reliability when measured individually. The PSI 90 measure uses the following design features to create a more reliable indicator of patient safety:

Each PSI component measure is smoothed or reliability-adjusted via Bayesian shrinkage to improve the stability of measure scores.
The composite measure combines information from multiple components (each component providing essential information about patient safety) which amplifies the signal strength and creates a more reliable composite measure.
The composite measure benefits from a much larger effective sample size compared to its individual components. This larger effective sample size reduces noise variance, thereby increasing the overall reliability of PSI 90. By combining the component measures, the PSI 90 composite measure achieves a median Spearman-Brown corrected ICC value of 0.82.

Since the last Consensus-Based Entity (CBE) Endorsement and Maintenance (E&M) review of PSI 90, CMS has implemented the following new minimum sample size requirement for reporting PSI 90:

At least 1 of the 10 components has at least 25 eligible discharges – AND –
At least 3 of the 10 components have at least 3 eligible discharges

The time lag is an inevitable limitation of claims data, as it takes time for claims to be processed by CMS and its contractors, and for the measure scores to be produced and validated. For v14 software, the performance period is July 1, 2021 – June 30, 2023 and the preview period for the measure’s results began in July 2024. Here is a summary of the steps that occur to develop the measure scores, using v14 dates as an example:

After the performance period ends (June 30, 2023), hospitals submit their claims and reconcile/make corrections with the Medicare Administrative Contractors.
CMS extracts the claims data in late September or early October (September 30, 2023) after the end of the performance period.
CMS reviews the data and develops an input data file for the software.
CMS selects the composite measures’ risk factors, calculates additional measure parameters, and updates the software with this information.
CMS computes and validates the hospitals’ scores and updates associated measure documentation and reports. These reports are shared with the hospitals in late July 2024 and early August 2024.

Organization

Mathematica

Staff Preliminary Assessment

CBE #0531 Staff Assessment

Importance

Importance Rating

Not met but addressable

Importance

Strengths:

Limitations:

While the literature review appears thorough and includes substantial recent evidence, the developer did not consistently provide information on the grading of the evidence from systematic reviews or clinical guidelines cited. A few of the component measures have minimal evidence cited (e.g., PSI 09, PSI 14, PSI 15).

Rationale:

The measure is rated as "Not Met, but Addressable" due to incomplete information on grading of systematic review evidence and guidelines for most component measures. It is also not clear from the submission which component measures are currently endorsed. A more thorough presentation of grading of evidence, and potentially more robust evidence for a few components, could elevate its importance.

Feasibility Acceptance

Scientific Acceptability

Scientific Acceptability Reliability Rating

Met

Scientific Acceptability Reliability

Strengths:

Data Sources and Dates: Data used for testing were sourced from claims data from Medicare fee-for-service (FFS) discharges from the universe of Inpatient Prospective Payment System (IPPS) hospitals during the period July 1, 2021 to June 30, 2023. The entities included in the analysis were characterized by 3,156 hospitals, 37.6% small (<100 beds), 33.9% medium (100-250 beds), and 26.7% large (>250 beds). The majority of the hospitals were located in urban geographical areas (75.4%; comprised of 35.7% “large” urban hospitals and 39.7% “other” urban hospitals), with a smaller subset of hospitals being located in rural areas (22.7%).

Limitations:

None.

Rationale:

Accountable Entity-Level Reliability: The developer conducted split-half reliability testing at the accountable entity-level. A PSI composite score was calculated for each split within each hospital and random effects model was fit to calculate the ICC. The Spearman-Brown formula applied. More than 70% of accountable entities meet the expected threshold of 0.6.

Scientific Acceptability Validity Rating

Met

Scientific Acceptability Validity

Strengths:

In general, two main findings support the validity assessment of PSI 90. First, the correlations with the healthcare-associated infection/complication measures, which have a greater construct overlap with the construct defined by PSI 90, are higher than the correlations with the 30-day readmission measures, which have a lesser construct overlap. This suggests the construct defined by PSI 90 is meaningfully distinct and more related to patient safety. Second, the correlations with the composite PSI 90 are higher than the correlations with the component measures, which also suggests the construct defined by PSI 90 is more comprehensive than the constructs of the components. Also supporting the validity assessment is the conceptual coherence of the weighting scheme. The weights for each component reflect the component’s volume (frequency of event) in the reference population and harm (excess risk of harms generated by experiencing the event and perceived severity of those harms). Entities may respond to PSI 90 by focusing resources on those components that are the most frequent and most harmful.
The developer conducted statistical risk adjustment, based on a conceptual model, selecting risk factors that have a significant correlation to the outcome. The developer reported c-statistics of 0.73-0.90, indicating good model discrimination.

Limitations:

Causal claims based on association (correlation) studies are prone to bias (i.e., confounding due to a common cause cannot be ruled-out). Additional support from mechanism studies that confirm the existence of a suitable mechanism capable of accounting for the observed correlation would strengthen the causal claim. Otherwise statements about the relative magnitude of the observed correlations and whether those magnitudes are greater or lesser than what one might anticipate are difficult to evaluate. For example, the developer highlights the strongest correlation between PSI 10 and the surgical site infection rate from colon surgery and abdominal hysterectomy. Even a focused, targeted empirical investigation of a mechanism capable of accounting for the relatively low correlation (r=0.12) and the other contextual mechanisms that explain why the correlation is not higher would be informative. Ultimately, the validity of PSI 90, including the adequacy of the risk adjustment, is a function of the validity and adequacy of the ten (10) component measures. The 2-year period of performance also lessens validity for the purpose of greater reliability.
Developer did not provide assessment or consideration of social risk factors.

Rationale:

The validity testing results support a moderate inference of validity for the measure, confirming that the measure accurately reflects performance on patient safety and can distinguish good from poor performance. The risk adjustment methods used by the component measures are appropriate and demonstrate variation in the prevalence of risk factors across measured entities, contribute to unique variation in the outcome, and show the impact of risk adjustment for providers at high or low extremes of risk. The model performance is acceptable.

Equity

Use and Usability

Use and Usability Rating

Met

Use and Usability

Strengths:

This measure is currently in use in CMS Hospital-Acquired Condition Reduction Program (HACRP), and is also reported on the Hospital Care Compare website.
The developer provides a summary of how accountable entities can use the measure results to improve performance, including the AHRQ toolkit, annual reports, and a range of interventions at the entity and clinician level that may be effective in reducing HACs identified in their evidence review (note, grading of evidence for guidelines and systematic reviews was not presented for most).
The developer reports that feedback is solicited via email and an online tool, and from TEPs, and uses this input alongside updated evidence to revise measure specifications.
The developer reports relative stability in performance from 2013 to 2021, and their rationale is that large changes in a composite measure are generally not expected. They also report some improvement in three of the component PSIs from 2017-2021.
The developer reports no unexpected findings.

Limitations:

While the developer states that feedback is used to update measure specification, specific instances of feedback are not provided.

Rationale:

For maintenance, the measure is actively used in at least one accountability application, with a clear feedback approach that allows for continuous updates based on stakeholder feedback. The developer provides evidence of opportunities for entities to improve performance, through their evidence review, through annual reports to entities, and through the availability of an AHRQ toolkit for performance improvement. Despite no large improvements in performance in the measure performance over time, the developer notes that large changes in performance are not expected in composite measures, and also notes that three component measures did show improvement in recent years. The developer reports no unexpected findings.

Committee Independent Review

Breadcrumb

Patient Safety Indicator (PSI) 90: Patient Safety and Adverse Events Composite

531 Patient Safety Indicator (PSI) 90: Patient Safety and Advers

PSI-90 public comment reply - American Medical Association

CBE #0531 Staff Assessment

support

Summary

support

Support

Approve, reluctantly

Worth a discussion

Support