PSI 90 is a composite of ten adverse event indicators that summarizes hospitals’ performance on patient safety for the CMS Medicare fee-for-service population. The timeframe used in the CMS Hospital Acquired Conditions Reduction Program (HACRP) and CareCompare public reporting are set within the Inpatient Prospective Payment Systems (IPPS) Final Rule annually. Typically, the performance periods use multiple months of claims data.
Measure Specs
- General Information(active tab)
- Numerator
- Denominator
- Exclusions
- Measure Calculation
- Supplemental Attachment
- Point of Contact
General Information
The Patient Safety Indicator (PSI) 90 composite measure captures hospital performance related to the occurrence of 10 preventable adverse events among hospitalized patients. The ten risk-adjusted PSIs that comprise PSI 90 are listed below.
- PSI 03 Pressure Ulcer Rate
- PSI 06 Iatrogenic Pneumothorax Rate
- PSI 08 In-Hospital Fall-Associated Fracture Rate
- PSI 09 Postoperative Hemorrhage or Hematoma Rate
- PSI 10 Postoperative Acute Kidney Injury Requiring Dialysis Rate
- PSI 11 Postoperative Respiratory Failure Rate
- PSI 12 Perioperative Pulmonary Embolism or Deep Vein Thrombosis Rate
- PSI 13 Postoperative Sepsis Rate
- PSI 14 Postoperative Wound Dehiscence Rate
- PSI 15 Abdominopelvic Accidental Puncture or Laceration Rate.
The rationale for reporting this measure is three-fold: (1) to encourage hospitals to implement best practices to minimize adverse events, (2) to help patients understand hospitals’ quality of care and make selection decisions, and (3) to allow payers to track hospital performance. PSI 90 is implemented in CMS’s Hospital Acquired Condition Reduction Program (HACRP). This program evaluates hospitals’ performance on patient safety measures and reduces Medicare payments for hospitals that perform poorly. Given the measure’s results impact payment, hospitals are motivated to implement safe hospital practices that minimize risks of harmful patient outcomes and improve care quality.
PSI 90 results are also publicly posted on https://www.medicare.gov/care-compare/?providerType=Hospital for CMS and Veteran Affair (VA) hospitals. Prospective patients can use these scores to evaluate hospitals’ quality performance as it relates to patient safety and use this information to make informed decisions regarding where they seek care.
CMS calculates PSI 90 using claims data from Medicare fee-for-service (FFS) discharges, using final action claims for inpatient stays paid under the Inpatient Prospective Payment System (IPPS).
Numerator
PSI 03: Discharges, among cases meeting the inclusion and exclusion rules for the denominator, with any secondary ICD-10-CM diagnosis code not present on admission for stage 3 or 4 (or unstageable) pressure ulcer, in the absence of a secondary ICD-10-CM diagnosis code present on admission for deep tissue injury or unstageable pressure injury at the same anatomic site.
PSI 06: Discharges, among cases meeting the inclusion and exclusion rules for the denominator, with a secondary ICD-10-CM diagnosis code for iatrogenic pneumothorax.
PSI 08: Discharges, among cases meeting the inclusion and exclusion rules for the denominator, with any secondary ICD-10-CM diagnosis code for fracture.
PSI 09: Discharges, among cases meeting the inclusion and exclusion rules for the denominator, with any secondary ICD-10-CM diagnosis code for postoperative hemorrhage or hematoma and any listed ICD-10-PCS procedure code for treatment of hemorrhage or hematoma. The ICD-10-CM specification is limited to postoperative hemorrhage or hematoma.
PSI 10: Discharges, among cases meeting the inclusion and exclusion rules for the denominator, with any secondary ICD-10-CM diagnosis code for acute kidney failure and any listed ICD-10-PCS procedure code for dialysis.
PSI 11: Discharges, among cases meeting the inclusion and exclusion rules for the denominator, with either:
- any secondary ICD-10-CM diagnosis code of acute postprocedural respiratory failure
- the last date of an ICD-10-PCS procedure code for a mechanical ventilation for greater than 96 consecutive hours is zero or more days after the first major operating room procedure, if the dates of both procedures are available
- the last date of an ICD-10-PCS procedure code for a mechanical ventilation for 24 - 96 consecutive hours is two or more days after the first major operating room procedure, if the dates of both procedures are available
- the last date of any ICD-10-PCS procedure code for an intubation is one or more days after the first major operating room procedure, if the dates of both procedures are available
PSI 12: Discharges, among cases meeting the inclusion and exclusion rules for the denominator, with a secondary ICD-10-CM diagnosis code for proximal deep vein thrombosis or a secondary ICD-10-CM diagnosis code for pulmonary embolism.
PSI 13: Discharges, among cases meeting the inclusion and exclusion rules for the denominator, with any secondary ICD-10-CM diagnosis code for sepsis.
PSI 14: Discharges, among cases meeting the inclusion and exclusion rules for the denominator, with any listed ICD10-PCS procedure code for repair of abdominal wall and with an ICD-10-CM diagnosis code for disruption of internal surgical wound.
PSI 15: Discharges among cases meeting the inclusion and exclusion rules for the denominator that include all of the following:
- any secondary ICD-10-CM diagnosis code for accidental puncture or laceration of a specific organ or structure during an abdominopelvic procedure; and
- a potentially related procedure for evaluation or treatment performed on an organ or structure related to where the reported accidental puncture or laceration occurred, and performed one to 30 days after the index abdominopelvic procedure; and
- without a principal ICD-10-CM diagnosis code (or secondary diagnosis present on admission) for accidental puncture or laceration of a specific organ or structure that matches the organ or structure of the potentially related subsequent procedure described above
Numerator details are outlined in the individual component's measure specifications, which are attached as Excel files in section 1.13. In each measure's specification attachment, the measure description tab shows the name of code sets included in the numerator, and the following tabs contain the lists of codes within each code set. Other code sets that are referenced as Appendix A-G are included in separate Excel files.
A complete list of the CMS PSI 90 and its component measure specifications can also be found on https://qualitynet.cms.gov/inpatient/measures/psi/resources.
The time periods are based on the particular year and use of the specifications. When CMS uses this measure in their programs, the measure is generally calculated using a two-year period. However, there have been exceptions to this timeframe which CMS has publicly shared.
Denominator
PSI 03: Surgical or medical discharges for patients ages 18 years and older. Surgical and
medical discharges are defined by specific MS-DRG codes.
Exclude discharges:
- with length of stay of less than 3 days
- with a principal ICD-10-CM diagnosis code for site-specific pressure ulcer stage 3 or 4 (or unstageable) or deep tissue injury at the same anatomic site
- with any ICD-10-CM diagnosis code for severe burns (≥20% body surface area)
- with any ICD-10-CM diagnosis code for exfoliative disorders of the skin (≥20% body surface area)
- with a principal ICD-10-CM diagnosis code assigned to MDC 14 Pregnancy, Childbirth & the Puerperium
- with a principal ICD-10-CM diagnosis code assigned to MDC 15 Newborns & Other Neonates with Conditions Originating in Perinatal Period
- with an ungroupable DRG (DRG=999)
- with missing gender (SEX=missing), age (AGE=missing), quarter (DQTR=missing), year (YEAR=missing), or principal diagnosis (DGNS_CD1=missing)
- with missing MDC (MDC=missing) when the user indicates that MDC is provided
PSI 06: Surgical or medical discharges for patients ages 18 years and older. Surgical and medical discharges are defined by specific MS-DRG codes.
Exclude discharges:
- with a principal ICD-10-CM diagnosis (or secondary diagnosis present on admission) of non-traumatic pneumothorax
- with any listed ICD-10-CM diagnosis code for specified chest trauma (rib fractures, traumatic pneumothorax and related chest wall injuries)
- with any listed ICD-10-CM diagnosis code for pleural effusion
- with any listed ICD-10-PCS procedure code for thoracic surgery, including lung or pleural biopsy and diaphragmatic repair
- with any listed ICD-10-PCS procedure code for potentially trans-pleural cardiac procedure
- with a principal ICD-10-CM diagnosis code assigned to MDC 14 Pregnancy, Childbirth & the Puerperium
- with a principal ICD-10-CM diagnosis code assigned to MDC 15 Newborns & Other Neonates with Conditions Originating in Perinatal Period
- with an ungroupable DRG (DRG=999)
- with missing gender (SEX=missing), age (AGE=missing), quarter (DQTR=missing), year (YEAR=missing), or principal diagnosis (DGNS_CD1=missing)
- with missing MDC (MDC=missing) when the user indicates that MDC is provided
PSI 08: Surgical or medical discharges for patients ages 18 years and older. Surgical and medical discharges are defined by specific MS-DRG codes.
Exclude discharges:
- with a principal ICD-10-CM diagnosis code (or secondary diagnosis present on admission) for fracture
- with any listed ICD-10-CM diagnosis code for joint prosthesis-associated fracture
- with a principal ICD-10-CM diagnosis code assigned to MDC 14 Pregnancy, Childbirth & the Puerperium
- with a principal ICD-10-CM diagnosis code assigned to MDC 15 Newborns & Other Neonates with Conditions Originating in Perinatal Period
- with an ungroupable DRG (DRG=999)\
- with missing gender (SEX=missing), age (AGE=missing), quarter (DQTR=missing), year (YEAR=missing), or principal diagnosis (DGNS_CD1=missing)
- with missing MDC (MDC=missing) when the user indicates that MDC is provided
PSI 09: Surgical discharges for patients ages 18 years and older, with any listed ICD-10-PCS procedure code for an operating room procedure. Surgical discharges are defined by specific MS-DRG codes.
Exclude discharges:
- with a principal ICD-10-CM diagnosis code (or secondary diagnosis present on admission) for postoperative hemorrhage or postoperative hematoma
- where the only operating room procedure is for treatment of postoperative hemorrhage or hematoma
- where the treatment of postoperative hemorrhage or hematoma occurs before the first operating room procedure, if the dates of both procedures are available
- with any listed ICD-10-CM diagnosis code for coagulation disorder
- with a principal ICD-10-CM diagnosis code assigned to MDC 14 Pregnancy, Childbirth & the Puerperium
- with a principal ICD-10-CM diagnosis code assigned to MDC 15 Newborns & Other Neonates with Conditions Originating in Perinatal Period
- with an ungroupable DRG (DRG=999)
- with missing gender (SEX=missing), age (AGE=missing), quarter (DQTR=missing), year (YEAR=missing), or principal diagnosis (DGNS_CD1=missing)
- with missing MDC (MDC=missing) when the user indicates that MDC is provided
PSI 10: Elective surgical discharges for patients ages 18 years and older with any listed ICD-10-PCS procedure code for an operating room procedure. Elective surgical discharges are defined by specific MS-DRG codes with admission type recorded as elective (TYPE_ADM=3).
Exclude discharges:
- with a principal ICD-10-CM diagnosis code (or secondary diagnosis present on admission) for acute kidney failure
- with any dialysis procedure that occurs before or on the same day as the first operating room procedure
- with any dialysis access procedure that occurs before or on the same day as the first operating room procedure
- with a principal ICD-10-CM diagnosis code (or secondary diagnosis present on admission) for cardiac arrest
- with a principal ICD-10-CM diagnosis code (or secondary diagnosis present on admission) for severe cardiac dysrhythmia
- with a principal ICD-10-CM diagnosis code (or secondary diagnosis present on admission) for shock
- with a principal ICD-10-CM diagnosis code (or secondary diagnosis present on admission) for chronic kidney disease stage 5 or end stage renal disease
- with a principal ICD-10-CM diagnosis code for urinary tract obstruction
- with any ICD-10-CM diagnosis present on admission of solitary kidney and any ICD-10-PCS procedure code for partial or total nephrectomy
- with a principal ICD-10-CM diagnosis code assigned to MDC 14 Pregnancy, Childbirth & the Puerperium
- with a principal ICD-10-CM diagnosis code assigned to MDC 15 Newborns & Other Neonates with Conditions Originating in Perinatal Period
- with an ungroupable DRG (DRG=999)
- with missing gender (SEX=missing), age (AGE=missing), quarter (DQTR=missing), year (YEAR=missing), or principal diagnosis (DGNS_CD1=missing)
PSI 11: Elective surgical discharges for patients ages 18 years and older with any listed ICD-10-PCS procedure code for an operating room procedure. Elective surgical discharges are defined by specific MS-DRG codes with admission type recorded as elective (TYPE_ADM=3).
Exclude discharges:
- with a principal ICD-10-CM diagnosis code (or secondary diagnosis present on admission) of acute respiratory failure
- with any listed ICD-10-CM diagnosis code present on admission for tracheostomy
- where the only operating room procedure is tracheostomy
- where a procedure for tracheostomy occurs before the first operating room procedure, if the dates of both procedures are available
- with any listed ICD-10-CM diagnosis code for malignant hyperthermia
- with any listed ICD-10-CM diagnosis code present on admission for neuromuscular disorder
- with any listed ICD-10-CM diagnosis code present on admission for degenerative neurological disorder
- with any listed ICD-10-PCS procedure code for laryngeal, pharyngeal, nose, mouth, or facial surgery involving significant risk of airway compromise
- with any listed ICD-10-PCS procedure code for esophageal surgery
- with any listed ICD-10-PCS procedure code for lung cancer
- with any listed ICD-10-PCS procedure code for lung or heart transplant
- with a principal ICD-10-CM diagnosis code assigned to MDC 4 Diseases & Disorders of the Respiratory System
- with a principal ICD-10-CM diagnosis code assigned to MDC 14 Pregnancy, Childbirth & the Puerperium
- with a principal ICD-10-CM diagnosis code assigned to MDC 15 Newborns & Other Neonates with Conditions Originating in Perinatal Period
- with an ungroupable DRG (DRG=999)
- with missing gender (SEX=missing), age (AGE=missing), quarter (DQTR=missing), year (YEAR=missing)
- with missing MDC (MDC=missing) when the user indicates that MDC is provided
PSI 12: Surgical discharges for patients ages 18 years and older with any listed ICD-10- PCS procedure code for an operating room procedure. Surgical discharges are defined by specific MS-DRG codes.
Exclude discharges:
- with a principal ICD-10-CM diagnosis code (or secondary diagnosis present on admission) for proximal deep vein thrombosis
- with a principal ICD-10-CM diagnosis code (or secondary diagnosis present on admission) for pulmonary embolism
- with any listed ICD-10-CM diagnosis code for heparin-induced thrombocytopenia
- where a procedure for interruption of vena cava occurs before or on the same day as the first operating room procedure
- where a procedure for pulmonary arterial or dialysis access thrombectomy occurs before or on the same day as the first operating room procedure
- where the only operating room procedure(s) is/are for interruption of vena cava and/or pulmonary arterial or dialysis access thrombectomy
- with any listed ICD-10-CM diagnosis code present on admission for acute brain or spinal injury
- with any listed ICD-10-PCS procedure code for extracorporeal membrane oxygenation (ECMO)
- with a principal ICD-10-CM diagnosis code assigned to MDC 14 Pregnancy, Childbirth & the Puerperium
- with a principal ICD-10-CM diagnosis code assigned to MDC 15 Newborns & Other Neonates with Conditions Originating in Perinatal Period
- with an ungroupable DRG (DRG=999)
- with missing gender (SEX=missing), age (AGE=missing), quarter (DQTR=missing), year (YEAR=missing), or principal diagnosis (DGNS_CD1=missing)
- with missing MDC (MDC=missing) when the user indicates that MDC is provided
PSI 13: Elective surgical discharges for patients ages 18 years and older with any listed ICD- 10-PCS procedure code for an operating room procedure. Elective surgical discharges are defined by specific MS-DRG codes with admission type recorded as elective (TYPE_ADM=3).
Exclude discharges:
- with a principal ICD-10-CM diagnosis code (or secondary diagnosis present on admission) or sepsis
- with a principal ICD-10-CM diagnosis code (or secondary diagnosis present on admission) for infection
- with a principal ICD-10-CM diagnosis code assigned to MDC 14 Pregnancy, Childbirth & the Puerperium
- with a principal ICD-10-CM diagnosis code assigned to MDC 15 Newborns & Other Neonates with Conditions Originating in Perinatal Period
- with an ungroupable DRG (DRG=999)
- with missing gender (SEX=missing), age (AGE=missing), quarter (DQTR=missing), year (YEAR=missing), or principal diagnosis (DGNS_CD1=missing)
- with missing MDC (MDC=missing) when the user indicates that MDC is provided
PSI 14: Discharges, for patients ages 18 years and older with any listed ICD-10-PCS procedure code for abdominopelvic surgery, open approach, or with any listed ICD-10-PCS procedure code for abdominopelvic surgery, other than open approach.
Exclude discharges:
- the last date of a procedure for abdominal wall reclosure occurs on or before the date of the first open abdominopelvic surgery procedure, if any, and on or before the date of the first abdominopelvic surgery, other than open approach, if any
- with an ICD-10-CM principal or secondary diagnosis code present on admission for disruption of internal operation (surgical) wound
- with length of stay less than two (2) days
- with a principal ICD-10-CM diagnosis code assigned to MDC 14 Pregnancy, Childbirth & the Puerperium
- with a principal ICD-10-CM diagnosis code assigned to MDC 15 Newborns & Other Neonates with Conditions Originating in Perinatal Period
- with an ungroupable DRG (DRG=999)
PSI 15: Surgical or medical discharges, for patients ages 18 years and older, with any ICD-10-PCS procedure code for an abdominopelvic procedure.
Exclude discharges:
- with a missing index abdominopelvic procedure date and/or missing all subsequent abdominopelvic procedure dates
- with a principal ICD-10-CM diagnosis code assigned to MDC 14 Pregnancy, Childbirth & the Puerperium
- with a principal ICD-10-CM diagnosis code assigned to MDC 15 Newborns & Other Neonates with Conditions Originating in Perinatal Period
- with an ungroupable DRG (DRG=999)
- with missing gender (SEX=missing), age (AGE=missing), quarter (DQTR=missing), year (YEAR=missing), or principal diagnosis (DGNS_CD1=missing)
- with missing MDC (MDC=missing) when the user indicates that MDC is provided
Denominator details are outlined in the individual component's measure specifications, which are attached as Excel files in section 1.13. In each measure's specification attachment, the measure description tab shows the name of code sets included in the denominator, and the following tabs contain the lists of codes within each code set. Other code sets that are referenced as Appendix A-G are included in separate Excel files.
A complete list of the CMS PSI 90 and its component measure specifications can also be found on https://qualitynet.cms.gov/inpatient/measures/psi/resources.
The time periods are based on the particular year and use of the specifications. When CMS uses this measure in their programs, the measure is generally calculated using a two-year period. However, there have been exceptions to this timeframe which CMS has publicly shared.
Exclusions
PSI 03:
- Length of stay of less than 3 days
- Principal ICD-10-CM diagnosis code for site-specific pressure ulcer stage 3 or 4 (or unstageable) or deep tissue injury at the same anatomic site
- Any ICD-10-CM diagnosis code for severe burns (≥20% body surface area)
- Any ICD-10-CM diagnosis code for exfoliative disorders of the skin (≥20% body surface area)
- Principal ICD-10-CM diagnosis code assigned to MDC 14 Pregnancy, Childbirth & the Puerperium
- Principal ICD-10-CM diagnosis code assigned to MDC 15 Newborns & Other Neonates with conditions originating in perinatal period
- Ungroupable DRG (DRG=999)
- Missing gender (SEX=missing), age (AGE=missing), quarter (DQTR=missing), year (YEAR=missing), principal diagnosis (DGNS_CD1=missing), or MDC (MDC=missing) when the user indicates that MDC is provided
PSI 06:
- Principal ICD-10-CM diagnosis (or secondary diagnosis present on admission) of non-traumatic pneumothorax
- Any listed ICD-10-CM diagnosis code for specified chest trauma (rib fractures, traumatic pneumothorax and related chest wall injuries)
- Any listed ICD-10-CM diagnosis code for pleural effusion
- Any listed ICD-10-PCS procedure code for thoracic surgery, including lung or pleural biopsy and diaphragmatic repair
- Any listed ICD-10-PCS procedure code for potentially trans-pleural cardiac procedure
- Principal ICD-10-CM diagnosis code assigned to MDC 14 Pregnancy, Childbirth & the Puerperium
- Principal ICD-10-CM diagnosis code assigned to MDC 15 Newborns & Other Neonates with Conditions Originating in Perinatal Period
- Ungroupable DRG (DRG=999)
- Missing gender (SEX=missing), age (AGE=missing), quarter (DQTR=missing), year (YEAR=missing), principal diagnosis (DGNS_CD1=missing), or MDC (MDC=missing) when the user indicates that MDC is provided
PSI 08
- Principal ICD-10-CM diagnosis code (or secondary diagnosis present on admission) for fracture
- Any listed ICD-10-CM diagnosis code for joint prosthesis-associated fracture
- Principal ICD-10-CM diagnosis code assigned to MDC 14 Pregnancy, Childbirth & the Puerperium
- Principal ICD-10-CM diagnosis code assigned to MDC 15 Newborns & Other Neonates with Conditions Originating in Perinatal Period
- Ungroupable DRG (DRG=999)
- Missing gender (SEX=missing), age (AGE=missing), quarter (DQTR=missing), year (YEAR=missing), principal diagnosis (DGNS_CD1=missing), or MDC (MDC=missing) when the user indicates that MDC is provided
PSI 09
- Principal ICD-10-CM diagnosis code (or secondary diagnosis present on admission) for postoperative hemorrhage or postoperative hematoma
- The only operating room procedure is for treatment of postoperative hemorrhage or hematoma
- Treatment of postoperative hemorrhage or hematoma occurs before the first operating room procedure, if the dates of both procedures are available
- Any listed ICD-10-CM diagnosis code for coagulation disorder
- Principal ICD-10-CM diagnosis code assigned to MDC 14 Pregnancy, Childbirth & the Puerperium
- Principal ICD-10-CM diagnosis code assigned to MDC 15 Newborns & Other Neonates with Conditions Originating in Perinatal Period
- Ungroupable DRG (DRG=999)
- Missing gender (SEX=missing), age (AGE=missing), quarter (DQTR=missing), year (YEAR=missing), principal diagnosis (DGNS_CD1=missing), or MDC (MDC=missing) when the user indicates that MDC is provided
PSI 10
- Principal ICD-10-CM diagnosis code (or secondary diagnosis present on admission) for acute kidney failure
- Dialysis procedure that occurs before or on the same day as the first operating room procedure
- Dialysis access procedure that occurs before or on the same day as the first operating
- room procedure
- Principal ICD-10-CM diagnosis code (or secondary diagnosis present on admission) for cardiac arrest, severe cardiac dysrhythmia, shock, chronic kidney disease stage 5, end stage renal disease, or urinary tract obstruction
- Any ICD-10-CM diagnosis present on admission of solitary kidney and any ICD-10-PCS procedure code for partial or total nephrectomy
- Principal ICD-10-CM diagnosis code assigned to MDC 14 Pregnancy, Childbirth & the Puerperium
- Principal ICD-10-CM diagnosis code assigned to MDC 15 Newborns & Other Neonates with Conditions Originating in Perinatal Period
- Ungroupable DRG (DRG=999)
- Missing gender (SEX=missing), age (AGE=missing), quarter (DQTR=missing), year (YEAR=missing), or principal diagnosis (DGNS_CD1=missing)
PSI 11
- Principal ICD-10-CM diagnosis code (or secondary diagnosis present on admission) of acute respiratory failure (ACURF3D*)
- Any listed ICD-10-CM diagnosis code present on admission for tracheostomy (TRACHID*)
- The only operating room procedure is tracheostomy (TRACHIP*)
- Procedure for tracheostomy (TRACHIP*) occurs before the first operating room procedure (Appendix A: ORPROC), if the dates of both procedures are available
- Any listed ICD-10-CM diagnosis code for malignant hyperthermia (MALHYPD*)
- Any listed ICD-10-CM diagnosis code present on admission for neuromuscular disorder (NEUROMD*)
- Any listed ICD-10-CM diagnosis code present on admission for degenerative neurological disorder (DGNEUID*)
- Any listed ICD-10-PCS procedure code for laryngeal, pharyngeal, nose, mouth, or facial surgery involving significant risk of airway compromise (NUCRANP*)
- Any listed ICD-10-PCS procedure code for esophageal surgery (PRESOPP*)
- Any listed ICD-10-PCS procedure code for lung cancer (LUNGCIP*)
- Any listed ICD-10-PCS procedure code for lung or heart transplant (LUNGTRANSP*)
- Principal ICD-10-CM diagnosis code assigned to MDC 4 Diseases & Disorders of the Respiratory System
- Principal ICD-10-CM diagnosis code assigned to MDC 14 Pregnancy, Childbirth & the Puerperium
- Principal ICD-10-CM diagnosis code assigned to MDC 15 Newborns & Other Neonates with Conditions Originating in Perinatal Period
- Ungroupable DRG (DRG=999)
- Missing gender (SEX=missing), age (AGE=missing), quarter (DQTR=missing), year (YEAR=missing), or MDC (MDC=missing) when the user indicates that MDC is provided
PSI 12
- Principal ICD-10-CM diagnosis code (or secondary diagnosis present on admission) for proximal deep vein thrombosis (DEEPVIB*)
- Principal ICD-10-CM diagnosis code (or secondary diagnosis present on admission) for pulmonary embolism (PULMOID*)
- Any listed ICD-10-CM diagnosis code for heparin-induced thrombocytopenia (HITD*)
- Procedure for interruption of vena cava (VENACIP*) occurs before or on the same day as the first operating room procedure
- Procedure for pulmonary arterial or dialysis access thrombectomy (THROMP*) occurs before or on the same day as the first operating room procedure
- Operating room procedure(s) is/are for interruption of vena cava (VENACIP*) and/or pulmonary arterial or dialysis access thrombectomy (THROMP*)
- Any listed ICD-10-CM diagnosis code present on admission for acute brain or spinal injury
- (NEURTRAD*)
- Any listed ICD-10-PCS procedure code for extracorporeal membrane oxygenation (ECMO) (ECMOP*)
- Principal ICD-10-CM diagnosis code assigned to MDC 14 Pregnancy, Childbirth & the Puerperium
- Principal ICD-10-CM diagnosis code assigned to MDC 15 Newborns & Other Neonates with Conditions Originating in Perinatal Period
- Ungroupable DRG (DRG=999)
- Missing gender (SEX=missing), age (AGE=missing), quarter (DQTR=missing), year (YEAR=missing), principal diagnosis (DGNS_CD1=missing), or MDC (MDC=missing) when the user indicates that MDC is provided
PSI 13
- Principal ICD-10-CM diagnosis code (or secondary diagnosis present on admission) for sepsis (SEPTI2D*)
- Principal ICD-10-CM diagnosis code (or secondary diagnosis present on admission) for infection
- Principal ICD-10-CM diagnosis code assigned to MDC 14 Pregnancy, Childbirth & the Puerperium
- Principal ICD-10-CM diagnosis code assigned to MDC 15 Newborns & Other Neonates with Conditions Originating in Perinatal Period
- Ungroupable DRG (DRG=999)
- Missing gender (SEX=missing), age (AGE=missing), quarter (DQTR=missing), year (YEAR=missing), principal diagnosis (DGNS_CD1=missing), or MDC (MDC=missing) when the user indicates that MDC is provided
PSI 14
- Last date of a procedure for abdominal wall reclosure (RECLOIP*) occurs on or before the date of the first open abdominopelvic surgery procedure (ABDOMIPOPEN*), if any, and on or before the date of the first abdominopelvic surgery, other than open approach (ABDOMIPOTHER*), if any
- ICD-10-CM principal or secondary diagnosis code present on admission for disruption of internal operation (surgical) wound (ABWALLCD*)
- Length of stay less than two (2) days
- Principal ICD-10-CM diagnosis code assigned to MDC 14 Pregnancy, Childbirth & the Puerperium
- with a principal ICD-10-CM diagnosis code assigned to MDC 15 Newborns & Other Neonates with Conditions Originating in Perinatal Period
- Ungroupable DRG (DRG=999)
PSI 15
- Missing index abdominopelvic procedure date and/or missing all subsequent abdominopelvic procedure dates
- Principal ICD-10-CM diagnosis code assigned to MDC 14 Pregnancy, Childbirth & the Puerperium
- Principal ICD-10-CM diagnosis code assigned to MDC 15 Newborns & Other Neonates with Conditions Originating in Perinatal Period
- Ungroupable DRG (DRG=999)
- with missing gender (SEX=missing), age (AGE=missing), quarter (DQTR=missing), year (YEAR=missing), principal diagnosis (DGNS_CD1=missing, or MDC (MDC=missing) when the user indicates that MDC is provided
Denominator exclusion details are outlined in the individual component's measure specifications, which are attached as Excel files in section 1.13. In each measure's specification attachment, the measure description tab shows the name of code sets included in the denominator exclusion, and the following tabs contain the lists of codes within each code set. Other code sets that are referenced as Appendix A-G are included in separate Excel files.
A complete list of the CMS PSI 90 and its component measure specifications can also be found on https://qualitynet.cms.gov/inpatient/measures/psi/resources.
The time periods are based on the particular year and use of the specifications. When CMS uses this measure in their programs, the measure is generally calculated using a two-year period. However, there have been exceptions to this timeframe which CMS has publicly shared.
Measure Calculation
- For each PSI component, calculate the observed rate as the number of discharges where the patient experienced the component event (numerator) divided by the number of discharges where the patient is at risk or “eligible” for the component event (denominator) per hospital.
- For each PSI component, use the component’s risk adjustment model (built using an external reference population and available in the CMS PSI software) to estimate the predicted probabilities of patients from each denominator-eligible discharge to experience the component event. Sum up these discharge-level predicted probabilities per hospital and divide by the denominator count per hospital to obtain the hospital’s expected rate. The expected rate reflects the rate of adverse events that a hospital would anticipate based on the hospital’s case-mix.
- For each PSI component, calculate the risk-adjusted rate as the observed rate divided by the expected rate per hospital, multiplied by the reference population rate (available in the CMS PSI Software). The risk-adjusted rate reflects the performance of a hospital treating its patients relative to the hypothetical average hospital treating patients with the same characteristics.
- For each PSI component, calculate the smoothed rate as the weighted average of the risk-adjusted rate and the reference population rate per hospital, where the weight is the hospital’s signal-to-noise reliability ratio. Smoothing adjusts risk-adjusted rates according to each hospital’s level of measure reliability, such that for hospitals with unstable estimates based on noisy data (e.g., due to small sample sizes), smoothing pulls the hospital’s risk-adjusted rate closer to the reference population rate.
- Calculate the PSI 90 composite score as the weighted average of the ten PSI components’ smoothed observed-to-expected ratios per hospital, which are the smoothed rates calculated in step 4 divided by the reference population rate. The weights for each component reflect the component’s volume (frequency of event) in the reference population and harm (excess risk of harms generated by experiencing the event and perceived severity of those harms). Information on harm and volume weights are included in Table 1 of the 7.1 Supplemental Attachment.
The measure is not stratified.
PSI 90 scores are calculated only if hospitals have (1) at least 25 eligible discharges on at least 1 PSI component and (2) at least 3 eligible discharges on at least 7 PSI components.
Supplemental Attachment
Point of Contact
Not applicable
Donta Henson
7500 Security Blvd
Woodlawn, MD
United States
Brittany Colip
Mathematica
600 Alexander Park Ste 100
Princeton, NJ 08540
United States
Importance
Evidence
See 2.1 – 2.2 Evidence of Measure Importance Attachment
Measure Impact
Patient and caregiver representatives from Mathematica’s internal Patient Family Advisory Board (PFAB) were asked to if they felt “PSI 90 is meaningful and produces information that is valuable in making care decision”. PSI 90’s background, purpose, measure definitions, and patient use cases/example scenarios were reviewed prior to polling the representatives. Opportunities to ask questions regarding PSI 90 were also offered to the representative. There was a total of four patient and caregiver representatives that opted to respond to the polling question. The representatives unanimously agreed that “PSI 90 is meaningful and produces information that is valuable in making care decision”.
Performance Gap
We assessed for performance gaps in the PSI 90 composite score using claims data from Medicare fee-for-service (FFS) discharges from the universe of Inpatient Prospective Payment System (IPPS) hospitals from July 1, 2021 to June 30, 2023. We used these same data in our testing activities which we describe in the scientific acceptability sections of this form. These data included a total of 3,156 hospitals and 13,702,080 discharges. Out of the 3,156 hospitals, 2,922 hospitals had data that meet the minimum sample size, whose distribution of PSI 90 scores is described in Table 1 (embedded in this submission form) below. The row “N of Persons / Encounters / Episodes” contains the number of unique denominator-eligible discharges across the 10 PSI components. Measure scores were calculated using the CMS PSI software version 14. Although many hospitals’ performance meet or exceed the national average (median PSI 90 score = 0.964), some hospitals fall behind, suggesting there is room for improvement. It is also crucial to continue monitoring all hospitals to ensure patient safety remains a priority.
Overall | Minimum | Decile_1 | Decile_2 | Decile_3 | Decile_4 | Decile_5 | Decile_6 | Decile_7 | Decile_8 | Decile_9 | Decile_10 | Maximum | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Mean Performance Score | 0.999 | 0.534 | 0.758 | 0.856 | 0.897 | 0.929 | 0.953 | 0.976 | 1.008 | 1.064 | 1.148 | 1.396 | 3.098 |
N of Entities | 2,922 | 1 | 292 | 293 | 291 | 292 | 293 | 293 | 291 | 292 | 293 | 292 | 1 |
N of Persons / Encounters / Episodes | 13,638,662 | 8,781 | 2,300,343 | 1,306,200 | 1,208,054 | 978,132 | 782,805 | 723,791 | 1,357,922 | 1,495,171 | 1,578,020 | 1,908,224 | 12,406 |
Equity
Equity
PSI 90 is a composite measure focusing on patient safety related outcomes. To understand disparities in patient safety, we share the PSI 90 disparity analysis from the Agency for Healthcare Research and Quality's (AHRQ) 2023 National Healthcare Quality and Disparities Report (NHQDR). Specifically, information contained in Appendix B (available at https://www.ahrq.gov/sites/default/files/wysiwyg/research/findings/nhqrdr/2023-qdr-appendixb-measure-category-5.pdf) addresses this issue. We examined disparities by gender, race/ethnicity, income, and metropolitan status.
Given we are utilizing AHRQ’s report for this analysis, we first share the bulleted information about the analysis prior to sharing the results.
- Component measures. The subgroup results below are reported at the individual component measure level. We report component measure information in this section because PSI 90 is not calculated at the patient or subgroup levels. Additionally, the composite scores are smoothed to the national average, which is not calculated at the subgroup level.
- Adjusted for gender. All 10 component measures are risk-adjusted for gender (as well as a range of other demographic and clinical risk factors; see section 4.4.4 for more details). Therefore, the component measure rates used to calculate gender disparities in the AHRQ report used a modified risk-adjustment model that excluded gender as a risk factor.
- Use of all payer data. The data (Healthcare Cost and Utilization Project [HCUP]) used within the AHRQ report captures all payers, which is appropriate for the disparity analysis to more broadly capture relevant patient subgroups. The rest of this submission form focuses on the Medicare Fee For Service (FFS) population given this is the population used to calculate hospitals’ pay for performance results in CMS programs.
- AHRQ software. The disparity analysis used the AHRQ version of the PSI software, whereas the rest of the submission form uses the CMS version. The two software sets are substantively aligned, however the AHRQ version is structured to run on the HCUP data, and the CMS version is structured to run on Medicare FFS data.
- Software version. The disparity analysis reports results using v13 of the software, whereas the rest of the submission form uses v14. While there are substantial differences between the software, the general results for patient safety should be aligned across v13 and v14.
The disparity results bulleted below indicate that care disparities exist for all but two (PSI 06 and PSI 08) component measures. This means that there are statistically significant differences (with a relative difference of at least 10%) across levels of at least one characteristic (gender, race/ethnicity, income quartiles, metropolitan status). However, the nature of disparities varied from component to component.
- Gender. The analysis reports better component measure rates for females compared to males for PSI 03, PSI 09, PSI 13, and PSI 14, but worse measure rates for females for PSI 15.
- Race. The analysis reports worse component measure rates for black patients compared to white patients for PSI 03, PSI 09, PSI 10, PSI 12, PSI 13, and PSI 15. Similarly, Asian and Pacific Islander (API) patients had worse component measure rates compared to white patients for PSI 09, PSI 11, and PSI 13, but API patients had better measure rates than white patients for PSI 03 and PSI 12.
- Ethnicity. The analysis reports better component measure rates for non-Hispanic white patients compared to Hispanic (all races) patients for PSI 13 and PSI 14, but Hispanic (all races) patients had better measure rates for PSI 09.
- Income. The analysis reports worse component measure rates for patients in lower quartiles of income compared to those in high quartiles for PSI 03, PSI 10, PSI 11, PSI 13, and PSI 14.
- Metropolitan status. The analysis reports worse component measure rates for less populated areas (e.g., small and medium metropolitan, micropolitan, noncore) compared to more populated areas (e.g., large fringe metro, large central metro) for some measures, such as PSI 03. However, other components, such as PSI 12, had the opposite pattern.
Even though there is no one particular subgroup with consistently poor component measure rates reported across all 10 PSI components, we will continue to monitor disparities particularly for black patients and patients in lower quartiles of income. We will focus on these groups given that among PSI components with significant disparities based on race and income, the directionality of the disparity was consistently worse for these subgroups.
Feasibility
Feasibility
This measure has been in the Centers for Medicare & Medicaid Services' (CMS) quality programs for nine years, and reporters have encountered minimal barriers when implementing the measure specifications, data abstraction, measure calculation, or performance reporting.
Administrative burden and cost associated with the PSI 90 data collection and entry is low. Hospitals generate the required data during routine patient care delivery processes and submit this information within their Medicare claim. Burden with generating a hospital’s score is low as it can be calculated with minimal resources using the publicly available CMS PSI software. Additionally, PSI 90 scores are calculated using only final claims data; thus missing data and the measure's susceptibility to inaccuracies are minimized. Patient confidentiality is also not a concern as PSI 90 scores are reported in aggregate at the hospital level.
Patient and caregivers from Mathematica’s Patient Family Advisory Board (PFAB) were asked to share what they believed were unintended impacts/consequences of PSI 90 on patients. PSI 90’s background, purpose, measure definitions, and patient use cases/example scenarios were reviewed prior to requesting feedback from the representatives. There was a total of four patient and caregiver representatives that provided feedback. No unintended consequence related to feasibility of implementing PSI 90 were expressed during the PFAB meeting.
No feasibility assessment was conducted as this is an established measure in CMS quality programs and final (Medicare) claims data generated during patient care delivery are used for measure score calculation. Therefore, no adjustments were made to the measure based on feasibility.
Proprietary Information
Scientific Acceptability
Testing Data
Testing analyses were conducted using claims data from Medicare fee-for-service (FFS) discharges from the universe of Inpatient Prospective Payment System (IPPS) hospitals from July 1, 2021 to June 30, 2023. Only discharge records from inpatient final action claims were included. Records were excluded if the total length of stay exceeded 365 days, if the patient was enrolled in Medicare Advantage at time of discharge, or if key data elements required to determine inclusion were missing or inconsistent. Measure scores were calculated using the CMS PSI software version 14.
To obtain hospital-level information (e.g., hospital characteristics), we supplemented the discharge-level claims data with the publicly available CMS FY 2025 IPPS Proposed Rule Impact File.
All testing results reported in this section are based on the data described in section 4.1.1 above, with the exception of the risk adjustment model coefficients reported in section 4.4.1a.
The risk adjustment models implemented in the version 14 CMS PSI software were fitted using claims data from Medicare FFS discharges from IPPS hospitals from July 1, 2020 to June 30, 2022 (software reference population). Therefore, the coefficient estimates provided in section 4.4.1a are based on this time period of data.
The testing data included 3,156 hospitals (universe of IPPS hospitals), comprised of small (<100 beds; 37.6%), medium (100-250 beds; 33.9%), and large (>250 beds; 26.7%) hospitals. The majority of the hospitals were located in urban geographical areas (75.4%; comprised of 35.7% “large” urban hospitals and 39.7% “other” urban hospitals), with a smaller subset of hospitals being located in rural areas (22.7%). Nearly all hospitals (98.5%) were acute care hospitals, with the remaining hospitals having missing hospital types.
See Table 2 in the 7.1 Supplemental Attachment for a table summarizing the distribution of hospital characteristics.
The testing data included 13,702,080 unique discharges that were denominator-eligible for at least one PSI component. The mean patient age among these discharges was 74.2 years (SD = 12.4), with a 53% female composition. 82% of these discharges were associated with white patients, 11% black patients, 2.2% Hispanic patients, 1.7% Asian patients, 2.5% North American Native patients, and 1.4% categorized as “other”.
Table 3 in the 7.1 Supplemental Attachment, additionally describes the demographic composition of the denominator-eligible discharges for each PSI component separately, as well as for the set of all 13,702,080 discharges.
Reliability
Component measure reliability:
We used a signal-to-noise approach to assess the precision of PSI component rates in distinguishing between hospitals according to their care quality. Signal-to-noise reliability assesses the extent to which variation in measure scores are attributed to true differences in quality across hospitals (signal) as opposed to error, or differences within hospitals (noise). Risk-adjusted PSI component rates were used in this analysis, as the signal-to-noise approach is not appropriate for smoothed rates, which are already corrected for each hospital’s signal-to-noise reliability. All references to PSI component rates in this section below assume risk-adjusted rates.
For each PSI component c and hospital h, we estimated a signal-to-noise reliability as:
reliability = signal variance / (signal variance + noise variance). The reliability estimate can range from 0 to 1, with 0 indicating no reliability where all variation in component rates is due to noise, and 1 indicating perfect reliability where all variation in component rates is due to between-hospital differences. Noise variance (one estimate per component per hospital) was estimated using the sampling variance of each hospital’s PSI component rate. Signal variance (one estimate per component) was computed iteratively using the empirical Bayes method (Morris, 1983) to estimate the between-hospital variance in each PSI component rate.
See Section 4.2.2 of the 4.2.3a Reliability Testing attachment for a full description of the computation, including formulas.
Morris, C. N. (1983). Parametric empirical Bayes inference: theory and applications. Journal of the American statistical Association, 78(381), 47-55.
Composite measure reliability:
The signal-to-noise approach is not appropriate for assessing the composite reliability, because the PSI 90 composite is constructed from smoothed PSI component rates, which already account for each hospital’s signal-to-noise reliability (see section 1.18, step #4 for an explanation of the smoothing process). Therefore, we used a split-half reliability approach and calculated an intraclass correlation coefficient (ICC) for each hospital using an estimated between-hospital variance component, error variance component, and the number of unique denominator-eligible discharges across all PSI components for each hospital. Similar to signal-to-noise, the ICC quantifies the amount of variation in measure scores due to between-hospital differences rather than within.
We obtained estimates of the between-hospital and error variance components from a simple, intercept-only random-effects model with no predictors. This random-effects model was fit to data made multilevel by creating random split-half samples, without replacement, of discharges (denominator-eligible for at least one PSI component) within each hospital and thereby producing two PSI 90 composite scores per hospital. ICCs range from 0 to 1, where 0 indicates no agreement or reliability, and 1 indicates perfect agreement or reliability. ICCs were additionally corrected using the Spearman-Brown formula to account for the fact that composite scores used in the random-effects model were computed using only half of the discharges from each hospital.
See Section 4.2.2 of the 4.2.3a Reliability Testing attachment for a full description of the method, including formulas. Note that these methods for assessing the component and composite measure reliability are consistent with methods used in the prior CBE maintenance review.
Component measure reliability: See Table 4 of the 4.2.3a Reliability Testing attachment for the distribution of signal-to-noise reliability estimates across hospitals for each PSI component. Note that component reliability results are only reported for hospitals with at least three denominator-eligible discharges for that component.
Composite measure reliability: See Table 5 of the 4.2.3a Reliability Testing attachment for the distribution of ICC estimates across hospitals for PSI 90. Note that composite reliability results are only reported for hospitals with a non-missing PSI 90 composite score (i.e., hospitals that met the minimum sample size requirements described in section 1.26; N = 2,922).
Table 2 (embedded in this submission form) below contains the distribution of ICC estimates across hospitals for PSI 90, where hospitals are grouped into deciles based on N, or the number of unique denominator-eligible discharges across all PSI components. For example, the “reliability” row of this table contains the mean ICC estimate among hospitals in the first decile of N, second decile of N, and so on. Similarly, the “mean performance score” row of this table contains the mean PSI 90 score among hospitals in the first decile of N, and so on. The row “N of Persons / Encounters / Episodes” of this table contains the sum of N among hospitals in each decile group. The minimum and maximum columns of this table contain the mean reliability, mean ICC, etc. of the hospital(s) with the minimum and maximum N, respectively.
The component reliability results show that signal-to-noise reliability estimates vary across the PSI components, although across the board, we observe that component reliability is low. This is aligned with testing results from the prior CBE maintenance review, which also found low reliability at the component-level. We observed the highest reliability in PSI 03, with a median estimate of 0.57 and 47% of hospitals having reliability estimates greater than or equal to the PQM threshold of 0.6. We observed the lowest reliability in PSI 14, with a median estimate of 0.04 and 0.03% of hospitals having reliability estimates greater than or equal to 0.6.
The composite reliability results, on the other hand, show that PSI 90 as a composite has acceptable reliability. This is also consistent with testing results from the prior CBE maintenance review. The median ICC was 0.82, with 78% of hospitals having ICCs greater than or equal to 0.6. This demonstrates the importance of the construction of PSI 90 as a composite measure that borrows strength from the collection of PSI components.
| Overall | Minimum | Decile_1 | Decile_2 | Decile_3 | Decile_4 | Decile_5 | Decile_6 | Decile_7 | Decile_8 | Decile_9 | Decile_10 | Maximum |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Reliability | 0.747 | 0.040 | 0.303 | 0.519 | 0.636 | 0.724 | 0.790 | 0.836 | 0.874 | 0.902 | 0.928 | 0.955 | 0.989 |
Mean Performance Score | 0.999 | 0.993 | 0.979 | 0.976 | 0.988 | 0.979 | 0.987 | 1.033 | 1.014 | 0.997 | 1.016 | 1.016 | 1.053 |
N of Entities | 2,992 | 2 | 293 | 292 | 293 | 292 | 292 | 292 | 292 | 292 | 292 | 292 | 1 |
N of Persons / Encounters / Episodes | 13,638,662 | 58 | 94,969 | 221,590 | 360,087 | 538,059 | 769,468 | 1,041,782 | 1,417,480 | 1,876,743 | 2,640,780 | 4,677,704 | 59,941 |
Validity
We assessed the construct validity of the PSI components and PSI 90 composite to investigate the extent to which these measures assess the construct of patient safety that they are intended to measure. Construct validity can be demonstrated using multiple forms of empirical evidence, such as correlations, predictive relationships, and group differences, that support a priori hypotheses about the construct.
We conducted correlational analyses to demonstrate evidence of validity. Specifically, we calculated Spearman’s correlations between risk-adjusted PSI component rates and other relevant but independently collected hospital-level measures of patient safety or harm events. We also calculated correlations between the PSI 90 composite scores and these same criterion measures. We used risk-adjusted rather than smoothed rates in the component validity analysis to maximize variability in component rates. The criterion measures were obtained from the public Hospital Care Compare data (available at https://data.cms.gov/provider-data/topics/hospitals) and included healthcare-associated infection rates (central line-associated bloodstream infections, catheter-associated urinary tract infections, surgical site infection from colon surgery, surgical site infection from abdominal hysterectomy, Methicillin-resistant Staphylococcus aureus bloodstream infections, and clostridium difficile infections), complication rate for hip/knee replacement patients, and 30-day readmission rates (for acute myocardial infarction patients, coronary artery bypass graft patients, chronic obstructive pulmonary disease patients, heart failure patients, after hip/knee surgery, pneumonia patients, and after discharge from hospital).
While we expect positive correlations between the PSI components and PSI 90 and these criterion measures given that they all capture patient safety and adverse events, we anticipated weak positive correlations across the board. PSI 90 is a complex, formative composite measure that accounts for various aspects of patient safety and therefore may not correlate strongly with any one criterion measure. We also expected the strength of correlations to differ according to the closeness in content to the PSI components in the specific aspect of patient safety assessed by the criterion measure.
See Table 6 of the 4.3.4a Validity Testing attachment for a correlation matrix of the PSI measures (components and the composite) and the criterion measures.
The validity analysis shows that as expected, PSI 90 is positively, although weakly, correlated with the criterion measures. This is also consistent with testing results from the prior CBE maintenance review, which also found weak, positive correlations between PSI 90 and these criterion measures. PSI 90 had the strongest correlation with the hip/knee replacement complication rate, Methicillin-resistant Staphylococcus aureus (MRSA) bloodstream infection rate, and clostridium difficile (C. diff) infection rate (all correlated at r = 0.12, p<.001). Overall, we observed stronger correlations between PSI 90 and the infection rates than between PSI 90 and the readmission rates. This is expected, given that infection rates are more directly reflective of in-hospital safety events, as is PSI 90, whereas readmission rates can further depend on events occurring outside of the hospital after a patient has been discharged.
Among the PSI component correlations, we observed the strongest correlation (r = 0.12) between PSI 10 and the surgical site infection rate from colon surgery and between PSI 10 and the surgical site infection rate from abdominal hysterectomy. Similar to PSI 90, the PSI components generally had stronger and more positive correlations with the infection rates than the readmission rates. We also note that correlations with the criterion measures were generally stronger among PSI 90 rather than the PSI components.
Overall, the analysis provides support for the construct validity for the PSI 90. The weak, positive correlations observed among PSI 90 and the criterion measures demonstrate that a broad, common construct of patient safety underlies these measures, but PSI 90 does not entirely overlap with any one measure of patient safety. Given that we observed stronger correlations between PSI 90 and the criterion measures than between the PSI components and the criterion measures, we emphasize the importance of the construction of PSI 90 as a composite.
Risk Adjustment
PSI components are risk-adjusted because some patients have higher risks of experiencing in-hospital adverse events, regardless of the hospital's quality of care (e.g., due to patients' preexisting conditions). Each PSI component is risk-adjusted (via separate risk adjustment models) in order to account for these differences in patient case-mix across hospitals. The risk adjustment models are used to generate component event rates that reflect what would be observed in a hospital if its observed level of care were applied to a case-mix of patients similar to the reference population, allowing for more standardized comparisons in event rates across hospitals.
The conceptual model for risk adjustment considers patients’ key demographic characteristics, severity of illness, clinical/comorbidities, and discharge-specific information (e.g., transfer-in as an indicator of recent health service use at a different facility as risk factors). The key criterion for risk factors include:
- Observability and validity using administrative data across hospitals
- Clinical relevance to the adverse event (i.e., outcome)
- Adequate prevalence to obtain reasonably precise estimates of risk
- Systematic variation by hospital
Risk adjustment models are developed annually as part of the measure production process. The approach to risk adjustment is to consider as many risk factors as theoretically justified and computationally practical, and to apply a feature selection technique to select an optimal set of risk factors among the candidate variables before fitting a final risk adjustment model for each PSI component (see section 4.4.4 for more details).
See Table 7 in 7.1 Supplemental Attachment for distributions of risk factor characteristics across hospitals. For example, for the risk factor Age_30_34 (indicator for whether a patient is 30-34 years old), the mean would indicate the average (across hospitals) percentage composition of discharges where the patient is 30-34 years old.
Table 7 additionally indicates whether each risk factor is included in the final risk adjustment model for each PSI component for version 14 of the specifications.
There is a risk adjustment model for each PSI component. Each risk adjustment model begins with a candidate list of risk factors, including sex, age category, age-sex interactions, Medicare Severity-Diagnosis Related Groups (MS-DRGs) aggregated across levels of comorbidities or complications, Major Diagnostic Categories (MDCs), 38 Elixhauser comorbidity variables, categories of unweighted counts of comorbidities, whether the patient was transferred from another acute care hospital, COVID-19 diagnosis, COVID-19-quarter, and COVID-19 diagnosis-quarter interactions. Several PSI components had additional candidate risk factors specific to those components – for example, PSI 03, 06, 08, and 15 additionally considered a flag indicating medical (as opposed to surgical) discharges.
From the candidate list of risk factors, variables were selected for inclusion in the final risk adjustment models using least absolute shrinkage and selection operator (LASSO) regression models. Using the selected risk factors, final risk adjustment models were fitted using generalized estimating equations (GEE) to account for nesting of patients within hospitals.
See attachment in 4.4.4a for the resulting list of risk factors selected for inclusion in each of the final risk adjustment models of the version 14 CMS PSI software, coefficient estimates from the GEEs, and their 95% confidence intervals.
Each component’s risk adjustment model was assessed for discriminative ability using the C-statistic, which is the probability that a randomly selected discharge with the PSI component event has a higher model-predicted probability of experiencing the event than a randomly selected discharge without the PSI component event. The C-statistic generally ranges from 0.5 to 1, with 0.5 indicating that the model classifies discharges with and without the event no better than random chance, and 1 indicating perfect classification. C-statistics between 0.7 and 0.8 are generally considered to be acceptable discrimination, and those between 0.8 and 0.9 are considered excellent discrimination (Hosmer & Lemeshow, 1995). We observed acceptable or excellent discrimination across all ten components, with the lowest C-statistic at 0.73 for PSI 12 and the highest at 0.90 for PSI 10. These results demonstrate that the risk-adjustment models are able to distinguish discharges with and without the respective PSI component event with adequate accuracy.
See Table 8 of the 4.4.5a Calibration and Discrimination attachment for all C-statistics.
Note that there is no risk-adjustment model for the PSI 90 composite. The individual PSI components are risk-adjusted, and the PSI 90 is a composite of those risk-adjusted (and smoothed) components. Therefore, we cannot report on the discriminative ability or calibration of the PSI 90 composite.
Calibration was assessed using a Hosmer-Lemeshow decile plot per risk-adjustment model. These plots are created by first grouping the discharges into deciles of model-predicted probabilities of experiencing the PSI component event. Then, within each decile, we calculated (1) the proportion of discharges in the decile group with the PSI component (observed risk) and (2) the mean model-predicted probability of experiencing the PSI component in that decile group (expected risk). These observed and expected risks are then plotted on the y and x axis, respectively, for each decile. Our resulting Hosmer-Lemeshow plots for all components show data points approximately lining up along the 45-degree diagonal, indicating that each decile group has a roughly equal observed and expected risk. This demonstrates that the risk-adjustment models are well-calibrated.
See the 4.4.5a Calibration and Discrimination attachment for all Hosmer-Lemeshow decile plots.
Hosmer DW, Lemeshow S. Confidence interval estimates of an index of quality performance model based on logistic regression. Statistics in Med. 1995;14(19):2161-72.
We can interpret the risk factors included in each final risk adjustment model (i.e., the results of feature selection using LASSO regressions) to be the set of risk factors that provide an optimal balance between model complexity and performance. These final risk adjustment models were evaluated on our testing data and shown to have satisfactory discriminative ability (C-statistics above 0.7 in all ten PSI components) and calibration (all ten PSI components’ decile plots showing approximately equal observed to expected risks across most deciles). These results suggest that the risk adjustment models are adequately able to predict patients’ risk of PSI component events using the patient or discharge characteristics (i.e., risk factors) included in the risk adjustment models.
Use & Usability
Use
The program reports measure scores at the hospital-level and applies to the inpatient care setting.
The program reports measure scores at the hospital-level and applies to the inpatient care setting.
Usability
Measured entities can improve the PSI 90 scores in a variety of ways. Annually, measured entities receive Hospital Specific Reports (HSR) from CMS with their PSI 90 score, detailing how their PSI 90 score compares to the national average, and patient level data related to the PSI 90 component events. Measured entities also have access to a guide to support the interpretation of the results listed in the HSR. These results can be used help measured entities understand which PSI 90 components are impacting their score, run further analyses, and compare their performance to other organizations. Additional instructions on how to use the HSR can be found on https://qualitynet.cms.gov/inpatient/measures/psi/reports.
The "Toolkit for Using the AHRQ Quality Indicators” provides resources for measured entities to use to improve their performance on the PSI 90 component measures. Improving clinical documentation and coding and implementing targeted clinical intervention were a few examples cited in the toolkit. More information about how hospital can use the "Toolkit for Using the AHRQ Quality Indicators " to improve their performance on the PSI 90 components can be found on https://www.ahrq.gov/patient-safety/settings/hospital/resource/qitool/index.html.
Additional processes and structures of care that have been shown to reduce the safety events included in the composite are documented in the supplemental importance attachment (question 2.2 Evidence of Measure Importance).
The public, including measured entities, can send questions or comments regarding PSI 90 and its PSI components to CMS via the QualityNet Question and Answer Tool. Input can also be submitted to AHRQ via Qlsupport@ahrq.hhs.gov, which is considered for inclusion in both the CMS and AHRQ versions of the PSI component measure specifications.
The CMS PSI support team triages, troubleshoots, and responds to technical inquiries related to methodology and rationale behind the indicators and general questions related to the use of the software. During a calendar year, CMS typically provides technical support to over 100 inquiries.
These inquiries commonly involve clarification regarding the technical specifications of the component indicators, as well as clarification about the population subject to inclusion in the PSI 90 composite, eligible admission types, the number of diagnosis fields used to calculate the component measures, and the Medicare fee-for-service date ranges used to calculate PSI rates.
In addition, CMS’s PSI support team gathers input on PSI implementation from technical expert panels convened to support PSI development and maintenance and offers recommended changes for CMS’s consideration. Specific suggestions for refining or enhancing the PSI specifications are addressed by CMS in consultation with AHRQ as needed, in its capacity as the original developer of PSI 90.
The CMS PSIs are updated annually, including updating indicator technical specifications in accordance with the latest coding guidance; suggestions from users and other stakeholders obtained through technical assistance, or workgroups; and the latest clinical and scientific research. CMS and AHRQ regularly reviews these sources, identifies possible indicator updates, and prioritizes updates for each indicator and software update based on expected impact on users.
As CMS reports in their 2024 National Impact Assessment of the Centers for Medicare & Medicaid Services (CMS) Quality Measures Report, from 2013 to 2021 PSI 90 scores have remained relatively steady (see table 9 in the 7.1 Supplemental Attachment). The best performing hospitals’ (10th percentile) rates ranged from 0.6 to 0.9 with most recent years at 0.8. The worst performing hospitals’ (90th percentile) rates ranged from 1.0 to 1.2 with most recent years at 1.2. The number of hospitals reporting each year slightly changed with the 3,106 being the lowest number to report and 3,422 being the highest number to report.
Given the composite is a summary score of risk and reliability adjusted component measure scores, large changes in the composite measure scores are not expected.
Improvement is more likely to be observed at the component measure level. The CMS report indicates improvement in three of the PSI component scores between 2017 and 2021 (PSI-15. Accidental Puncture or Laceration rate, PSI 06. Iatrogenic Pneumothorax Rate, PSI 12. Perioperative Pulmonary Embolism or Deep Vein Thrombosis Rate). Two additional PSIs had positive score trends until 2020 and 2021 (PSI 13. Postoperative sepsis rate and PSI 11. Postoperative respiratory failure rate).
Centers for Medicare & Medicaid Services. (2024) 2024 National Impact Assessment of the Centers for Medicare & Medicaid Services (CMS) Quality Measures Report. https://www.cms.gov/medicare/quality/measures/national-impact-assessment
Patient and caregivers from Mathematica’s Patient Family Advisory Board (PFAB) were asked to share what they believed were unintended impacts of PSI 90 on patients. PSI 90’s background, purpose, measure definitions, and patient use cases/example scenarios were reviewed prior to requesting feedback from the representatives. There was a total of four patient and caregiver representatives that were asked to provide feedback. No unexpected findings or unintended impacts on patients were identified.
Additionally, we reviewed implementers submission to ServiceNow and did not identify any comments regarding unintended impacts on patients.
Comments
Staff Preliminary Assessment
CBE #0531 Staff Assessment
Importance
Strengths:
- Logic Model: A clear logic model is provided for each of the 10 patient safety indicator (PSI) components, depicting the relationships between structures of care (inputs, e.g., staffing ratios and turnover, improved training and supervision) and processes of care (activities, e.g., risk assessments, appropriate selection of treatments and medication), and the component measure focus, as well as identifying specific downstream harms/impacts associated with poor performance on each PSI (e.g., increased costs and mortality for PSI 03). These models demonstrate how the measure's implementation will lead to the anticipated outcomes.
- Evidence and Literature Review: The measure is supported by a comprehensive, literature review including systematic reviews clinical practice guidelines and recommendations that may demonstrate a net benefit in terms of improved outcomes and cost/resource related to these PSIs. Systematic reviews were cited supporting PSI 03 (Pressure Ulcer Rate), PSI 06 (Iatrogenic Pneumothorax Rate), PSI 08 (In-Hospital Fall-Associated Fracture Rate), and PSI 11 (Postoperative Respiratory Failure Rate). Evidence grading was referenced only for PSI 03 (fair-quality evidence was found that advanced static support surfaces reduced pressure ulcer rate; and evidence was of insufficient quality to draw reliable conclusions about other interventions).
Guidelines or recommendations were cited for all but PSI 15 (Abdominopelvic Accidental Puncture or Laceration), but grading was not presented for any guidelines cited.
Evidence for the effects of structure of care was presented for most PSIs, and was largely observational. - Performance Gap: Data from the universe of Inpatient Prospective Payment System (IPPS) hospitals from July 1, 2021 to June 30, 2023 shows a performance gap, with decile ranges from 0.758 to 1.396, indicating variation in measure performance across the target population. In addition, while the mean of observed to expected ratio of this composite measure is near 1 (0.999), deciles 7-10 have mean scores above 1, with decile 9 = 1.147 and decile 10 = 1.396.
- Patient Input : Description of patient input supports the conclusion that the measured outcome is meaningful with at least moderate certainty. Four patient and caregiver representatives who completed the poll, unanimously agreed that “PSI 90 is meaningful and produces information that is valuable in making care decision”, following review of PSI 90’s background, purpose, definitions, and example scenarios.
Limitations:
- While the literature review appears thorough and includes substantial recent evidence, the developer did not consistently provide information on the grading of the evidence from systematic reviews or clinical guidelines cited. A few of the component measures have minimal evidence cited (e.g., PSI 09, PSI 14, PSI 15).
Rationale:
- The measure is rated as "Not Met, but Addressable" due to incomplete information on grading of systematic review evidence and guidelines for most component measures. It is also not clear from the submission which component measures are currently endorsed. A more thorough presentation of grading of evidence, and potentially more robust evidence for a few components, could elevate its importance.
Feasibility Acceptance
Strengths:
- The developers indicate that the necessary people, tools, tasks, and technologies present no burden or barriers for the implementation of this claims-based measure. All data elements are available in electronic form and are generated during usual patient care. There are no fees or licensing requirements associated with this measure.
Limitations:
- None.
Rationale:
- This maintenance measure meets all criteria for 'Met' due to its clear and implementable data collection strategy, and transparent handling of licensing and fees, ensuring practical implementation within the health care system.
Scientific Acceptability
Strengths:
- Data Sources and Dates: Data used for testing were sourced from claims data from Medicare fee-for-service (FFS) discharges from the universe of Inpatient Prospective Payment System (IPPS) hospitals during the period July 1, 2021 to June 30, 2023. The entities included in the analysis were characterized by 3,156 hospitals, 37.6% small (<100 beds), 33.9% medium (100-250 beds), and 26.7% large (>250 beds). The majority of the hospitals were located in urban geographical areas (75.4%; comprised of 35.7% “large” urban hospitals and 39.7% “other” urban hospitals), with a smaller subset of hospitals being located in rural areas (22.7%).
Limitations:
- None.
Rationale:
- Accountable Entity-Level Reliability: The developer conducted split-half reliability testing at the accountable entity-level. A PSI composite score was calculated for each split within each hospital and random effects model was fit to calculate the ICC. The Spearman-Brown formula applied. More than 70% of accountable entities meet the expected threshold of 0.6.
Strengths:
- In general, two main findings support the validity assessment of PSI 90. First, the correlations with the healthcare-associated infection/complication measures, which have a greater construct overlap with the construct defined by PSI 90, are higher than the correlations with the 30-day readmission measures, which have a lesser construct overlap. This suggests the construct defined by PSI 90 is meaningfully distinct and more related to patient safety. Second, the correlations with the composite PSI 90 are higher than the correlations with the component measures, which also suggests the construct defined by PSI 90 is more comprehensive than the constructs of the components. Also supporting the validity assessment is the conceptual coherence of the weighting scheme. The weights for each component reflect the component’s volume (frequency of event) in the reference population and harm (excess risk of harms generated by experiencing the event and perceived severity of those harms). Entities may respond to PSI 90 by focusing resources on those components that are the most frequent and most harmful.
- The developer conducted statistical risk adjustment, based on a conceptual model, selecting risk factors that have a significant correlation to the outcome. The developer reported c-statistics of 0.73-0.90, indicating good model discrimination.
Limitations:
- Causal claims based on association (correlation) studies are prone to bias (i.e., confounding due to a common cause cannot be ruled-out). Additional support from mechanism studies that confirm the existence of a suitable mechanism capable of accounting for the observed correlation would strengthen the causal claim. Otherwise statements about the relative magnitude of the observed correlations and whether those magnitudes are greater or lesser than what one might anticipate are difficult to evaluate. For example, the developer highlights the strongest correlation between PSI 10 and the surgical site infection rate from colon surgery and abdominal hysterectomy. Even a focused, targeted empirical investigation of a mechanism capable of accounting for the relatively low correlation (r=0.12) and the other contextual mechanisms that explain why the correlation is not higher would be informative. Ultimately, the validity of PSI 90, including the adequacy of the risk adjustment, is a function of the validity and adequacy of the ten (10) component measures. The 2-year period of performance also lessens validity for the purpose of greater reliability.
- Developer did not provide assessment or consideration of social risk factors.
Rationale:
- The validity testing results support a moderate inference of validity for the measure, confirming that the measure accurately reflects performance on patient safety and can distinguish good from poor performance. The risk adjustment methods used by the component measures are appropriate and demonstrate variation in the prevalence of risk factors across measured entities, contribute to unique variation in the outcome, and show the impact of risk adjustment for providers at high or low extremes of risk. The model performance is acceptable.
Equity
Strengths:
- Each PSI component measure's performance was empirically tested across multiple sociocontextual variables, including gender, race, ethnicity, income, and metropolitan status.
- Disparities results were drawn from the Agency for Healthcare Research and Quality's (AHRQ) 2023 National Healthcare Quality and Disparities Report (NHQDR). The analysis used all payer data from the Healthcare Cost and Utilization Project (HCUP).
- The developer reported that the most consistent disparities shown were lower performance for Black vs. White patients (PSI 03, PSI 09, PSI 10, PSI 12, PSI 13, and PSI 15), Asian/PI patients vs. White (PSI 09, PSI 11, and PSI 13), and lowest quartile of income patients relative to highest income quartile (PSI 03, PSI 10, PSI 11, PSI 13, and PSI 14).
Limitations:
- The years of data and analytic approach were not specified in the submission.
The developer notes they will continue to monitor disparities for the most affected subgroups, but they do not specifically address unintended consequences or apply an interpretation for the results.
Rationale:
- The rating for Equity is Not Met But Addressable. While the measure partially addresses equity in health care outcomes for each component measure, the developer could provide additional information in the submission itself describing methods and exploring the interpretation of the disparities findings and how they might be used to improve health care.
Use and Usability
Strengths:
- This measure is currently in use in CMS Hospital-Acquired Condition Reduction Program (HACRP), and is also reported on the Hospital Care Compare website.
- The developer provides a summary of how accountable entities can use the measure results to improve performance, including the AHRQ toolkit, annual reports, and a range of interventions at the entity and clinician level that may be effective in reducing HACs identified in their evidence review (note, grading of evidence for guidelines and systematic reviews was not presented for most).
- The developer reports that feedback is solicited via email and an online tool, and from TEPs, and uses this input alongside updated evidence to revise measure specifications.
The developer reports relative stability in performance from 2013 to 2021, and their rationale is that large changes in a composite measure are generally not expected. They also report some improvement in three of the component PSIs from 2017-2021. - The developer reports no unexpected findings.
Limitations:
- While the developer states that feedback is used to update measure specification, specific instances of feedback are not provided.
Rationale:
- For maintenance, the measure is actively used in at least one accountability application, with a clear feedback approach that allows for continuous updates based on stakeholder feedback. The developer provides evidence of opportunities for entities to improve performance, through their evidence review, through annual reports to entities, and through the availability of an AHRQ toolkit for performance improvement. Despite no large improvements in performance in the measure performance over time, the developer notes that large changes in performance are not expected in composite measures, and also notes that three component measures did show improvement in recent years. The developer reports no unexpected findings.
Committee Independent Review
support
Importance
agree with staff regarding limited evidence base for a number of the component measures
Feasibility Acceptance
agree with staff
Scientific Acceptability
agree with staff
agree with staff
Equity
agree with staff
Use and Usability
agree with staff
Summary
Support, with the few caveats expressed by staff regarding additional equity studies
Summary
Importance
Agree with staff recommendations.
Feasibility Acceptance
Agree with staff recommendations.
Scientific Acceptability
Agree with staff recommendations.
Agree with staff recommendations.
Equity
Agree with staff recommendations.
Use and Usability
Agree with staff recommendations.
Summary
I agree with staff recommendations.
support
Importance
Support staff comments on addressable formula updates.
Feasibility Acceptance
Feasibility demonstrated.
Scientific Acceptability
This was met.
This was met (thank you for the proactive work with the patient advisors and the reflections on the findings.)
Equity
Though optional, this was pursued by developers.
Expertise mentioned in the staff comments are supported by this reviewer.
Use and Usability
Use/Usability is met.
Summary
The use of the patient advisors was well documented. Similarly, using the findings for improvements or further review of QM data was strong.
Support
Importance
Agree with staff
Feasibility Acceptance
agree with staff
Scientific Acceptability
Agree with staff
Agree with staff
Equity
Agree with staff
Use and Usability
Agree with staff
Summary
Agree with staff. This measure does have flaws but is all we have for certain measures. Until a replacement is found, either for individual measure or another composite, we must keep this measure.
Approve, reluctantly
Importance
Agree with staff assessment
Feasibility Acceptance
It's been collected for a while now...
Scientific Acceptability
Agree with staff assessment
Agree with staff assessment
Equity
More imformation about how the variables impact the measure and what to do about them would be helpful.
Use and Usability
Already in use
Summary
Don't love this measure but don't have good reason not to approve it
Worth a discussion
Importance
Agree with staff
Feasibility Acceptance
Agree with staff
Scientific Acceptability
Agree with staff
Agree with staff
Equity
Agree with staff
Use and Usability
not met but addressable; the use of the measure to improve outcomes is important but the capacity to use the metrics may vary by hospital dependent on size of hospital and the ability to control for events such as nurse turn over rates. If the capacity to use the measure is variable then smaller hospitals may have more of a negative impact from using the measure
Summary
AMA concerns need to be addressed; consider a case minimum limit
Support
Importance
Concise, well written summary of evidence supporting rationale for each of the adverse outcomes.
Feasibility Acceptance
Used in CMS quality programs for nine years.
Required data entered by hospital as part of claim. Calculation of measure is done by publicly available CMS PSI software to reduce hospital burden.
Four representatives from Mathematica’s Patient Family Advisory Board provided feedback. (% of all members?). No unintended consequences related to feasibility were reported.
Non-proprietary
Scientific Acceptability
Overall 0.747
The median ICC was 0.82, with 78% of hospitals having ICCs greater than or equal to 0.6.
Construct Validity testing. Correlation between risk adjusted PSI component rates and other measures of hospital level patient safety and harm events.
Strong supporting statement: Overall, the analysis provides support for the construct validity for the PSI 90. The weak, positive correlations observed among PSI 90 and the criterion measures demonstrate that a broad, common construct of patient safety underlies these measures, but PSI 90 does not entirely overlap with any one measure of patient safety. Given that we observed stronger correlations between PSI 90 and the criterion measures than between the PSI components and the criterion measures, we emphasize the importance of the construction of PSI 90 as a composite.
Equity
Summarized relevant findings from AHRQ 2023 National healthcare Quality and Disparities Report.
Use and Usability
Used for public reporting and payment program.
Actions of measured entitles to improve performance clearly described.
Approaches to gather and integrate feedback clearly described.
Patient Family Advisory Board re: no unexpected findings or unintended consequences repeated in this section.
Summary
Excellent submission.
Public Comments
531 Patient Safety Indicator (PSI) 90: Patient Safety and Advers
The American Medical Association (AMA) notes that testing demonstrated that reliability is very poor for the individual measures in this composite and the minimum intraclass correlation coefficient (ICC) was 0.04 for the composite. It is not clear if applying a case minimum would sufficiently improve the reliability of this composite but ask that the committee discuss whether it should be required. In addition, because this measure is based solely on administrative claims, hospitals struggle to be able to drive improvements at the point of care from these data given the delayed distribution of measure scores. As a result, the AMA questions whether this measure is truly useful for accountability and improvement purposes.
PSI-90 public comment reply - American Medical Association
Thank you for your feedback.
As safety events, Patient Safety Indicators (PSI) are typically rare events that naturally show lower reliability when measured individually. The PSI 90 measure uses the following design features to create a more reliable indicator of patient safety:
Since the last Consensus-Based Entity (CBE) Endorsement and Maintenance (E&M) review of PSI 90, CMS has implemented the following new minimum sample size requirement for reporting PSI 90:
The time lag is an inevitable limitation of claims data, as it takes time for claims to be processed by CMS and its contractors, and for the measure scores to be produced and validated. For v14 software, the performance period is July 1, 2021 – June 30, 2023 and the preview period for the measure’s results began in July 2024. Here is a summary of the steps that occur to develop the measure scores, using v14 dates as an example: