E&M Guidebook version 2.0

Suggest changing the title of the E&M Guidebook to Measure Endorsement & Management Guidebook. (or something like that affect). The acronym E&M is well-known for Evaluation & Management CPT Codes. (E&M Codes)

Thank you, Debora Contos

The E&M Guidebook is well put together and excellent tool to endorse recommendation to give Quality Healthcare measures for health providers giving care to follow.  I look forward in assisting the team with meaningful measures, particularly in area like CVD, Kidney disease, metabolic syndrome that will show up in the Peak65 zone populations and bring a great burden on Medicare.  Prevention and/or slowing the progression of comorbidies ( like prediabetes/diabetes, CKD,CVD, obesity, hyperlipidemia) will be my focus. 

Endorsement Should Explicitly State Whether a Measure is or is not Appropriate for Use in Public Reporting Programs and Pay for Performance Systems.

It is inappropriate for the Committee to simply “endorse” a measure without specifying how the measure will be used.  Many of the measures that are coming to the Committee are adequate for use in quality improvement efforts, but are not sufficiently valid or reliable to be used for public reporting or in pay-for-performance systems. 

The Committee should explicitly vote whether to endorse a measure (a) for confidential reporting only; (b) for public reporting; and/or (c) for use in pay-for-performance programs.

Developers of Measures Intended for Use in Public Reporting or Payment Programs Should Be Required to Estimate Classification Error Rates Rather Than Simplistic “Reliability” Statistics.

A measure that is sufficiently reliable for determining whether a hospital’s or physician’s performance is higher or lower than average may not be sufficiently reliable for classifying hospitals and physicians into deciles of performance and modifying payments on that basis.  If a measure is intended to be used to classify hospitals, physicians, or other providers into categories, the developer should explicitly estimate the classification error rates.

The Committee Should Not Endorse Measures “With Conditions” Unless the Conditions Have Been Explicitly Defined. 

If there is going to be a category for “Endorse With Conditions,” then there needs to be a more organized process of identifying the conditions and voting on them.  Measure #2687 was “Endorsed with Conditions” but there was no process at the meeting for determining what the conditions should be.  Battelle unilaterally decided to reclassify the measure as “Endorsed,” which is not what the Committee voted for.  Changes need to be made to the process in order to avoid this problem in the future.

If the Committee cannot achieve consensus either to endorse without conditions or to not endorse at all, and if no conditions have been identified that can be endorsed, then the measure should be tabled, and Battelle should develop options for conditions and bring them back to the Committee for discussion and voting at a subsequent meeting.

The Full E&M Committee Should Continue to Vote on All Measures.

The full E&M Committee should vote on all measures, and there should not be separate meetings of the Advisory Group and Recommendations Group.  Merely increasing the proportion of the Committee members who serve on the Recommendation Group does not assure that the full range of knowledge and perspectives represented on the Committee are reflected in the votes on measures.  Under the new process, the members of the Recommendation Group do not participate in and therefore do not benefit from the discussion at the Advisory Group meeting, and the Advisory Group members, after having spent considerable time reviewing and discussing the measures, have no direct input on whether the measures are endorsed. 

The process described in the draft guidebook is not only less efficient than the previous process, it can result in less consensus than the previous approach.  If the 1/3 of the Committee who are part of the Advisory Group reach a completely different conclusion about a measure than the 2/3 of the Committee who are part of the Recommendations Group, then there is not 75% agreement among the members of the full Committee. 

Participation by all of the Committee members in the Q&A session with the developers would enable all questions and concerns to be identified prior to the endorsement meeting so that the developer, staff, or other Committee members can address them before an endorsement vote is taken.  This will likely lead to endorsement meetings that are shorter and therefore more likely to achieve a quorum of the full Committee. 

Reducing the Number of Voting Members is Not an Appropriate Way to Achieve a Quorum.

If the goal of this new approach was to make a quorum easier to achieve, that could be done more easily and appropriately by simply changing the quorum threshold.  80% of 40 members was 32, and 80% of 25 members is 20, so the new quorum is only half of the full committee.  The specified attendance quorum for the meeting is only 60%, so using the same quorum for voting by the full Committee would require 24 members (60% of 40 members), which would be a larger proportion of the full committee and allow everyone an opportunity to vote.  

It would be much easier to achieve a quorum if Battelle surveys the Committee members in advance of the meeting and then selects a date that will work for the largest number of Committee members, rather than simply announcing a date that is convenient for Battelle but is not workable for the Committee members. 

If it turns out that the meeting falls a few votes short of a quorum, there is nothing wrong with contacting the unavailable members after the meeting to obtain their votes. Obtaining a few votes offline is far preferable to excluding a large subset of the Committee members from voting at all.

NOTE: There is an error in the wording about the voting quorum in the new document.  On page 25, it says that the voting quorum is “at least 80% of all active Recommendation Group members present [emphasis added] who have not been recused.”  The previous version said the quorum was at least 80% of all active committee members who have not been recused, so it appears that the word “present” is an error. 

We appreciate the opportunity to comment on the proposed updates to the Endorsement and Maintenance (E&M) Guidebook for reviews starting with the Fall 2024 cycle.

While the goal is to ensure the quality measurement review process is reliable, transparent, attainable, equitable, and meaningful, the updated guidebook raises several concerns. These issues are informed by our participation on several E&M committees in the 2023-2024 term year. 

Defining Consensus & Endorsement Decision Outcomes

Endorsed with Conditions

The guidebook describes several scenarios that can result in different measure endorsement decisions. However, scenario two, in which if 75% or more of committee member votes are distributed across ‘endorse’ and ‘endorse with conditions’ that results in an endorsed with conditions outcome, is misguided. Batelle could just as easily apply that thinking the other way so that a measure is not endorsed. This leads to inconsistent endorsement outcomes with the E&M committee’s deliberations.

As a result, 75% or more should be required for each endorsement outcome: 'endorsed,' 'endorsed with conditions,' 'not endorsed,' and 'endorsement removed.' 

‘Endorsed with conditions’ is described as a situation in which the E&M committee agrees by 75% or greater that the measure meets the endorsement criteria, but committee reviewers have conditions they would like addressed when the measure comes back for maintenance. If these recommendations are not addressed, the developer/steward should provide a rationale for consideration by the E&M committee. There needs to be greater clarity provided to committee members regarding the timeframe for the conditions to be met so that committee members fully understand that the measure will not come back for at least 5 years. The conditions placed on the measure must be clear to both the committee members and the developer. 

Although there is a list of non-negotiables that cannot be conditions, it is not all-encompassing, nor can it be. As a result, it is highly subjective, and it is up to the staff or committee to be well-versed in these non-negotiables and to enforce them when appropriate.

A tiered conditions timeframe should be instituted, given that not all conditions carry equal weight. Some may be required to avoid any unintended consequences (e.g., add an exclusion for depression screening for patients who already have a diagnosis of depression). These need immediate attention and are relatively easy to address. Others may be suggestions for the future (e.g., expand to another data source). 5 years is too long to allow measures to remain endorsed if conditions are placed on their endorsement. 

Endorsement Removed

One of the stated reasons for removing endorsement is that “there is no longer a meaningful gap in care, or the measure has plateaued (i.e., no significant change in measure results for accountable entities over time).”  However, performance gap is an element of the evaluation rubric and should be considered during the measure’s maintenance review. 

Annual Updates

The guidebook states that when an endorsed measure is not being re-evaluated for continued endorsement, measure stewards have the option to submit an annual update of the measure specifications to Battelle. Annual updates should be a requirement. If a program were to use the STAR database to identify CBE-endorsed measures, the language or specifications might be outdated. This leads to some of the challenges of differing measure versions being used in different programs.

The American Medical Association (AMA) appreciates the opportunity to comment on the updates to the Partnership for Quality Measurement (PQM) Endorsement and Maintenance Guidebook. While we agree with some of the changes to the process, we are extremely concerned with timing and apparent lack of developer input on the revisions to the measure evaluation criteria. We believe that it serves as a good example of the lack of transparency and collaboration that needs to occur to improve the process. The packaging of the materials and edits to the original draft were poorly put together— only high-level notes were provided and no detailed summaries or redlines included. Therefore, it is not clear whether many organizations such as measure developers will realize the significance of some of the revisions.

All stakeholders involved should be provided clear information on any potential changes, provided sufficient time to review and comment, and given a reasonable timeframe on when the revisions will be implemented. It appears that the revisions such as those proposed to the measure evaluation criteria were made in a silo without input from measurement experts and measure developers, which is a significant concern. Measure development requires collaboration and input from multiple parties such as clinical and measurement experts, end users, patients, and others. The AMA also believes that the endorsement process should follow this model too. We are particularly concerned to see the change to the scientific acceptability criteria where it appears that developers must provide patient- or encounter-level testing at the time of initial endorsement both for reliability and validity. It is our understanding that this change would go into effect in the next cycle with the intent to submit information due on October 1, 2024. Any developer who planned on submitting a measure for endorsement in the fall will have either completed or be in the process of finalizing their testing results as measure development and testing is a multi-year process. Developers need sufficient time to incorporate any changes into their development and testing plans. These changes will have a significant impact on whether developers will be able to meet the endorsement requirements and may prohibit groups from submitting measures that can greatly contribute to advancing quality. We do not view these changes as insignificant and the unintended consequence of discouraging measure submissions must be avoided. 

The AMA urges the PQM to be thoughtful about making such changes without any advance warning and without input from measure developers on whether this shift is feasible in less than six months. We made similar comments on our letter on the initial version of the Guidebook and are disappointed to see that this practice of limited advanced notice continues. 

We urge PQM to mirror many of the processes and policies of the previous Consensus-Based Entity (CBE). Any interested party was able to follow every step of the process and know which stakeholders were involved in the endorsement reviews. To date, we have yet to see this level of detail provided. We recommend PQM publish the final list of those individuals who are on the Advisory Group and the Recommendation Group, including the co-chairs, for each cycle prior to the start of the measure reviews. We also believe that the current summaries provided is insufficient due to the lack of detail and specific information provided. Transcripts and/or recordings for the meetings should also posted to the website- similar to what the former CBE released.

We also continue to request that the PQM reconsider the current timeline for the two endorsement maintenance cycles. For example, public comment occurs at the same time as much of the work for Pre-Rulemaking Measure Review (PRMR) and Measure Set Removal (MSR) and overlaps with other activities such as public comments on proposed rules. These overlapping timelines lead to a significant burden for external stakeholders. We are extremely concerned that it will lead to reduced public input on one or more of these activities and urge the PQM to change the timing of the MSR and endorsement comment periods to avoid the months when proposed rules are also released. 

The AMA urges PQM to ensure that the endorsement maintenance process is transparent, involves input from multiple stakeholders prior to proposing changes, and provides sufficient advance notice of changes. Otherwise, there continues to be great risk of compromising its integrity and discouraging participation in the process. 

Thank you for the opportunity to comment. 

Dear Battelle Team, 

On behalf of Health Services Advisory Group, Inc. (HSAG), we appreciate the opportunity to review and comment on the updated Endorsement and Maintenance (E&M) Guidebook, which was released for public comment on June 5, 2024. For your consideration, we have provided a separate attachment with in-line comments. 

Thank you for the opportunity to comment.