Skip to main content
Back

Blog

Anna Greenberg and Anthony Dale

Measuring what matters in hospitals

Hallway health care – gurneys with sick patients lining hospital corridors – is one of the most graphic representations of the pressures on Ontario’s health system today. We know that the equivalent of more than 10 large, 400-bed hospitals are filled to capacity each day by patients who don’t need the level of services hospitals are designed to provide. They are waiting for more appropriate placement in long-term care, rehabilitation, home care and assisted living.

Tackling these and other system challenges requires an accurate assessment of their scope and scale. Every health care system needs to measure how it is doing so it can improve and so Ontarians know whether the system is moving in the right direction and if they are getting good value for their money.

To properly support the priorities for the system, it is important that what we measure helps us identify and focus our efforts on the most pressing concerns. It is also important not to overwhelm health care professionals with a burdensome requirement to measure too much. As health quality guru Dr. Don Berwick wrote in 2016, excessive and mandatory measurement “is as unwise and irresponsible as is intemperate health care”. Measurement should provide meaningful information on performance, so providers can begin the quality improvement process, if required.

The Ontario Hospital Association and Health Quality Ontario are working together to stop excessive measurement and prioritize what matters. The objective is to ensure that investment and efforts to measure and report performance indicators in hospitals support better patient outcomes and are aligned with Ontario’s health system strategy.

Depending on the size and type of institution, senior leaders in hospitals in Ontario have identified more than 300 indicators that cross their desks from different organizations, in different report formats, on different time cycles and sometimes with overlapping indicators using different definitions. This is in addition to what a hospital may choose to collect on their own to manage and improve care and operations. Some of these quality improvement indicators are duplicative or not aligned with one another. Additionally, hospitals are asked to individually calculate and submit indicators when they might be more efficiently monitored centrally.

When we asked hospital leaders what was useful in all this reporting for managing and improving quality, the number of indicators shrunk by about a third. This didn’t mean that there weren’t gaps (e.g. what truly matters to patients), but it did highlight that there was a lot of measurement and reporting happening that was clearly of low value.

One of the key issues we identified is that our system tends to add new measures but is reluctant to retire measures even when they are not actively being used to make decisions or improvements. We don’t have clear criteria for when a measure should be modified or retired. And even when we add measures, we don’t always have a clear plan for how that data will be used. In the worst cases, there is data being collected routinely and no one is using it or looking at it.

Working closely with senior hospital and system leaders we have committed to:

1. Significantly reduce what we currently measure in hospitals, retiring measures of little to no value;
2. Revisit what we measure on a routine basis (i.e. annually) to make sure it continues to be of value;
3. Exercise much more discipline when we add measures about the plan for how it will be used;
4. Distinguish between what is important enough to be publicly reported vs. what is a lower priority but should still be monitored vs. what is low value and should be retired.
5. Recommending a centralized system where indicator performance is monitored and communicated back to providers or stakeholders when an issue has been identified.

An emphasis on a smaller number of system measures necessarily requires more discipline. It forces us to make important choices at the individual hospital and system levels, and it acknowledges that improvement can happen with the right focus. The hard work of quality improvement is not measurement. It’s the incremental and iterative steps frontline health care professionals and teams take to make patient care better. Measurement is vital to knowing whether these improvements make a difference. But excessive measurement can be a needless distraction from the hard work of improvement.

Measurement has always been a critical part of the delivery of quality care and as Ontario embarks on a restructuring of care delivery, it is clear the province’s hospitals need to be well-aligned to measure what matters.

Anna Greenberg is Interim President and CEO, Health Quality Ontario and Anthony Dale is President and CEO, Ontario Hospital Association.
 

Previous Article Caregiver distress: A crisis grows
Next Article Advance care planning – we’re paying attention now
Print

7 comments on article "Measuring what matters in hospitals"

Beth Gorbet

Why measure? Any one can tell that things are not working efficiently. With the aging population, things are only going to get worse without a major adjustment of the system. One of the approaches is not that difficult, though the government and the insurance agencies are baulking. Focus on preventative approaches to fend off illness. There will still be plenty of folks needing doctors and hospital care but much can be curtailed before it gets out of hand. More support needs to be given to the alternative health care practices particularly those that cause no harm.


Glencia Brookes

This is indeed a significant step towards making meaningful changes in healthcare; especially when no one understands why or how they are being used, and have no impact on quality care/best practice for patients


Carole Ann Alloway

I agree with Beth that we should be focussing on wellness in the community, which includes prevention and resilience. However, we should also include front line workers in developing metrics. Both Anna and Anthony know my story and I believe if the nurses had been given the authority and encouragement to use their own judgement and do a wellness check before discharge, my husband would not have returned to the ED with pneumonia. But the 3-day hospital stay was foremost in their minds to the exclusion of the whole patient assessment.


Ross Male

This is an important initiative. Many of my patients have observed that nurses in hospitals seem to spend more of their time at their desks and computers than with patients. Much of this is for quite appropriate documentation pertinent to patient care, but I suspect a lot is also related to "measurement mania". I suspect my nursing colleagues are no happier about these added priorities and work than the patients, but I will let them speak to that.

I look forward to when this breath of fresh air and sensibility about measurement extends beyond hospitals to the Long Term Care Sector, where there is also a need to review and retire measurements. As Anna and Anthony observe, excessive measurement in any environment can be a distraction from the hard work of improvement we really want to do.


Adrian Brown

Timely piece about quality and relevance of measuring various outcomes. Another frustration for many of us is accessing the data to review, audit and change as necessary our practices. As an example, the Choosing Wisely Campaign has many logical recommendations, but assessing the impact on the system is challenging when data would come from O.H.I.P. billings, I.C.E.S., BORN etc..... Public system, public data, therefore, more transparency and access to data required.


Dr Asad Qamar

This is very good initiative as hospitals are overburdened with their KPIs. I think if the target is achieved for an indicator and it sustains for next 6 months then hospital should either stop it or do it less frequently and jump to new indicator. I would suggest that hospitals should focus more on outcome indicators rather than structure or process indicators


Shawn Tracy

"The hard work of quality improvement is not measurement. It’s the incremental and iterative steps frontline health care professionals and teams take to make patient care better." Perhaps this is merely unfortunate wording, but measurement is in fact part and parcel of the iterative process that frontline staff and QI teams take to make patient care better. It is a false dichotomy to separate the two... either one without the other doesn't make much sense. And even a cursory glance at the list of tasks that HQO and OHA have committed to tackling would suggest that measurement is indeed hard work!!

Leave a comment

Name:
Email:
Comment:
Add comment

Name:
Email:
Subject:
Message:
x

Let’s make our health system healthier

Join Our Patient, Family and Public Advisors Program

Patients, families and the public are central to improving health quality.


Man smiling

Sign up for our newsletter

Are you passionate about quality health care for all Ontarians? Stay in-the-know about our newest programs, reports and news.

Health Quality Connect - Health Quality Ontario's newsletter - on an iPad and a cell phone