Nov 30, 2023
How do you know if the changes your team is making are leading to improvement? Simple – you measure.
Successful measurement is a cornerstone of successful improvement. Measurement allows quality improvement (QI) teams to demonstrate current or baseline performance, set goals for future performance, and monitor the effects of changes as they are made.
Measurement does not have to be difficult or time-consuming. The key is to pick the right measures so your team can quickly see the results and adapt their interventions accordingly. This puts less strain on resources and more focus on outcomes.
Here are nine tips to help make your measurement journey a simple one.
Table of contents
- Tip 1: Use the right type of measurement
- Tip 2: Engage others in developing a measurement plan
- Tip 3: Use a driver diagram to determine what to measure
- Tip 4: Only measure what is needed
- Tip 5: Select across the family of measures
- Tip 6: Keep data collection simple
- Tip 7: Show the data over time
- Tip 8: Ask questions about the data
- Tip 9: Understand variation and aim to reduce it
- Where to go from here
Tip 1: Use the right type of measurement
Three types of measurement are used in health care: measurement for improvement, measurement for accountability, and measurement for research. However, a common error is to use the wrong type of measurement for the situation.
Each type of measurement has a different aim, uses a different type of data, and has a different testing strategy. Without fully considering or understanding how to use each of the types of measurement, you run the risk of not having the success you want to see or of creating a future challenge for your team.
The table below compares the different characteristics of the three types of measurement:
Tip 2: Engage others in developing a measurement plan
QI is a team-based activity – it is important to engage the team of people who are directly involved in or will be affected by the process being improved. Who better to determine what should be measured than the people that know the process best?
Part of team engagement can be the development of a shared measurement plan. Map out your process and determine what you should be measuring together.
Tip 3: Use a driver diagram to determine what to measure
A driver diagram is a visual display of what “drives” or contributes towards a team’s goal. This tool helps organize all aspects of an improvement project, so they can be discussed and agreed upon by the team.
There are three components to a driver diagram:
- Aim or desired outcome: A specific, measurable, and achievable goal
- Primary drivers: High-level or big bucket items that must occur in order to achieve your desired outcome
- Secondary drivers: Actions that must occur in order to achieve your primary drivers
Check out the image below to see an example of a driver diagram.
Once you understand what steps need to be taken in order to achieve your goal, you can then measure those actions, activities, and processes to see if the work you are doing is resulting in an improvement.
Tip 4: Only measure what is needed
As said by Dr. Brent James: “Each data point costs roughly $1 once you factor in the value of people’s time. So don’t waste people’s time collecting data that you’re not going to use!”
Measurement takes time and effort to do well, so ensuring that you only collect data that will be used is one of the most important things you can do. When in doubt, look at the table included in Tip 1 to see what types of data are needed for each type of measurement.
Tip 5: Select across the family of measures
Another thing to keep in mind is the types of measures your team is using for their improvement project. Known as the Donabedian model, this classification system lists three basic types of measures: outcome, process, and balancing.
The table below lays out the different characteristics of each type of measure.
To have an effective strategy, it’s important to include at least one of each type of measure in your tests of change.
Tip 6: Keep data collection simple
You’ve probably heard of the acronym KISS – keep it simple and straightforward – in your day-to-day, but have you considered applying that adage to your improvement project?
Data collection doesn’t have to be complex and complicated. In fact, it can be as simple as using a pencil and piece of paper to track how often something happens.
A good way to engage with your colleagues about QI is to start with something small and simple to collect, then use the data you’ve collected as part of team discussions on. Designing complicated, automated data-capturing systems is often a poor use of resources because your measurement plan will evolve over time as you continue to do quality improvement.
Tip 7: Show the data over time
In real-world situations, the results of our efforts are often difficult to spot because of the random variation that exists. In the context of quality improvement, however, variation is embedded into the process through Plan-Do-Study-Act (PDSA) cycles.
Through PDSA cycles, improvement leaders test small and incremental tests of change, which makes it easy to spot when improvements are being made. In fact, analyzing data and comparing it to your predictions and aim is a major component of quality improvement work.
Collecting data throughout your process and showing data over time is necessary for monitoring the process throughout your improvement efforts.
Tip 8: Ask questions about the data
In the book The Tyranny of Metrics, author Jerry Z. Muller makes the argumentment, “Measurement is not an alternative to judgement: measurement demands judgement. Judgement about whether to measure, what to measure, how to evaluate the significance of what’s been measured, whether rewards and penalties will be attached to the results, and to whom to make the measurements available.”
Measurement in and of itself is not the solution to the quality problem. Instead, measurement is the answer to a question. Your measurement strategy should be informed by the questions you seek to answer and data needs to be considered in the context of the system in which it was collected.
Tip 9: Understand variation and aim to reduce it
Any rational plan is a prediction of future performance and outcomes. To manage a system, we need to make predictions about its future.
The more predictability (or stability) in a process, the more confident we can be in our predictions of its future performance. On the other hand, the more variation in a process, the less confidence we can have on predicting its future performance.
In health care, the reduction in variation is the desired outcome of nearly all quality improvement initiatives.
According to Walter Shewhard, there are three essential components of knowledge:
- The data of experience in which the process of knowing begins
- The prediction of data that one would expect to get in the future
- The degree of belief in the prediction based on evidence
Based on these components, Shewhart introduced the Statistical Process Control to visually represent the amount of variation within a process, which you can see in the table below:
Where to go from here
Quality improvement is a continuous journey and as a result, measurement needs to evolve. As one part of a process has been improved and the new mechanisms are well established, the opportunity arises to focus on the next area or aspect that could improve.
This cyclical process means that your next step is to return to Tip 1 and begin the process again, identifying the appropriate type of measurement for the new aspect, engaging others in developing the new measurement plan, and so on.
Continuous improvement is never done – it’s an ongoing process of incremental improvement that, over time, can yield great rewards, provide new opportunities, and engage partners and staff in new and exciting ways.
We wish you all the best on your measurement and improvement journey!
With special thanks to Doug Campbell, Patrick Falastein, Sonia Vanderby, Alvin Yapp, and Mallory Clarkson for helping make this blog possible.
