Does the PDSA Model Work? Part 3 of 5

It is widely promoted in healthcare, but does the PDSA model work? This is the third in my series of blog posts on the Plan-Do-Study-Act (PDSA) model, and its applicability to quality assurance and quality improvement (QA/QI) in healthcare. As I described in the previous post, the separation between the Plan, Do, Study, and Act stages is not really clear, and the goal of each stage is also not very clear. In addition, it’s not clear how the PDSA model is related to QA/QI metrics.

I think it would be hard to actually do evidence-based research on the PDSA model. I believe this is why if you look for evidence for or against the PDSA model – like cost-savings or reduction in harm as a result – you probably won’t find it in the scientific literature. I remembered that one of my favorite successful evidence-based QA/QI efforts which was accomplished by Intermountain Health never mentioned the PDSA in its evidence base. I looked long and hard, and I just couldn’t find studies where they calculated the cost of implementing the PDSA model, and compared that to how well the model worked at reducing some bad outcome.

Still, I wanted to assemble some evidence – at least anecdotal evidence. Is this really a good model for QA/QI – not just because “Harvard says so”?

According to Customer Experience, Does the PDSA Model Work?

You will find the PDSA model all over the internet. It is endorsed by all the big public health and healthcare places. But does it work? Is it evidence-based? Do places that implement it actually improve their healthcare? Given my experience as a patient at local hospitals in Boston who have embraced the PDSA model, I say, “Definitely not!” In fact, the whole United States has embraced the PDSA model, and we have some of the worst healthcare quality in the world.

The customer that had approached me about helping with QA/QI was a healthcare system that had already implemented the PDSA over a period of several years. They had hired high-powered trainers and experts to come on-site over a period of years to help them implement continuous improvement through the PDSA. I asked them how the experience was, and they reported that it was very paperwork-intensive, and in the end, it did not relate to their context. More importantly, they said they thought it wasn’t worth any improvement they might have seen. To quote one physician,

“We have been throwing ourselves into the PDSA for the last five years. You’d think we’d have the highest quality healthcare in the world, but in reality, it’s hard to tell if we’ve measurably changed.”

So does the PDSA model work? Well, apparently, not if you are a healthcare system that fully embraces it – leaving me to question if it works for anyone.

There Appears to be no Evidence-base for the PDSA in the Literature

I figured out that the PDSA is based on an old idea called “Scientific Management Theory”. This is a management theory that applies to situations like factories, where there are a few leaders and a bunch of workers doing repetitive things, and they can be easily controlled in a big system. Since the PDSA is based on a management theory (especially one with “scientific” in its name), I think if the theory is any good, when we benchmark how much the effort at doing QA/QI actually results in improved quality, we should have a positive result.

But my search of the scientific literature largely brings up praise of the PDSA, and explainers of how to implement it. There are no cost-benefit analyses. I did not look very hard when I did my search this year, but in the multiple times I’ve done this search over the years, I have not found anyone who was trying to actually measure each stage of the PDSA model, and benchmark whether the cost and effort was worth it.

The best, clearest criticism of the PDSA model is this article published in BMJ Quality & Safety called, quite bluntly, “The problem with Plan-Do-Study-Act cycles”. Without doing a formal analysis, they insist that the PDSA cycle can bring a return-on-investment (ROI). However, they warn that the vast amount of resources needed to implement the model might not be available. Therefore, a “half-baked” effort might be wasted effort, and the authors encouraged making sure the resources are there before launching on a series of PDSA cycles.

Hints from AHRQ about the PDSA Model

If you think about it, you can invest more resources as the BMJ authors suggest to ensure you get an ROI for your PDSA effort. But if you invest these extra resources, the cost goes up, and it’s harder to get an ROI. This forces us to have to focus on QA/QI problems that will save us the most money, not ones that kill the most people. So it becomes hard to answer the question, “Does the PDSA model work?” For what exactly?

But this idea of investing a lot of resources into large projects seems to be directly contradicted by AHRQ. In the United States (US), we have a federal agency called the Agency for Healthcare Research and Quality (AHRQ), which funds grants and directs initiatives to study and improve the quality of healthcare in US facilities. Of course, AHRQ itself is terribly underfunded, so don’t think you can pay for a PDSA cycle with an AHRQ grant!

But I noticed that they said something interesting on the web page about trying to use the PDSA model to “implement health literacy tools”. I will paraphrase their advice here:

• Single Step: Each PDSA contains only a single step of the entire implementation.
• Short Duration: Each PDSA cycle should be short as possible (in the health literacy situation, AHRQ says it can be as short as an hour)!
• Small Sample Size: A PDSA will likely only involve one or two clinicians, and once feedback is obtained and the process refined, the “implementation can be broadened to include the whole practice”.

This description totally contradicts the idea of implementing routine PDSA cycles in an entire healthcare system, as well as the idea of dumping a lot of money and resources into a PDSA effort. The PDSA process described on AHRQ’s health literacy site sounds like trying a recipe over and over to iterate to improve the taste – which is only possible if you are working alone. It reflects the situation back when the Scientific Management Theory was created, where factory leaders could change the lighting in the factory, or change the length of breaks for their employees, and then measure how much productivity went up.

Asking, “Does the PDSA work?” gets at the most philosophical question, and that is: What is the goal of QA/QI in healthcare? To save money? Or to save people? Even so, it seems that even if you are optimistic, it would be hard to pull together a body of evidence that says the PDSA works. If this makes you sad, turn your frown upside down! I offer an alternative to the PDSA model in my next blog post in the series!

 

Updated June 16, 2021. Added blog post menu July 19, 2021.