Change Project 911: When Your Rapid-cycle PDSA is not Working

Mat Roosa, LCSW-R
NIATx Coach

Rapid-Cycle Plan-Do-Study-ACT (PDSA) is a powerful tool for improvement that can enable a team or organization to achieve its short-term goals and move toward long-term success. But sometimes, PDSA change cycles do not yield the desired results.

Here are a few questions to consider when your change project does not achieve the goal.

What are the lessons learned from “failure”?

Rapid-Cycle PDSA has been called a “no-fail” method. The lessons learned from change that does not achieve the desired result can yield as much information as a highly successful change project. Finding out what does not work enables a team to avoid future investments in ineffective strategies and focus on efforts with a high return on investment.

Was the goal realistic?

We often recommend a “stretch goal” for a project that pushes the team toward a result that might seem unattainable. Stretch goals can energize a team toward greater achievement. However, sometimes a lack of information or an overabundance of enthusiasm can result in an unattainable goal. Recalibrating the goal toward a more realistic expectation can clarify the level of success the change achieved. 

What does the early data tell us?

Some change teams make the mistake of waiting until the “Study” phase of PDSA to look at the data collected. However, an initial review of the data during the “Do” phase may uncover the need to restructure the change or reconsider the data plan. These adjustments can rescue some change projects from heading too far in the wrong direction.

Are we experiencing unexpected variables?

Confounding variables can have a big impact on change project results. Teams should conduct some form of environmental scan to consider factors such as seasonal events, economic trends, political or social events, changes in staffing, or other variables affecting the people being served or the people providing the service.

Was our aim statement hypothesis correct?

Increase A from B to C by date D through strategy E.

Teams can consider a number of assumptions related to this equation when a change project is not yielding the desired results:

• Is E actually a primary driver of A? Perhaps other strategies will have a better impact on the thing that we are trying to change.

• Is C too high? See our discussion of realistic goals above.

• Do we need more time? An adjustment to D may allow the change to unfold in a manner that creates better understanding of the impact of the change, or achieves greater results.

• Is A the key indicator of success? Are we measuring the right thing? Maybe there are better ways to understand the impact of strategy E. Maybe we are having an effect on a different goal.

• Is our data source valid and reliable? Is our chosen measure giving us accurate information about the thing that we are seeking to change? Are all participants following the measuring and reporting process consistently? Sometimes participants in the data collection process have a different interpretation of the data collection rules. (Oh, I thought we were only counting attendance for people who showed up on time…etc.)   

Rapid-cycle change projects should always yield valuable results, even when they do not achieve the desired goal. Taking some time to consider the questions above will result in more reliable results that can serve as a compass to guide your ongoing change project journey.  

 

About Change Project 911

Change Project 911 is a monthly blog post series covering common change project barriers and how to address them. Has your change project hit a snag that you’re not sure to tackle? Share your issue in the comments section below, or email Change Project 911 at matroosa@gmail.comWe’ll offer solutions from our team of change project experts!

About our Guest Blogger

Mat Roosa was a founding member of NIATx and has been a NIATx coach for a wide range of projects. He works as a consultant in quality improvement, organizational development and planning, and implementing evidence-based practices. His experience includes direct clinical practice in mental health and substance use services, teaching at the undergraduate and graduate levels, and human service agency administration. You can reach Mat (Change Project SOS) at matroosa@gmail.com.

Change Project 911: Counting what Counts: Addressing the challenge of incomplete data collection

Mat Roosa, LCSW-R
NIATx Coach

“Help! We don’t know if our change is an improvement!”

 

At the foundation of all quality improvement work lies data.

 

Imagine driving down a twisty road at night and having your headlights turned off for a portion of the journey. That’s what happens when we try to manage a change project without consistent data access.

 

It can be helpful to think about the data needed to steer a change in three stages:

 

Data at the beginning of the change journey: Baseline

The only way that we know if a change is an improvement is by measuring before the change, and comparing that measure to ongoing data collection during and after the change. We all know this. And yet, too often teams rush to implement changes and fail to collect baseline data. They are then left confused about the impact of the change and may be at risk of sustaining new activities that soon demonstrate little or no benefit.

 

Data during the change journey: Data-driven change management

As we drive along, we keep gathering data by looking down the road as far as we can see. Each turn in the road reveals new data to interpret and incorporate into our effort to steer safely. A failure to regularly collect data blinds a change team’s effort to interpret the change as it evolves.  

 

Data toward the end of the journey: Sustainment

At the conclusion of the change project, the team must ask whether they want to abandon, adopt, or adapt the change project based upon the data collected. The best way to sustain a successful change is through regular data checks that ensure that the new practice is firmly established and continues to have the desired effect.

 

This focus on data can all seem like a lot of work. However, focusing on a few key factors can help ensure that data collection continues for the duration of the project and beyond. The following tips can help you make sure that you count what counts:

Keep the data simple: If you have the choice between a perfect measure that is complex and a “good enough” measure that is simple, pick the good enough measure. To keep the entire team engaged in the project, keep the data clear and understandable to all team members. A simple line graph helps the team to track the trend.

Use existing data sources: Most teams have access to a range of existing data sources that they can use to steer the project without adding any additional burdens to the system. 

Assign a data coordinator. Placing one team member in charge of managing the data can ensure accountability. Each time the team meets, the data coordinator can make sure that the data is available and current. It can also help to have a second party assigned to the data coordination task, so that the data production process does not stop if the coordinator is not available.

Frontload the data effort. Too often, the data measurement components of a change project are developed and implemented in a mad scramble as a change project commences. Careful consideration of data well before the project begins will ensure consistent data.

Train people in data collection. There’s often an assumption that all participants in the change project have a clear understanding of data collection definitions and procedures. “Oh, I thought we were counting it this way!” is a frequent comment as teams realize that they failed to train the team on the specific data collection details. This confusion can result in a need to restart the change or throw away a portion of the data collected.

There are many different ways to count things. Engage the team in generating ideas about what data metrics to use and how to collect them. Even simple measures can be collected in different ways and require team dialogue. The team can help to consider how best to measure your change to make sure that you count what counts.

Make data collection an essential part of your change project from the beginning, and you and your team will have a built-in tool for seeing if your change is an improvement!  

 

About Change Project 911

Change Project 911 is a monthly blog post series covering common change project barriers and how to address them. Has your change project hit a snag that you’re not sure to tackle? Share your issue in the comments section below, or email Change Project 911 at matroosa@gmail.comWe’ll offer solutions from our team of change project experts!

About our Guest Blogger

Mat Roosa was a founding member of NIATx and has been a NIATx coach for a wide range of projects. He works as a consultant in quality improvement, organizational development and planning, and implementing evidence-based practices. His experience includes direct clinical practice in mental health and substance use services, teaching at the undergraduate and graduate levels, and human service agency administration. You can reach Mat (Change Project SOS) at matroosa@gmail.com.

The Treatment Challenge in Jail Settings: Detox and Withdrawal or Continuing Medication?

Jessica Vechinski, MSW
Project Manager, Justice Community Opioid Innovation Network (JCOIN)
Center for Health Enhancement Systems Studies, UW-Madison

Opioid use disorder (OUD) is prevalent among people in the criminal justice system—estimates show that as many as 50–65 percent of people entering the system have an OUD or other substance use disorder (SUD). Yet, access to high-quality treatment that includes medications for opioid use disorder (MOUD) is limited, with fewer than one percent of jails and prisons providing access to medications. This lack of access leads to high recidivism rates and preventable overdose deaths. The risk of death within the first two weeks after release is 12 times higher for individuals with OUD due to fatal overdose, often related to loss of tolerance during incarceration.

A new National Institutes of Health and National Institute on Drug Abuse study conducted by the University of Wisconsin’s Center for Health Enhancement and Systems Studies (CHESS) and George Mason University’s Center for Advancing Correctional Excellence! are testing ways to expand use of medications such as buprenorphine, naltrexone, and methadone for OUD in criminal justice settings. The study, titled Justice Community Opioid Innovation Network (JCOIN), is exploring two evidence-based coaching interventions for disseminating MOUD in justice-involved populations: the NIATx model for process improvement and Extension for Community Healthcare Outcomes (ECHO).

As part of the NIATx intervention, sites are asked to identify a project “aim.” A recurring primary aim that jails have identified is establishing policies and procedures for continuing a prescribed buprenorphine regimen when a person enters the jail setting. Jails and prisons around the country face multiple barriers to use of MOUD for persons with SUD, forcing those individuals into detox rather than continuing care.

This is a topic of considerable debate within jails, justice systems, and state governments. Our study team feels that providing justice-involved individuals with continuing MOUD care is ethical medical practice. Like diabetes, heart disease, or cancer, SUD is a chronic condition caused by behavioral, psychological, biological, and environmental factors. When a person with diabetes enters jail, they receive insulin and continuity of care. But a person on a prescribed buprenorphine regime for their SUD often has no choice upon entering jail other than to suffer through detox and withdrawal.  

Two chronic conditions. One receives continuity of care, but the other does not. Why is that?

Barriers to MOUD in jail settings

Stigma

The view that addiction is a moral failing and not a medical condition is a top reason, combined with the view that public resources should not go to treating a moral failing. A second factor is the pervasive misconception of MOUD as “just replacing one drug with another.” A third significant barrier that keeps jails from providing buprenorphine is the risk of diversion: the person receiving buprenorphine will sell or give the medication to other incarcerated individuals. Yet jails that have administered buprenorphine either by strips or crushed pills have found ways to decrease diversion. Strategies include having the individual eat crackers and drink water or conduct post-dose “mouth checks.”

Lawsuits against jails: Addiction is a disability

The evidence for continuing buprenorphine in jail settings far outweighs the reasons for discontinuing care. The most substantial support comes from cases of justice-involved individuals suing jails on the grounds that addiction is a disability, and that being denied treatment is a violation of the Americans with Disabilities Act. The Act requires that public places or services be accessible to all, including those recovering from alcoholism and drug addiction. Justice-involved individuals have claimed that denying medication while incarcerated is discrimination based on their disability. Even though justice-involved individuals are winning their cases and states are starting to take note, many jails disagree with this position.

One last point to consider

Isn’t rehabilitation one of the main reasons for incarceration? By not allowing an individual who has taken the responsible steps to be on a prescribed buprenorphine regimen to continue care, the jail is not providing adequate healthcare or an environment conducive to rehabilitation.

What do you think?

Should jails be required to continue providing medication to those already on a prescribed buprenorphine regimen when entering the jail?

About our Guest Blogger

Jessica Vechinski is a member of Center for Health Enhancement System Studies (CHESS) at the University of Wisconsin. She serves as the Project Coordinator for a five-year study with the Justice Community Opioid Innovation Network (JCOIN), an initiative funded by NIDA/NIH. The study is testing the combination and dosages of two evidence-based strategies to implement or improve Medications for Opioid Use Disorder (MOUD) programming within justice settings around the country. You can reach Jessica at jvechinski@wisc.edu.

Change Project 911: The Incomplete Walk-through

Mat Roosa, LCSW-R
NIATx Coach

Understand and involve the customer.

This is the first and most important of the five NIATx principles. Much of our NIATx work involves working to understand the customer/client/patient experience—because the customer experience is the critical factor in all service delivery. Strategies to understand and involve the customer can include client interviews, focus groups, or including clients on a change team.

 
The walk-through, one of the five essential NIATx tools, is a role-play exercise designed to give staff the chance to walk in their clients’ shoes. The walk-through allows staff members to focus on a specific part of the service process and gather critical insights into what it feels like to schedule an appointment, find the program, sit in the waiting room, fill out the paperwork, or complete an assessment session.
 
Walk-throughs have helped thousands of people understand the customer experience and identify improvement opportunities. That’s the good news.
 
And now for the bad news: Too often, people just don’t finish the walk-through.

Doing an incomplete walk-through is like taking half of your antibiotics. “Half” of a walk-through can be worse than no walk-through because it can reinforce inaccurate assumptions and may lead you to think that you know things that you do not know. Improvement efforts are difficult to achieve with such a hazy vision.
 
So, what do we mean by an “incomplete” walk-through? The tips that follow will ensure that your walk-through is complete and will help you gain the clear and critical vision required for meaningful improvement.
 
Don't look at it, do it.
Looking at your waiting room from behind the reception window and sitting in the waiting room for 20 minutes waiting for an appointment are two very different experiences. Observing is not the same as doing. The complete walk-through duplicates the client experience.
 
Stay in character...both of you.
As you plan the walk-through, develop a character that includes all of the core details needed to complete the process. If you’re seeking admission to a mental health service, show up with your demographic information and symptoms clearly defined, and stay in character for the entire process. Try to understand the experience through your character’s lens. What would it be like to be having this experience, given this background and current need? If a staff member who is completing part of the process with you starts to step out of character by saying, "At this point in the process, we would typically do X," gently remind them that you want to follow the normal process. We want them to avoid commentary about 'X' and to just do 'X.'
 
Do a chunk of the process. (Not all of it.)
People tend to skip parts of the walk-through process because they have limited time and are trying to do too much. If you only have 90 minutes, don't try to walk-through a process that takes three hours. Plan ahead and dedicate your energies to a manageable part of the process. Select something that you suspect has some challenges that warrant further attention.
 
Do every part of that chunk. (Don’t skip steps.)
Now that you have been selective in your choice for a walk-through, be diligent about completing every part of the sequence. Remember that you are trying to understand both the nuts and bolts of the sequence and how the sequence feels. Skipping some of the forms on the intake clipboard will not give you a true impression of what it feels like to do that paperwork. Many of the stressors that our clients experience are cumulative. Each of our forms or data gathering processes might be easy to complete. But stressors can emerge when a client is asked to complete eight sets of forms and provide the same address and phone number on four of them.
 
Be open to learning. (You don’t already know it.)
 "We can skip this part. I already know how this works" is a phrase often heard in an incomplete walk-through. Work to maintain a humble and curious posture when doing the walk-through. Assume that there are many things that you do not know about the process. This can be challenging when exploring familiar territory, but remember that you are familiar with the staff experience side of the process. The process looks very different from the other side of the desk.
 
Visit the NIATx website to learn more about the walk-through and other process improvement tools. 

About Change Project SOS

Change Project 911 is a monthly blog post series covering common change project barriers and how to address them. Has your change project hit a wall that you're not sure how to tackle? Share your story in the comments section below, or email Change Project 911 at matroosa@gmail.com. We’ll offer solutions from our team of change project experts!

About our Guest Blogger

Mat Roosa was a founding member of NIATx and has been a NIATx coach for a wide range of projects. He works as a consultant in quality improvement, organizational development and planning, and implementing evidence-based practices. Mat also serves as a local government planner in behavioral health in New York State. His experience includes direct clinical practice in mental health and substance use services, teaching at the undergraduate and graduate levels, and human service agency administration. You can reach Mat (Change Project SOS) at matroosa@gmail.com.

Behavioral Health Resources for Service Members and Veterans

The Defense Health Agency’s (DHA) Psychological Health Center of Excellence (PHCoE) provides psychological health expertise to the military community by continuing to improve psychological health care, readiness, and prevention of psychological health disorders. PHCoE utilizes Implementation Science, which helps clinicians implement empirically supported treatments and provides Clinical Support Tools which assist providers in the use of Department of Defense (DOD)/Veterans Administration (VA) clinical practice guidelines for psychological health conditions, including provider, patient, and family guides.

PHCoE provides Prevention and Outreach Support to clinicians and the military community in the prevention and early detection of psychological health disorders through its inTransition program and the Psychological Health Resource Center (PHRC). The PHRC is a 24/7 resource center for service members, veterans, family members, and providers seeking psychological health resources, in addition to other resources. The inTransition program is a free, voluntary, and confidential coaching program that can help connect any service member or veteran with behavioral health care, regardless of discharge status, time in service, or time since separation. inTransition can bridge potential gaps during a service member or veteran’s transition to a new provider or it can provide guidance for someone seeking care for the first time. inTransition is a telephonic program that is available globally 24 hours a day, 7 days a week, 365 days a year. The inTransition coaches assist with identifying each service member or veteran’s needs, helps to create an action plan to meet those needs, navigate the appropriate health care system(s), and provide additional support resources as needed.

The bulk of inTransition’s cases involve service members separating from service. This transition from military to civilian life can be a very difficult time. Service members have a vast number of changes to make in their lives and the lives of their families. It is not a surprise that Service members and veterans often put their behavioral health care, or health care in general, on hold while meeting the challenges of this life transition. Veterans listed their top five difficulties associated with transitioning to civilian life as (Zoli et al., 2015):

• Navigating the VA’s system of care (60%)
• Obtaining a job (55%)
• Adjusting to civilian life (41%)
• Financial issues (40%)
• Skills translation (39%)

 Of the eligible veterans to receive behavioral health care with the Department of Veterans Affairs from 2002-2015, only 62% of those obtained VA health care (Dept. of Veterans Affairs, 2015). Post-9/11 veterans that do not seek mental health support at the VA do so for three main reasons (National Academies of Sciences, Engineering, and Medicine, 2018):

• They don’t know that the VA offers mental health care benefits.
• They are unsure how to apply for VA mental health benefits.
• They are unaware of their eligibility status with the VA.

inTransition can help eliminate these deterrents from getting connected to behavioral health care and assist with the other above needs as well. The inTransition coaches are familiar with and trained in military culture. They can help service members and veterans find transition-related resources (i.e. employment, financial, VA benefits, etc.) and are experienced in navigating the VA medical system.

The need to maintain or get connected to behavioral health care is vital. The inTransition Program is available 24/7/365 to assist service members, veterans, and providers on how to navigate the path for a successful connection to care.

Visit www.pdhealth.mil/intransition or call:

800-424-7877: Inside the U.S.
800-424-4685: Outside the U.S. toll-free
314-387-4700: Outside the U.S. collect

 

References:

Analysis of VA Health Care Utilization among Operation ... (2015). https://www.publichealth.va.gov/docs/epidemiology/healthcare-utilization-report-fy2015-qtr1.pdf

C. Zoli, R. Maury, & D. Fay, Missing Perspectives: Servicemembers’ Transition from Service to Civilian Life — Data-Driven Research to Enact the Promise of the Post-9/11 GI Bill (Institute for Veterans & Military Families, Syracuse University, November 2015)

National Academies of Sciences, Engineering, and Medicine. Evaluation of the Department of Veterans Affairs Mental Health Services. 2018. doi:10.17226/24915