"When a measure becomes a target, it ceases to be a good measure." Charles Goodhart (1975)[i]

From the Telegraph, UK: “In February 2010, the first report into the scandal of appalling nursing at Stafford Hospital — where patients were left in filthy sheets, their dressings unchanged while nurses shouted at and mocked them — concluded that in order to achieve the coveted Foundation status, the hospital trust’s management had become obsessed with meeting government targets rather than looking after the patients…Nurses elsewhere endlessly repeat this complaint. Everything has to be documented. Everything is driven by ‘performance targets’ which have to be audited.”[ii] “ Meanwhile, nurses were instructed by senior nurse colleagues to falsify waiting times, and to claim that patients had been seen more quickly than they were.”[iii]

From the healthcare example, we can see the effects of Charles Goodhart’s insightful quote on the use of targets and unintended consequences. Dr. W. Edwards Deming was more direct and to the point; “Fear invites wrong figures.”[iv]

When leaders and managers employ the use of numerical goals, the intent is usually to cause improvement as opposed to using the measure for judgment. Deming noted three unintended consequences of numerical goals: 

“Will a numerical goal be achieved? Anybody can achieve almost any goal by:

• Redefinition of terms

• Distortion and faking

• Running up costs”[v]

The unintended consequences of goals and targets where there is no method is pervasive throughout industry, government, healthcare and education. We have had several attempts to improve education over the last three decades. This has led to teachers being fired and charged criminally.[vi] Donald T. Campbell warned of this problem in what has become known as Campbell's law:

"The more any quantitative social indicator (or even some qualitative indicator) is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor."

Associates in Process Improvement (API) has attempted to strike a balance on the use of goals and targets:[vii]

“Advice about the use of numerical goals in setting an aim for an improvement project could include “Never use them” or “Always provide one and let other people figure out how to achieve it.” This book’s authors recognize the abuses associated with numerical goals and the potential unintended consequences when people are held responsible for results they are not capable of achieving…We do not believe these instances of poor practice or abuse should preclude beneficial use of numerical goals.

We also recognize some hardy souls need only the challenge of a numerical goal to find ways to actually improve the system. Our experience indicates that a middle ground between these two extremes is achievable and useful.

First and foremost, numerical goals must be connected to methods for achieving the goals. Leaders should understand that to improve a stable system beyond the current level of performance a fundamental change is needed…here are some initial considerations to develop methods for achieving numerical goals:

• Observe other organizations that have accomplished similar goals.
• Give some basic concepts or ideas that could feasibly result in achieving the goal.
• Draw out ideas from participants themselves by asking questions such as, “What would it take to get a 50 percent reduction in time to ship an order?” From the healthcare example above; what would it take to reduce wait time and patient harm?
• Ask experts on the changes being considered what level of improvement is possible.

Numerical goals can also be a convenient way to communicate expectations. What are the consequences of not meeting the numerical goal? Are small and incremental improvements expected, or are large breakthrough changes necessary? If the numerical goal is used well, it communicates not only the expectation but also the support that will be offered. Large changes to big systems usually require investment of time and capital. When first using a goal to break the current bounds of the status quo, leaders must furnish:

• An explanation of the need for and feasibility of the goal
• Assurances the goals will be used to cause new thinking and learning, and not for judgment

Systems thinking informs us that one can create the illusion of improvement by only focusing on one measure of the system. Improvement requires that we focus on multiple measures to mitigate the unintended consequences noted by Deming. Specifically, three types of measures can be used to ensure a fundamental change in patient waiting time:

• Outcome Measures – Waiting Time
• Process Measures – Number of patients scheduled; number of drop in patients
• Balancing Measures – Cost per patient; patient satisfaction. 

Please note the bold and underlined statement used earlier; First and foremost, numerical goals must be connected to methods for achieving the goals. This is directly from our learning from Deming and has been verified in our work experience in API. A project where the solution is unknown, may not have a method. This has to be developed. Deming would chide his audiences; “If you have a method, why did you not do it last year? There can be only one reason; you were goofing off.” Of course this got a laugh, but there is a lesson here as well. The people with whom we are working usually have full time jobs. Taking time to develop, test and implement changes in the unknown, is already a daunting task. What they need is help from sponsors, not expectations for more judgment; they already have plenty of that. So if the sponsor is showing up and removing barriers and doing their part, they will have a good idea of when the team is to be finished. Deming would call this “substituting leadership.”

References:

---

[i] Goodhart, C.A.E. (1975). "Problems of Monetary Management: The U.K. Experience". Papers in Monetary Economics (Reserve Bank of Australia) I.

[ii] http://www.dailymail.co.uk/health/article-2275943/NHS-Why-nurses-stopped-CARING.html#ixzz3NJS41dHF

[iii] http://www.telegraph.co.uk/health/healthnews/9851763/Mid-Staffordshire-Trust-inquiry-how-the-care-scandal-unfolded.html

[iv] W. Edwards Deming. The New Economics for Industry, Government, Education (p. 94)

[v] W. Edwards Deming. The New Economics for Industry, Government, Education (p. 43).

[vi] http://www.nytimes.com/2010/06/11/education/11cheat.html?pagewanted=all&_r=0

[vii] Langley, Gerald J.; Moen, Ronald D.; Nolan, Kevin M.; Nolan, Thomas W.; Norman, Clifford L.; Provost, Lloyd P. (2009-06-03). The Improvement Guide: A Practical Approach to Enhancing Organizational Performance (Kindle Locations 1916-1918). Wiley Publishing. Kindle Edition.