Part 2: Complexity of the Real World Has Outpaced the Myth of the Linear Method – The Need for a Science-Based Methodology Based on Questions.

Part 1 (see link to Part 1 here), presented some information on the use of linear models that typically reflect the ideal journey of a development or learning journey. Unfortunately, the real world typically deviates from this ideal view. What sort of model should we use that would more reflect our learning journey?

Learning from Deming – The Analytic Study

Deming offers some help on distinguishing two types of studies for learning from data and our experience in the real world. He classified studies into two types (1975), depending on the type of action that will be taken[i]:

1.       Enumerative study: one in which action will be taken on the universe that was studied (e.g., conducting a census, or sampling materials for a decision on acceptance or pricing).

2.       Analytic study: one in which action will be taken on a cause system to improve performance of a product, process, or system in the future (e.g., a study to select a future raw-material supplier or using a Shewhart control chart to learn and improve a process).

Note the key word in definition of analytic study – future. For Deming, the problem is prediction. The Foreword to the book, Quality Improvement through Planned Experimentation by Moen, Nolan and Provost (2009), Deming discusses how the results of studies for improvement are used to predict:

Why does anyone make a comparison of two methods, two treatments, two processes, two materials? Why does anyone carry out a test or an experiment? The answer is to predict; to predict whether one of the methods or materials tested will in the future, under a specified range of conditions, perform better than the other one. Prediction is the problem, whether we are talking about applied science, research and development, engineering, or management in industry, education, or government. The question is, what do the data tell us? How do they help us to predict?

What sort of model should we follow that helps us with the problem of prediction? When in doubt we can usually fall back on the methods used in science where questions become our guideposts for learning as we make predictions about the effect of our changes. Questions are powerful.

Science begins and ends in questions, but does not end in the same question in which it began..[ii]

Thinking about how to phrase useful questions is no easy feat. Dennett (2009) has offered a good description of the challenge of getting to useful questions[iii]:

“…anybody who has ever tackled a truly tough problem knows that one of the most difficult tasks is finding the right questions to ask and the right order to ask them in. You have to figure out not only what you don’t know, but what you need to know and don’t need to know, and what you need to know in order to figure out what you need to know, and so forth. The form our questions take opens up some avenues and closes off others, and we don’t want to waste time and energy barking up the wrong trees.”

This quote leads us to some observations about the utility and power of questions as we make changes and improvements:

•       When making improvements, questions lay out the journey of learning to develop, test and implement changes.

•       A good question can give us the opportunity to view many possible answers. People in the same process have different experiences and perspectives. These differences may produce very different answers and predictions relative to a well-crafted question.

•       Curiosity is critical to the crafting of questions. People who are wedded to the current way of doing things can have their thinking impacted as they journey with others on the process of answering the questions. With technical change, social consequences follow, questions help the adult learner with self-discovery, a powerful process for social change.

•       Questions expand our thinking and lead to more questions and possibilities for making changes. Answers usually end this process of thinking, discovery, and learning.

PDSA – Learning Engine for Analytic Studies

Moving from linear methods that do not include the need for questions and corresponding predictions to a methodology that is useful for an analytic study is essential. One such methodology is Deming’s PDSA cycle. Deming’s learning journey that led to the PDSA cycle began with his study of C.I. Lewis’s book, Mind and the World Order (1929) and ended with the publication of the PDSA cycle in Deming’s book, The New Economics (1994). Deming honored Dr. Walter Shewhart by calling the PDSA cycle, The Shewhart Cycle for Learning and Improvement[iv]. Deming’s journey of learning that led to the PDSA cycle was documented by Moen and Norman (2009) in the paper, Circling Back: Clearing up myths about the Deming cycle and Seeing How it Keeps Evolving[v].

Associates in Process Improvement (API) in 1994 added the rigor and importance of questions in the Plan phase of the PDSA cycle[vi]. Figure 1 describes this contribution:

Figure 1: PDSA Cycle

When developing questions for a PDSA cycle, and individual or team is sometimes tempted to utilize yes/no type questions. Table 1 describes moving the yes/no type question to an inquiry based question[vii].

Table 1: Moving the Yes/No Question to an Inquiry Type Question

Why is it important to pose the inquiry based question? When we convene a team of people to make an improvement, these people have had different experiences and interpretations in the system. More importantly, they may come from very different parts of the system. If a yes/no type question is posed, people could all answer yes or no for various reasons, based on their experiences while making predictions and miss the opportunities for sharing their experiences and interpretations. We surface these different experiences during the act of making predictions and explain our theories behind our predictions. An inquiry type question forces the sharing of different perspectives during the act of making predictions, which in turn could change how we collect data or carry out a test. Consider the example:

Using an exercise to teach the use of the PDSA cycle, questions were developed and people were asked to predict what impact exercise had on blood pressure. A test was then carried out by having people run up and down stairs, then collecting the data on the different blood pressures. As predictions were then discussed, a nurse posed the theory that women generally have higher blood pressures than men. This led to the need to stratify the data into male and female categories to test this theory. During the Do part of the PDSA cycle, we observed that men, some with beer bellies, were huffing and puffing during the brief experiment. The women in the experiment were all doing very well and is was obvious that the women had been working out. During the Study, the theory of women having higher blood pressures than men was updated with the caveat, it depends on the health and physical shape of those in the experiment. Act – In the future, further stratify the data by history of physical exercise and health. 

API adopted the PDSA cycle as an engine to drive learning. The Plan includes a plan for data collection. In Do, data are plotted during the collection or test providing information. During Study, the predictions are considered relative to the information collected in Do, creating knowledge. Action is then based on knowledge.

Built into the journey of moving from data to knowledge in the PDSA cycle, is idea of deductive and inductive learning[viii]. From Plan to Do is the deductive approach. A theory is tested with the aid of a prediction. In the Do phase, observations are made and departures from the predictions are noted. From Do to Study, the inductive learning process takes place. Gaps and surprises (anomalies) to the prediction are studied and the theory is updated or thrown out as needed. Action is then taken on the new learning to improve our ability to predict, the essence of the analytic approach. Figure 2 describes this iterative process of learning.

Figure 2: PDSA – The Iterative Nature of Learning and Improvement

 Model for Improvement and PDSA

The PDSA cycle is designed around a very tactical idea of developing, testing, and implementing proven changes. The Model for Improvement uses three broad questions to frame a project. The PDSA cycle is driven by the third question, What change can we make that will result in improvement? The Model for Improvement uses three broad questions to frame a project. The PDSA cycle is driven by the question, What change can we make that will result in improvement? which allows iterative PDSA’s for learning from changes.  Often the PDSA is misused by trying to frame a whole project using a single large PDSA instead of multiple, iterative PDSA’s. The latter are far more effective for learning the impact of changes as we work toward the overall objective of the project, which is captured in the first question of the Model, What are we trying to accomplish?  The second question, how do we know a change is an improvement?, provides the measures which tell us from our changes whether we are accomplishing our objective framed by question one of the Model.  Figure 3 describes the Model for Improvement.

Figure 3: Model for Improvement – Three Questions and PDSA Cycle

 Conclusion

Part 1 presented some information on the use of linear models that typically reflect the ideal journey of a development or learning journey. Unfortunately, the real world typically deviates from this ideal view.

·         The work of Dr. Jeff Conklin described how two developers following the same model did not perceive the world in the same way at the same time. One saw a problem, the other a solution.

·         Margaret Wheatley has observed that the fixation on linear models persists because people merely match their real-world results to model given. The myth of the linear model working continues.

·         The distinction between complicated and complex systems:  Complicated things can be figured out with time, like how a calculator works. Complexity operates in open systems, the dynamics of which are changing as well as the experience of the observer. This idea was captured in the quote from Heraclitus: It is impossible to step into the same river twice.

Part 2 has presented Deming’s distinction between analytic and enumerative studies. As Deming observed, “the problem is prediction.” In moving from a linear model to a model that will be useful for addressing the dynamics of the analytic environment we are better prepared to deal with the discoveries and challenges presented by the real world.  

In recent years, various models for improvement have been developed. Some with a focus on using various improvement tools in some prescribed order. This practice usually leads to a waste of valuable time applying tools rather than targeting the learning around useful questions and making improvements –quickly.  The good news, people are becoming aware of prescriptive models that emphasize the use of tools rather than focusing on improvement[ix]. Improvement professionals should encourage:

·         The use of methods that are useful in analytic studies. How do these methods lead to questions and enhance our ability to predict?

·         A method that employs questions that lead to more appropriate data collection and tests of changes.

·         Testing as the foundation of science and critical to carrying out an analytic study. Models that put an emphasis on implementation without adequate testing should be avoided.

·         The use of the scientific method which includes the ideas of deductive and inductive learning. Models that have the scientific method built in should be encouraged. PDSA was developed with this idea in mind.  

·         Avoidance of focusing on the use of tools rather than learning and improvement.

Finally, I am reminded of a conversation with a very bright individual who had led several successful improvement efforts: “Cliff, following an array of tools is easy. Developing the necessary questions is hard.” I couldn’t agree more. Thinking is very difficult.  Being involved in the activity trap of using tools creates excitement and the illusion of progress. Ultimately, the management will be asking the question, “What have you done for me lately?” If asked this question, you will want to show a list of improvements that were executed quickly. If forced to show all the activity and no results, this is called a reduction in force. Far too many improvement professionals were forced out of a job in the last recession. President Truman observed, “History does not repeat itself, we just keep on making the same mistakes.” Time to get focused on the science of improvement and the questions that drive tackling the analytic problem and our ability to impact the future.

References:

---

[i]     Moen, Ronald; Nolan, Thomas; Provost, Lloyd. Quality Improvement Through Planned Experimentation 3/E McGraw-Hill (2009)

[ii]     Six Easy Pieces: Essentials of Physics Explained by Its Most Brilliant Teacher Audible – Abridged

Richard P. Feynman, ©1963, 1989, 1995 The California Institute of Technology; (P)2005 Perseus Publishing. Note: I remember hearing these lectures and have attributed the quote to Feynman here, but have been unable to really nail it down. Would appreciate a more accurate attribution. Please contact me: cnorman@apiweb.org

[iii]    Breaking the Spell – Religion as a Natural Phenomenon, Daniel C. Dennett (2006, Viking Press, p. 19)

[iv]    W. Edwards Deming. The New Economics for Industry, Government, Education, Second Edition (p. 132). Published by MIT (1994).

[v]     Moen, R.D. Norman, C.L. Circling Back: Clearing up myths about the Deming cycle and Seeing How it Keeps Evolving, Quality Progress, American Society for Quality, November, 2010

[vi]    Langley,G. J., Nolan, K. M., Nolan, T. W., 1994. "The Foundation of Improvement." Quality Progress, ASQC, June,1994, pp. 81-86.

[vii]   Langley, G. J., Moen, R. D., Nolan, K. M., Nolan, T. W., Norman, C. L., Provost, L. P., 2009. The Improvement Guide: A Practical Approach to Enhancing Organizational Performance (JOSSEY-BASS BUSINESS & MANAGEMENT SERIES)

[viii]   Langley, Gerald J.; Moen, Ronald D.; Nolan, Kevin M.; Nolan, Thomas W.; Norman, Clifford L.; Provost, Lloyd P. (2009) The Improvement Guide: A Practical Approach to Enhancing Organizational Performance. Wiley Publishing.

[ix]    RIP SIX SIGMA !!!!!, Published on March 2, 2017 by Satyarth Pandey, Linkedin Post