It is really interesting to watch Scrum become more popular and hear folks’ responses to why we do this or that in the framework. Some feel that Scrum is just another project management technique. Others feel that it is a set of leadership characteristics. Others yet feel that it is a simple framework that allows for product delivery. What many folks don’t know (and should) is that Scrum is a natural technique that is a best fit for complex situations. Deeply rooted in complexity science and studies in physics and biology, Scrum allows for teams to achieve punctuated equilibrium as a way to inspect and adapt and thus evolve a product (with the customer’s help); additionally, it provides management with a leadership structure to allow for this very important experimentation. So, it’s not just another project management technique, but a very natural, primitive way to work.

As you will learn in any certified Scrum training, Ralph Stacey’s complexity model is one of many theories that form the foundation of Scrum effectiveness. Based on the work that Ken Schwaber and Jeff Sutherland did with several firms back in the 1990s, complexity theory and punctuated equilibrium are a couple of the main ideas underlying Scrum and its effectiveness as a management and leadership approach. 

Stacey, as well as Oguinnake and Ray, prove that in systems that are chaotic and complex (each defined by its own set of parameters and overlapping constraints), a simple, non-prescriptive framework that allows for emergence and empirical learning is the best approach to take. In software development projects, complexity is impacted by changing requirements, changing technical landscape (or new slash emerging techologies) as well as the people working on the projects (people are unbelievably complex beings). Those of us who have applied Scrum (or other iterative and incremental techniques) to complex projects have come to understand the value in getting started and creating an output around which teams can learn and react (or inspect and adapt). We also value and appreciate the power in giving teams high level goals for a sprint, and allowing them to manage their work (and their complex selves) in order to get the job done. The two-pronged approach of timeboxing (‘subtle pressure’), in addition to the lofty goal, described in 1986 by Takeuchi and Nonaka in a Harvard Business Review study entitled “The New New Product  Development Game”, leads to time advantages and product innovations as a result.

It was with great delight that I picked up the latest HBR, November 2007, 21 years after Takeuchi & Nonaka’s article, in which these ideas are replayed all over again. In “A Leader’s Framework for Decision Making”, David J Snowdon and Mary E Boone discuss the leader’s need to adjust her style in increasingly complex situations. In complex systems, since the agents and the system constrain one another, especially over time, we cannot predict what will happen. We in the Scrum community have seen this played out many times, and see the idea becoming more accepted over time. If you accept that the future of a complex system cannot be predicted, then that leads one to an alternative mode of leadership - command and control bloat and corrupt the system. Snowden an Boone, just like Stacey, propose that leaders in complex environments should let the best patterns emerge and resist regressing to more controlling forms of management.

What I found particularly interesting in this study is the matrix of Stacey’s system complexity levels (simple, complicated, complex and chaotic) overlayed with the leader’s job, danger signals, and proper responses to those danger signals.  

In complex systems, the leader’s job is to “create environments and experiments that allow patterns to emerge” and “increase levels of communication and interaction”. That is exactly what we do with Scrum teams. The aforementioned leader – whether it is executive or ScrumMaster – allows the team to discover what works for them, what is the appropriate course of action, the best engineering method, the best approach for the customer. According to the authors, and based on my observations, the number one danger signal is the “temptation to fall back into habitual command and control mode”; in Scrum we call this “reverting to form”.  The leader’s response, by our authors, is to “be patent and allow time for reflection”, while “use approaches that encourage interaction so patterns can emerge”. In Scrum, leaders do this by allowing the team to figure out the best solution, as well as to employ an iterative and incremental approach to regularly inspect and adapt; the rhythm and lofty goal of ‘potentially shippable product’ allows for teams to create the pattern of reliable product delivery, which affords the organization potential return on investment, or, at the very least, the ability to assess its inventory for value and potential marketability.

What I found compelling about this article were the details around even the simple systems. The danger signals are “complacency and comfort, desire to make complex problems simple and no challenge of received wisdom”, among others. The response to these situations were to “stay connected without micro-managing  and create communication channels to challenge orthodoxy” among others. In Scrum, we do this by empowering the team to self-manage. ScrumMasters and executives of Scrum development organizations do not micromanage, and they seek the challenging knowledge discovered by teams in the retrospective. This communication channel is the heart of change; like a heartbeat, it keeps the organization moving forward, rhythmically inspecting and adapting its processes as it learns. I can imagine that while these are effective leadership responses to the danger signals in the most simplistic environments, they are even more important in the complex. While the HBR authors didn’t exactly specify the ability to scale one level’s practices and danger signals to the next, we feel that it wise to do so. The more complex the system, the more we search for patterns. Stopping to inspect what’s emerging is a wonderful way to achieve this. Our authors are quick to point out that the leader’s style should adapt to the given situation; Mike Cohn wrote about this in a Scrum context in an article called “Situational Leadership” which is in line with Peter Hersey's Situational Leadership approach.

I could go on and on about HBR’s publication this month, and how it illustrates what the founders of Scrum learned by studying and observing over a decade ago. What is compelling and exciting about this article is its translation of complexity theory and appropriate leadership response in the context of general business practice. It is not IT or new product development specific, and that is exciting as we think about the uses of Scrum in non-technological environments. Just as lean practices make good business sense, and have proven themselves across manufacturing and software, the values of Scrum as a leadership style do the same. I encourage you to read this article and think about your business and leadership, and how you can reduce complexity therein by embracing emergence, empiricism and the simple fact that we cannot predict the future.