Foundations

Empiricism: transparency, inspection and adaptation

The Scrum Guide sums up its theory in one sentence: knowledge comes from experience, and decisions are made on the basis of what is observed. This article explains what that implies, why Scrum distinguishes three pillars (transparency, inspection and adaptation) and how each event in the framework puts them into practice.

Follow the recipe or taste the dish

A cook making pancake batter for the hundredth time follows the recipe without thinking: the same quantities and the same movements give the same result. The same cook working out a new dish goes about it differently. They try, taste, correct the seasoning and taste again. They cannot write the recipe before cooking, because they do not yet know what will work.

Specialists in industrial process control distinguish these two situations. A process is called "defined" when it is understood well enough that repeating the same steps produces the same result. When the process is too complex to be described in this way, it is steered "empirically": what comes out of it is measured frequently, and adjustments are made. Ken Schwaber recounts in his books that, in the mid-1990s, he submitted the software development methods of the day to researchers at the chemical company DuPont, among them Babatunde Ogunnaike, co-author of a textbook on process control. Their view, as he reports it, was that software development was being managed as a defined process when it called for empirical control.

Scrum is built on this idea. The 2020 Guide states that "Scrum is founded on empiricism and lean thinking." In philosophy, empiricism refers to the doctrines that derive knowledge from experience, such as those of John Locke or David Hume. The Guide uses the word in a practical sense: a way of conducting work whose outcome cannot be known in advance.

When a detailed plan is no longer enough

The Guide intends Scrum for complex problems. The authors of A Scrum Book give a simple test: a complex problem is "one that you don't fully understand until seeing its solution, and that you can't derive the solution from first principles ahead of time." Developing a new product is the typical example. It is easy to explain why a product succeeded once it has succeeded, but nobody could be sure of it beforehand.

In such cases, a long and detailed plan gives an impression of control that it does not deliver. The Guide puts it this way: "In complex environments, what will happen is unknown. Only what has already happened may be used for forward-looking decision making." From this Scrum derives an iterative, incremental approach. The team moves forward in small steps, produces at each step something that can be examined, and decides on the next step according to what it has seen. The Guide sees this as a way to optimize predictability and control risk, since a wrong direction is discovered at the latest by the end of the Sprint.

Three pillars that hold one another up

Transparency

The work and the way it is done must be visible to those doing it and to those receiving it. In Scrum, important decisions are based on the perceived state of the three artifacts: the Product Backlog, the Sprint Backlog and the Increment. If these artifacts give a false picture, the decisions based on them diminish value and increase risk. That is why the Guide pairs each artifact with a commitment, which says what it should be read against: the Product Goal for the first, the Sprint Goal for the second and the Definition of Done for the third.

Inspection

The artifacts and the progress toward the goals must be examined frequently and diligently, so as to detect variances and problems. The five Scrum events give this examination its cadence.

Adaptation

If some aspect of the process drifts outside acceptable limits, or if the resulting product is unacceptable, what is being done or produced must be adjusted, and as early as possible so that the deviation does not grow. The Guide specifies that adaptation becomes more difficult when the people involved are not empowered to decide. That is one of the reasons it wants a self-managing team.

The Guide links the three notions in two sentences: "Transparency enables inspection" and "Inspection enables adaptation". It adds that "inspection without transparency is misleading and wasteful", and that "inspection without adaptation is considered pointless". The three pillars therefore depend on one another, as the following example shows.

Four chances to inspect in every Sprint

The Guide presents each event as a formal opportunity to inspect and adapt the artifacts. The Sprint serves as a container for the four others.

So two things are examined in every Sprint: the product, at the Sprint Review, and the way of working, at the Sprint Retrospective. The authors of A Scrum Book stress the second. In their words, "The Sprint in Scrum is first a cycle of process improvement and second, a cycle of regular delivery."

The three pillars can be recognized in this example. Transparency comes from the fact that the session really took place and that everyone watches the same recordings, instead of discussing a document. Inspection is regular and looks at what the students actually do. Adaptation is immediate, since the rubric changes in the very next cycle. If the four teachers had written the whole course before trying it, they would have discovered these problems in January, in front of all the classes at once.

Empiricism, PDCA and lean thinking

The introduction to A Scrum Book compares the pair "inspect and adapt" to two other cycles. The first is PDCA (plan, do, check, act), associated with the name of W. Edwards Deming, who preferred to say PDSA, replacing "check" with "study". The second is what the architect Christopher Alexander calls the fundamental process. In all three cases, you act, look at the result and correct. The exact history of PDCA is told in the article PDCA and the A3 report (Lean corpus, in French).

A Sprint can be read as one turn of this wheel. Sprint Planning corresponds to planning, the work of the Sprint to doing, the Sprint Review and the Sprint Retrospective to studying the results, and the decisions that come out of them to acting. Scrum adds a fixed, short cadence of one month or less, and a second, faster turn, the Daily Scrum, which happens every day.

Lean thinking, the second source named by the Guide, contributes the concern to reduce waste and focus on the essentials. The two sources meet: work that nobody has examined for six months and that turns out to be useless is a waste that a short cycle helps to limit.

The patterns themselves are to be applied empirically. The book recommends introducing only one change at a time (the One Step at a Time pattern), so that you know what to credit an improvement to, and stepping back when the change achieves nothing.

Limits and pitfalls

Transparency easily turns into surveillance. When what is made visible is used to judge individuals, everyone learns to present a reassuring board, and transparency recedes. The Guide in fact links the pillars to the Scrum values: without respect and courage, nobody shows work that is in trouble (see The spirit of the game).

Inspection without adaptation is the most common trap. A Sprint Review reduced to a demonstration that people applaud, and a Sprint Retrospective that notes the same problems from one Sprint to the next, keep to the schedule and change nothing.

Measures can mislead. A team whose velocity is turned into a target learns to inflate its estimates, and the figure stops telling you anything about the work (see Goodhart's law, in French).

A Sprint does not have the rigor of a scientific experiment. There is no control group, several things change at once, and people readily credit an improvement to the last decision made. Scrum's empiricism is a discipline of regular observation, useful for correcting a course. It cannot firmly establish relationships of cause and effect.

Nor does empiricism spare you from forecasting. Scrum plans a great deal, every Sprint and every day; it only declines to take a distant plan for granted. Some kinds of work lend themselves poorly to this, because you cannot produce within a month a partial result worth examining, or because a mistake there is irreversible.

Finally, the idea that empirical control suits complex work better than a detailed plan is consistent with process control theory. The effectiveness of Scrum itself, however, remains poorly established by independent studies, as the article Criticism and the state of the evidence explains.