The value stream

Estimation and velocity

Every team gets asked when the work will be ready, and an answer given in hours almost always turns out to be wrong. Scrum's patterns propose estimating in relative points, then forecasting from what the team actually finished in the last few Sprints: this is "yesterday's weather". This article explains these practices, the calculations that go with them and the reason why velocity must not become a target.

Comparing rather than counting hours

This way of working has a name in A Scrum Book (2019) by Jeff Sutherland, James Coplien and their co-authors: the Estimation Points pattern. A pattern describes a recurring problem, the forces at play and a solution proven by practitioners.

The problem is this. The team needs reliable estimates so as to take neither too much nor too little work into a Sprint (the fixed-length work period, one month at most) and to announce plausible dates. Estimating in hours runs into several obstacles. It takes a long exercise, which people are reluctant to repeat when the situation changes. It assumes you know how many hours a week are really spent on the planned work, whereas meetings, emails and interruptions take up a share that nobody measures. Finally, it invites stakeholders (customers, management, users) to redo the calculation on the basis of full weeks, then to dispute the difference. The authors add that people are poor at estimating in absolute units but good at comparing.

The solution is to estimate with unitless numbers, in relative terms, starting from a simple item that everyone understands and that serves as a reference. The estimate covers all the work needed for the item to be truly done. The authors highlight an indirect benefit: estimating forces the team to think about the problem early.

The most widespread technique is planning poker, devised by James Grenning in 2002 and popularized by Mike Cohn in Agile Estimating and Planning (2005). Each developer secretly picks a card, everyone turns theirs over at the same time, those who gave the extreme values explain themselves, and the round is repeated until agreement. The cards roughly follow the Fibonacci sequence (1, 2, 3, 5, 8, 13), and the growing gap between values is a reminder that uncertainty increases with size.

The people who do the work estimate it

The Pigs Estimate pattern answers a simple question: who gives the figure? Two situations often distort estimates. A well-meaning manager estimates in the team's place, and their figure becomes hard to challenge. An expert estimates work that others, less experienced, will carry out. In both cases, the team does not feel responsible for a figure it did not produce.

The solution is for the people who will do the work to estimate it, all together, so that every skill in the team carries weight in the consensus. The Product Owner, accountable for the value of the product, estimates value; the Developers estimate effort. Nobody else can impose or challenge an estimate in advance, and it remains a forecast that the team revises when it learns something new.

The name comes from a story that Ken Schwaber and Mike Beedle tell in Agile Software Development with Scrum (2001). A chicken suggests to a pig that they open a restaurant serving ham and eggs. The pig refuses, because the chicken would only be involved whereas he would be committed. The Scrum Guide dropped this vocabulary in 2011. Its 2020 edition states the same rule plainly: the Developers who will be doing the work are responsible for sizing it.

What velocity measures

The note Notes on Velocity is not a pattern but a clarification. A team's velocity is the sum of the estimates of the items it has finished during a Sprint. The authors draw several consequences from this.

• Velocity does not measure effort. In a stable team whose Sprints have a fixed length, effort is constant; what varies is the amount of work finished.
• It belongs to the Developers and characterizes the whole team, never an individual.
• It serves as a forecast, like a weather report: in the authors' words, "it is not a target or a guarantee for stakeholders."
• It has an average and a spread. A team that fills its Sprint according to its average velocity should expect to finish everything only one time in two. Its commitment is to the Sprint Goal, and not to the full list.
• Reducing the spread matters more than raising the average, because a steady velocity makes forecasting possible. Raising velocity does not, in any case, guarantee that the team produces more value.

Forecasting the next Sprint

Yesterday's Weather

The Yesterday's Weather pattern starts from a human failing. Out of pride, or under pressure from "stretch" goals set from outside, teams take on more work than they can finish. They then cut corners to keep up, or they disappoint. The solution fits in one sentence: "In most cases, the number of Estimation Points completed in the last Sprint is a reliable predictor of how many Estimation Points of work the team will complete in the next Sprint." The authors link this rule to J. Scott Armstrong's work on forecasting by extrapolation (1984).

Running Average Velocity

A single Sprint can be exceptional, in either direction. The Running Average Velocity pattern describes the danger of reacting to it too quickly: a one-off dip triggers emergency measures, and a chance rise becomes the required norm. The solution is to calculate velocity as the average of the most recent Sprints, three at least. The authors prefer this running average to the average of all Sprints since the beginning, which keeps the trace of abandoned practices. They point out its drawback: an abnormal Sprint weighs on several calculations, then causes a jump the other way when it leaves the window.

Updated Velocity, or recognizing a new level

The running average has a weakness for a team that is improving: it always lags behind a rising series. Conversely, a single good Sprint does not prove that progress has been made. The Updated Velocity pattern sets a rule: a new velocity is recognized only if the team holds the new level for three Sprints in a row. The authors connect this caution to the change model of the therapist Virginia Satir, in which anything new opens a period of disorder before a new balance is reached.

Let us return to the insurer's team. It automates part of its checks, and the next three Sprints give 31, 33 and 32 points. After the first, it does not change its forecasts. After the third, the new level is confirmed, and it uses 32, that is, 96 divided by 3.

With several teams

When several teams work on the same product, the Product Owner needs an overall velocity to forecast. The Aggregate Velocity pattern examines the false solutions. If each team estimates "its" items on its own scale, the figures cannot be compared, and averaging them would be like mixing miles and kilometers without knowing the conversion rate. Items also end up assigned to a team long before they are built, which takes away the Product Owner's freedom to reorder them.

The solution is for all the teams to estimate all the items together, from a common reference and a common scale. The aggregate velocity is then the sum of the teams' velocities. Three teams that finish 25, 30 and 20 points on the same scale have an aggregate velocity of 75 points, and a 300-point Product Backlog represents about four Sprints. The authors warn against the temptation to use this common scale to compare teams with one another.

The Specialized Velocities pattern deals with the case where teams have specialized by part of the product, for example a car's multimedia system and its braking. Having the multimedia team estimate the braking would be time wasted. The solution is for each specialized team to estimate and build the items in its own area, asking others for their view when needed. The pattern sets two conditions: specialization follows the product's domains and never techniques (no testing team), and no team can use it as grounds to refuse an item.

Velocity is not a target

Imagine that management asks the insurer's team to raise its velocity from 25 to 30 in the next quarter. The simplest way to get there requires no extra work: an item estimated at 3 points yesterday just needs to be given 5. Velocity goes up, the amount delivered does not change, and forecasts become useless since the unit has changed along the way. Other shortcuts exist, such as declaring done a piece of work that has not been checked.

This mechanism is described by Goodhart's law (in French): an indicator is informative as long as nobody has an interest in moving it, and it goes awry as soon as it is made a target with stakes attached. The book's authors take the same line. The note on velocity states that it is not a target. It adds that lack of effort is rarely the problem, so asking a team to "work harder" is not the solution. The Running Average Velocity pattern warns that excessive pressure leads to overtime and doctored figures.

Comparing teams by their velocity makes no more sense. A team at 50 points and another at 20 deliver as much if the first gives 5 points to what the second estimates at 2.

Limits and pitfalls

The 2020 Scrum Guide mentions neither velocity, nor points, nor planning poker. It asks only that the Developers size the work, and it notes that the more they know about their past performance, their upcoming capacity and their Definition of Done, the more confident their forecasts will be. Everything this article describes therefore belongs to complementary practices, which you can do Scrum without adopting.

Velocity adds up estimates, not measured results. Converting points into days ("one point is worth one day") cancels the benefit of the method. Yesterday's weather also assumes a stable team and Sprints of the same length: every change in membership disrupts the series.

On the evidence, some restraint is in order. The authors assert that relative estimation is more accurate than estimation in hours, but they also refer to a paper by Magne Jørgensen (2013), a researcher who specializes in estimation, whose title indicates that the result depends on what you compare with and how you compare. The accuracy of plus or minus 20% that they attribute to forecasts based on velocity is a practitioners' observation. Some teams in fact give up points and count items split to a similar size, an option the note mentions without recommending it. The article Criticism and the state of the evidence takes stock of these questions.