Dev Manager Metrics: Tracking Average Point Value to Manage Uncertainty and Risk

Story Points -The Base Metric

Story points are, functionally, an estimate of the overall effort required to implement the item. That effort can be broken down into a combination of complexity and sheer effort aka hours needed. How that breakdown can be calculated is a topic for another post. For today, let's just focus on Story Points and what they explicitly, and implicitly represent.

Explicitly, as stated above, Story Represents an estimate of the overall effort. Know what humans are terrible at? Estimation. Seriously. Well, we are kinda good at it in certain circumstances. You can read about it here, here, and here. When it comes to estimating how much effort a task will take, the more complex the task, the more we're likely to have an incorrect estimate.

And that brings us to what's implicit in Story Points: uncertainty. We are certain that we are uncertain. And we are uncertain about how much uncertainty we have. Story Points can be looked at as a measure of uncertainty. The average story point value in a sprint can be used as a measure of the uncertainty in the sprint.

The Fibonacci Sequence and Uncertainty/Risk

For teams using the Fibonacci sequence for their Story Points, each time you move up a value, the overall effort doubles. With the doubling of the effort comes an increase in the associated uncertainty. The amount of uncertainty that comes with each point value is up for debate but I'd suggest a 75% increase is not unreasonable.

Point Value	Uncertainty %	Certainty %
1	5%	95%
2	9%	91%
3	15%	85%
5	27%	73%
8	47%	53%
13	82%	18%
21	144%	0%

With a starting uncertainty of 5% for a 1-point story, we can see that by the time we hit an 8-point story, our uncertainty is almost equal to our certainty. This means an 8-point story in our sprint has almost equal odds of being completed or not. I don't know about you but I'm not big on managing my teams based on a coin flip.

A synonym for uncertainty is risk. Uncertainty in completing a story is the risk that the story won't be delivered by sprint end.

Note: These numbers are different for each team based on a number of factors. These are meant for illustrative purposes.

Tracking Uncertainty in Each Sprint

As a dev manager, if you track the average point value for each sprint, you're starting to track the uncertainty that's inherent in each sprint. From the data above, we can see that if our sprint has an average story point value of 3, we have 15% uncertainty. That's another way of saying we have a 15% risk of our sprint not completing. When you get to an average of 5 points, there's a 1 in 4 chance your sprint isn't completed.

Tracking the average point value can help you help your team manage their workload and tracking the 4-sprint average can help you see how that management is helping the team.

Reducing Risk for Your Development Teams

The number 1 technique for reducing uncertainty is the refinement process, specifically splitting large stories into smaller stories. An 8-point story may not simply get split into two 5-point stories. With a bit of creativity and some thought, it could be split into 5 2-point stories. The risk for a 2-point story to not be completed is less than 10%. I'll take a 1 in 10 chance that a 2-point story doesn't get delivered versus a 50% chance an 8-pointer doesn't every day of the week!

Managing the scope of a sprint is another way to manage uncertainty. Four 5-point stories are not the equivalent of 10 2-point stories despite both totaling 20 story points. If the team has to pull in larger stories, reducing the sprint velocity is a valid way to manage the risk associated with the higher story points.

Dev Team A's Journey

We've met Dev Team A before. They got a new manager at Sprint 4 who started them doing some things differently.

Here we can see where Dev Team A was running with a consistently high average point value. They were taking in multiple 8-point stories and struggling to complete them. At Sprint 4, their new dev manager suggested they start taking on smaller stories and splitting the larger stories. As they got better at splitting and refining stories, their average point value dropped.

When their manager looks at their 4-sprint average for their average point value, he can see that, so far, they're trending down towards around 2.5. This represents ~12% uncertainty in the sprint. While he's looking to get that number down to around 2, he's happy it's down from 3.8, which equates to around 20% uncertainty.

Conclusion

As a dev manager, you can never remove uncertainty from your sprints. Things happen that are beyond your control. Your team may discover something while working on a small story that blows everything up. It happens. You can, however, measure and track the amount of uncertainty your teams are bringing into each sprint and gain some control around the risk to each story, the sprints, and the projects.