Planning and Estimation
Transcript
Made automatically from the video's sound, so it may contain mistakes. Click a line to play from there.
Welcome back to The Explainer. In our last session, we looked at how Scrum, Kanban, and XP balance cadence, work in progress flow, and technical discipline. Well, today we are tackling what is arguably the ultimate friction point for software teams, planning and the estimation.
We're gonna put textbook theory right up against real world practice and separate the actual empirical research from practitioner dogma. So first up, let's talk setting the scene. To do that, we're gonna cover four main areas.
We'll start with the story point trap, then look at visualizing true flow, shift over to forecasting instead of guessing, and wrap up with outcome-based roadmaps. Alright, part one, the story point trap. Let's look at how incredibly good intentions from early agile theory went terribly wrong when put into corporate practice.
What's absolutely fascinating here is this expert confession from Ron Jeffries, the actual inventor of story points. He essentially admits they were designed to hide time from management. You see, back in the early extreme programming days, teams estimated in ideal days, but stakeholders just couldn't grasp why 50 ideal days of work couldn't be squeezed into a three-week calendar block, so they changed the term to points simply to stop the arguments.
It was a defensive move, not some scientific measurement of complex work. And this perfectly illustrates how Martin Fowler's original concept of velocity got corrupted. It was meant to be a simple localized capacity gauge based on the idea of yesterday's weather, meaning whatever we got done last week is roughly what will get done next week.
Simple, right? But in the corporate world, this internal team gauge was weaponized. It turned into a cross team management KPI. Management started comparing team A's velocity directly to team B's, basically treating abstract relative points like a universal currency.
So what happens then? Well, El Yahoo Goldrat hit the nail on the head. Tell me how you measure me, and I will tell you how I will behave. When management turns abstract point guessing into a formal KPI, you trigger a classic Goodhart's law pathology.
You get point inflation. If a developer's bonus or performance review is tied to hitting a certain number of points, they're naturally going to inflate the estimates. So you end up with hours wasted in planning poker, debating Fibonacci numbers, producing thrown estimates that suddenly harden into rigid deadlines, all just to feed a completely dysfunctional metric.
Visualizing true flow. This is the empirical antidote to all that estimation waste we just talked about. Instead of assigning developers multiple tasks at once and burning them out, empirical Dora research shows us a much better path.
We actually have to limit our active inventory. In tech, our inventory is invisible. If you don't limit work in progress, tasks just pile up out of sight. People constantly context switch, work takes longer to finish, and quality drops.
By setting strict whip limits, we prioritize completing current work before anyone is allowed to pull in anything new. The mantra here is stop starting and start finishing. To actually fix flow, we have to map out the value stream and expose the hidden barriers causing downstream rework.
Step one is tracking lead time, which is how long it takes from a request to actual delivery. Step two is process time, the actual time spent working on it without interruptions. And finally, we measure percent complete and accurate, telling us how often a piece of work moves downstream without needing to be kicked back for corrections.
This is exactly how we expose the real bottlenecks. By focusing on these four core metrics from the Kanban Guide and Scrum.org, teams can shift completely away from abstract points and look at actual system health. Work in progress tells us our current load.
Throughput is how many items cross the finish line. Cycle time tells us how fast they move. But crucially, look at work item age. This should be inspected daily.
We look at this in our stand-ups to catch items before they stall out in the pipeline, rather than waiting for a sprint review to realise we failed. And this is where visualizing flow lets teams practically apply Little's law. A cumulative flow diagram shows you exactly where your work is piling up.
If your in testing band is getting wider and wider, you've got a hidden cue. You've got blocked work. You don't need better estimates to fix that. You need to swarm that testing bottleneck and clear it out.
Let's move on to part three. Forecasting over guessing. It is absolutely time to move away from deterministic guessing and shift towards statistical reality.
Alan Hollop's brilliant pile of blocks metaphor really changes the game here. On the left, you see teams wasting hours debating whether a stories are five or an eight on the Fibonacci scale. But on the right, we use story counting, an idea championed by Martin Fowler.
If you use Bill Wake's invest criteria to slice your work into roughly equal small pieces, say no more than a couple of days' work each, you don't need points at all. You just count the stories. A pile of different sized blocks reaches the exact same height as a pile of average size blocks.
Counting similarly sliced stories works just as well, if not better, than guessing arbitrary numbers. Now, a standard burn down chart assumes a naive, linear rate of progress. But let's be honest, software development is never a straight line.
That's where Monte Carlo probabilistic forecasting comes in. It takes a team's actual historical daily throughput and runs thousands of random simulations. It doesn't ask how big is this item, it asks, based on our historical system data, what are the odds we finish these 25 items by a specific date?
This kind of statistical forecasting replaces those hard, anxiety-inducing deadlines with explicit service level expectations and confidence ranges. Instead of a manager demanding a rigid date based on a guessed velocity, the team can say, based on our data, we are 50% confident we can deliver this in five days, and 85% confident we can deliver it in eight days.
It completely changes the conversation from a commitment trap into a very mature discussion about risk management. But what about the nature of the work itself? Because numerical estimates fail to capture uncertainty, this shapeup hill chart metaphor shifts our focus entirely from counting what is not done to discovering what is actually solved.
The left side of the hill is the uphill battle of figuring out what we are even doing, the unknowns. The peak is when we finally solved the core problems. The downhill slope is just execution.
You can't estimate the uphill part accurately because you haven't solved it yet. This visual helps everyone see exactly where the real risk lies. Let's wrap up with part four, outcome based roadmaps. We're gonna scale this up from the team level to macro level product strategy.
Marty Kagan from the Silicon Valley Product Group strongly warns us that typical Gantt roadmaps are purely output focused. They just dictate features and dates. But recent empirical frameworks like Space and DevX prove that developer productivity thrives on solving actual problems, not just typing out predetermined code.
So instead, an outcome roadmap, using something like Jan Abasto's Now Next Later Horizon model, focuses on the actual business value we are trying to achieve rather than fixating on rigid timelines. To break free from being just another feature factory, management must prioritize results over upfront feature lists.
Empowered teams need deep context. Think about Scrum.org's evidence based management. When teams understand the why, the product vision and the prioritised business objectives, they can figure out the what and the how much better than a stakeholder trying to guess what feature might work six months from now.
And when dates are truly necessary, the team makes high integrity commitments based on actual discovery. General Gorge Patton once said, don't tell people what to do, tell them what you need accomplished, and you'll be amazed at the results.
This timeless quote perfectly captures a truly agile, outcome driven product culture. If management acts as a command and control center, dictating a roadmap of features, they strip the team of their ability to innovate.
Tell the engineers the problem you need solved and let them use their expertise to actually solve it. So I'll leave you with this provocative thought to consider. Take a hard look at your metrics, your estimates, and your roadmaps.
Are you just trying to build a better deterministic guessing machine in a feature factory? Or are you building a system optimized for empirical flow and real business value? Think on that.
Next time on The Explainer, we will tackle DevOps and continuous delivery. See ya then.