Planning and Estimation
Podcast · Planning and Estimation

Why Story Points Failed Software Teams

by ZumiLabs · 48m
0:00 48m

Transcript

Made automatically from the audio, so it may contain mistakes. Click a line to play from there.

Host 1

Picture this. You are trapped in a two-hour refinement call on a Tuesday afternoon.

Host 2

Always a Tuesday.

Host 1

Always. And you are just staring at this grid of exhausted faces on your screen. Like these are eight brilliant engineers.

Host 2

People who can architect highly complex systems in their sleep.

Host 1

Right. Geographically distributed systems. But right now, they are agonizing over a single Jira ticket.

Host 2

Just absolutely agonizing over it.

Host 1

I mean, they are debating whether this highly ambiguous, poorly defined ticket is a three or a five on the Fibonacci scale. The tension is just palpable.

Host 2

Oh, I've been there. The back end engineer is arguing it should be a three, right? Because the database schema is already in place.

Host 1

Exactly. But then the front end engineer pushes back and insists it's an eight because, well, the UI components don't even exist yet, and the design system needs a massive overhaul.

Host 2

So what happens? They inevitably just split the difference. They compromise on a five purely to end the conversation and get off the call.

Host 1

Yeah, just to escape the meaty. And meanwhile, lurking in the background of this call, off camera and on mute, is a VP of engineering.

Host 2

Oh no. Oh yeah.

Host 1

And what are they doing? They are quietly updating a massive Excel spreadsheet to track the team's velocity so they can report upward to the board on Thursday morning.

Host 2

If you are listening to this right now, your blood pressure probably just spiked.

Host 1

I don't mind it. Because you have lived this exact scenario. You sit there thinking, is this really the best use of our collective brain power? Like arguing over imaginary numbers.

Host 2

It really is wild when you step back and look at it.

Host 1

But here is the ultimate irony, the absolute kicker behind this entire familiar scene. Ron Jeffries, the man widely credited with inventing story points in the first place, has publicly apologized for them.

Host 2

Yes. He has gone on record saying he is deeply sorry he ever came up with the concept.

Host 1

Which serves as a really fascinating plot twist in the history of software engineering.

Host 2

It really does. And to understand why he apologized, you know, you really have to understand why he invented them to begin with.

Host 1

Right. Let's go back to the origins.

Host 2

Back in the early days of extreme programming, teams were estimating work in what they called ideal days.

Host 1

Ideal days.

Host 2

Yeah. Essentially, this was a measure of how long a task would take if absolutely no one interrupted you, like zero interruptions.

Host 1

No meetings, no slack messages, no production fires to put out, just pure unadulterated coding time.

Host 2

Exactly. But stakeholders and managers just couldn't wrap their heads around the fact that an ideal day might actually take, you know, three actual calendar days to complete in the real world.

Host 1

Because they just latched onto the word days.

Host 2

Right. They would look at an estimate of five ideal days and demand the feature be shipped by Friday.

Host 1

That it's just human nature, honestly. If you give a manager a unit of time, they immediately map it to a calendar.

Host 2

They absolutely do. So Jeffries and his colleagues decided they needed to change the unit. They stopped calling them days and started calling them points.

Host 1

Points.

Host 2

Yeah. The entire original purpose of Story Points was to obfuscate duration. It was a deliberate shield.

Host 1

So they were invented specifically so that managers wouldn't pressure teams over time estimates.

Host 2

Exactly. It was a way for developers to say, this is the relative size of the effort without handing a weapon to management to use for arbitrary deadlines.

Host 1

Wow. And the tragic irony is that those points became the exact weapon of micromanagement they were designed to prevent.

Host 2

Precisely. That VP with the spreadsheet tracking velocity as a performance metric, that is the nightmare scenario Jeffries was trying to avoid.

Host 1

It is a classic case of a tool being stripped from its original context and just turned into a mechanism for corporate control.

Host 2

It is the modern software development ritual. I mean we call it the Fibonacci Poker Circus.

Host 1

The Fibonacci Poker Circus. I love that. Well, let's step out of the circus tent for a bit. Welcome to Deep Dive Episode Three of our series Ways of Working from Agile to AI Native. I am so glad you're with us today.

Host 2

It is great to be here.

Host 1

Now, in our last deep dive, we mapped out the fundamental mechanics of Scrum, Kanban, and XP. The boards, the daily rhythms, and the roles that get code out the door. So we aren't going to rehash basic scrum roles or how continuous integration works today.

Host 2

Right. We're moving past that.

Host 1

Today, we are looking at the messy, emotionally charged realities of planning and estimation. We have a massive stack of sources to guide us through this.

Host 2

We really do. We are drawing on the wisdom of Martin Fowler, the provocative arguments of Alan Hollib and the hashtag no estimates movement, Marty Kagan's insights from the Silicon Valley Product Group, Basecamp's ShapeUp methodology, and hard empirical research from Google Cloud's Dora team.

Host 1

Plus the space in DevX frameworks. It is a lot of ground to cover.

Host 2

Huge amount of ground.

Host 1

So our mission today is to unpack story points, the widespread misuse of flow metrics, work in progress limits, and how to build modern product roadmaps that actually balance a developer's need for autonomy with the executives' desperate need for predictability.

Host 2

We will cover the mechanics and real world practices, the anti-patterns backed by evidence, and of course, give you pragmatic team takeaways.

Host 1

Okay, let's unpack this. Why do we even estimate in the first place if it causes so much pain and dysfunction? Like, are we just gluttons for punishment?

Host 2

It's a completely fair question. Especially when you look at the abysmal track record of software estimates.

Host 1

Right. They are notoriously bad.

Host 2

They really are. But Martin Fowler has a great piece called Purpose of Estimation, and he makes a crucial foundational point. Estimation is not inherently evil.

Host 1

Okay, so it's not just corporate torture designed to make developers miserable.

Host 2

No, surprisingly not.

Host 1

Okay.

Host 2

But it is valuable only when it informs a significant business decision.

Host 1

A significant decision.

Host 2

Meaning if the estimate doesn't physically change the action we are about to take, we shouldn't be wasting time calculating it.

Host 1

Exactly.

Host 2

Fowler gives two main examples of when an estimate is genuinely useful. The first is resource allocation at a macro level.

Host 1

Macro level. Okay, like what?

Host 2

Well, a company has a fixed amount of money and a fixed number of people, right? But they have ten great ideas. To decide whether to fund project A or Project B, executive leadership needs some rough sense of the cost versus the benefit. That makes sense. Yeah, you simply cannot make a strategic bet if you don't know if the bet costs ten thousand dollars or ten million dollars.

Host 1

That makes total business sense. I mean, I can't walk into a car dealership and buy a car without knowing if it's $30,000 or $300,000. The estimate changes my decision to purchase.

Host 2

Right.

Host 1

So what is the second scenario?

Host 2

The second is coordination. This happens all the time in complex multi-team systems. Say team A is building a brand new checkout flow for an e-commerce platform.

Host 1

Okay.

Host 2

But they absolutely cannot launch it until team B finishes building a new payment gateway API.

Host 1

Well, dependencies.

Host 2

Exactly. If team B estimates that their API will take roughly two months, and team A estimates the checkout flow will only take one month, team A knows they shouldn't start working on the checkout flow today.

Host 1

Because they finish early and just be sitting around.

Host 2

Right. They should go build something else for a month so they aren't just sitting around waiting for the API to be ready.

Host 1

Let's play that out for the listener because that is a very real pain point. If team A starts too early, they finish their part and then the code just sits there, it goes stale.

Host 2

And they have to context switch away.

Host 1

Yeah, and then in two months, when the API is finally ready, they have to refamiliarize themselves with their own code base. So in that specific scenario, spending a day estimating the rough timelines saves the company a massive amount of wasted effort.

Host 2

It creates boundaries for coordination.

Host 1

But notice what Fowler is talking about here. He is talking about broad macro level estimates.

Host 2

Right, not the micro stuff.

Host 1

Right, not the micro stuff.

Host 2

Exactly. He is not talking about the Tuesday refinement call where a team is estimating every single tiny ticket. And he points out this massive psychological trap we fall into with estimates, which is deeply rooted in behavioral economics.

Host 1

Loss aversion.

Host 2

Yes.

Host 1

I see the behavioral economics angle here. Humans are naturally optimistic creatures, and software developers might be the most optimistic of all.

Host 2

Oh, without a doubt.

Host 1

When you ask a developer how long something will take, they envision a world where they write perfect code on the first try, the servers never crash, the third party API documentation is entirely accurate, and the product manager never ever changes their mind.

Host 2

That optimism bias is exactly why our estimates are almost always too low. We estimate for the happy path.

Host 1

And the happy path rarely exists in software engineering.

Host 2

Almost never. When reality inevitably hits, you know, when you discover a legacy database issue that takes three days to unravel, or a dependency conflict that completely breaks the build, the estimate is missed.

Host 1

And this is where loss aversion kicks in. In psychology, loss aversion means that the pain of losing something is psychologically twice as powerful as the pleasure of gaining something. Right. So when an estimate is set, it creates an anchor. It sets an expectation in the minds of management.

Host 2

And when the team misses that estimate and the schedule slips, management perceives that delay as a painful, tangible loss. Even though the estimate was just a guess made in a clean shirt before anyone wrote a single line of code, the human brain doesn't care. It feels like money or time was stolen from them.

Host 1

This triggers anxiety for the stakeholders. Which in turn triggers pressure. Management comes down on the team demanding to know why they are late on a date they literally made up.

Host 2

Exactly.

Host 1

The developers feel the heat, they feel attacked. So what happens next, Print? To protect themselves, the developers pad their estimates.

Host 2

Always. A ticket that should be a three becomes an eight.

Host 1

Yep. Management sees these inflated estimates, they push back, and you end up in this vicious low-trust cycle of negotiation where everyone is lying to each other about the numbers.

Host 2

Which perfectly sets up a brilliant analogy Fowler uses to describe this phenomenon. He says, plans and estimates are like a lettuce.

Host 1

A head of lettuce. Yes. Think about your own refrigerator. You buy a beautiful crisp head of lettuce on Sunday. If you use it on Monday or Tuesday, it's great.

Host 2

Very crisp.

Host 1

But if you look at it after a week, it's getting rather wilted and sad. And if you leave it in the CRISPR dwarf for a couple of months, it is completely unrecognizable slime.

Host 2

It is a highly visceral, accurate way to describe the shelf life of a software estimate. If you estimate what you are doing tomorrow, it's crisp. You know what you're doing. Right. If you estimate what you're doing in six months, it's slime. The underlying conditions will have changed so drastically that the estimate is not just useless, it is actively misleading.

Host 1

So we started with ideal days multiply by three, then we changed the name to story points to hide the time for managers. Managers figured it out anyway and started tracking velocity.

Host 2

And now we are left holding wilted lettuce.

Host 1

Exactly. But wait, if points are broken and estimates degrade into slime so quickly, how do we predict the future? Businesses still require some level of forecasting.

Host 2

They do.

Host 1

This brings us to Alan Holeb and the hashtag no estimates movement. Let's break down Holeb's stance, because on the surface, it sounds like he is telling developers to just walk into the CEO's office, slam their laptops on the desk, and refuse to estimate anything ever again.

Host 2

It sounds incredibly radical, I know. But Holeb's argument is rooted in a fundamental truth about the nature of software engineering that most traditional management frameworks completely ignore.

Host 1

Okay, what's the truth?

Host 2

He makes a distinction that is vital for anyone leading a technology team to understand. Unless you are building the exact same website for the exact same client with the exact same technology stack you've used before, everything in software development is an act of invention.

Host 1

Invention, not construction.

Host 2

Exactly.

Host 1

We live to use the metaphor of building a house in this industry. Like we need a strong foundation, or we are architecting the system. When you build a physical house, you have blueprints. You know exactly how many bricks, how many feet of copper wire, and how many hours of labor it will take.

Host 2

It is a highly constrained, predictable environment.

Host 1

Right. But building software is nothing like building a house. You do not actually know what you are building until you start building it.

Host 2

Because the requirements change the moment the user sees the first working prototype and realizes it's not what they actually wanted. The technology behaves in ways you didn't expect.

Host 1

Exactly. Because if it was a solved problem, you wouldn't be paying engineers to write custom code. You would just buy an off-the-shelf SaaS product. The very fact that you are writing custom code means you are traversing unknown territory.

Host 2

Therefore, Hollib argues that because software is invention, upfront, long-term estimation is nothing more than guesswork. And making rigid business plans based on guesswork doesn't just create the dysfunction and pressure we talked about. It is actively negligent.

Host 1

Negligent? That's a strong word.

Host 2

He views estimates as a form of waste in the lean manufacturing sense. We spend countless hours arguing in refinement meetings over numbers that we know will be wrong the second a developer opens their IDE.

Host 1

Let me push back on Hollip here. I hear the logic, but I can hear every project manager and executive listening right now screaming at their speakers.

Host 2

Oh, I'm sure they are.

Host 1

You cannot run a publicly traded company or a startup with a burn rate on vibes. If we aren't spending hours arguing over whether a ticket is a three or a five, what do we actually do in these planning meetings? How do we break the work down so developers know what to type today?

Host 2

If we abandon relative pointing, the alternative is focusing incredibly heavily on how we slice the work.

Host 1

Slicing. Okay.

Host 2

And this brings us to a concept many listeners will know, but few execute well. The Agile Alliances Invest criteria for user stories. This remains the gold standard for what a good piece of work looks like before a developer touches it.

Host 1

Let's do a quick refresher for the listener. Independent, negotiable, valuable, estimable, small, and testable. Invest.

Host 2

While they are all critical for a healthy backlog, for our specific goal of ditching story points, we need to hyperfocus on the letters S and T, small and testable.

Host 1

Small and testable.

Host 2

Ron Jeffries, our friend who apologized for story points, and another Agile practitioner named Neil Killick propose an ultimate trick for sizing work that completely bypasses the Fibonacci sequence.

Host 1

Let's hear the trick. I'm ready.

Host 2

Do not estimate the size of a story. Do not assign a number to it. Instead, focus all that intellectual energy in your refinement meeting on slicing the story down until it is so small that it only requires a single acceptance test to prove it works.

Host 1

A single acceptance test. That is tiny. Let's ground this in a concrete example so the listener can see the mechanics of this. Let's invent a ticket.

Host 2

Okay, let's hear it.

Host 1

A junior product manager creates a ticket that says, as a user, I want to manage my user profile. In a traditional scrum team, they might look at that and call it an eight point ticket. How do we apply this slicing trick?

Host 2

First, we look at manage my user profile and realize it is a massive, multifaceted desire. It includes uploading an avatar, changing a password, updating an email, and writing a bio. So the first slice might be as a user, I want to edit my display name.

Host 1

Okay, is that testable? Yes. Is it small? It feels smaller. But is it a single acceptance test? Like what if they enter special characters? What if they try to use an existing name? What if the database rejects the length?

Host 2

You're hitting the exact friction point. It's still too ambiguous. So we slice it again. The new story becomes as a logged in user, I can save a plain text display name under 50 characters and see a visual success message.

Host 1

And that is an atomic unit of value. I can write one automated test for that. I log in, I type John Doe, I hit save, I check for the success banner, done.

Host 2

Exactly. It forces you to strip away all the nice to have's and focus on the absolute core requirement. You stop debating complexity and start debating scope.

Host 1

I use the elephant analogy for this. If someone asks you to estimate exactly how much an adult elephant weighs, you are going to guess wrong. And you are going to spend hours debating the bone density of elephants with your colleagues.

Host 2

Right, which gets you nowhere.

Host 1

Exactly. Stop trying to guess the weight. If your goal is to eat the elephant, just focus on taking one consistent bite-sized piece at a time.

Host 2

And if you can train your team to do that consistently, something magical happens to your metrics. You transition from estimating into Fowler's concept of story counting.

Host 1

Story counting. This is where we throw away the crystal ball and start looking at empirical data. If we aren't using points, we are just counting the tickets.

Host 2

Yes. Fowler points out that if a team gets rigorous about the invest criteria and they are consistently slicing their stories to be roughly within the same order of magnitude, meaning they are all generally small, the law of averages takes over.

Host 1

Walk me through the math on that. How does the law of averages help here?

Host 2

You do not need to assign Fibonacci numbers because over the course of a few weeks, the stories that end up being slightly bigger and the stories that end up being slightly smaller will statistically cancel each other out.

Host 1

Oh, I see.

Host 2

Let's say you have ten sliced stories. Maybe two of them end up taking three days because of an unforfine bug. Maybe two of them take only half a day because the logic was simpler than expected. The rest take about a day. Over a large enough sample size, the average regresses to the mean.

Host 1

Statistics to the rescue. So instead of arguing if a ticket is a three or a five, you simply look at your historical data. You count how many of these uniformly sliced stories the team finishes per week. In flow metrics, this is called your throughput.

Host 2

Right.

Host 1

Let's put a scenario together. Over the last month, a team completes an average of eight stories a week. Their throughput is eight.

Host 2

Now the business comes to you and says, We have this massive new epic for the Q3 launch, and we've collaboratively sliced it down into 40 small stories. When will it be done?

Host 1

Right. The classic question.

Host 2

You do not need to sit in a room and estimate each of those 40 stories. You simply divide 40 by your historical throughput of eight, and you can say with mathematical confidence, it will take us about five weeks.

Host 1

That is incredibly liberating for an engineering manager. I mean, think of the hours saved. No more poker cards, no more exhausting debates about whether complexity is the same as effort. Just slice it small and count the throughput.

Host 2

It removes the emotion and the ego from the equation. It is just empirical data. You are forecasting based on actual historical performance, not optimistic guesses.

Host 1

Well, let's complicate this though. Okay. Because even if a story is sliced small, there is still the underlying nature of the work itself. Throughput assumes all work carries a somewhat similar risk profile.

Host 2

Yes, it does assume that.

Host 1

But Basecamp's methodology called ShapeUp has a really compelling counter-argument. They argue that numerical estimates, whether you're using points, hours, or even story counting, fail entirely because they do not capture technical uncertainty.

Host 2

This is a vital nuance. ShapeUp notes that a four-hour task you have done ten times before is a wildly different beast than a four-hour task involving a brand new framework you have never touched.

Host 1

Yeah, if I'm building a basic login form for the hundredth time, four hours is four hours. My throughput metric holds up.

Host 2

Right.

Host 1

But if I'm integrating a bleeding edge AI library that has zero community documentation, four hours could easily turn into four days of crying at my keyboard because of a dependency conflict. A number or a single ticket count does not communicate that risk to management.

Host 2

To solve this communication gap, ShapeUp introduces a brilliant visual metaphor called the Hill chart.

Host 1

The Hill metaphor completely changed how I view daily stand-ups. Imagine a bell curve or a literal hill. Every single piece of work you ever do has two distinct phases.

Host 2

Right.

Host 1

First, you have the uphill phase. You are at the bottom left, looking up at the peak. This is where you are figuring out the approach, battling the unknowns, reading the API documentation, and realizing the database isn't structured the way you thought it was. It's the I have no idea what I'm doing yet phase.

Host 2

This is where all the risk lives. This is where projects go to die. It is pure discovery.

Host 1

You are pushing the boulder uphill. But then eventually you reach the peak. You have the aha moment. You know exactly what files need to change, you know exactly what the logic needs to be, and you've verified the API works. From the peak, you look down the other side. That is the downhill phase.

Host 2

The downhill phase is pure execution. The unknowns are entirely eliminated. It is just typing, testing, and deploying. It's just a matter of putting in the hours.

Host 1

So mechanically, how does a team use this hill chart to replace traditional status updates?

Host 2

Instead of a manager coming to a developer and nagging, are you 50% done with that five-point ticket? A question which means absolutely nothing, by the way.

Host 1

Right, because 50% of the code could be written, but if you haven't solved the core architectural problem yet, you are zero percent done in reality.

Host 2

Exactly. So instead of percentages, the team looks at a visual hill chart. The developer plots a dot representing their current task on the hill. If the dot is placed on the uphill slope, everyone in the organization instantly knows this work is still highly uncertain. We are still figuring it out. Do not expect a delivery date yet.

Host 1

And if the dot is on the downhill slope.

Host 2

Then everyone knows it's just a matter of time. The risk is gone.

Host 1

It provides status without the nagging. But more importantly, it solves a massive psychological hurdle on engineering teams. Developers are notoriously terrified to admit, I don't know how to solve this.

Host 2

Oh, absolutely terrified.

Host 1

They will sit at their desks spinning their wheels in silence for days rather than ask for help because they feel like they should know the answer.

Host 2

The uphill phase normalizes not knowing. It builds it directly into the culture. It acknowledges that the first half of software development is being confused and discovering the path.

Host 1

Which is huge for psychological safety.

Host 2

By visualizing that a task is stuck on the uphill side for three days, senior engineers or technical leads can immediately see who needs help getting over the hump without the developer having to raise a red flag and feel incompetent.

Host 1

And it fundamentally changes how teams sequence their work. We talked about the inverted pyramid earlier.

Host 2

ShapeUp advises teams to tackle the biggest unknowns first. You sequence the backlog so that you are getting the scariest, most ambiguous tasks over the hill at the very beginning of your sprint or cycle.

Host 1

You do not leave the risky database migration for the week before launch.

Host 2

Never. You push the biggest boulders over the peak first, leaving only the predictable downhill execution for the end of the cycle.

Host 1

Which brings up another mind-bending concept from the ShapeUp methodology. If we aren't asking developers how long will this take, what question do we ask the business? They introduce the concept of the appetite.

Host 2

The appetite is a complete inversion of traditional planning. An estimate asks the engineering team, how long will it take to build the specific list of features? An appetite turns to the business stakeholders and asks, how much time do we want to spend on this problem?

Host 1

How much is this worth to us? That completely reframes the conversation.

Host 2

It does. If the business decides a new reporting dashboard is only worth two weeks of investment, the appetite is two weeks. This flips the dynamic to fixed time variable scope.

Host 1

Fixed time variable scope. So the deadline is a hard, unmoving constraint, but what we actually deliver within that deadline is entirely flexible.

Host 2

The team starts building. At week four, they hit a massive technical snag.

Host 1

Okay.

Host 2

In a traditional estimate-driven model, the project manager pushes the deadline back to week eight. In an appetite-driven model, the deadline remains week six. Instead of extending time, the team is empowered to aggressively hammer down the scope.

Host 1

They cut things out.

Host 2

Yes. They negotiate with the business to cut a tertiary feature, ensuring that the core value, the absolute essence of the problem is still delivered within the original six weeks.

Host 1

It forces ruthless prioritization. You cannot keep adding scope creep if the appetite is fixed. Okay, so we've established how to plan without points in estimates. We slice things small with invest. We use historical throughput for story counting. We use Hill charts to communicate uncertainty, and we use appetites to fix our time boxes. Right. But this leads us directly into the dark side of this deep dive.

Host 2

A dark side of metrics.

Host 1

Because even if a team adopts everything we just talked about, executive leadership loves a KPI. They love a key performance indicator. And when they take these agile flow metrics and turn them into performance targets for bonuses or reviews, things go off the rails incredibly fast.

Host 2

This brings us back to Alan Hollib's critique. He emphatically states that velocity or throughput is a gauge, not a control.

Host 1

A gauge, not a control.

Host 2

The thermometer analogy is perfect here.

Host 1

A thermometer tells you the temperature of the room. It gives you neutral empirical information. If you take a lighter and hold it under the thermometer to make the mercury rise, you haven't actually made the room any warmer. You've just broken the thermometer.

Host 2

That is exactly what happens when management says, our team velocity was 40 points last sprint. I need you to hit 50 points this sprint, and your quarterly bonuses are tied to hitting that metric.

Host 1

And what do developers do?

Host 2

They are highly intelligent, rational actors operating within the system you designed. They will simply inflate their estimates. A ticket that was a three yesterday is suddenly a five today. A five becomes an eight. The total points go up, the spreadsheet looks green, the bonus is secured, but not one single extra line of valuable code was shipped to the user.

Host 1

There is a famous economic principle at play here. Goodhart's law.

Host 2

Goodhart's law states that when a measure becomes a target, it ceases to be a good measure. And Iliahu Goldrat, the father of the theory of constraints, phrased it even more bluntly when he said, Tell me how you measure me, and I will tell you how I will behave.

Host 1

If you measure me by points, I will give you points. Let's look at the actual empirical research on this because it is damning. The ACMQ published research around the space framework in 2021, and later the DevX framework in 2023.

Host 2

Right.

Host 1

These frameworks were designed specifically to counteract the toxic ways we measure developer productivity.

Host 2

The space framework explicitly warns against using simplistic metrics like commit counts, lines of code, or velocity as productivity KPIs. We've seen the disastrous outcomes historically. When IBM measured productivity by lines of code, developers intentionally wrote massive, bloated, inefficient programs because they were quite literally being paid by the line.

Host 1

When speed or commit count becomes the target, the destructive outcomes are immediate. Developers churn out high quantities of low value code. They skip writing unit tests.

Host 2

Because tests slow you down.

Host 1

Exactly. They abandon practices like test-driven development or TDD. Because writing robust tests takes time and they need to ship points. They incur massive technical debt, which eventually slows the entire company to a crawl, all to satisfy a spreadsheet metric in the short term.

Host 2

Which is why we need to look at empirical evidence of what actually works to improve flow, rather than just pressuring people to type faster. And there is no heavier artillery in the software world than the Dora research from Google Cloud.

Host 1

Dora, the DevOps research and assessment team. These researchers have spent years surveying tens of thousands of engineering professionals to find out what actually drives high-performing technology organizations. And they have a lot to say about how work physically flows through a system.

Host 2

One of their most critical findings relates to a natural but incredibly flawed instinct that untrained or rookie managers have when a project falls behind.

Host 1

Let's paint the scene. A project is delayed. There's a massive backlog of work. So the manager looks at their team and says, Okay, Alice, I know you're deep in the payment gateway code, but I also need you to start investigating the user profile bug. Oh, and can you also jump on this architectural review for next quarter? Just multitask. We need to increase our overall utilization.

Host 2

It seems incredibly logical to an untrained manager. If everyone is working on three things at once, everyone is busy. And if everyone is busy, more things must be getting done.

Host 1

But the Dora research proves that is completely false.

Host 2

It proves that this approach is disastrous. Forcing developers to context switch between multiple tasks actually drastically lengthens lead times and burns teams out. You are destroying the system's flow.

Host 1

Let's use an analogy hall of references, which he borrowed from Goldrat's theory of constraints. Imagine three hikers on a narrow mountain trail. They are walking single file.

Host 2

Okay, picture them.

Host 1

The product, the software we are building, is not delivered until the very last hiker crosses the finish line.

Host 2

The system is only as fast as its constraint.

Host 1

So imagine the hiker in the middle of the line is our developer, Alice. Alice is multitasking. She's jogging forward to close a gap, stopping, jogging backward to help the hiker behind her, jumping up and down, expending massive amounts of energy. She is incredibly busy and highly utilized. But does her busyness make the final hiker cross the finish line any faster?

Host 2

No. Because they are on a narrow path. Her bouncing around is just wasted energy. In fact, her erratic movement might cause the person behind her to trip.

Host 1

That is exactly what context switching is in software engineering. You are burning massive amounts of cognitive energy, unloading one problem set from your brain and loading another. You feel exhausted, but the overall flow of value to the customer hasn't sped up by a single second.

Host 2

So what is the empirical solution? Dora is very clear on this. Strict work in process or whip limits.

Host 1

Whip limits. Let's break down the mechanics of how a whip limit physically works on a team.

Host 2

Think of a standard Kanban board. You have columns like to do, in development, in testing, done. A whip limit is a hard, non negotiable cap on the number of items allowed in a specific column at any given time. Let's say you have a team of four developers. You set the whip limit for the in development column to three.

Host 1

Okay. So a maximum of three tickets can physically exist in that column.

Host 2

Now Alice finishes coding her ticket. She moves it to in testing. There are now only T tickets in development. The system has an open slot.

Host 1

Makes sense.

Host 2

Meanwhile, Bob, who is also developing a feature, hits a wall. He needs an answer from the database administration team, and they won't reply until tomorrow. Bob is blocked. In a traditional system without whip limits, Bob would just pull a brand new ticket from the to-do column to stay busy.

Host 1

He becomes the middle hiker jumping up and down. He starts a second task.

Host 2

But with a whip limit, let's say the column is already full because two other developers pulled tickets. Bob cannot pull new work. The board physically forbids it. He is forced to stop.

Host 1

I can hear the executives panicking.

Host 2

Like, what does Bob do?

Host 1

Did you just sit there and read Reddit on company time?

Host 2

This is the genius of the whip limit. He doesn't sit idle, he is forced to look downstream. He sees Alice's ticket sitting in in testing. He reaches out and says, Hey Alice, I'm blocked on my feature. Can I help you write test automation for your ticket so we can move it to done?

Host 1

It forces collaboration instead of isolation.

Host 2

It forces the team to adopt the mantra stop starting, start finishing. By artificially capping capacity, you instantly expose the bottlenecks in your system. If the in testing column is always hitting its whip limit, it becomes glaringly obvious that QA is the bottleneck, or that developers are throwing poor quality code over the wall.

Host 1

This connects directly to a broader lean practice that Dora strongly recommends for exposing these hidden delays. Value stream mapping, or VSM.

Host 2

Value stream mapping is an exercise where you gather every single person involved in the delivery process, from the business stakeholder who had the original idea to the developer, to the QA tester, to the ops person who deploys it. You physically map out on a whiteboard every single step the work takes.

Host 1

And as you map those steps, you measure three crucial flow metrics for each step.

Host 2

Lead time, process time, and percent complete and accurate, or percent CA.

Host 1

Let's define those because they reveal so much hidden waste. Lead time is the total elapsed time a ticket sits in a specific state, from the moment it arrives to the moment it leaves. Process time is the actual time a human being spent actively working on it.

Host 2

The gap between lead time and process time is usually shocking to organizations. A ticket might sit in a ready for QAQ for three days. That is your lead time. But it only takes the QA engineer 30 minutes to actually run the tests. That is your process time. Wow. Value stream mapping exposes that you don't need developers to type faster. You need to eliminate the three days of wait time.

Host 1

But the real diagnostic tool is that third metric, percent complete and accurate. Let's say a team is coding incredibly fast. Their throughput is through the roof, but QA keeps finding bugs and kicking the code back. How does value stream mapping expose that dysfunction?

Host 2

That is exactly what percent CA measures. It represents the percentage of time a downstream step receives work that it can process without having to send it back upstream for rework or clarification.

Host 1

So if QA receives 10 tickets and kicks five of them back to development because they fail basic tests, the percent CA for the development step is 50%.

Host 2

Yes. And this proves mathematically why tracking just velocity is so deceptive. A team might be producing huge amounts of code, looking highly productive on a spreadsheet, but if their percent slash CA is low, they are just producing a massive volume of defects. They are flooding the downstream process with poor quality, which causes massive rework and clogs the entire value stream.

Host 1

In addition to these, there are four core flow metrics that modern agile teams use to manage this process instead of points. We already talked about throughput. Let's quickly hit the other three. Work item age.

Host 2

Work item age is simply how long a specific currently active item has been in progress. It is a leading indicator. If a ticket has been in progress for 15 days, but your average is five, you know you have a blocked item that requires intervention today before it ruins your cycle time.

Host 1

And cycle time?

Host 2

Cycle time is the total elapsed time from the moment work begins on an item to the moment it is finished and delivered. Throughput is how many you finish. Cycle time is how long one takes from start to finish.

Host 1

And teams visualize this using a CFD, a cumulative flow diagram. I love CFTs. It's a stacked area chart that shows how many tickets are in each column over time. If you see the testing band getting wider and wider, while the done band stays flat, you don't even need to talk to the team to know that testing is bottlenecked.

Host 2

And teams use these metrics to set an SLE as service level expectation. An SLE is a forecast based on historical cycle time. For example, 85% of our stories are completed in eight days or less.

Host 1

So, bringing this all together, if we abandon long term estimates, if we refuse to commit to arbitrary points, and if we ruthlessly limit the work in progress to optimize flow, how does a CEO or a product manager actually plan a roadmap for the year?

Host 2

This is the million dollar question. And it is where Agile Purism meets the harsh reality of running a commercial business. To answer it, we have to look at the work of Marty Kagan and the Silicon Valley Product Group, or SVPG.

Host 1

Marty Kagan, yeah.

Host 2

Kagan argues that the traditional corporate roadmap, what he disparagingly calls the feature request spreadsheet, is fundamentally flawed.

Host 1

We all know that spreadsheet. It is a list of 50 features handed down from sales, marketing, and the CEO, each with a hard deadline attached. Build a chatbot by Q2. Add Apple Pay integration by Q3. Redesign the homepage by Q4.

Host 2

Kagan explains that managing by a feature spreadsheet creates what he calls mercenary teams.

Host 1

Mercenaries versus missionaries. Let's break this down. What is a mercenary team in this context?

Host 2

A mercenary engineering team is a group of developers who are just handed a list of requirements and told to build it by a certain date. They have zero context. They don't know why they are building the chatbot. They just know they have to code it to satisfy the spreadsheet.

Host 1

So they just do the work and go home.

Host 2

Exactly. They act as mercenaries. They do the job, they are paid for, no questions asked. If they ship the chat bot in Q2 and it fails to increase sales, they do not care. They shipped it on time. The failure is the product manager's problem.

Host 1

It is the ultimate demoralizing environment.

Host 2

It is the just shut up and code mentality.

Host 1

In contrast, strong product companies build missionary teams. These teams are not given a list of features to build, they are given a customer problem to solve, and they are empowered to figure out the best way to solve it.

Host 2

So instead of telling the team, build a chatbot by Q2, leadership says, our customer support wait times are too high, and we are losing customers. Your mission is to figure out how to reduce the wait time by 20%. And this leads to SVPG's alternative to traditional roadmaps, shifting from output to outcome.

Host 1

Output versus outcome.

Host 2

Output is shipping of specific feature on a specific date. The chatbot. Outcome is solving a business objective, reducing the wait time by 20%. Modern product roadmaps shouldn't be a list of features, they should be a prioritized list of business outcomes.

Host 1

Let's explore why that matters. If the team is focused on the outcome, they might look at the problem and realize a chatbot is a terrible idea that will just frustrate users more.

Host 2

Exactly.

Host 1

They might dig into the data and realize that 80% of the support calls are just people asking how to reset their password. So they just redesigned the password reset UI, which takes two days, and they solve the problem. If they were mercenaries, they would have wasted months building a useless complex chatbot. Because they are missionaries focused on outcomes, they solve the actual business problem.

Host 2

This approach aligns perfectly with Scrum.org's evidence-based management or EBM framework. EBM measures value in terms of current value and unrealized value, pushing teams to run small experiments to see if they are actually moving the needle on the business outcome, rather than just celebrating that they shipped a feature.

Host 1

Frameworks like Janna Basto's Now Next Later format are perfect for this. Instead of a 12-month timeline, the roadmap is divided into three buckets. Now is the highly defined work we are doing today. Next are the problems we plan to tackle in a few months, which are a bit fuzzy. Later are the broad strategic goals for the future. It communicates direction without committing to false dates.

Host 2

It allows the team to iterate to use the shape up methodology to experiment and discover the best solution, rather than being locked into a predefined estimated feature that might not even work.

Host 1

I am fully on board with outcome roadmaps. But I am going to play devil's advocate again, because there are executives listening who are saying that's great for optimizing a password screen, but businesses do have hard deadlines. It is not just arbitrary middle management pressure.

Host 2

That is entirely true. The real world has hard constraints.

Host 1

There are regulatory changes. GDPR compliance goes into effect on a legally mandated date. The SEC demands a financial reporting overhaul by a specific date. Or you have a massive marketing launch tied to a physical event like the Super Bowl. You cannot just tell the SEC or the chief marketing officer, we are an empowered missionary team focused on outcomes, so we'll deliver the compliance feature when it feels right.

Host 2

No, you certainly cannot. And both Fowler and SVPG validate this reality. There are times when hard date-driven commitments are absolutely necessary for the survival of the business. But the difference between a high performing agile team and a chaotic feature factory is how you arrive at those commitments. Kagan calls them high integrity commitments.

Host 1

High integrity commitments, as opposed to low integrity guesses made by developers held hostage in a refinement meeting. So how do we make a high integrity commitment for a hard deadline?

Host 2

You do not make it on day one. When the business needs a hard date, you utilize the concept Holub mentioned, a proof of concept phase. You give the team four to six weeks to actually start the work to get over the hill in shape up tones.

Host 1

To clear the unknowns.

Host 2

To battle the legacy API, to test the new database architecture, to see how the system actually behaves.

Host 1

Yes.

Host 2

And during those weeks, you are not asking for estimates. You are measuring their actual throughput. You are gathering real empirical data on how fast they can solve this specific type of problem in this specific code base.

Host 1

And then you use probabilistic forecasting. This is where we bring in the heavy mathematical artillery. Tools like Monte Carlo simulations, which were pioneered in the agile space by experts like Daniel Vicante.

Host 2

Let's explain what a Monte Carlo simulation actually does in this context, because it is brilliant. You have your historical throughput. Let's say over the last 10 weeks, your team completed five, eight, four, seven, nine, five, six, eight, four, and seven stories per week.

Host 1

So a highly variable output, which is normal.

Host 2

You don't just take a simple average, because an average hides the extremes. A Monte Carlo simulation takes those actual historical weeks and randomly selects one to simulate a future week. It does this over and over again thousands and thousands of times to simulate a thousand possible futures for your project based solely on your past performance.

Host 1

So it runs 10,000 simulations of how the next two months might go. And it doesn't spit out a single date.

Host 2

Exactly. It might tell you, based on 10,000 simulations of your actual throughput, in 50% of the scenarios, you finished all 40 stories by November 1st. In 85% of the scenarios, you finish by November 15th. And in 95% of the scenarios, you finished by November 30th.

Host 1

You take that data to leadership. You have an adult mathematically sound conversation about risk. If it's a regulatory deadline and missing it means the company is fined millions of dollars, you look at the 95% confidence interval. You staff the project and aggressively cut scope so that the work fits within the November 30th projection.

Host 2

That is a high integrity commitment.

Host 1

It is. Based on empirical math, simulated risk, and reality. It is not based on a developer being pressured into saying next Friday, just to end a meeting. But to successfully execute this, to transition from feature factories tracking velocity to empowered teams using probabilistic forecasting and outcome roadmaps. It requires a massive tectonic shift in corporate culture. It requires a fundamental redefinition of leadership versus management. SVPG draws a very sharp, distinct line here.

Host 2

Leadership is about inspiration, context, and direction. Leaders must provide a crystal clear product vision and strategy. They define the North Star. They set the outcomes. We need to reduce checkout abandonment by 10% to hit our revenue targets. They define the destination, but they absolutely do not dictate the path.

Host 1

They don't hand the team a spreadsheet that says build a chatbot.

Host 2

Right. And managers. Their role completely transforms. In this model, managers must step away from command and control oversight. They stop updating spreadsheets with JIRA points. Instead, they focus entirely on the people and the system environment.

Host 1

SVPG breaks it down into staffing, competence, and coaching.

Host 2

A manager's job is to make sure the team has the right mix of engineering skills, to ensure they are technically competent, and most importantly, to coach them. A manager's job is to look at the value stream map, spot the bottleneck in QA, and go fight the organizational battles required to fix it. They help developers connect the dots across different teams. They manage the flow of the work through the system. They do not manage the individual workers.

Host 1

Which brings us to the end of this massive deep dive into the realities of planning and estimation. We have covered a staggering amount of ground today. From the original syn of Story points to the mathematics of flow metrics to the culture shift of outcome roadmaps. If you are an engineering lead, a product manager, or a developer listening to this, and you are exhausted by the Fibonacci poker circus. Let's synthesize this stack of sources into three pragmatic, actionable takeaways you can bring to your team tomorrow. Number one.

Host 2

Kill Velocity as a performance KPI. If your organization is using story points or commit counts to measure developer productivity, you are incentivizing the wrong behaviors and inviting good Hart's Law into your culture. Shift to story counting. Train your team to slice stories down using the invest criteria until they are roughly the same size and focus on creating a smooth, predictable flow of value. Measure throughput, cycle time, and work item age, not arbitrary numbers.

Host 1

Number two, enforce whip limits. Take the empirical lessons from Dora to heart. Stop the context switching madness. Multitasking does not make your team faster, it grinds them to a halt and burns them out. Put hard physical limits on your Kanban board for the number of items in progress. When a column is full, the team must swarm to finish old work before pulling new work. Expose the bottlenecks and force collaboration.

Host 2

And number three, pivot to outcome roadmaps. Move away from brittle feature date Gantt charts. Use tools like Jana Bastou's Now Next Later format to communicate direction. Frame your roadmaps around business outcomes, empowering your teams to act as missionaries to solve problems rather than mercenaries typing code. Reserve hard, high integrity estimates backed by Monte Carlo simulations only for when the business genuinely requires them for critical coordination or regulatory compliance.

Host 1

Those are three powerful transformative moves. Thank you so much for joining us on this deep dive. Next time, in our fourth deep dive of the series, we are taking the code we've just meticulously planned and mapped out, and we are going to look at how it actually gets safely, securely, and continuously into the hands of the customer in our deep dive on DevOps and continuous delivery.

Host 2

But before we go, I want to leave you with a final thought to mull over during your next refinement meeting. We have established today through evidence, behavioral psychology, and experience that upfront estimates are mostly fiction. Because software development is an inherent act of invention. But ask yourself this if your company's entire budgeting and planning process relies on an illusion of absolute certainty, are you actually funding innovation? Or are you just funding the people who happen to be the best at making up numbers?

Host 1

See you next time.

Save to playlist

Manage playlists →

Share