Product backlog items’ sizing has long since become routine for numerous agile teams. These days, you’d be hard pressed to find a developer, Product Owner, or Scrum Master who’s never heard of the Story Point concept.

The Internet is full of explanations, guidelines, step-by-step instructions, videos on how to do proper estimation using Story Points. This topic shows up in many SM/PO certifications. So, for me, it sounded like a done deal: Story Points were the industry standard.

Recently, I’ve begun to rethink the widespread adoption Story Points.

For one thing, many customers start using Agile Poker because of the Relative session. According to app users, relative estimation is ideal for teams that struggle with Story Points because, in Relative session, users estimate without thinking at all of the estimation value—just the relative effort of each item as compared to the rest.

Some teams decide to switch to Poker sizing after several sprints. Others like Relative so much that they choose to stay with Relative as their main estimation technique.

Moreover, I always wondered why the Story Points topic gets so much attention, so many comments, whether it’s Mike Cohn’s blog post, a YouTube video, or a conference speech.

Finally, there’s my team. Imagine my surprise when developers from the Agile Poker team—the very people who create this estimation software for thousands of people, showing them how it’s done correctly—told me that they think of man-days while estimating in story points. Period.

All this inspired me to write a post to structure my knowledge and, hopefully, help others and their teams with this popular (but deceptively simple) topic.

The story of Story Points

To understand the whole concept of Story Points, it’s crucial to know why it was developed in the first place.

We suck at estimating.

But what should you do as a horrible estimator? The Scrum guide (up until the 2020 release) said that estimates should be provided by people that will be doing the work, but it didn’t say how.

And what did Agile enthusiasts do after reading the scrum guide back in 2008 when they had zero step-by-step practical instructions?

They started estimating Product Backlog Items (PBIs) in time. And, admittedly, they did it wrong. Mostly because:

Again, we suck at estimating.
Estimates are treated as a commitment by the stakeholders.
As a result of points 1 and 2, estimations were time consuming or teams intentionally overestimated.

The story point is an abstraction that solved these problems.

Story Points have no units, so it can’t be related to any commitment. If you asked the right questions (for example ‘How is this compared to that thing we did last time?’) it could save a significant amount of time in the estimation process because you just compare one thing to another. And if you fail with the estimation, it’s okay as long as you’re consistent.

Here’s why.

If you say that a particular user story is three times more complicated than the reference one, but in reality, it’s only two times “bigger,” it’s still OK if you estimate all similar stories that same way. Because your frame of reference remains the same.

Where did story points come from?

Story Points was the brainchild of Extreme Programming (XP), and was widely adopted by Agile practitioners.

As Ron Jeffries describes, the original “Points” for estimating “Story” were based on “Ideal Days” to finish a specific PBI. These could be converted to work days by multiplying by the “load factor.” This would give you the actual implementation time.

For example, a story estimated as three “Ideal Days’ multiplied by a load factor of three yields an estimate of nine days.

But stakeholders were confused with three ideal days estimation versus nine non-ideal days, so Ron’s team started calling “Ideal Days” “Points” or “Story Points.”

And that’s how the Story Point concept came to be—time estimation, that evolved into a different, very popular estimation tactic.

What is a Story Point?

A story point is an abstract measure of effort required to implement a user story. In simple terms, it is a number that tells the team the difficulty level of the story. The difficulty could be related to complexity, potential risks, and the effort involved. Let’s briefly analyze each measure.

Effort or volume. How much is there to do?

What amount of work is there to do?
Is it big? Is it small? Is it average?
How much effort will it take to complete the job?

Complexity. How difficult is the work?

How complex is the work compared to other stories of the same volume?
Is it easy or difficult to do?
What are the dependencies and circumstances?
Are there any unusual practices involved?

Risks. What are the unknowns?

What are our “known unknowns?”
What is our experience with similar stories?
What are we uncertain about?
Does this work challenge our skills?

Story Point estimation is typically performed at Product Backlog Refinement Sessions. Product Backlog is evaluated by the team responsible for the actual development and testing work.

Benefits of using Story Points

“Story Points are faster, better, and cheaper than hours, and the highest performing teams completely abandon any hourly estimation as they view it as waste that just slows them down.”
Sutherland, J. (2013). Story Points: Why are they better than hours?

Let’s review the benefits of using Story Points.

Estimating in Story Points is fast. Teams who estimate in days tend to take the discussion a few levels deep. If you estimate in days, fear of commitment makes people hesitant to reach consensus.

As Jeff Sutherland observed, one CMMI level 5 company determined that story point estimation cuts estimation time by 80%, freeing teams up for more productive project activities.

It’s Relative. Mike Cohn best described it in Agile Estimating and Planning:

“Because the estimate for each feature is made relative to the estimates for other features, it does not matter if our estimates are correct, a little incorrect, or a lot incorrect. What matters is that they are consistent. And Indeed, ‘there is credible evidence that humans are good in relative estimation compared to absolute’.”
Cohn, M. (2005). Agile Estimating and Planning. (1st ed.) Prentice Hall.

Story Points remain the same for everyone. It doesn’t matter if a Senior or Junior developer will deliver the story. What matters is that the story is X times bigger than the reference one.

Estimating in Story Points allays team members’ fear of commitment. When estimating using hours, you make a precise time commitment. Estimating in Story Points prevents giving an exact time commitment. Nobody knows precisely how many hours you are appointing to a specific Item.

Eliminates the need for frequent re-estimation. When planning with Story Points and velocity, we essentially take those points as size-based estimates. We don’t expect the size of the story to tell us the time needed to complete it. If estimates are in days, the team finds itself in the position of needing to frequently re-estimate future stories based on current knowledge.

When teams talk about “days of work,” it implies a level of specificity that isn’t real. We avoid this problem when using Story Points. The fact that a team learned to work twice as fast gets reflected in their velocity. As long as the story sizes are consistent (read the previous points) between stories, no re-estimation is necessary for planning to occur.

Story Points encourage cross-functional collaboration. Mike Cohn explains this very well in his book ‘Agile Estimation and Planning.’. Agile teams include people from different disciplines like programmers, analysts, testers, designers, or product owners. Agile projects benefit when each team member thinks about a built feature first and then their role as a specialist contributor. As a result, estimating in story points helps teams learn to work cross-functionally.

Because a Story Point is a single number, all team members need to consider the big picture when estimating—to think of the item as a complete feature. They need to shed any departmental mindset about how much time a programmer or tester or UI person would take and any effort to add that to their estimate.

At the end of the day, scores are made up. They’re just numbers.

It’s the conversation and collaboration that happens during the estimation session that brings value and helps team members gain a common understanding of stories. The team behavior takes precedence over individual preferences as conversations result in team building, with team members constructively criticizing one another, and engaging in healthy debate.

Weaknesses of using Story Points

“I like to say that I may have invented story points, and if I did, I’m sorry now.”
Ron Jeffries, Extreme Programming (XP) co-founder and one of the original signatories of The Agile Manifesto, in a tweet dated May 23, 2019.

Story Points are not intuitive. My job provides me with the opportunity to communicate with lots of Agile Poker users, Scrum Masters, Product Owners, and Tech leads. I regularly hear that they don’t understand Story Points.

Not only does the person advocating for Story Points have to repeat basic principles several times, but they find even explaining Story Point sizing (e.g. 3-5-10-…) leads to questions, and sometimes arguments:

Should we go for another round?

Why don’t we just take the bigger number?
I will do this in 3 SP, why are we putting 5 SP?
Why can’t we use the average of 3 and 5 SP?
What is the difference between 1, 2, and 3 SP?
What is the lowest SP value, 0.5 or 1?

Have you ever heard somebody say “this item feels more like a 3 to me” even though all other members estimate as 2? I’ve experienced such dead ends when trying to reach consensus.

Ultimately, it slows down the process and counteracts the ‘Fast’ benefit of Story Points.

By comparison, the T-Shirt sizing scale is much easier to adopt.

“Scrum Masters often guide the team to the desired answer rather than let the team arrive at their answer based on conversation and understanding of the work. Even worse, I have witnessed several Scrum Masters declaring the story point value once the story is read.”
Cook, L. (2017). Pitfalls of Planning Poker.

Not truly relative / Translating Story Points to hours. Do you remember the history of Story Points? It evolved from time estimates, and even today, it is still mixed with time estimates. Teams convert time to Story Points and back because time estimates are easy to understand.

But why is that?

Imagine a team estimating numerous PBIs, one by one. The group is supposed to compare each item to a baseline item (typically a ‘1’). This is exactly where the problem lies.

Participants have to figure out how much bigger the item is. 2x? 10x? The human brain tries to simplify the task by creating an image of each size on the scale and unconsciously translates it to time intervals (e.g. 2–3 days for size ‘2’).

The whole Story Point concept turns into absolute estimations without you even noticing.

It’s funny how time estimates brought relative Story Points back to absolute estimates. This slows down the process, canceling out most of the benefits of this approach.

Adjusting Story Point estimate with a specific developer to work on it in mind. We see teams continually estimating with a specific team member in mind: a PBI might be 3 Story Points for a senior developer, but 8 Story Points for a junior developer. So the developer is selected first, and then the Story Points value is set.

It’s so easy to go with 3 SP in this case (and numerous Agile Poker users do so), but it’s not the correct approach. Story Points estimation is a team commitment. What will happen if the junior developer has to pick up the story?

Resizing unfinished issues. The Story Points concept doesn’t provide a solution of what to do with stories that are moved to the next Sprint. Many teams re-estimate the story and put remaining estimates, but you shouldn’t have to to re-estimate at all.

Reestimating the reference issue will endanger future planning sessions. As a result of Sprint Planning, the team will know all necessary tasks to complete the issue. The estimation of these tasks is in hours. So, in the next Sprint, the team will know how much time is still necessary to complete the PBI. The fact that the PBI was not completed reflects the velocity rather than the Story Points value.

Avoid adjusting reference PBIs. Revising reference PBIs leads to re-evaluating the team’s Story Point values and could seriously mess with your planning process.

Summing up Story Points is artificial. Summing up the relative value of Story Points is the subject of much debate. Eight 1-point tasks take much less time than one 8-point task! Point totals across sprints are not comparable because of this math problem.

Most of us try to overlook this problem, but some frameworks address it head on. The exceptionally popular Weighted Shortest Job First (WSJF) prioritization framework suggests calculating the score based on 4 Story Points criteria. You sum and divide relative values to get another relative value for comparison purposes. It scares me, to be honest. But apparently it works.

Story Points vs Bugs and Tasks

From guides, you know that only User Stories should be sized. From real life, you know that teams estimate all possible issue types. Full stop.

Should I use Story Points?

My short answer is YES! 2 main points to support this:

Story Points concept has stood the test of time. Despite attracting thousands of haters from the moment of its introduction, the concept survives and prospers.
There is no better alternative for Backlog sizing.

So, the right question to ask is “How do I use Story Points?”

As the 2020 updates to the Scrum Guide reflect, the industry is becoming more flexible, and the community is okay with non-strict rules.

So, even if you make every possible mistake pointed out in the previous section, as long as your estimation sessions work for you, you’re fine. However, as a professional, you should understand the drawbacks of not using the framework as intended. Then, if you make a conscious decision to bend or break certain rules, that’s your call.

How do I make estimation work better?

At some point, every Agile team faces the need for estimation, and the vast majority is or will be using Story Points. Here are some recommendations from Agile Poker users as to how they address estimation challenges:

Start out with Relative estimation. As mentioned before, Story Points are often translated into time. Usually, it starts from the very first estimation, where estimators have to somehow think of some magic items to estimate effort.

Suggestion: avoid Story Points completely at the beginning. Use the Relative estimation method of Agile Poker instead, where your team can group user stories into columns according to their relative weight and only then assign the story points values.

This approach has worked for numerous Agile Poker users, who, after several Relative sessions, switched to Planning Poker, Bucket Sizing, or stayed with Relative.

Know the basics and adjust accordingly. There must be someone who knows the basics of Story Points estimation and can present it to the team.

If you have read this blog post until this point, maybe that person is you!

But this person must understand all the consequences of modifying the by-the-book process (and most teams do modify).

Prevent estimate inflation. In Agile Poker, we implemented two features to help keep estimations consistent:

Reference issues are constant stories with different Story Points values used as a baseline for estimates.
Triangulation issues constantly change and display recently finished stories from the same project (with the same labels and components) to give users more context of a selected Story Point value.

Submit your personal estimates

More theory on this topic might be found here.

Try different estimation methods. There are many different estimation methods for Story Points estimates, but Planning Poker is considered the default one. It might be limiting for some teams. For example, many of our clients use Bucket Sizing with a voting option as a faster alternative. My team and I use Async Poker, which enables estimators to go through the list of stories on their own, get into details if needed, and have time to explain the choice in the comment section.

Discuss incorrectly Story-Pointed issues in retrospective. Every now and then, the team Story Points an issue where it is clear that the estimate was entirely off. It is important to discuss these issues and try to learn, so future estimations are more accurate.

Once my team stopped creating test tickets, considering it as a part of stories, the reference issues were still the same. Retrospective helped to figure this out and improve our planning.

Adopt. Cliche, but still, no one knows your team better than you. If you see problems or difficulties with Story Points estimation that is not answered in the books and theory, keep adjusting them until you solve those problems.

Conclusions

Story Points is simple in theory but difficult to apply. It’s well described, but rarely used as intended. The Story Points concept arose from time estimates, but Story Points scores aren’t connected with time estimates.

In short, use Story Points to the extent they’re helpful to you. If you find Relative estimation (or another method) easier, do that. Because the end goal is to keep your team on schedule.

I hope this article will help you with sorting out the concept and help with a quick, reliable and pleasant planning process.

Benefits and weaknesses of using story points

Subscribe for updates