Mike, I respectfully disagree.  While the people estimating the epics may know that 20, 40 and 100 are WAG’s, those who are given those estimates very, very often do not.

I wrote about this specific problem back in June of this year at http://practicalagility.blogspot.com/2011/06/fibonacci-must-die.html .

As I say, in my experience the large numbers cause much more harm than good.  I have seen items that were estimated at 40 broken down into many smaller items whose aggregate estimate was much lower.  I have seen items estimated at 20 turn into triple-digit monsters when the deferred breakdown finally occurred.

I coach groups to use a much smaller set of values, and to split items more and sooner.  Does that create some waste?  Yes, in purist Lean terms.  However, finding out that an epic that was estimated at 20 breaks down into many, many items totalling well over 100 would seem to violate the Last Responsible Moment, would it not?

I totally agree with you Mike.

There are still many companies (organizations) that during company-wide budgeting or their capital planning period still require rough order of magnitude “estimates” for how long it will take to develop the product and how much it will cost. Especially, those companies that do Portfolio and Program Management Therefore, if the folks estimating using story points do not have 20, 40, 100 story point cards as you so aptly stated “you would need to put no more effort into this than you need”. 

Additionally, for though folks who do throw out the higher value cards I would venture a guess they have not come to understand and come to grips with the difference between estimating and making a commitment.

I personally don’t advocate using the 100 point card, but I find the 20 and 40 useful. I find that some people get confused about when to use story points, and maybe that is the issue here.
I find that for release planning - thinking about 3 - 6 months of effort, I don’t want to spend a lot of time getting into that much detail. It is enough to know that a 20 or 40 means it is big and we need more information.
If I am talking about the next iteration - that is completely different.

+1 Dave - this has been a bone of contention that’s been borne out time and again in every day situations by many coaches in the field. It even prompted Schwaber to make a tongue-in-cheek observation about how we’re perverting nature by messing around with the Fibonacci sequence and Golden Ratio.

Practically speaking, I’ve found teams to use common “groupings” of lower numbers when estimating and planning a Sprint, eg. 3,5,8. 13 and 21 tend to be brought out when stories aren’t well-decomposed, with 21 the high-watermark of uncertainty. 40, 100 and ? or infinite are almost never used except as comic relief, followed by quick conversation of whether the feature needs to be better developed and deconstructed.

IMHO, defending the large numbers is a bit like the famous moment in the Spinal Tap mockumentary where Nigel Tufnel defends the notion that his amps are louder because the volume controls “go to 11”.  When the interviewer asks why not just keep the 1-10 scale, making “10” the loudest, Tufnel blankly responds: “because they go to 11”.  Circular logic and hilarity ensue.

Chris R. Chapman
President & Founder, Derailleur Consulting

Mike

I agree that large numbers are helpful.  Dave, I agree the big numbers are wrong.  I coach that.  I respectfully disagree that that they should never be used.

I consider anything over a 5 (you say 3, let’s not argue about that one. I like 3 or less but draw the line at 5) in need of being broken down before being included in an iteration.  Bigger numbers are budgetary, and the law of averages starts to come in.  So I am not surprised by your experience that some 40s are smaller and some 20s are bigger once more work is done.  I’d be more surprised if it did not happen.

We need to coach teams and their stakeholders on what estimates are and are not.  Large numbers are helpful for budgetary estimates. If larger numbers are not allowed and a budgetary estimate is needed, you are back to being in the analysis paralysis *phase*, breaking down epics up-fornt, delaying other valuable well understood work.

Larger numbers represent risk.  If a given epic has a large number, the team should decide if they can afford having the risk, or if they know enough to even break it down.  Sometimes we don’t know enough to break a story down and in those cases a budgetary number is all that can be done before more is learned.  Other times we choose to not break epic stories down so that well understood work can start, buying time to do the breakdowns in parallel.

@Dave R.

I mentioned in my first comment there is an appropriate time to play planning poker and use the values 20, 40, and possibly 100.

On the other hand I agree using Fibonacci numbers or using story points, especially values as large as 20. 40 and 100, during iteration (sprint) planning is counter-productive.

In fact during iteration (sprint) planning I steer teams away from playing planning poker all together when making commitments to get done what needs to get done. Instead I advise them to task out and then derive their level of effort for what it will take to get a story done in hours. This is where making the distinction between making a commitment and deriving an estimate come into play that Mike has articulated many times to us.

I’m one of the guys not using large numbers, because in my experience anything above 13 is coffee cup reading anyway. Epics of size 20 or larger are usually so wooly that my team can’t even with a certain uncertainty say if it’s a 20 or a 40 (or even 100). For me it’s enough to call it “epic, to be split” without giving it any size. As soon as it gets nearer I ask the Product Owner to split it properly into User Stories that can be estimated.

BTW: I don’t use Planning Poker, I use the Team Estimation Game or Magic Estimation. Both methods
apply Story Points *after* the Stories were estimated relatively to each other (w/o any numbers). If it turns out that there are Stories beyond 13 I take them out to process as described above.

Hi Dave—
I’m sorry your experience has been different from mine. I have not had much trouble getting most bosses, clients and customers to understand the inherent uncertainty in rough estimates. As I said in the post though, these aren’t numbers I’ll use all the time but I have no idea why anyone would get rid of them completely. Just because they aren’t useful in all situations doesn’t mean we should never use them.

Hi Russell—
I think you make a good point about the importance for separating estimating from committing. An estimate is merely the first step in arriving at a commitment and it is what it is. People can’t say “change the estimate.” My estimate is what I think. Someone can request that I change a commitment I make based on my estimate. But my estimate remains what it is.

Hi Janet—
Fair enough. I don’t use a 100 card very often but I’ll keep it around. I find 20 and 40 to be very helpful. It is not out of the realm of experience for most teams to estimate those with enough accuracy that they can be helpful is putting together a set of rough estimates and creating, as you say, a release plan from them. And you’re right—I’d never advocate a 20 or 40 coming into an iteration (unless the team had a velocity like 150-200, which is not normal given how most teams estimate).

Hi Chris—
I’m sorry you don’t find the larger numbers useful. I’ve pointed out that they aren’t useful in all contexts but that doesn’t mean I’d want to get rid of them in all contexts. To continue your Spinal Tap analogy, that would be like saying that since an 11 isn’t helpful on an amp, we should get rid of all 11s.

Thanks for the comments, James.

You make a very good point about educating stakeholders. Sure, some will never get it but executives and stakeholders are much smarter than software teams joke about them being and I can almost always get through to them about how to use estimates. Often a good way is to ask someone, “how much money will we make on this project next year?” They’ll likely give a range and I can point out that even with that range they can make appropriate decisions. So, too, with an estimate of the delivery date. Early on, knowing that a project will take 9-12 months is often more than sufficient for the early questions like “should we do it?”

Good point, too, about larger numbers indicating risk. I teach teams to use the numbers as buckets—so a 20 is really anything too big to be called 13. We do that to help mitigate the risk. Sure the 20 could turn into something we wish we called 22 or 25 or even 32. But if most 20s are “truly” 14-20, this averages out quite well since, as you say, the “law of averages starts to come in.”

Hi Russell—
I’m with you: I don’t like teams to play Planning Poker during sprint planning. I’ve blogged about that previously. Planning Poker is helpful when we need team consensus on an estimate and we need that when estimating product backlog items. but don’t really when estimating a task that one person will do during iteration planning.

Hi Sven—
What you’re doing (not using big numbers) may be perfectly fine. The only time I’d change that is if my boss, client or customer wanted a rough estimate (“Are we talking summer or fall to be done with this?”). In that case I can take the extra effort of splitting a bunch of big stories into small ones (less than 8 or 13 in your case) or I can learn to be better at estimating the slightly larger stories. I find plenty of cases where we need to give a rough estimate and using 20s, 40s, and sometimes even 100s, can be helpful.

Hi

I sometimes use the 100 card to demonstrate when a story is so vague that it is impossible to estimate.  These ‘100’ stories stick out like saw thumbs in the backlog.  It’s surprising how quickly a list of ‘100’ stories becomes a hot topic with the product owner and senior stakeholders.

We also suggest stories with a max of 13 points be taken into a sprint, so any 20’s or 40’s need to be broken down to 2 or more sub stories.

Hi Leon—
That sounds like a very reasonable approach and very similar to what I do. Thanks for sharing.

Mike,

I enjoyed this one, and really like that you specifically called out “contract development” as an exception.

I can attest the the fact that clients requesting contract development work are rarely satisfied with these rough estimates—even though we know they are relatively accurate. Typically on a first pass, we estimate out a few core feature sets in detail, and then assign large numbers to other similar-sized feature sets (20, 40, etc). Unfortunately, we never seem able to sell this approach, and the prevailing mindset seems to want a locked in fixed price estimate.

Personally, I find this instinct counter-intuitive. On any significant project, attempting to get a more accurate estimate (up front) is usually misleading. The client is going to change things as the project progresses. Features will be trimmed or expanded. New features will be discovered and added.

I agree with you that these large numbers are useful for big new projects. But they are damn hard to sell. The whole estimation process is one of the hardest things about Agile to sell to an outsider. At the end of the day, most folks just seem to want a fixed price.

Thanks, Michael. The key with contract development is to acknowledge that it is all about shifting risk. A client hires a contracting organization to develop some software so they can avoid the risk of doing it themselves. The contract organization charges a premium for taking on that risk. One of the challenges is that the client often thinks they’ve eliminated more of the risk than they have—there may be financial penalties if the contractor does not deliver on time but there is often a huge penalty to the client company of not having the product when they expected.
But, you’re right—no matter what we put in a contract, there will usually be some features trimmed or expanded. And that’s somewhat as it should (and needs to) be. As we learn more about the thing being built (by building it), new requirements emerge. The challenge of course is writing contracts that accommodate emergent requirements.

Mike, my shop is one that throws out the large point cards. The main reason is that our managment does not have a feel for what a 40point estimate is. We are very focused on providing estimates that have can have some support. This means we create estimates by breaking down projects to a granular level that can be estimated under 40pts. We then add up those Epics to get our SWAG. The variance in actual to estimate when done this way tends to be more accurate because we have already produced a first pass analysis. Thanks for your posts.

Hi Scott—
My point isn’t that everyone needs to use the large cards. It is merely that sometimes they are helpful—-IF we’ve gotten to the point where we’ve educated the recipients of our estimates about the inherent uncertainty in them and IF there is no need for a more precise estimate built from smaller stories. There’s a cost to breaking stories down. Often teams just assume that management would rather pay that cost (i.e., spend that time) rather than have a less precise estimate. To do our jobs well, we need to offer management or other recipients of our estimates that choice. You may already be doing that from what you write.

And you’re welcome for the posts. Thanks for taking the time to comment on them.

I’ll weigh in with Dave Rooney on this one.

I often find I have to explain the difference between macro-level estimation to support go / no-go decisions and micro-level estimation or “sizing” to support short-term planning. Many people seem to think “estimation” is all the same. The best thing about this post is that it explicitly calls out the difference between macro- and micro-level estimation.

IMHO Planning Poker is most useful for short-term planning within the scope of a single team. I don’t think it’s the right tool for macro-level estimation, for instance when “your boss wants to know the general size of a new project being considered.”

IME Planning Poker is most effective when we recognize certain strengths and limitations:

(1) Fibonacci series corresponds with the way things tend to vary in size in nature. No reason to assume software follows different laws of physics from everything else.

(2) Humans are hard-wired to make accurate gut-feel guesses about the relative sizes of things, provided the range of sizes is within a single order of magnitude. With cards ranging from 1 to 100 or more, we side-step our natural strength in this area. Given 1, 2, 3, 5, 8, 13, the 13 provides the warning flag about “risk” or “unknowns.” There’s no additional value to be gained by including a 100 or a 200 card.

Teams that haven’t yet reached a high level of proficiency in splitting stories will naturally create stories that vary significantly in size. Planning Poker can be very helpful in those cases, provided we use it at the right level of detail for the right kind of planning.