Mike,

does that mean that Epics should fit into release or sprint so we can better measure created value?

No it doesn’t. An epic is still what it has always been—a description of work that will take longer than a sprint or two to complete. Epics are generally big enough that value can be ascribed to them. For example “Support tables” in a word processor would be an epic. We could do ROI analysis on that feature and decide if it would be worth it. Even that could get tricky though if some of the underlying work to support tables enabled us to support another few feature (e.g., charting). The point I’m trying to make is that no matter how badly teams want to measure “business value created” per sprint, it is essentially impossible on that small a timescale.

Could you give some examples of ROI analysis for the feature. Steps of analysis and real examples will be appreciated.

Hi Denis—
See Chapter 10 of Agile Estimating and Planning, which describes how to do the financial analysis and provides an extended example.

Agreed. We have done business value estimation with great success. I think what you’ve hit on is critical. Business value estimates work best with more course grained features and not individual stories.

Nice post.

Thanks, Brandon.

Hi Mike,

I seem to disagree with your conclusion. Are you willing to investigate with me where I am wrong or misunderstood you? I do support your argumentation that it is difficult to measure value. Your examples are strong in illustrating that it is difficult to measure these accurately, especially at the detailed level of user-stories.

However, I always build on Gilb’s Measurability Principle: “Anything you need to quantify can be measured in some way that is superior to not measuring it at all.”. And it works, we’ve always been able to create transparency and insight there were others failed.
This measurability principle also holds for user-stories, independent of the level of granularity. For sure, for epics and themes it may seem ‘easier’ to define self-supporting business-cases; especially when they need to be supported with market numbers, etc. However, even then things are related and measuring business-value is just an agreement on the measurement procedure too.
It just depends on the agreement you make: how can we approach the thing we want to measure to such extend that we can use it credibly enough in our decision making.

As such, I always recommend to use some way of value measurement (e.g. assigning value points) to a user-story; that is feasible, at least relative to others. With the story-points aligned, it is then even feasible to create some kind of ROI indicator (value-points/story-points). Just as long there is agreement that this kind of indicators are credible enough for the decisions taken with them, it works.

The reason I recommend this, is that pre-sprint priorisation and also in-sprint priorisation on story level is wishful, so help is needed. I see it as too easy to limit value to the epic level. After all, some user-stories (within the same epic) are more important than others; their relative importance need to be made explicit. So, assigning value points at story-level do help. Furthermore, just as with user-stories (An invitation to a conversation) also value-points help in bringing a new perspective to a story, just as the ‘how-to-demo’ does.

Could you reflect on my ideas, as I seem to think that value measurement at user-story levels is a prerequisite to prioritise properly and transparent?

(an illustration of this principle and how we’ve applied this is included in: [link removed as it no longer worked]

I’m totally agree with Rini. As far as you are aware that it is only an indicator, it is interesting to add business value to a story. BTW Mike, thanks for your books and PDFs/Slides. You are a great source of information.

Mike,

Great post, as usual.  However it seems that if your PO is doing his part, measuring business value is not really needed.  Does this add process where the instinct of the team and product owner would decide what is done?

It seems like this would take some of the ownership of the team away and replace it with a mathematic formula of product management.

Thanks,
Steve

I think one of the keys here is I think Mike is talking about “Business Value”.  That would typically be a dollar amount.  Some amount of new revenue, cost savings, cost avoidance, something like that.  I have to agree with Mike that tacking that sort of value at the story level isn’t that practical.  Sure, there may be some stories it will work for, but for most, it probably won’t work. 

Now, there are definitely other ways of tracking other metrics to aid in setting priority that may sometimes be referred to as value.  I think this is what the last few comments are hinting at.  I personally really like the technique taught by Pragmatic Marketing which includes using an evidence number (market evidence, so things like number of clients asking for this or experiencing this problem, number of won/lost deals because of this item, number of prospects interested in this, etc) and impact (which is a custom scale that is really more of a business strategy alignment).  If you multiply those two numbers together you will get a number that will aid in prioritizing your requests.

Steve, if you only have a single product you are ever considering and you have a great product owner, then yes, this may not be necessary.  A few problems I’ve seen though…

1) If you ever have to consider items across products, these numbers can definitely help in the discussion of priority of items in that case.
2) These sorts of numbers/metrics can really help convey value and priority to the rest of the team as well.  Again, if your PO is doing a great job conveying this, may not be necessary, but I very often hear other members of teams asking why certain items are more important than others and not understanding the priority.

Because of these, we are looking at implementing a more mathematical value scheme based on the Pragmatic Marketing approach I described above, but it wouldn’t necessarily be the final priority, it would be more of a guide.  As a PO, if you are arranging things in an order other than suggested by these numbers, obviously you should be able to explain right away why the priority is different than the numbers show.

Rini—
My opposition is largely to making detailed decisions with imprecise information. Taking a number of “value points” (VPs) and dividing it by story points (SPs) can lead to incorrect decisions under the circumstances I described. Again, the best example is the cost of raising a cow and then selling it for beef and for leather. I could make a decision to split cost equally but that’s as arbitrary as deciding to split it 100%/0% or 80/20 or 63/37, etc. The proper way to make the decision “should I invest in raising this cow?” is to consider both benefits together and dividing that by the full cost.

Even if I did invest in coming up with a better “value point” estimate of the value of beef and the value of leather, I would still need to add them together to do the proper analysis. Things can be decomposed too far.

I’ll take a look at your IEEE paper again as soon as I can. I remember reading it when it came out a few years ago.

Hi Pablo-
Thank you. I’m glad you like all the resources we’ve made available here and our other sites. I appreciate knowing you’ve found them useful.

Hi Steve—
I suppose you’re right that there is a risk that the PO becomes simply an automaton doing whatever some analysis spreadsheet says but I haven’t seen that happen. I totally encourage product owners to assess the value of epics or themes. My favorite approach is Karl Wiegers’ Relative Weighting or good ol’ financial analysis based on discounted cash flow models. We can predict how much value a large thing (epic, theme) will bring the organization. But when we start looking at that large thing and saying “let’s do this 2-day user story first and then this 3-day user story followed by the other two-day story” that’s where we go wrong.

Hi Matt—
I think the example techniques you give are excellent and they support Gilb’s Measurability Principle mentioned by Rini. They give us a better way to make decisions. They may not be perfect but they are better than nothing.

By the way, we do have tools on here for helping product owners make these types of relative priority decisions. Check out http://www.mountaingoatsoftware.com/tools and look at the theme screening, theme scoring and especially the relative weighting tool. Relative weighting is very similar to Planning Poker in that we estimate the relative benefit of getting a feature from 1-9 and then the relative pain of not getting the feature on a 1-9. These are added and compared to the cost. But, again, I recommend this at an epic/theme level rather than tiny user stories.

Hi Mike, I also fully acknowledge the fact that prioritizing by using business value is not always easy and the only thing to do. I have, however, always had success with asking our customers to provide business value to a user story/use case. But in addition they are asked to tell us if it will be show stopper for releasing the software to production if that user story is not implemented. Show stopper here meaning: if the release will truly be impossible to use by the end users. This builds on trust, of course. My Product Owner can by no means be dishonest and just claim something is a show stopper if it is not true. But agile builds on trust :o) - so dishonesty never becomes a problem.

Hi Leise/DK:
Please not that I am not saying that you shouldn’t prioritize on business value. I am saying that discrete amounts of business value cannot reliably be calculated for items that are very small. Your mention of use cases shows this—use cases are typically much, much larger than a typical user story and so can have value or ROI calculations done on it.  Similarly, categorizing items into “showstopper or not” is an alternative method. So, that, too points out that you are using alternatives to putting specific values on small items. If you could have put a valid value (ROI) on each item you wouldn’t need to call anything a “showstopper.” You could define a dollar amount—anything causing a loss more than $x—and each of those would be the equivalent of a showstopper.

I think I would have titled this “Estimate Business Value at the Feature level not at the Story level”

Hi Jason—
Now you tell me! ;) Yes, that would have been a better title.

Great Article Mike!

I agree with applying business value at the epic level. Business value at this level is an estimate on how much value can be realised if developed and only realised once in production. Prioritising a backlog at the epic level is easier than smaller finite stories and attempting to calculate value at these smaller story levels is uneccessary work which may never be used if the story is deprioritised.

I think at a suitable time when the epic does become broken down to smaller feature set stories the update of business value should be attributed too. Reason being one feature set maybe more valuable than another, or put another way one feature set maybe deemed to have little business value. In which case its plausable that that feature set of low value could be deprioritised to a later date if at comes to light. In terms of your cow you might consider leather has a higher ROI, maybe 100x more value, than beef so much more that actually we’ll decide to just develop leather and deal with beef at a later. In this example would seem like a lot of beef is going to waste and so from a prioritisation view business value alone I dont think should be the only factor to prioritising a backlog. You could go further own to the finite story level but maybe some common sense approach here is better.

Essentially I think “an agile approach to defining business value” (perhaps that should have been the title) seems to make sense

Thanks, Steve.

For a somewhat academic paper on the four factors that should be considered when prioritizing, see this article from XP2006.

Mike, while my company’s approach is to estimate BV at the feature level (agree with 100% much easier to facilitate prioritization), I have a more subtle question.

Should BV be relative scoring by project or simply driven by strategic factors that align and transalte into?

Perhaps this is more of a maturity question in that newer orgs / teams adopting agile start with relative scoring for BV and migrate to a strategic factor approach to standardize thinking across the enterprise.  I am interested in your thoughts on the pros & cons of each approach.

Hi Paul—
Almost all decisions are relative. Should we do this or should we do that? So relative comparisons are often sufficient. However, I’ve seen many organizations struggle with assessing the relative merits of let’s say feature A1 in project A vs. feature B3 in project B. Their values differ in large measure because of the project the features are part of. Say A1 is adding a feature to Word that will increase Word revenue by 1% and that B3 is a feature on my www.PlanningPoker.com site that will increase traffic there by 25%. Well, in relative value, A1 wins because of how lucrative Word is compared to my site.

So usually there’s a high-level decision about how much will be put into each project. This is often expressed in staff levels: we’ll put 50 people all year on Word and a part-timer occasionally on PlanningPoker.com. Then within those projects there is usually a relative estimation of the value of the features on just that project.