Mike, do you retroactively have them assign the points?  I have a blog entry at http://bit.ly/d23Ij6 where I make the case that I’ve never seen a team able to accurately size a “defect.”  If they do it retroactively this makes more sense to me.

- Bob -

What I’m referring to here is something like a team saying “Let’s reserve 40 hours for bugfixing this sprint” and then calling that (for example) 6 points because based on their experience 6 points would typically be that same amount of work. (Numbers entirely for example only.)

There has been some strong debate in our Phoenix Scrum User Group (http://phxsug.org) meetings about this very topic.  The description here is interesting, independently capturing the positives and negatives of the topic.

My take is that that the team and stakeholders need to know both their capacity for new work and their capacity for bug fixing.  I recommend that the story points for bugs should not be “lumped in” with the overall story point velocity.  The points burned fixing bugs needs to be visible so that the team and stakeholders are fully aware of the cost of low quality.

Maybe have two story point burndowns each sprint.  One shows new work and the other shows bug fixes.  Over time, the bug fix starting points should be less and less as the team increases the quality of the code base.

Let’s say that we reserve those 6 story points / 40 hours for bug fixing. Does it really tell product owner that suspending bug fixing for some sprints will allow her to add 6 points of “feature” stories for the sprint capacity?

That would mean that fixing bugs (which is very specific part of development) is easily interchangeable with new stuff to be done. I’m not convinced for 100% that this is the case, but I have a feeling that after suspending bug fixing PO and the team would need to find out teams new velocity during 1-2 sprints. This would lead to conclusion that after all estimation on bugs was useless.

What do you think about it?

We discovered that fact a couple of sprints ago. Our rule is that if a bug is reported out of sprint, through a user using the software for example, it’s treated like a user story - estimated and points assigned as normal. If however the bug is a result of a code review defect or qa hammering on the code during a sprint, it is dealt with in sprint, as part of the user story that generated the bug, and is not assigned any points; since the story estimate takes it into account.

I think it is good to add point estimates to bugs.  It may well impact how bugs are prioritized - if it is known to be simple a program manager may say yes I want these 6 first then…  If then know the first 2 are likely to take a bunch of time, they may say, ok I am not going to get these 4 for awhile…  They might just accept that, or maybe say lets shift more hours to bug fixes this sprint.  Or they might say well if it is that big an issue maybe we could do x instead…

In practice I rarely do it for emergent bugs we are going to fix in the current sprint (due to importance), but we do it for bugs that are in the backlog.  I sometimes will have us estimate a current bug if I think it is going to be a big deal - to help determine what we really want to and what the impact may be on velocity.  We do not have many emergent significant bugs so it isn’t much of an issue for us.  We have one backlog for all stories, new features or bug fixes.

We do have more difficulty accurately estimating bugs compared to new stories, but we still provide actionable estimates (they are not perfect, but are usable).

Hi Marcin-
The only reason why a product owner couldn’t suspend the six points of bugfixing per sprint and get instead 6 points of new functionality is if the items were incorrectly estimated. I’m not sure I buy the premise that fixing bugs is a very specific part of development. Most programmers have developed skills in both fixing defects and in adding features.

Hi Alan—
Yes, if a team is making a planned effort to burn through a big amount of legacy bugs they may want to create a separate burndown of bug points. I usually just do this by graphing defect inflow and outflow as described in this article from Better Software magazine years ago.

Hi John-
Absolutely. In order to prioritize we need to know the cost of things. For example, I can only prioritize between a 65” plasma TV and a sandwich if I know that the plasma TV costs a lot more than roast beef on rye.

You’re also right about not doing this for bugs found in the current sprint: If a bug is found in the code the team is working on, it should be fixed right then. Ideally by the tester yelling across the team room, “Hey, Mike—there’s a bug in the loan value calculation you just wrote. To see what I mean, enter…” This isn’t estimated or given story points; it’s just fixed right then.

Good post. One of my doubts in assigning story points a bug fix task is that it could be not so easy to do that. I mean, there are some bugs that you have very little idea about why it’s happening. If that’s true, you would have no clue about how to estimate that bug fix.

What would you do in that kind of situation?

Thanks, Hugo. Note that in the post I very carefully avoided recommended putting a story point cost on each bug. In most situations that is very hard (which a number of people have said) and not worth the time it would take to do since you might be able to fix it just as fast as you could estimate it. Some bugs—big ones, nasty ones, etc.—warrant a discrete estimate just on that bug. Often though we have a bug database with say 200 bugs we want fixed. What I’m referring to in this post is a team saying “let’s spend 40 hours fixing bugs this sprint.” Then during the sprint when a programmer has an hour to spare late in the day (and doesn’t want to start anything new) the programmer fixes a quick bug or two from the database. The next day another programmer grabs a bug that takes 2-3 hours. And so on. The team tracks those hours a bit (tick marks on a whiteboard often suffices). And the question was whether to give themselves some number of points for work of this nature.

IMHO if fixing a “bug” will result in business value, then it deserves points just like any other task that results in business value. If fixing the “bug” doesn’t result in business value, then what is the reason to fix it at all?

Dave-
I won’t argue with you but I will point out the argument that is made in favor of not assigning points: If we give a team points for finishing a story and then later give them more points for fixing the bugs in the story, it can be argued that the number of points earned exceeds the full nature of the work.

I thought the original context of the post was a set of old bugs that the team inherited when they moved from traditional to agile methods. With that context in mind, I suspect many of the bugs in the tracking system would not need to be fixed at all.

The pattern you mention in your comment is different. It’s a team that generates new bugs in each sprint. Wouldn’t that pattern indicate that there is an opportunity for the team to improve its quality-related practices? The question to ask might be, Why are we generating new bugs in each sprint? What can we do differently?

Hi Dave—
Yes, the context in the post is different (a set of old bugs). And that’s why I said I agreed with you that the bugs should, in this case, be estimated.

The issue of why new bugs are occurring is a different, and very important, one. That does need to be addressed by the team.

Another thought…regarding the question of “later give them more points for fixing the bugs,” doesn’t that depend on the nature of the “bugs?” Thanks to the feedback loops inherent in agile methods, stakeholders will refine their understanding of business needs based on the team’s delivery of solution increments. This is a good thing. If this results in changes to the “requirements” that were previously delivered, is that really a “bug?”

I use a similar approach, but I tell teams to assign points to bug fixes *after* they’ve fixed them. Usually people have lots of trouble using the same scale points that they use for functionality, so what I’ve done is creating a separate scale (small bugs, medium bugs, large bugs…). That way, if you on average do 80 story points and 45.000 bug points, if this sprint you got 60 story points but made 61.000 bug points you can still say you had a good amount of work done. Of course, if the team hasn’t got problems using the same scale, the maths are way simpler :)

Thank you for sharing your thoughts, Mike. You rock!! :)))

No, in that case it wouldn’t be a bug. Bug/Feature is always a gray area. My favorite little toy I ever got at a conference was 10-15 years ago. It was a heavy coin, like a silver dollar. On one side it said “bug” and on the other it said “feature.” I loved that idea as a way to settle the bug/feature debate. I’ve still got the coin.

Interesting article, we are exactly facing the situation you describe. We agreed that we will reserve a number of story points for each sprint for fixing bugs. To handle the work I created a bug backlog, I gave each bug a priority and had a software engineer give an esitmate for fixing the bug.

I personally like to add bugs I want to have fixed as tasks to the scrum board. This way it is easy for the software engineers to pick up bugs. It also means we can move the tasks on the scrum board and discuss them in the daily stand up.

Thank you for the post Mike! I tried to treat tasks like bug fixing as a card with points on it, which in my mind basically becomes estimating a time boxed item. The problem I found was that it was having unintended consequences on my velocity and sort of washing out the fluctuations of a teams velocity over time. For example: if I know my team normally completes about 10 points of work per sprint and I want to re-purpose every 5th sprint to be legacy bug fixing then I would be tempted to put a 10 on that task and have that be the only task for every 5th sprint. Each time we do that sprint of bug fixing my velocity would be 10 exactly. But I knew the velocity would be 10 weeks ago when we planned the bug fixing sprints. And worse, if I use that average of 10 to plan it’s not really an accurate reflection of how the team is performing now, it’s a reflection of how the team was performing prior to creating the task for the bug fixing sprints (which may have been months ago). Maybe your team is really performing at a 5 or a 15 now? Maybe the point values you assign to a time boxed task should be adjusted sprint to sprint based on the last X number of sprints average velocities? But, why not just truly time box the work, don’t estimate it, and don’t measure velocity during that period measure something like bug found/fixed rate. Then you can tell your stakeholder(s) something like “during sprints that we are doing new work we can do about 9 points of work, if we remove a bug fixing sprint you will have ~15 more bugs in product, but you will get ~9 points of new work in that time”. I guess what I am asking is: Should velocity really be used to measure the team during all parts of a development cycle or only for new development work? Maybe there are better ways of showing that the team is making forward progress depending on the type of activity being worked on?

Hi Angel—
It’s not surprising you’ve found it difficult to use the same scale for bugs and new features. Studies have shown we are only good at estimating things across about one order of magnitude. Often bugs are smaller and they are often very difficult to estimate in advance.

Also, thanks for your kind words.

Like every use of story points I guess “what you intend to use them for” determines “how to use them”. In this post we seem to talk about using them for predicting the pace of future development.

It seems to me that “the level of ambition of quality” should be the distinguishing question between if we should have the “bug-points” included in velocity, or not.

If you have made a significant rise in the level of quality - then your new-development leave very few “new” bugs. In that case you will get more useful predictions if you let the amount of bug-fixing be included in velocity. It seems like this is the situation you talk about, Mike.

In this situation it can be argued that there are no distinction between an “old bug”, a “change request”, or a “requirement of a new feature”. They are phenomenologically the same thing: they are observable in the same way (system not behaving as wished), they are addressed in the same way (by humans changing code), they are verified in the same way (through testing), and they compete about the same attention and resources (developer/tester focus, test environments etc).

On the other hand - if you keep the same (low) ambition of quality as when the “old bugs” where written, then you should get no extra story points. The fixing of the old bugs is probably en par with the production of new bugs as you develop new functionality. So all you do is to uphold a “steady flow”. In this case bug-fixing is just “late development” so giving extra points for bugfixing would be “inflated velocity”.

There is one situation I have not been able to wrap my head around completely. If you have a set of bugs that have been “accepted” (i e business have decided “do not fix those, keep on going on the backlog”) at one release. Then, at some later point of time, those bugs are lifted up on the backlog and given priority - should those be counted into velocity? They are still “late development” - but doing them do still constitute an “alternative cost” that could have been spent on proceeding along the product backlog. What is your experience and opinion on that situation?

How do you know the size of the defect?  Quite often they’re icebergs - a little bit of work to find the problem and a ton of effort to fix it - or inverse icebergs, a ton of effort to find the problem and a 5 minute fix.  If you assign points to something of unknown effort, it’s essentially meaningless.

If you do know how much effort goes into repairing the defect, I would suggest that it isn’t a defect at all, but it’s another story.

What I have used successfully is a separate prioritized queue for defects.  During any given sprint the Product Owner has the option of allowing people to pull from the defect queue, with a minimum of one developer and one tester and a maximum of the whole team.  The defects are visible on the board, but they aren’t counted in any way towards the team’s velocity in the sprint.  This is similar in spirit to Ron Jeffries’  concept that a defect is a “negative feature”.

Thanks, Robert. It’s good to know we’re in agreement. ;)

Hi Bryce—
I can’t argue with your points. You do a good job of making the counterargument to my position. I wrote the blog because this is not really a cut and dried situation and the answer really depends on what the team wants to measure with their velocity. Of course the good news is that the problem should not be a persistent one—after all, the bug database (“bug backlog”) shouldn’t be so massive that it takes years to work through. Once the team is “caught up” (and remaining bugs thrown away as too low priority to fix) this problem goes away, which is, of course, the real goal.

Hi Dan—
Thanks for commenting. As always, you’re right and the desired use of velocity will determine which approach the team (and product owner) take here. That’s something always worth remembering.

You’re also right that bugs and new features are the same. I often make that point in my consulting work with this example: Suppose a customer calls your tech support line and hears, “Press 1 to report a bug or press 2 to request a new feature.” The caller would think, “I don’t know which it is. I just want different behavior than now.”  Since to a user, a bug or a feature request is the same, we really want our development process to treat them the same. To some extent that is my point here in that I’m saying “group these X bugs together without individual estimates and just call fixing them to be worth Y points because those Y points are part of our ability to turn ideas into product.”

As for the late development bugs (bugs fixed long, long after they were made): I just go ahead and give those points and treat them just like a regular ol’ user story on the backlog. The idea here is that two years ago we left a bug behind and it wasn’t worth fixing. So we claimed full velocity. That slightly overstated velocity back then. Probably not a big deal. Then two years later we do fix that bug and we give ourselves say 2 more points. We’ll we’re again overstating velocity. But since I tend to work with average velocities and tend to expire ancient history (i.e., our velocity during the height of the Roman Empire is irrelevant) this never seems a big deal.

A key thing for me in all of these discussions is: Will the difference lead me to behave differently? If it won’t, I want to measure things in the simplest manner possible. Counting the bugfix in velocity here overstates velocity (when averaged over a long number of sprints) but likely by such a tiny amount that it doesn’t matter. Since that is easier than making any adjustment and the adjustment wouldn’t lead me to different behavior, I’m good with the simplification.

Hi Dave—
This topic has gone in a lot of different directions because of all the comments. So let me restate the context which I’m assuming:

The team says something like “we’ll spend 40 hours bug fixing this sprint and that’s about 3 story points.” Any time a programmer wants to fix a bug, he or she grabs one and starts in on it. If it looks nasty, they may defer it until another sprint so that it can be truly estimated and we can be sure the PO wants it at that cost. The technique I’m describing here is a good one when a team has, say, 400 bugs that need to be fixed. In such cases, a lot of those are likely fairly quick wins and the others will be found and left for “first class treatment” as product backlog items.

I find it useful to allocate capacity to bug-fixing rather than estimating bug fixes in points. By allocating capacity, I mean: 1 developer (or dev pair) assigned to fixing bugs for that sprint. At some level, it still means translating that 1 dev pair for 1 sprint into a points number, but I feel that this is cleaner than saying velocity was increased because we fixed X number of defects. I would rather say that we have a lower capacity for developing new features because some capacity is allocated to bug-fixing.

This description fits our approach very well!

We have well defined categories of bugs - and yes, these are almost all pre-Agile.  (We have a lot of pre-Agile code!)

“Blockers” are when the software is failing to perform as it should, with no workaround, and a client is “bleeding” in full view of their clients.  We’ve decided that these have to “crash” a sprint - depending on the resources required we might abnormally terminate.  The good news is that as we hack out the worst of the legacy code, these are less common.

Next level down is “critical bugs” - again a software fail, but the support team has a workaround that the user can get on with.  Following some uninspiring “bug bashing” Sprints the team suggested spending 10-25% of a sprint on resolving criticals.

- we have about 40 of these, and we pin them up near the Sprint Board
- we review them from time to time, trying (as best we can) to size them
- we also look for “targets of opportunity” linked to other stories in the sprint
- they are the lowest priority on the sprint compared to development
- having them displayed does help drive the TDD process in the team

This has been working well - a couple of the developers enjoy breaking up the sprint with bug-bashing.  So far new “criticals” are topping up the “reservoir”, but I’ll ensure the level is maintained at ~30.

Hi Guy—
I love that you’ve found an approach that works for you, and it’s nice that we take similar approaches. Good luck getting through the last 30-40.

Hi Mike! Very surprising to find that your POs pass software which has bugs. To put it another way, in your original example, if the PO suspended bug fixing for six weeks, you would have 0 story points of software at the end because none of it would work as intended.

Scrum requires a fully functional, discreet increment of shippable software at the end of each iteration. If your POs are passing software which contains bugs, then you really do have no idea how long development will take as your developers are not actually completing any features. Bugs are unfinished, or non-functional code and all are ascribable to feature development. The BEI of the feature the dev was working on when the bug cropped up is the one which should take the hit from the fix, as until then the original feature should have been completely functional as outlined in the design document. All development required to get a feature functional should be under the feature’s container in order to get a proper velocity, even if it means rework of what was considered to be a complete increment of functionality.

Hi Paul—
First, to be clear, these aren’t “my POs.” These are product owners I work with in all sorts of companies, some quite new to Scrum/agile and others with lots of experience and good results.

However, even so, I will completely defend the practice of “passing software which has bugs.” Not all bugs are worth fixing. A bug that happens to a small set of users once every decade and would cost 20 hours to fix might not be worth fixing. Each fix needs to be prioritized with a conscious decision. Of course for most bugs this is an instantaneous process and carried out by the programmer immediately upon hearing of the bug—-“whoa, that’s a nasty crash. I’ll fix it right now.” But there are plenty of bugs that don’t warrant fixing. I’ll go so far as to say every commercial software is shipping with bugs. Some bugs are known but weren’t worth fixing. Other bugs weren’t even worth looking for because they must happen so infrequently.

All of this, though, is irrelevant as the blog post was about a team which already has a large bug backlog, often because of becoming agile partway into the life of a product. As for the process you describe, I completely agree except there often is no design document and doesn’t need to be. The team should fix bugs they find in the sprint they find them. (Barring the exception that some bugs may be too expensive to justify fixing.)

Mike,
Thanks for sharing. I believe that your post specifically addresses the scenario where the team inherits a list of defects, not the scenario in which the team is having problems creating relatively defect-free code. Feel free to correct me if I’m wrong. In the latter scenario, I believe the sizing of freshly minted defects can create a false sense of velocity, or mask the effort to get quality code http://wp.me/pxikP-18 I would be interested in your thoughts about sizing new defects vs. letting the reduced velocity be the indicator that there is a problem.

Hi Dan—
You are correct that this post is about teams that inherit a list of defects. You are also correct that putting points or new defects will artificially overstate velocity. If a defect is found in the sprint the feature is being developed, fix it right then. If the defect is found within a “few” sprints, it goes on the product backlog so that the PO can decide if it’s worth fixing. It may be given an estimate then (so that the product owner knows what it will cost to fix) but it should not be counted in velocity.

The reason I put “few” in quotes above is because at some point it is actually better to count the bugfix in velocity. I tend to work a lot with rolling average velocities and I’m rarely interested in velocity from more than a year ago. So if a team created a bug 13 months ago and we find it today, I am inclined to give it points and count them in velocity today. Yes, this overstates velocity if we look at the full credit the team got 13 months ago plus the little bit they’ll get for the fix this sprint. But, the value that really should be reduced is the one from 13 months—-and that one is so old we’re not using it any longer in velocity calculations. So, that’s just a slight variation on the idea but in general: No credit for fixing bugs found in user stories the team already took credit for.

Dan-

Mike gives good advice in response to your scenario if there is a steady stream of defects being fixed, steadily, an iteration or two after they were discovered.

Anyone-

In a different scenario though, what do I do if we normally fix defects as soon as they are found (with no estimate) but have a backlog of defects that came from some unnatural event (a real bad iteration). Further, I’m not fixing all of them right now, and I’m not fixing them in a steady stream each iteration (and this is outside of my control). I’m fixing 4 this iteration. I’ll fix none next iteration. Maybe later we’ll fix another batch of them. And I want to know what my velocity is as we go through the next few iterations and I need to know when we’ll be done with this release and all the defects. To me, it doesn’t matter when this batch of defects were introduced or discovered (this release or last). It’s quantifiable work to do. I think I need to estimate that work, and don’t see the downsides of doing so. What am I missing?

Hi Dan—
See my reply to Dan on 29 Nov 2010. (Up two comments above) I address your situation there, especially in the second paragraph.

Other opinions for Andrew?

I break bugs down into the following three categories and assign points as follows:

1)Legacy bugs
I define a “legacy bug” as any bug that was introduced prior to the team adopting Scrum.  Those go on the backlog as user stories and are estimated in points and prioritized by the PO.

2)Scrum Era bugs
Any other “bug”, introduced since adopting Scrum, is fixed within one sprint of it’s discovery(this sprint or next) and no story points assigned, unless both the PO and dev team agree to defer the bug beyond that next sprint(Deferred bugs).  If either the dev team or PO chose to have the bug fixed this sprint or next, then the bug appears as a task for the sprint, and is estimated in hours.

3)Deferred bugs
These go on the backlog and are estimated in Story Points and prioritized by the PO.

While this logic is somewhat complex, all other solutions to the “should I assign story points to bug fixes” are also complex, and usually have much larger risks of bad effects.

Further, I don’t subscribe to the “bugs can’t be well estimated” theory.  Here’s why. 
#1.  I encourage team members to take some time to investigate a bug, up to an hour to estimate a bug that appears to be nasty.  This helps reduce estimation inaccuracies.
#2.  I encourage teams to then “inspect and adapt” on sizing bugs, just as they would for User Stories.  #3 If the team still can’t be terribly accurate after a few iterations, then I would probably just encourage them to spend more time investigating nasty bugs to obtain a better estimate—this would be an extreme situation in my opinion(very low quality product).  Note that they can only investigate for an estimate, the time is not meant to spend diagnosing or fixing the bug.

I also don’t agree with a regular time-boxed bug fixing period because I feel it hinders transparency and makes it too easy for teams to sweep new bugs under the rug.  “We’ll have time to fix this later in the bug fixing sprint or in the bug fixing time-box next sprint.”

Having said all of the above, while PO’s are certainly allowed to prioritize bugs into sprints, the dev team is also allowed to pick any bug it feels needs fixing (whether it be on the backlog or not), and bring that into the sprint as a task (they gain no story/velocity points for this, regardless of whether the bug was on the backlog or not).  Only the dev team can decide how much product backlog to bring into a sprint, so if they decide to spend ~ 50% of their sprint bug fixing things not prioritized by the PO, so be it, but it will be visible because their velocity will be lower.

So, in short, Legacy and Deferred bugs can be fixed either
a) When they are prioritized into the sprint because the PO perceives business value gained (story points added into velocity), OR
b) When they are brought into a sprint by the dev team because the dev team perceives value, and no story points are added into velocity as a result of fixing these bugs.

According to Mike’s User Stories Book:
“...the developers will estimate how much work they’ll be able to do per iteration. We call this velocity…” 

Further if you measure velocity in User Story Points, then each User Story must be
“...valuable to either a user or purchaser of a system”

So, velocity is meant to be an estimate of a team’s iteration capacity to deliver User Stories that are of value to users and stakeholders outside of the dev team.  I believe the above strategies that I describe do just that.

I could be wrong though, and I encourage others to let me know if they think I am!

My get feel is always to treat bug investigation/fixing as undesirable in the same way as other undesirables such as CiFails, CodeReviewFails and environmentFails.  So I would normally record these tasks as “unplanned” work for discussion in retrospectives. I normaly have three categories at the story level: project (stuff the customer wants directly), engineering (TechnicalDebt et al.) and defects and each can contain tasks which may be planned (counts towards velocity) or unplanned (“background noise”).  The Product Owner has to understand that sometimes things get in the way of delivering explicit buisiness value and whilst they are at liberty to bring large chunks of “unplanned” defect fixes into a sprint, they have to be cognizant of the negative impact it is likely to have on velocity.

Having said that, in the scenario above where there are historical bugs being taken over by a new team, the bugs are effectively new feature requests since the new team has only a single production baseline (i.e. “working software”) to work from so everything is a new feature.  So in this case I would say all the bug fixes should be “planned” work.  It may still, however, be desirable to distinguish between defect, project and engineering stories but all should count towards velocity for the new team.

Hi Richard—
I agree. The key distinction for me is whether fixing the bug represents something the team already took credit for (by saying the feature was done in the first place) or whether it’s old.

Hi Mike,

Great post! 1st paragraph on 29 Nov 2010 is what I was looking for. I think I know now how to handle newly found defects and legacy ones!

Regards,
Ch

Thanks, Christophe. I’m glad you found it helpful.

Can “yesterday’s weather” be helpful with migrating from block-allocating time to legacy defects to comparing them with similar defects to generate a level-of-effort in points, even if they’re a different kind of points? Admittedly, the value may jump or drop dramatically after a cursory investigation of the symptoms due to difficulty in reproducing or subtlety of effect. However, on average, defects with a particular general sort of symptoms in a particular part of the code are likely to require similar effort, no?

Hi Lupestro—
I don’t see how the idea of “Yesterday’s Weather” is related. Yesterday’s Weather is the idea that the team will do the same amount of work (usually measured in story points) in this sprint as they did in last sprint. It’s a very simplistic sprint planning technique.

But, of course, your idea of comparing relative effort in points is valid—that’s the idea behind points.