Polishing Software and the Death of a Thousand Cuts

Any large software project will typically use some form of bug/issue tracking software to keep tabs on all of the bugs and opportunities for improvements that get identified during the course of the project. This database may contain thousands of issues, prioritized by importance. Inevitably, there will never be enough time to address all of them, so as release time approaches, the triaging process typically gets stricter, with the classification of issues into things that there is still time to fix, and ones that won't be fixed in the release.

The question that this blog post will try to address is: what should we do with all of those issues?

Issue Triaging

There are two main types of software releases: products that are released once, and products that are maintained with subsequent versions. This distinction is very important when it comes to bug tracking, because issues that are found in a single release project can be treated very differently to a release with multiple versions.

With a single release, such as a typical game, an issue that doesn't get dealt with for the release will possibly get handled in a patch, but otherwise will never be resolved and can basically be forgotten about. Since patches tend to focus on serious issues or issues that are found after release, any issue found before release but deemed too low priority to resolve is probably never going to get fixed.

With products that have multiple versions, an issues can't be forgotten so easily. If you defer an issue from the current release, that will still leave it open for fixing on the next release. Just because something is low priority now (compared to other issues) does not mean it will still be the case when the next software version rolls around. Particularly for software that needs to remain backwards compatible with previous versions, issues can never truly go away unless they are fixed.

So, this all sounds fairly obvious and straightforward. The problem arises with a pattern that tends to emerge with a lot of low priority issues, which is that they stay low priority, and get shifted from release to release without ever getting fixed. Your issue database gradually fills up with hundreds or thousands of these issues that are never important enough to spend time fixing, but still exist in your software. What can you do about them?

• Keep them in the system - a bug is a bug is a bug.
• Remove after some number of releases without being fixed. Kind of like a 'three strikes' system or similar, where you say that if it didn't become a high enough priority after two, three, whatever releases, it never will, so close it as a "won't fix".
• Raise the priority of the issue after each release. This would mean that it eventually becomes important enough to fix, but in practice this artificial gaming of the triage system doesn't really work, since people will recognize that they're missing out on fixing more important issues in favour of ones that are marked as high priority, but really aren't.
• Mark as "won't fix" immediately and forget about it unless it gets raised independently again.

It's this final option that bothers me. The idea is that if an issue is not worth fixing now, it's not worth fixing at all, so just mark it as "won't fix" if it doesn't make the cut in your triaging. This makes for a much cleaner issue database, but is it actually a good idea?

Software Polish

Many pieces of software do what they're designed to do, but may have clunky interfaces, various minor behavioural quirks, and so on. We would tend to think of these as good software, but not great software (bad software is a program that can't even perform the job it was designed for!). The difference between good and great software is typically what we think of as polish. Getting rid of all of those little annoyances, cleaning up the UI, streamlining the user experience, these are things that rarely involve big issues, but are rather a collection of lots of tiny issues. Typically, none of those issues will be a big deal on their own, but when you accumulate a lot of them, you end up with software that feels unpolished. With games, you'll say that they needed another 3 or 6 months to finish it. With versioned software, you call it Vista (snap!).

So software being unpolished can be thought of as being like a death of a thousand cuts. None of those cuts is a big problem on its own, but they add up, and you eventually reach a point where you realize that you're in trouble. How can you avoid this downward slide into unpolished software, or probably more realistically, how can you make your project take the uphill march that ends in a polished product?

Issue Tracking is Polish Tracking

This is where your issue tracking is your friend. If you're accumulating lots of low priority issues, this is a warning sign that your product is unpolished. By just marking them all as "won't fix", you lose this important information and get a false sense of the quality of the software. Not only do you have the appearance of less open issues that need resolving than is actually true, but you also add confusion as to which issues you chose not to fix because they weren't genuine bugs, and which weren't fixed purely due to triaging.

Perhaps having all of these issues in the database is telling you that you're not spending enough time fixing issues, and that's why they're accumulating.

And this brings us to one more option for dealing with a large number of low priority issues, one that I left off the list above: spend more time polishing, and less time adding features on your next release!

Every developer prefers adding awesome new features over fixing bugs, and customers certainly like new toys, but it's also true that developers like to feel pride in their work, and customers like to use software that doesn't annoy them. It can be a hard balance to meet, but when your low priority issues start to accumulate, rather than ignoring them, it might be time to recognize that your software is becoming less polished, and you need to put more effort into fixing those issues.

Of course, there are always trade-off based on the number of developers you have, the time until the next release, and how keen your customers are to get new feature X. Maybe you need to shift your priorities, maybe you need to hire more developers, maybe you need to extend your release times, or maybe you need to accept that your software will become less polished. The key point is that this should be a conscious decision, well thought out and based on the available evidence. Marking off issues as "won't fix" that aren't actually fixed distorts your data and makes it harder to have a true picture of your current situation. And if you don't properly understand your current situation, you're less likely to make a good decision for the future.