Betting, probabilities, and risk in software development
Over the years I’ve come to realize that an important aspect of practical software development is that it’s a bit like gambling.
Let me elaborate.
On any given project we’re faced with a lot of choices, options and decisions. Any particular feature, enhancement or change we are making can have a multitude of aspects we need to consider and solve for. Our team at OK GROW! experiences this as well and we’d certainly love to solve all of them.
The problem is, we can’t. There just isn’t enough time to solve every problem we come across. No solution we create will be “perfect” in the sense that it cannot be broken, someone cannot throw it a curve ball and so on.
So what are we to do?
This is where the gambling part comes in.
I think it helps to think about about writing code like placing a bet. In effect, that’s what we are doing: placing a bet. We are betting our time, energy, effort, code, tests, additional complexity perhaps, etc. All of this ends up boiling down to money anyway (i.e., time is money). So when we write code, we are placing a bet and we hope that bet pays off, that it ends up being worth it.
As with gambling, we seek to assess the probabilities and risks before we place the bet.
Much of what we do in thinking about implementing some feature, enhancement, or even a bug fix involves assessing all of the various cases or situations that could arise. We ask a lot of “What if…?” questions to make sure we are covering everything. Usually what we’re looking for are negative or failure cases. But not every “what if” case is created equal. Each has three things for us to consider:
The probability it will occur
The down-side risk if it does occur
The size of the bet necessary to address it
This process is collaborative with the technical/development team and the business/product team.
In most software development projects “the business” is the group that is both paying for and guiding what should be developed. They also have to live with the results and pay some of the costs if things don’t work well. They can help in estimating the probabilities of something happening with your app as well as the downside risk or cost if it does.
The development team however is who is best positioned to predict the size of the bet or the cost to address a certain problem. This team might also have something to say about probabilities and risks if the issue is a of a technical nature.
When probabilities of something happening are high, the downside risk is large and the bet is small, the decision-making is generally pretty easy.
Things get tricky with “edge cases” or “corner cases” - situations that we already recognize are uncommon or unlikely. As developers we often want to build the perfect solution. We want to cover all cases. Sometimes even the business gets into this game…creating the perfect piece of software.
The reality however, is that there are limits, and success depends on being careful about not placing too many large bets on low probability cases and knowing how to handle and/or limit the downsides when not placing the bet.
How can we best handle this?
First, it’s important to acknowledge that we are trying to predict the future here and if we could do that perfectly, there would be no gambling involved. We would just know.
So when we’re assessing probabilities, risks, and bet sizes, what we’re doing is making guesses…hopefully reasonably educated and informed guesses.
But let’s go for it.
When thinking about probabilities, percentages are a common and useful way to go. Lower percentages indicate something having a lower probability, and higher percentages a higher probability. In gambling it is common to use odds in the form of a ratio or fraction (e.g., 1-1, 3-2, 9-1, etc.) These are not exactly the same things but, they’re close enough for our purposes here. And, for our purposes, odds are just a different way of saying a percentage (e.g., 50%, 40%, 10%, etc.) We’ll stick with percentages because they are simpler and more widely understood by most people.
When thinking about bet sizes…or the effort…some people use “T-shirt sizes” (e.g., large, medium, small, etc.) Another common approach these days is using “story points” from a Fibonacci sequence (1, 2, 3, 5, 8, 13, 21, etc.) which represent a scale of increasing size, complexity and/or uncertainty for completing something. Basically a 1 is an easy, well-understood “slam dunk” and a 21 is a “gosh…we really have no idea…break this down or give us more details please” kind of thing. Either of these can work for this but I’m going to suggest using points for this approach for two reasons. First, it’s a number which will become useful a little later here. Second, the idea of a non-linear scale for this estimation is very useful. When to jump from let’s say an 8 point to 13 point estimate you’re communicating a bit more than going from say a medium to large.
Risk is another one that people often measure in T-shirt sizes. And that’s probably fine for a start, but I’m going to suggest using points in much the same way we do to estimate development effort for a story. Using the same Fibonacci sequence gives a number and perhaps a bit more information in that number. So a pretty low-risk item is a 1, 2 or 3. Then 13, 21, etc. indicate pretty high risk.
Keep in mind these are just tools to be able to make some quick assessments about moving forward. They’re not intended to be mathematically precise measurements. It’s a guessing game. We’re trying to optimize the time to make a decision and hoping to increase the quality of our decisions and our success by placing more of the right kinds of bets.
Okay, with this ground work in place, let’s make some decisions.
First, let’s lay out some random examples:
What can we do with these numbers?
I like to see if I can combine multiple numbers into one to simplify decisions like this. So let’s give that a try. I first thought about just adding or multiplying them with the idea that larger numbers would indicate which items we ought to bet on. The problem is that larger bets will also increase the total and that’s not really what we want. We want the bet to reduce the total in proportion to its size. So I started with adding the probability and risk and dividing that by the bet. Here’s what we get:
This isn’t too bad. But I felt like just adding probability and risk together didn’t really express their magnitude well. So I changed to multiplying probability and risk and still dividing that total by the bet to get this:
That’s more like it. In this table I can see quickly see by the summary column which items might be the best bets to place. Items A and E bubble to the top. And when I look at the detail, this kind of confirms things.
Item A has a high probability (80%) and a high-ish (13) risk if we don’t address it and very low cost of addressing it (3 points). We should probably make this bet.
Item E has a much lower probability (40%) but a pretty high (21) risk if we don’t address it and medium-ish cost of addressing it (8 points). We might want to make this bet.
Just to be sure about this, let’s sanity check a couple others.
Item C has a high probability (80%) but a pretty low downside risk (3) and the cost of addressing it is pretty darn high (21). This seems like something we might be willing to take the risk on for now anyway.
How about Item D? Well, it has a pretty high risk (21), the cost is medium-ish (8) but the probability is pretty low (20%). Here again, the total (53) suggests maybe we could go either way. Which brings us to…
Here is the final, sorted list of bets we could place (i.e., items we could potentially work on):
When we look at all of these together. The items we probably should address (with our limited resources) start to clearly bubble to the top.
When developing software we need to recognize the reality of our limited resources as well as the reality that we can’t deliver perfect software. We try to think about and consider all of the aspects of what we’re creating including all of the negative cases both broad and edge/corner cases. But this decision-making is complicated by other factors including the probability of those things happening, the downside risk if they do happen and the cost to address or fix them and, finally, the fact that we often need to look at multiple different things and pick only a subset to address. I’ve proposed a simple decision-making method to help us relatively quickly make a decision and, hopefully, improve the overall success of the bets we place in developing software.
Need help optimizing development solutions? Get in touch!