Waltzing With Bears

I used to be an avid rock climber, ice climber, mountaineer, and backcountry skier. I say used to, because having kids has slowed me down a bit. In those sports, risk management is very important. Ignoring it can easily have deadly consequences. I still teach mountaineering classes for the Colorado Mountain Club. I always tell my students that technical skills are necessary and easy to teach. They just take a little practice. They aren't the most important thing. The most important skill required for mountaineering and the hardest thing to teach is good judgment aka risk management.

As a LabVIEW consultant, risk management is also important. Of course, I am always concerned with managing my own level of risk, but good risk management skills have other benefits. Being able to discuss risk management with my clients helps me to convince them to launch new projects. It also helps me convince clients of the value of updating and maintaining existing projects. I've also used risk management to help sell clients on better ways of working. I've become very well-versed in risk management.

Steve Watts talks about the book Waltzing With Bears all the time and speaks very highly of it. It's been sitting on my shelf for a while. I finally got around to reading it. Overall I liked it. I think maybe I built it up a little too much in my head, because it felt like there was something missing. We'll get to that part, but first, let's talk about what I liked about the book.

The Good

There were several parts that really resonated with me.

The Opening

There is a story at the beginning of the book that really resonated with me. It talks about the idea that you can't judge someone's decision making based on the outcome. You can only judge it on the knowledge and experience that they had at the time. I always tell my mountaineering students: "Just because you were successful doesn't mean you made good decisions." This book is all about how to make good decisions.

The Nano Percent Date

I immediately identified with the idea of the nano percent date. When someone asks me how soon I can have something done, I'm always tempted to just do the math on the happy path in my head and blurt that super-optimistic answer. I catch myself wanting to do that all the time. That super-optimistic date is the nano percent date: the date at which if absolutely nothing goes wrong you could finish the project by that date. It's unrealistic. It's an almost, but not quite zero, possibility hence the nano-percent date.

The nano percent date is the earliest you could deliver the project, but not the most likely date. The most likely date is much further out. Worse than that though, is the fact that the probability graph has a very long tail. Let's say the nano percent date is 2 months. The most likely date might be 5 months. The long tail says that there's a chance it could take 12 months or more. If you promise 2 months, but it ends up taking 12, that results in a very unhappy customer.

How do you deal with this? Well, the book gives 2 pieces of advice. The first one is to give a more conservative estimate somewhere around a 50% chance of being done and to put bounds on it. Make it clear when you are likely to be done, but also what the optimistic and pessimistic answers are as well. I think that is very good advice. Honesty and transparency go a long way to building trust. The other piece of advice is to build in some margin in both the schedule and budget. Have a contingency fund set aside and plan to deliver your project well before the desired deadline (that really means if your deadline is fixed, you have to start it early enough.)

Defining Risk

The book does a good job of defining and quantifying risk. They work off of the common definition of risk as the probability of something going wrong times the consequence. This common definition is really useful because it helps us to prioritize risks. Understanding that risk has these 2 components is useful when talking about mitigation.

Mitigating Risk

The book does a good job of discussing the various mitigation strategies. Mitigation really comes to lowering one of the 2 components of risk. We can either lower the probability of something bad happening or lower the consequences if it does happen. If we are lucky, maybe we can find something that addresses both factors. The key point that I picked up from the book was that all mitigations have a cost. It is kind of like insurance. You pay now, to avoid paying much more later. It is worth bearing that in mind when talking about mitigations.

The Could Be Better

I mentioned that there was something missing from the book and that something is a pragmatic approach to quantifying risk. The authors certainly try, but I feel like they fall short. I feel like as software engineers often do they overly complicate things.

The author present this Monte Carlo simulation in an Excel Spreadsheet, which is great. It is certainly much better than nothing and much easier than crunching all the numbers itself. My problem is it feels overly complicated and it requires you to input probability curves for the various risks. I'm a small consultancy. I have some data, but enough to make it statistically significant. Sure I could input some data, but I'm not sure it would give me relevant output.

The Pragmatic Approach

The approach presented certainly works if you have the data to make it meaningful and are willing to take the time fill it out and explain it to your audience, but there are more practical tools.

A Simple Risk Spreadsheet

I watched Steve give a presentation on this subject a few years ago. He showed a very pragmatic solution and it very much mimics what I've learned to do through rock climbing and mountaineering.

You really just need a simple spreadsheet. You list all the risks and for each one you rank the probability and consequence. No probability graphs, just a single number 1-5 for each. If you want a little more granularity you could use 1-10. Multiply them together and that is the severity for each risk. You can also have a column where you list any mitigations and maybe even have separate columns for the mitigated risk scores, and maybe even the cost of the mitigation. Yes it is not as accurate, but it is quick and easy.

It lets you make quick decisions and rank risks in terms of priority. You can have limits for any individual risk and add them up and have a limit on the overall risk. Any time either goes over the defined limit then you need to do some mitigation. Ask yourself "What could we do to lower the probability or consequence of this risk to an acceptable level?"

Using Process

Here at SAS, I rarely do one of these risk diagrams. It is certainly in my tool box, but generally I find I don't need that level of formality. There are certain recurring risks in software. Running over schedule or budget, introducing bugs, misunderstanding of requirements, building the wrong thing, needing to roll back to a previous version, integrating changes from various developer, scope creep, etc. We simply build mitigation for those common problems into our process.

At SAS we believe in "Small Steps and Quick Feedback" Our process helps to eliminate many of those risks. The quick feedback ensures that we are building the right thing, because customers constantly have input. And if we do build the wrong the thing, because we take small steps, it is easy to throw that small step away. We use CD to continuously deliver working software to our customers for evaluation. We use Unit Testing, TDD, and CI to make sure that we catch as many bugs as we can before they make it to our customers. We prefer pair and ensemble programming so it lessens the issues caused by integrating changes from various developers. Pairing is builtin code review so that helps with lowering the number of bugs. It also helps to make sure everyone stays on the same page which lowers the risk of staff turnover, at least as far as important and unique knowledge walking out the door.

When it comes to schedule and budget, working iteratively and actively involving your clients has lots of advantages. Our clients help us prioritize work so we get the most important things done first. If the project loses funding, we've at least got the most important things done. If we run out of schedule or budget it is never a surprise. They are involved the whole way so they know far in advance if there is going to be an issue and are able to plan better better.

I want to learn more

We've got a couple upcoming opportunities where you can learn more about this topic.

Selling Your Ideas: The Economics of Software Development

I am doing a presentation for Empathy in Tech in June. As I'm writing this, it isn't listed on their Eventbrite page, but it should be soon. We'll talk about risk management and how you can use it help sell your ideas. It's free. I highly encourage you to check it out.

GDevCon NA Workshop

I am teaming up with DSH Workshops to give a workshop at GDevCon NA. We are going to talk about the entire software development lifecycle. We will certainly talk about risk and some tools to manage it. You can sign up using the button below. NOTE: You'll need to purchase a ticket to GDevCon NA. Once you select a ticket then you will see an option to purchase the workshop as well.