The Right Amount of Technical Debt
How do you determine the right amount of technical debt on a project? Both too much and too little can slow you down.
JB Rainsberger just put up a very interesting blog post about technical debt. Basically, it is wrestling with this contradiction around technical debt. On one hand, if we decide to eliminate technical debt by making sure all our code is absolutely perfect, it takes a really long time to ship anything. On the other hand, if we just throw caution to the wind and our only goal is to ship code and we let the technical debt pile up, at some point the weight of the debt starts to slow us down. So as developers, we need to both incur technical debt to get things out the door promptly, and at the same time, we have to pay down technical debt to ship things promptly. The result is we all end up arguing about what the right amount of technical debt is: Are we spending too much or not enough time cleaning up?
If I understood the article correctly, JB's take is that managing technical debt is inevitable. We can't just take either extreme and either avoid all technical debt or let ourselves drown in it. He views the arguing over the right amount of technical debt as a sign of a lack of trust among developers. Trust is very important and there is something else at work here as well.
Be clear about what you are optimizing for.
Something that we don't do enough as software developers is take a step back and look at the big picture. We get caught up arguing over the details and lose sight of what's important. The first paragraph made an assumption that delivering code promptly and working iteratively was important. Why is that? We have to ask ourselves why we work that way and what are we trying to achieve? I think if we as developers can learn to step back and focus on what is important, then that, combined with a little trust, will go a long way to alleviating arguments over how much technical debt is the right amount.
Optimize For Learning
I recently recorded a podcast interview for the LabVIEW Experiment with Kyle Griffin Aretae. As I am writing this, it has not been released yet. We talked about Agile and why it is misunderstood. He mentioned that there are various things that you can optimize for and that a large part of the disagreements within the Agile Community are because different people are optimizing for different things.
Through that conversation, it became very clear to me that I personally am optimizing for speed of learning. It stems from the fact that a lot of my projects are R&D projects. No one has ever done this particular task before so it is all new and all about learning. As part of that, the customer doesn't know exactly what they want. I don't know if what they think they want is even possible. There are a lot of unknowns. The only way to find out what works is to experiment. My whole development process revolves around taking small steps and after each step getting working software into my customers' hands so they can try it out and we can get feedback. I want to be able to do that as often as possible - that is what I am optimizing for.
Fix Today's Problem and Tomorrow's
Before our interview, I ran into Kyle at a local meetup. We had a very interesting conversation. He asked a question something like "What is the overarching goal of Agile?' I blurted out "Fixing the customer's problem." Kyle corrected me, and rightfully so. He said, "It's not just about fixing the customer's problem today, but also fixing it tomorrow and into the future". If we focus on just fixing today's problem, we lose sight of the big picture and may be tempted to take on too much technical debt, which would slow us down when it comes time to solve tomorrow's problem.
Finding Balance
Finding balance is definitely something I struggle with. Specifically, I struggle when it comes to cleaning up code. Since I optimize for learning, I write a lot of code that gets thrown away. Does it make sense to spend a ton of time polishing code that may get thrown away? If so how much time? Is there a limit? A point of diminishing returns? Is that time better spent getting the code into the customer's hands quicker? What if that code ends up not getting thrown away?
From my point of view, I see a tendency in the LabVIEW community to over-engineer things. I look at some of my fellow LabVIEW programmers straightening every wire and say to them "You should stop spending so much time polishing things and actually ship them." At the same time, I've had other LabVIEW programmers look at my code and say I operate too loosely, that I take on too much technical debt and I should slow down and spend more time cleaning things up. Perhaps both sides are right and we are just optimizing for different things therefore our particular balance points are different.
Back to JB
Towards the end of his article, JB talks about how the correct balance or the right amount of technical debt can vary between teams and projects. In the end, it's all about experimenting and constantly adjusting. Just as I am constantly learning about my customer's needs and the domain, I am also constantly learning about exactly how much technical debt is appropriate. If I notice that I'm slowing down then that is a good sign that I either need to take on a little more debt or spend some time paying down the existing debt. Usually, in my case, it is the latter. The good news is software is malleable and I've spent a lot of time honing my refactoring skills.
