What is the Value Proposition of CI/CD?

Do you use CI/CD? If not, why not? If so, what benefits do you see?

Samuel Taggart

Nov 8, 2023 • 7 min read

As some of you may know I am running a CI/CD in LabVIEW workshop at the end of the month. I was talking to some members of my Mastermind Group and I got a little pushback. They said "You haven't done a great job of selling the value of CI/CD." So this post is hopefully going to outline the benefits of CI/CD.

2 Types of Problems

In general, when writing software, there are 2 types of problems we can run into: "doing the thing wrong" and "doing the wrong thing".

Doing the thing wrong

The first type of problems are your classic software bugs. We intend to do some calculation, but there is an error in our code and we come up with the wrong value. We're very clear about what we intend for the code to do, but the code is just not doing it. We generally catch these problems with some kind of testing, usually unit testing or integration testing. Occasionally these might slip through our testing and get caught in production if they deal with some kind of edge case that we didn't account for.

Doing the wrong thing

The second problem we run into is the situation in which the code is doing exactly what we as the developer intended, but that isn't what the end user wanted. We're doing the wrong thing. The problem isn't in the code, but in our communication with the end user. We can't catch these with tests, we need to get feedback from our customers actually using our software to catch these. That means we need to get the code out the door and into the customer's hands before we can find out we did the wrong thing.

Catching Problems Early

I find that catching both types of problems early makes them easy to fix. The longer it takes to identify that there is a problem, the more changes that have been made to the software in that time frame. If the software works and I create a new version and a feature suddenly stops working, I have to look through all the changes between the working version and the new broken version. Sorting through a handful of commits is much easier than sorting through hundreds. The other advantage of finding out there is a problem quickly is that it is a lot easier to remember something I just did than something I did last month. The memory and the context are much fresher in my head. That makes the problem easier to solve.

How do we catch them early?

CI/CD is the easiest way to catch both of these types of problems early. The CI part contributes to catching the traditional software bugs and the CD part contributes to catching miscommunication or bad requirements.

If the CI part of our system runs our tests every time we push, then we will immediately detect any regressions. If we are also using TDD, then we are also immediately making sure any new code we write works as expected. If we are pushing fairly often, then the number of commits to sort through should be small and the changes should be fairly recent so remembering what you were thinking when you made the changes should be easy. This helps to ensure we build the thing right.

Testing cannot help us with catching communication or requirements problems. We can only get that feedback from our customers actually using our software. For that, the CD part of the system can help us by making it very easy for us to release the software to our customers. This makes it easy for them to install the latest version and verify that we understood their requirements and that we built the right thing.

Ideally, the customers would be installing new software every time we make a release. That doesn't always happen. Making frequent releases is still valuable in those cases. Let's say our customer is on version 1.0.0 and we release 1.0.1, 1.0.2,1.0.3, 1.0.4. They upgrade from 1.0.0 to 1.0.4. and notice that somewhere along the way there was a problem. Because we did frequent releases, we can have the customer go back and install and test 1.0.1,1.0.2,1.0.3. We can then figure out exactly which version broke things, and narrow our search to just the changes introduced in that version. Otherwise, we'd have to look at all the changes between 1.0.0 and 1.0.4.

Why Automate?

Aside from preventing the 2 major causes of software problems, CI/CD also gives us all the benefits of automation. These are the selling points that we use to sell automated test and measurement systems to our customers.

Automated systems are

faster
more repeatable
less prone to human error
less prone to people forgetting to run them

Challenges

Nothing is perfect. There are some challenges.

I Don't Have Unit Tests

Sometimes I hear people say "Well I would consider CI/CD but I don't have any unit tests." Well there is no better time to start writing them than now, but you can also automate plenty of other things besides unit tests:

building executables
building installers and packages
building documentation
running VI analyzer
running something like pylavi which can check for VIs with breakpoints, or with compiled code included
running the DQMH module validator

So you can still catch a bunch of problems with CI/CD even if you don't have unit tests. If you use a tool like my lunit or vitester G-CLI tools they both pass by default if they don't find any tests. So you can add a testing step to your CI pipeline even if you don't have any tests yet and then if in the future you add tests they'll automatically be run for you.

What if I can't deploy all the time?

This is also something I hear a lot. I have 2 relevant stories to share here.

The importance of having a testing environment

I used to work in the nuclear industry. We built monitoring systems for nuclear power plants. These sat inside containment. There was limited network access and no real way to update them over the network. Updating them required shutting down the reactor, having someone don a radiation protective suit, and go into containment with a USB stick. That didn't happen very often. And when it did, we certainly wanted to make sure we got it right.

Now our software development team there was not very agile, but we did find a way to make sure that we could be ready to deploy at any time. We built a test system in our lab that was as close to the real system as we could get. We could deploy to that whenever we wanted. That gave us the ability to frequently test our code out on real hardware. We weren't doing CI/CD at the time, but it would have worked out really well if we had. We could have set up our CI/CD pipeline to deploy to testing either every night or every release or something similar. Being unable to deploy directly to production doesn't need to stop you from doing CI/CD.

The Ability to Respond Quickly in an Emergency

I was at the software crafters conference recently. There was a guy there who was a consultant who worked for a major retailer. They were very restrictive about when updates could be deployed. It could only happen once a month in the middle of the night and no updates from October until January due to the holiday season. Of course, this customer had much higher deployment failure rates than their other customers due to the large batch size - when you deploy many changes at once the chance of one failing is much higher. Also, the mean time to restore was also very high due to the batching because it was harder to tell which of the many changes caused the problem. He was looking for advice on how to sell them on deploying more often.

I gave him some very specific advice - to sell the customer on the ability to respond quickly in case of an emergency. Now the customer certainly didn't want any updates during their busy season. However, you have to imagine that if they found a showstopping bug during the busy season, they would want it fixed as quickly as possible. That is the key - building the ability to respond quickly in an emergency. My advice was to talk the customer into allowing them to deploy more often during the non-busy season. The selling point was that this is practice. They are building the muscle to be able to push a bugfix very quickly. Then if it is ever needed during the busy season, they will be prepared. I'm not sure if the developer was actually able to sell it to the customer, but to me, it made a lot of sense.

Getting Started

So if you have looked at CI/CD in LabVIEW, you have realized there are a lot more challenges than what I've posted above, particularly the technical challenges of simply setting it all up and making it work. I've been working hard on trying to lower that barrier of entry.

I want to do it myself

If you are the type of person who likes to do things yourself, I've got some resources to help you. I've put together a set of G-CLI tools and I've been documenting my setup and the things I've learned along the way. Check out https://sas-gcli-tools.gitlab.io

I like to do things the easy way

There are a lot of little details to get right and it can be daunting. That's normal. To make it easier, I put together a half-day workshop, where we'll sit down and tackle it as a group. We'll get you set up with a runner and get a template running on your machine. We'll explain how it works. In the end, you'll be able to take that template and apply it to any project to get a CI/CD pipeline working in a matter of minutes. You'll also know how to change it to reflect your software process.