So for my first #BlogADayMay, I’m doing a rant I’ve had saved up for a while. Continuous Deployment, which is one of the new shiny hawtness going around, isn’t always the right thing.
Yes, it’s shocking – not every technique is universally applicable. In this post, I want to explore some of the reasons why you may not want to do Continuous Deployment.
First, though, let me make something clear: Continuous Deployment is not a bad thing to do. There are many situations where it is the ideal thing to do, and many more where moving towards Continuous Deployment is a good idea (even if you fall short and don’t ever achieve it). It’s just not 100% applicable.
Why wouldn’t you use Continuous Deployment?
Here’s some reasons not to use Continuous Deployment; there are others.
Work In Progress (WIP)
WIP is a big one. In order to use Continuous Deployment – or even less extreme versions, such as the ‘always work on master/no feature branches’ crowd – you need to be comfortable with having unfinished work in your production code. That takes a lot of support infrastructure to make happen – mainly in the form of Feature Toggles.
There’s a lot of benefit involved in limiting Work In Progress, and reducing how long work stays in-progress, but it can’t be eliminated for any development pipeline that involves more than one person. You will need to be very comfortable with managing Feature Toggles to be able to do Continuous Deployment.
(Sidebar: it’s very easy to not limit WIP when using Feature Toggles. It takes discipline to go through and turn the features on for all users, and the WIP is not finished until the Feature Toggle is removed. The worst part is when you end up with multiple Feature Toggles in the same area. But I’ll save that rant for another post)
Testing Process
If you are going to do Continuous Deployment, you better have confidence in your automated testing for regression testing. Now, automated testing is an area where it’s always good to invest – but you shouldn’t think of doing Continuous Deployment until you’ve done release after release with zero bugs being found via your pre-release testing process.
You also need to make this pretty fast. If you’re doing Continuous Deployment, you’re essentially pushing to production as a result of every commit to the master/release branch. This will happen via a build pipeline that will test & verify the changes before releasing – and if this gets stopped at the very last stage, it will get rolled back. If this takes time, there will be other changes in the pipeline – and they all need to get rolled back and re-committed.
If your build pipeline takes hours, you’re not going to get benefit from Continuous Deployment.
Deployment Environment
This is a bit overlooked in a lot of the stuff I’ve read promoting Continuous Deployment, but… not all deployment environments are amenable to Continuous Deployment in the first place.
Serverside and web development is (very) amenable. But it’s not the be-all and end-all of software. There’s other platforms, such as mobile and desktop. There is software that is hard to update, such as embedded software in devices that aren’t connected on-line. There is software used in regulated environments, where new versions must be certified before being released.
If you can’t just deploy a change anytime you want to, you can’t do Continuous Deployment.
Note that even in these environments, you can still get a lot of benefit in having a master branch that’s ready to be released all the time.
Customer Acceptance
We write software to be used by people. Some people don’t cope well with change. Especially when you write software used by people to do their work, change management is a big deal. I’ve known users of software to go into rages because of a screen change, because it reduced their productivity because their muscle memory was now broken.
Frequent change can seriously piss your userbase off. You can manage this with Feature Toggles – deploying the software but not turning features on – but if you have features that stay turned off for months while you get ready to roll them out to users, you have WIP that builds up, and you lose a lot of the value of Continuous Deployment.
Cost-Benefit Analysis
Finally – doing Continuous Deployment can be expensive. It takes support infrastructure, discipline, and vigilance.
If your rate of delivery isn’t that high, or your customers don’t want continuous updates, or for any other reason where you just can’t get the value, then it’s just not worth doing.
Wrap up
There’s nothing in this post that represents some great insight. It boils down to: “Don’t just believe the hype; stop and consider”.
If you can do it, do so. If you can’t, consider progressing towards it. Start somewhere where you have control and the impact is low – maybe refresh a test environment on a daily basis, or produce nightly builds that you can push out to mobile devices via TestFlight.
Share this:
Software is too expensive to build cheaply.... 
I understand what you’re saying, but the reasons you give to not use CI sound like they’re mostly reasons related to a bad implementation of the CI process itself. So you *almost* could have summed up the article by saying, “It’s not worth using CI if you don’t have it implemented correctly or if your project isn’t large enough to justify the costs”, and I would agree with that. But once you scale your dev team up past a certain size, CI (when implemented correctly) is by far the best method I’ve ever *personally* experienced deploying with. It still requires attention to things you mentioned like Code Toggles, well written unit tests, and devs that care about quality code. But one thing I know for sure that CI has over other methods of deployment: it’s much easier and far less disruptive to rollback small incremental changes than a major release.
Also, your “Customer Acceptance” section is mostly true regardless of the method of deployment, and I’d argue that can be a bonus for using CI: You can know earlier on in the process what things customers hate (and can react/fix things sooner) rather than waiting until a large feature has been built to learn that you have to go back to the drawing board.
I believe I took great pains to point out that Continuous Deployment (which is _not_ CI – that’s only part of the Continuous Deployment program) is generally a laudable goal to work towards. What I’m trying to point out is that there are legitimate reasons why an organisation may not be able to – or even want to – deploy their code into production several times a day.
There are types of software, and problem domains, that are not amenable to Continuous Deployment. This is not a matter of “implementing it correctly” or “being large enough” – it just can’t be achieved in certain conditions.
As for the Customer Acceptance part – particularly in enterprise software, a lot of organisations prefer to do changes in big, and relatively infrequent, batches. They can set aside for training, they learn the new stuff once, and then they get on with using the tools provided to do their actual work. A lot of enterprise software users prefer to use absolutely shit software _that they know how to work with_ instead of being regularly being bumped down a competence level because the UI and processes get changed. Again – this is part of the problem domain, and if your target audience is like that, Continuous Deployment is not for you.
Finally – Continuous Deployment doesn’t actually eliminate the “Big Bang Release” problem. Sure, you get incremental delivery of functionality – but that means your “Big Bang Release” is flipping the Feature Switch on, instead.
I can, personally, see zero reasons not to have good engineering practices such as:
* good unit (and other types of) tests
* automated deployment processes (manually or automatically initiated)
* the _ability_ to push out small incremental changes quickly (needed for bug fixes, at least, even if you do bigger feature releases)
* devs that care about quality code
“There are types of software, and problem domains, that are not amenable to Continuous Deployment. This is not a matter of “implementing it correctly” or “being large enough” – it just can’t be achieved in certain conditions.”
We ARE talking about cloud based software here, right? If that’s true, I can’t think of any situations right off where it could not possibly be achieved. I agree there are situations where it probably doesn’t make sense (project size/cost), but it should always be achievable if it is determined to be more beneficial than whatever method it replaces.
No. And that’s part of the problem. I’m talking about software. Cloud based software is only part of the overall domain of software.
I’m also saying that, even within the tighter domain of server-based software (which Cloud is a subset of), there are problem domains that are not amenable to Continuous Deployment. And it’s not just about project size or cost.
Alright, what are some more specific examples of those “problem domains” *within* server-based software that are not amenable to CI/CD then? I don’t see any mention of that in your article.
Generally, any domain where work-in-progress isn’t acceptable, or where releases take time to vet. Think anything that requires external certification, for starters (e.g. HIPAA compliance).
And you’re right. I unintentionally left the problem domain part out of the post.