Stop using Jenkins for Continuous Deployment (CD)

Jenkins pipelines can be employed for Continuous Deployment (CD) capabilities. But just because you can doesn’t mean you should.

It's not that it's impossible; some teams manage to make it work. However, the critical question is whether you should. More often than not, the answer tends to be no. While Jenkins excels in Continuous Integration (CI) scenarios, utilizing it for CI/CD proves to be a less-than-ideal workflow, often leading to more complications than initially anticipated.

We have seen this scenario repeatedly occur over the past two years of talking to customers: An engineering team uses Jenkins for testing code and building deployable artifacts (their CI process). They decide to employ Jenkins pipelines for executing CD, and while it seems promising initially, it introduces a host of problems and technical debt to the entire CI/CD process.

In the following sections, we'll delve into why Jenkins remains a popular solution, where it falls short in CD, and why it's advisable to move away from using Jenkins for CD capabilities.

Why is Jenkins so popular in the first place?

Let's be clear that we don't discredit Jenkins as a tool. Jenkins stands out as an open-source automation server equipped with plugins for streamlined building, deploying, and automating projects. It remains a highly favored tool in the DevOps realm, boasting over 15 million downloads.

What many developers appreciate about Jenkins is its maturity and proven resilience. Designed with flexibility in mind, Jenkins provides an extensive Software Development Kit (SDK) covering virtually every imaginable aspect. Whether you're looking to oversee an AWS environment or integrate with GitHub, there's a plugin tailored for that specific purpose. While this adaptability has its drawbacks, it is the primary factor behind Jenkins' popularity and its success in numerous organizations.

Where does Jenkins fall short?

To be blunt, Jenkins is starting to show its age as software. Originally developed in 2004, it wasn't designed to be cloud-native. Although developers and DevOps teams have found ways to make it work, its architecture, rooted in a master node and multiple build agents, reflects an outdated model from the era of data centers and static servers.

While this approach isn't necessarily bad, the reliance on plugins for workarounds introduces additional configurations, often seen as a hassle. For instance, configuring containers with the Jenkins Kubernetes plugin involves working with YAML inside a Groovy file, increasing the risk of errors. The variety in plugin syntax further complicates matters.

Managing Jenkins is considered challenging, with its requirement for Groovy scripting adding complexity compared to more modern tools. Despite the usefulness of Jenkins' SDKs, some plugins are limited and just have basic functionality. That’s fine until you realize that most plugins are often not well documented and you need to script your way through a majority of use cases. On top of that most users complain about buggy plugins that are a hassle to manage from a dependency standpoint. Some plugins unfortunately can’t be patched for security vulnerabilities due to inter-dependencies.

The user interface is notably outdated, demanding significant human resources for operation. Despite attempts to reduce script reliance, Jenkins remains script-centric. This results in the DevOps team being burdened with maintaining infrastructure, updating plugins, and troubleshooting.

In reality, it’s estimated that sustaining Jenkins requires the daily efforts of 2 to 5 engineers. While valuable and proficient in various tasks, the limitations of relying solely on Jenkins become evident as operations expand, leading to increased maintenance, operational challenges, and potential disruptions.

Why does it make sense to move your CD process off Jenkins?

We firmly believe that software delivery and deployment should be separate from your CI provider, which is usually purpose-built for running unit tests and compiling build artifacts. There are some major motivations for this separation, among them being:

• Deployment is frequently a much longer running process than a standard code change, encompassing staged releases across multiple environments with multiple rounds of integration testing.
• A separate CD system can make it easy to manage drift in microservice dependencies by allowing them to be tested in tandem and rolled back independently.
• Separating CD enhances your security posture by not requiring master creds in a building service like Jenkins or a cloud service like Circle-CI.

A central CD service drastically simplifies managing polymorphic infrastructure providers, as a pull-based architecture naturally can extend to multi-cloud or on-prem delivery models.

Jenkins functions as a CI server, heavily dependent on scripts. The Jenkins pipeline functionality, consisting of a suite of plugins, facilitates the implementation and integration of CD pipelines within Jenkins. This situation presents challenges in establishing a smooth and comprehensive CI/CD pipeline.

Putting aside Jenkins plugins, let’s quickly break down a few core concepts of a deployment pipeline:

• Artifacts
• Applications or services
• Secrets
• Deployment workflow
• Stages
• Ability to rollback
• Software release strategy (Blue/Green, Canary, etc.)
• Tests and tooling (security and monitoring)
• User Groups & Users (RBAC)

Jenkins pipeline functionality contains none of those out-of-the-box. Instead, you have to write custom scripting to manage or perform all the above tasks. You get the point, Jenkins pipelines are just another way of hardcoding together a solution with scripts.

Your and your team's time is far too valuable to spend it maintaining deployment pipelines. The reality is that scripting together your pipeline introduces more complexity, which increases the chances of pipelines failing.

Sure, if you want to you can manually script a Canary deployment to a Kubernetes cluster by heavily editing some Jenkinsfiles, but it’s not easy and ultimately a pain. Remember what was mentioned earlier; just because you can, doesn’t mean you should.

Plural

We believe that CDs shouldn't be challenging. You shouldn’t be coding and maintaining your CD process as your applications and services evolve. There are plenty of CD tools on the market that integrate and complement Jenkins for CI.

At Plural we evolved our product to easily integrate with CI tools such as Jenkins. ​​This model allows you to let your CI system do what it does best, test code, and build deployable artifacts, and then allow our products CD capabilities to handle the longer-duration and security-intensive task of deployment from there.

Our CD capabilities are based on a scalable, secure, agent-based pull architecture. It doesn't require direct access to any of the clusters it deploys to, meaning it can manage workloads in any cloud, on-prem, on the edge, or even on a local laptop running KIND.