With the upcoming end-of-life for Bamboo Cloud, I’m in the market for a new build server setup. For this¹ experiment, I’m returning to an old favourite – Jenkins – paired with a potential new favourite – Docker. In this post, I describe how I’ve set up a Jenkins server in a Docker container, using the Multibranch Pipeline plugin to automatically configure a simple build².

I’m looking to set up a Jenkins server via Docker to address some key points:

• I want most of the configuration for the server to be under version control.
• I want the ability to run the build server locally on my machine when I’m experimenting with new features or configurations
• I want to easily be able to set up a build server in a new environment (e.g. on a local server, or in a cloud environment such as AWS)

Docker addresses these points pretty well, as far as I can see. I’m no Docker expert, though – this is literally the first thing I’ve tried to do with this. But it’s a good starting point.

For this article, I want to get a Jenkins server, managed with Docker, to the point where I can easily set up and teardown a container locally. For reference, I’m using Docker for Mac 1.12 and Jenkins 2.7.1 (the stable versions at the time of writing), and much of the information here came from a superb set of articles from Maxfield Steward of RiotGames.com and the documentation for the Official Jenkins Docker Image.

Steps

Go and install Docker

You will need to go and get Docker. That’s a given. I used Docker for Mac, because I’m a Mac user. Your mileage may vary.

You’ll also want Docker Compose, if you follow these instructions. Docker Compose makes working with multiple related containers easier. It comes as part of the install bundle for the Mac and Windows versions, but needs to be obtained separately for Linux.

Create Docker Files

Following the tutorial from RiotGames, I created a working directory with the following layout:

• jenkins_docker
  • .env
  • docker-compose.yml
  • jenkins_data/
    • Dockerfile
  • jenkins_master/
    • Dockerfile
    • conf/
      • plugins.txt
  • jenkins_nginx/
    • Dockerfile
    • conf/
      • jenkins.conf
      • nginx.conf

This setup creates:

• a ‘data volume container’ to hold the Jenkins Home directory and the Jenkins logs directory.
• a container for the Jenkins Server itself
• a container for nginx, to be used as a reverse proxy in front of Jenkins.

Let’s go through each file, starting with the Dockerfiles

data/Dockerfile

This creates a container to manage the data volumes – one of the recommended best practices for using Docker and Jenkins. By moving the Jenkins Home directory into a volume, I can easily create, destroy, and recreate the main Jenkins container – something that needs to be done when upgrading the image.

This container will hold the ‘unmanaged’ Jenkins content – things I set up through the UI (such as security & users) – as well as the build artefacts.

master/Dockerfile

This is my configuration of the Jenkins server. It’s pretty close to the example from the RiotGames tutorial, but I’ve added the section between lines 13 and 18, where I select which the plugins to be installed. If you omit this, Jenkins will prompt to install plugins for you on startup. (Actually, even with this, it prompts to install – but these will already be selected and available).

My personal plugins list, at this time, consists of the common plugins, plus the Bitbucket Branch Source Plugin, because we use Bitbucket. This plugin will allow me to create new jobs for Jenkins without having to configure Jenkins itself – an important step in keeping as much Jenkins configuration in source control as possible.

If you’re not sure what plugins you wish to use, I’d advice commenting these lines out for now, then generating a plugins.txt file and enabling the lines again later.

nginx/Dockerfile

This is a basic nginx configuration, based on the RiotGames tutorial. Rather than embed it directly, I direct interested readers to the relevant gist.

docker-compose.yml

The docker-compose.yml file ties these three containers together. The data container provides the volumes used by the master container to store Jenkins data, whilst the nginx container provides a suitable reverse proxy.

One point to note is that it’s possible to change which port on the machine running the Docket container is used to expose nginx. The default – specified in a .env file – should be 80, but that’s not suitable for a lot of developer machines (such as my own), so we need to override it.

Lastly, note that the master container mounts the present working directory as /backup – this is used later to make it easy to extract data from the container.

Build and Start The Containers

We can now use Docker-Compose to build the containers, and then start them.

$ docker-compose build
$ docker-compose up -d

At this point, the containers should be running, and you should be able to go [http://localhost/] to see your Jenkins instance (add a port to the URL as needed). When you get there, you’ll be asked to enter the initial administration password – which will be inside your container. So how do you see it? With docker-compose exec!

$ docker-compose exec master cat /var/jenkins_home/secrets/initialAdminPassword
<randomly generated password here>

Once you’ve entered the initial admin password, you will, as previously mentioned, be prompted to install plugins. After that, you’ll need to make your first user, and your first job.

Backing up Jenkins Data

An important step is to know how to back up and restore your Jenkins data. The exact way to do this depends on how you’ve configured your Docker containers – assuming you’ve used a data volume container like I did, you can issue a command to extract data for backup purposes:

$ docker-compose stop
$ docker-compose run --rm master tar cvfz /backup/backups.tgz /var/jenkins_home
$ docker-compose up -d

If you need to restore this data, you run the exec in reverse:

$ docker-compose stop
$ docker-compose run --rm master bash -c "cd /var/jenkins_home && tar xvfz /backup/backup.tgz --strip 1"
$ docker-compose up -d

The server that you run the Docker containers on can then be configured to perform the backups on a regular basis via a cron task or similar.

Deploying to a Production Server

This step I haven’t yet gotten around to; I’ve still got several months to get off Bamboo Cloud, and I haven’t decided exactly where I’ll host the Jenkins server (assuming that’s what I end up with).

One likely place, however, will be on Amazon’s OpsWorks infrastructure – we use that for our production servers now, so putting a new instance up there will be straight forward. It will require building a new Chef recipe for deployment, but that’s reasonably straightforward.

Where to from here?

At this point, if you’ve been following along, you’ve got a suite of Docker containers that you can use to run a Jenkins server. I’d suggest committing the files you’ve created along the way to version control, so you can keep track of the changes you’re making.

After that – create some jobs in Jenkins! Get building. Along those lines, my next post in this series will be about what’s involved in getting a multi-branch pipeline configured for Maven. Until then, happy reading.