Jenkins Deployments in Kubernetes with Docker and Groovy

In upcoming posts, we’ll show you how to connect a dockerized Jenkins server like the one we will work with today to a cloud instance, so it can actually do some enterprise-level work. But first, we’ll introduce the concepts we’ll need to execute this task via a look at dockerizing a Jenkins server so that it can create a user automatically upon startup in a container.

Why?

Kubernetes enforces the best practice of immutable container images. This means that when Kubernetes looks to a container registry (i.e. Docker Hub) and pulls a given Docker image, it does so with the expectation that the image it is pulling is either pre-configured or able to configure itself within the larger Kubernetes system.

In order to satisfy this need for autonomy among containers, we will need to configure the dockerized Jenkins server automatically upon startup in a Kubernetes Pod. To do so, we will need to use Jenkins' Groovy Hook callback feature. This is a pair of callbacks that are triggered upon successful startup or failure to start up the Jenkins server respectively.

Self-contained and repeatable

When we are done, with just the kubectl CLI, we’ll be able to spin up any number of Jenkins instances as a unified deployment in Kubernetes. Each will independently configure an identical admin user according to the Groovy Hook script we provide in our single Docker image.

Why Groovy?

Groovy is derived from Java, and looks familiar to many for that reason. It is written to interact with Java prototypes directly, which makes it a very convenient option for scripting in Jenkins, as Jenkins runs on Java. Most importantly for us though, is the fact that the Jenkins callback functionality that we need to use for automating our deployment is only directly accessible with Groovy.

Automating Jenkins User Creation in Docker

As called out above, this is the first in a series of posts on this topic. For the sake of clarity, we will first introduce automating the process of creating a user in a dockerized Jenkins server. In future posts, we will detail the process for connecting a given instance of this Jenkins server with a given cloud via a Jenkins plugin.

First, let’s create a directory to house our work:

$ mkdir jenkins-docker && cd jenkins-docker

Next, we’ll have to define our Jenkins server image in a Dockerfile.

Let’s create our Dockerfile, like so:

$ touch Dockerfile

Then we will need to paste the following into our new Dockerfile:

# Dockerfile

FROM jenkins/jenkins:lts

# Skip initial setup
ENV JAVA_OPTS -Djenkins.install.runSetupWizard=false

COPY plugins.txt /usr/share/jenkins/ref/plugins.txt
RUN /usr/local/bin/install-plugins.sh < /usr/share/jenkins/ref/plugins.txt

COPY init-user.groovy /usr/share/jenkins/ref/init.groovy.d/init-user.groovy

Notice that we pull the latest Jenkins image as our base. From there, we disable the setup wizard, so we can automate configuration; we copy and install our plugins; and we will copy our init-user.groovy file into the “~/init.groovy.d” directory.

All files ending in .groovy in “~/init.groovy.d” will be executed in alphabetical order when the Jenkins startup callback is triggered.

Before we can build the above image, we will need to create our plugins.txt and init-user.groovy files so that they can be copied into the Docker image. To do so, we can run:

$ touch plugins.txt

We then need to paste the following in the body of plugins.txt.

# plugins.txt

credentials-binding:latest

The plugin(s) listed here (there will generally be more) are installed by line 9 in the above Dockerfile, and this file is available for that command to be called because it is copied into the image just before it in line 8.

$ touch init-user.groovy

And finally, paste the following into the body of init-user.groovy:

#!/usr/bin/env groovy

import com.cloudbees.plugins.credentials.Credentials
import com.cloudbees.plugins.credentials.CredentialsScope
import com.cloudbees.plugins.credentials.domains.Domain
import com.cloudbees.plugins.credentials.impl.UsernamePasswordCredentialsImpl
import com.cloudbees.plugins.credentials.SystemCredentialsProvider

String sshUsername =  System.getenv()['SSH_USERNAME'] ?: "admin"
String sshPassword =  System.getenv()['SSH_PASSWORD'] ?: "admin"

String sshUserCredentialsId = java.util.UUID.randomUUID().toString()

Credentials sshCredentials = (Credentials) new UsernamePasswordCredentialsImpl(CredentialsScope.GLOBAL, sshUserCredentialsId, "VM SSH credentials", sshUsername, sshPassword)
SystemCredentialsProvider.getInstance().getStore().addCredentials(Domain.global(), sshCredentials)

Notice the required chaining of events, as we need access to the credentials plugin, which is made available by the above installation, in order to set system credentials automatically upon startup of the Jenkins server.

Running it

Prereqs:

• You will need to have a running instance of Docker to complete this exercise.

First – still within our jenkins-docker directory – we will have to build our image, like so:

$ docker build -t jenkins-with-admin-user .

Then we can run the image. We are running this in “detached mode,” as indicated by the -d. We are also exposing ports 80 and 50000 so Jenkins can receive and send communications.

$ docker run -d -p 80:8080 -p 50000:50000 --name jenkins jenkins-with-admin-user

(Note: Exposing ports in Kubernetes will require an Ingress or LoadBalancer Service. We are running it directly in Docker for testing purposes here.)

Once the Jenkins server is up, we can check on our success by execing into our running Pod, like so:

$ docker exec -it jenkins bin/bash

Finally, we can confirm that our admin user was created by navigating to “~/var/jenkins_home” and viewing the contents of credentials.xml, which is now populated with our newly minted creds! Success!

jenkins@6c5affe545ce:/$ cd var
jenkins@6c5affe545ce:/var$ ls
backups  cache  jenkins_home  lib  local  lock  log  mail  opt  run  spool  tmp
jenkins@6c5affe545ce:/var$ cd jenkins_home/
jenkins@6c5affe545ce:~$ ls
config.xml                     identity.key.enc                             jenkins.telemetry.Correlator.xml  nodes        secret.key.not-so-secret  userContent
copy_reference_file.log        init.groovy.d                                jobs                              plugins      secrets                   users
credentials.xml                jenkins.install.InstallUtil.lastExecVersion  logs                              plugins.txt  tini_pub.gpg              war
hudson.model.UpdateCenter.xml  jenkins.install.UpgradeWizard.state          nodeMonitors.xml                  secret.key   updates
jenkins@6c5affe545ce:~$ cat credentials.xml
<?xml version='1.1' encoding='UTF-8'?>
<com.cloudbees.plugins.credentials.systemcredentialsprovider plugin="credentials@2.3.12"></com.cloudbees.plugins.credentials.systemcredentialsprovider>
<domaincredentialsmap class="hudson.util.CopyOnWriteMap$Hash"></domaincredentialsmap>
<entry></entry>
<com.cloudbees.plugins.credentials.domains.domain></com.cloudbees.plugins.credentials.domains.domain>
<specifications></specifications>

<java.util.concurrent.copyonwritearraylist></java.util.concurrent.copyonwritearraylist>
<com.cloudbees.plugins.credentials.impl.usernamepasswordcredentialsimpl></com.cloudbees.plugins.credentials.impl.usernamepasswordcredentialsimpl>
<scope>GLOBAL</scope>
<id>7a005d0d-b81b-4e7a-9698-80d671865c17</id>
<description>VM SSH credentials</description>
<username>admin</username>
<password>{AQAAABAAAAAQEGS59jaUSYKyZ7X8ik6lFuJag1Rm78kouy4lZ6699G4=}</password>

Now, this Docker image can be built any number of times, and we can be assured that there will be an admin user available when the container is spun up.

To wrap up, let's push this image to our Docker Hub repo so that it can be pulled by Kubernetes, like so:

$ docker login
$ docker commit jenkins <your_dockerhub_username>/jenkins-with-admin-user</your_dockerhub_username>
$ docker push <your_dockerhub_username>/jenkins-with-admin-user</your_dockerhub_username>

TL;DR

Kubernetes requires that the container images it pulls be pre-configured or able to configure themselves when they are spun up so that any number of container images can be spun up at a moment's notice. In order to satisfy this requirement with a dockerized Jenkins server, we need to use Jenkins' inbuilt Groovy Hook callback, which is triggered upon the startup of the Jenkins server.

‍Keep reading... Automating Jenkins Deployments in K8s with Docker and Groovy Part II