Note: This post is now outdated. GNU Mailman now has a new container image for running tests and this time it just works.

Being one of the core developers of GNU Mailman, I took up the task to setup a Continuous Integration server for Mailman. While looking for a service using FOSS and was free to use I stumbled upon Gitlab CI. There were several other options to choose from, but none of them fulfilled our requirement of using FOSS. Gitlab is a FOSS and its source code is available on Gitlab as well as Github.

Now, the model of Gitlab CI is different from the others you would have encountered before like Travis-CI or Circle-CI. Gitlab CI only provides an interface to view the results of the tests associated with each commit or Pull Request. The actual tests are run using gitlab-ci’s runners which can be run on any other system 1. The latest gitlab-ci-multi-runner is written in Go and can run tests for multiple repositories 2.

Gitlab CI can be used to run tests in a couple of ways:

  • locally
  • using Docker container
  • using Docker container and executing job over SSH
  • connecting to remote SSH server
  • using a Parallels VM

Since we wanted to build Pull Requests, considering the security implication of running a running un-trusted code inside a server/local-machine, I decided to go with the 3rd option. Docker running in unprilidged mode and with no root access, even inside the docker container ,could do much less harm. So, for this setup I needed the following things:

  • A docker image inside which your tests are gonna run
  • The above image have ssh-access
  • A postgres database image linked to the above container

I created a new docker image called ‘mailman-runner’ to run the tests. Its Dockerfile6 is also available on Github if you would like to view to suggest edits.

Install Docker first

	curl -sSL https://get.docker.com/ | sh

We need to mount a data volume into our gitlab-ci-multi-runner container to be used for configs and other resources:

	docker run -d --name multi-runner --restart always \
	-v /var/run/docker.sock:/var/run/docker.sock \
	-v /PATH/TO/DATA/FOLDER:/data \
	ayufan/gitlab-ci-multi-runner:latest

The -v option is used to add a data volume to the container. Here we add a data volumen and a docker-socker to the container so that it can spawn new containers to run tests in.

Now, register the runner:

 1docker exec -it multi-runner gitlab-ci-multi-runner register
 2
 3Please enter the gitlab-ci coordinator URL (e.g. http://gitlab-ci.org:3000/ )
 4https://ci.gitlab.com/
 5Please enter the gitlab-ci token for this runner
 6xxx
 7Please enter the gitlab-ci description for this runner
 8my-runner
 9INFO[0034] fcf5c619 Registering runner... succeeded
10Please enter the executor: shell, docker, docker-ssh, ssh?
11docker-ssh
12Please enter the Docker image (eg. ruby:2.1):
13mailman-runner:latest
14If you want to enable mysql please enter version (X.Y) or enter latest?
15
16If you want to enable postgres please enter version (X.Y) or enter latest?
17
18If you want to enable redis please enter version (X.Y) or enter latest?
19
20If you want to enable mongo please enter version (X.Y) or enter latest?
21
22Please enter the SSH user (eg. root):
23runner
24Please enter the SSH password (eg. docker.io):
25runner
26
27INFO[0037] Runner registered successfully. Feel free to start it, but if it's
28running already the config should be automatically reloaded!

Note: The ssh user and password in the above configuration was created by default in the mailman-runner docker image.

After this, the configuration file (config.toml) would look somewhat like this:

concurrent = 4

[[runners]]
  name = "second-runner"
  url = "https://ci.gitlab.com"
  token = "XXX"
  executor = "docker-ssh"
  [runners.ssh]
    user = "runner"
    password = "runner"
  [runners.docker]
    image = "mailman-runner"
    privileged = false
    volumes = ["/cache"]

PS:

  1. Now, Gitlab CI has a concept of shared runners where you can use some public runners to run your tests. This update was added recently.

  2. Previously, one runner could be used for only one repository. You needed to start another instance of the same runner to run tests for another reposiroty. However, now you can share runners across all your repositories even if they belong to a group.