Fun with WSL and local GitLab Runner

I was looking for a solution to run a GitLab pipeline locally. I haven’t really figured out an easy way, but apparently one could use the gitlab-runner tool to run individual jobs. Although you can install all tools for Windows I wanted to run the tools a bit more isolated. Therefore I decided to use wsl.

This is what I had to do!

  • install Ubuntu distribution
  • install gitlab-runner tools
  • install docker
  • run the gitlab commands

The list is quite short, but I spent quite some time figuring out how I can make caching happen.

In a nutshell I run an Ubuntu VM using wsl in which I can execute my pipeline jobs using gitlab-runner. The runner is spinning up Docker containers to execute the jobs as declared in .gitlab-ci.yml.

Ubuntu / WSL

First I had to install the Ubuntu WSL distro. Although the command line tells me where to find the distros (i.e. the Microsoft Store) I had a bit a hard time finding it. But the link WSL | Ubuntu helped me out as there is a link to directly get to the proper distro.

I have a complete Ubuntu environment ready in seconds and the integration with Windows works really well. I start WSL by typing wsl -d Ubuntu in my command line.

Ubuntu, ready in seconds

Install the tools

First of all I installed gitlab-runner:

sudo apt install gitlab-runner

Then I installed docker, which is a bit of a pain if you just want to get started quickly. I basically followed this guide and it worked well: How To Install and Use Docker on Ubuntu 20.04 | DigitalOcean

WSL 2

I first tried to run docker on my VM, but it failed. I had to upgrade my distro to WSL 2 by invoking this command:

wsl –set-version Ubuntu 2

After launching the VM again, I was able to run docker commands.

Docker

When I run my GitLab pipeline, I want to use Docker as executor. GitLab runner basically spins up containers (as per image defined in the .gitlab-ci.yml) and executes the job. The Docker daemon doesn’t start automatically, this is not hard to configure, but to first test my setup I had to start it manually by invoking sudo service docker start

I verified my setup by running docker run hello-world. If it works, it will print something like:

running a container in a VM running on Windows. Cool!

Running GitLab

Although it reads pretty simple, I spent quite some time understanding how to use the gitlab-runner tool. My main issue was to ensure the cache is working between the job executions. All the builds runs in a container and my initial assumption that caching just works was wrong. The tool tells me that instead of a distributed cache a local cache is used, but it never worked.

The trick is to mount a volume, so that the cache created inside the container is persisted on the host.

So, to actually run a job from my pipeline I navigated to a project with the .gitlab-ci.yml in it and executed the following command:

sudo gitlab-runner exec docker build-web –docker-volumes /home/gitlab-runner/cache/:/cache

Where build-web is the job I want to run and /home/gitlab-runner/cache the directory on the host system where the cache should be stored. By default the runner will put the cache in the /cache directory in the container.

Final Thoughts

I was hoping that I can execute the whole pipeline using the command line. Seems with gitlab-runner I can only run a single job. Still good to test stuff – definitely good to learn more about how GitLab runners work. And maybe this guide helps someone setting up their local GitLab runner.

Leave a Reply

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.