Fun with WSL and local GitLab Runner

I was looking for a solution to run a GitLab pipeline locally. I haven’t really figured out an easy way, but apparently one could use the gitlab-runner tool to run individual jobs. Although you can install all tools for Windows I wanted to run the tools a bit more isolated. Therefore I decided to use wsl.

This is what I had to do!

  • install Ubuntu distribution
  • install gitlab-runner tools
  • install docker
  • run the gitlab commands

The list is quite short, but I spent quite some time figuring out how I can make caching happen.

In a nutshell I run an Ubuntu VM using wsl in which I can execute my pipeline jobs using gitlab-runner. The runner is spinning up Docker containers to execute the jobs as declared in .gitlab-ci.yml.

Ubuntu / WSL

First I had to install the Ubuntu WSL distro. Although the command line tells me where to find the distros (i.e. the Microsoft Store) I had a bit a hard time finding it. But the link WSL | Ubuntu helped me out as there is a link to directly get to the proper distro.

I have a complete Ubuntu environment ready in seconds and the integration with Windows works really well. I start WSL by typing wsl -d Ubuntu in my command line.

Ubuntu, ready in seconds

Install the tools

First of all I installed gitlab-runner:

sudo apt install gitlab-runner

Then I installed docker, which is a bit of a pain if you just want to get started quickly. I basically followed this guide and it worked well: How To Install and Use Docker on Ubuntu 20.04 | DigitalOcean

WSL 2

I first tried to run docker on my VM, but it failed. I had to upgrade my distro to WSL 2 by invoking this command:

wsl –set-version Ubuntu 2

After launching the VM again, I was able to run docker commands.

Docker

When I run my GitLab pipeline, I want to use Docker as executor. GitLab runner basically spins up containers (as per image defined in the .gitlab-ci.yml) and executes the job. The Docker daemon doesn’t start automatically, this is not hard to configure, but to first test my setup I had to start it manually by invoking sudo service docker start

I verified my setup by running docker run hello-world. If it works, it will print something like:

running a container in a VM running on Windows. Cool!

Running GitLab

Although it reads pretty simple, I spent quite some time understanding how to use the gitlab-runner tool. My main issue was to ensure the cache is working between the job executions. All the builds runs in a container and my initial assumption that caching just works was wrong. The tool tells me that instead of a distributed cache a local cache is used, but it never worked.

The trick is to mount a volume, so that the cache created inside the container is persisted on the host.

So, to actually run a job from my pipeline I navigated to a project with the .gitlab-ci.yml in it and executed the following command:

sudo gitlab-runner exec docker build-web –docker-volumes /home/gitlab-runner/cache/:/cache

Where build-web is the job I want to run and /home/gitlab-runner/cache the directory on the host system where the cache should be stored. By default the runner will put the cache in the /cache directory in the container.

Final Thoughts

I was hoping that I can execute the whole pipeline using the command line. Seems with gitlab-runner I can only run a single job. Still good to test stuff – definitely good to learn more about how GitLab runners work. And maybe this guide helps someone setting up their local GitLab runner.

Run EasyTax2014 with Docker

I just created a Docker image that runs the EasyTax2014AG application. It was pretty straight forward to write the Dockerfile. I have based it on the official Ubuntu image because I wanted the Dockerfile to be transparent so that you can actually trust there was nothing installed that tries to steal your data in some way.

FROM ubuntu:14.04
MAINTAINER Christian Menz <christianmenz@gmail.com>

RUN apt-get update
RUN apt-get -y install python-software-properties
RUN apt-get -y install software-properties-common
RUN apt-get -y install unzip

#
# Install Oracle JDK
#
RUN add-apt-repository -y ppa:webupd8team/java
RUN apt-get update
RUN echo debconf shared/accepted-oracle-license-v1-1 select true | debconf-set-selections
RUN echo debconf shared/accepted-oracle-license-v1-1 seen true | debconf-set-selections
RUN apt-get -y install oracle-java7-installer
RUN apt-get -y install oracle-java7-set-default
RUN apt-get clean

# Install some X libs & stuff (not sure what is actually needed)
RUN apt-get -y install libxtst6 libgtk2.0-0 libgdk-pixbuf2.0-0 libfontconfig1 libxrender1 libx11-6 libglib2.0-0 libxft2 libfreetype6 libc6 zlib1g libpng12-0 libstdc++6-4.8-dbg-arm64-cross libgcc1

# Download and extract EasyTax2014
RUN mkdir /EasyTax
WORKDIR /EasyTax
RUN wget http://www.hwis.ch/Downloads/EasyTax2014AG_unix_1_1.tar.gz
RUN tar -zxvf EasyTax2014AG_unix_1_1.tar.gz

RUN mkdir /root/TaxData

# Run the app
CMD ./EasyTax2014AG/EasyTax2014_AG

Here are the instructions to build and run the container

docker build -t christianmenz/easytax2014ag .

xhost +

docker run -it --rm -v /tmp/.X11-unix:/tmp/.X11-unix -v /home/<yourusername>/EasyTax2014/:/root/TaxData -e DISPLAY=unix$DISPLAY christianmenz/easytax2014ag

Note: the important thing for you to change is the location where you will actually want to store your data. When using the application please make sure you save the data in the mounted directory (/root/TaxData). If you miss that point you could end up having lost all your changes.

OK, maybe I should do my taxes now.. :/