Getting started with GitLab pipelines

Author: Stuart Maynes

This article aims to give you the knowledge needed to configure and test basic CI/CD pipelines using GitLab.

CI/CD pipelines are the top-level component of continuous integration, delivery, and deployment. Pipelines comprise of jobs and stages. A job defines what to do, for example, compile code. A stage determines what jobs are run, for example, a test stage could contain jobs to run unit tests for the frontend application and the backend application. Jobs that are within the same stage are executed in parallel.

A small Python app is used in this article to demonstrate pipelines however, the instructions can be adapted to the language of your choice. A link to the Python app is at the bottom of this article.

Requirements

To follow along you will need a GitLab account and for the later section ‘testing locally’ you will need to install Docker and gitlab-runner.

Pipeline configuration

Pipeline configurations are written in a file called .gitlab-ci.yml using the Yaml syntax. This file must be located at the root of the application.

The pipeline configuration for our app is as follows. It’s a small app and the configuration is simple.

image: python:3.8

stages:
  - test

test_app:
  stage: test
  script:
    - python -m pip install -r requirements.txt
    - pytest

The first line specifies the Docker image needed to run the jobs. We can use an image from Docker Hub by specifying the image-name and an optional tag. Above we have requested the Python image and used the 3.8 tag to make sure we are getting a version compatible with our app. If the tag is omitted then by default the latest version of the image will be used.

Next, the stages of the pipeline are defined. Our app contains one stage but a more sophisticated app may have more, such as build, test, deployment. Stages are used to group jobs, which are defined later.

The line beginning test_app defines a job that is part of the test stage. The script section is where the commands for the job are defined. The script section can take a single line or, as in the case above, an array of commands. Our job is made up of two commands first the dependencies listed in requirements.txt are installed using pip. Next, using Pytest the unit tests are run.

Testing the pipeline

At this point, it is worth making your first commit and pushing it to your repository on GitLab. GitLab will detect the presence of the .gitlab-ci.yml file and runners will handle the pipeline.

You can view the progress of the pipeline by navigating to the pipelines section of the repo you’ve just pushed to. This section page can be found under CI/CD which is situated underneath ‘merge request’ on the sidebar in GitLab.

Testing the pipelines locally

Testing pipelines can be tedious if you’re having to make changes, make a commit and push to GitLab and wait for the results each time. Luckily, however, there is a better way to test jobs.

If you haven’t already, install Docker and gitlab-runner. Once this is installed you can use it to run specific pipeline jobs locally. This greatly speeds up the feedback cycle.

Installation instructions

sudo curl --output /usr/local/bin/gitlab-runner https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-darwin-amd64

Then grant the correct permissions with

sudo chmod +x /usr/local/bin/gitlab-runner 

Once gitlab-runner is successfully installed, it can be used to run individual jobs.

gitlab-runner exec docker <job-name> 

To run the test_app job we defined earlier

gitlab-runner exec docker test_app 

Conclusion and further reading

Hopefully, you found this article useful and you didn’t have much trouble getting your pipeline up and running. We defined a simple pipeline and used gitlab-runner to test it locally.

Personally, I have found that using gitlab-runner is much more convenient than pushing changes to GitLab each time. However, once I did find that I was getting different results locally compared to GitLab. I could get PHPStan to run locally, but it always failed on GitLab without providing any information about the issue.

If you have any questions or comments please feel free to reach out to me on Twitter @stuartmaynes.

Further reading

GitLab’s documentation is comprehensive and worth reading if you want to continue your journey with pipelines and CI.

The code used is available on GitLab for you to use.