Table of Contents
Introduction
GitLab CI/CD is the continuous integration and continuous delivery system built into GitLab. It lets you automatically build, test, and deploy your code whenever you push changes to a Git repository hosted on GitLab. In this chapter you will see how GitLab CI/CD is structured, how pipelines, jobs, and runners work together, and how to describe your automation using the .gitlab-ci.yml file.
Core Concepts: Pipelines, Stages, and Jobs
GitLab CI/CD is driven by pipelines. A pipeline is a set of jobs that run in a defined order after a trigger such as a code push or a merge request.
A pipeline is divided into stages. Typical stages are build, test, and deploy, but you can choose any names that match your workflow. Jobs are the individual tasks inside a stage. For example, you might have one job to run unit tests and another to run integration tests. Jobs in the same stage can run in parallel, while stages run in sequence according to their order.
You declare stages and jobs inside .gitlab-ci.yml. At a minimum, you list all the stages you intend to use and then assign each job to one of these stages using the stage keyword.
stages:
- build
- test
- deploy
build_app:
stage: build
script:
- make build
run_tests:
stage: test
script:
- make test
deploy_prod:
stage: deploy
script:
- ./deploy.sh
In this example, the build_app job runs first, then run_tests, then deploy_prod. The pipeline succeeds only if each stage completes successfully.
In GitLab CI/CD, jobs in the same stage run in parallel, and stages run in order as listed under stages:. If a job fails and is not marked as allowed to fail, the pipeline stops at that point.
The `.gitlab-ci.yml` File
The heart of GitLab CI/CD is the .gitlab-ci.yml file stored in the root of your repository. GitLab automatically reads this file on every push. The file uses YAML syntax and defines your pipeline configuration.
A .gitlab-ci.yml file usually contains global configuration blocks and one or more job definitions. YAML indentation is significant, so each nested key must be indented consistently with spaces.
At the top you can define vocabulary shared by all jobs. The stages list is required if you use custom stage names. You can also set variables that all jobs can use with the variables keyword.
stages:
- build
- test
variables:
APP_ENV: "test"
NODE_ENV: "test"
Each job is a top level key, with a name you choose. Inside a job you typically specify at least a stage and a script. The script is a list of shell commands that will be executed on the runner.
test_app:
stage: test
script:
- echo "Environment is $APP_ENV"
- npm install
- npm test
GitLab evaluates .gitlab-ci.yml on each commit and shows you whether its syntax is valid. There is also a CI Lint tool in GitLab that can validate your file before you commit it.
GitLab Runners
Jobs do not run on the GitLab server itself. They run on GitLab Runners. A runner is a process that fetches jobs from GitLab and executes them in some environment. Runners can be shared by many projects or specific to a single project or group.
From GitLab’s perspective, the runner is an executor that can use different backends. For example, it can run jobs inside Docker containers, directly on a Linux machine using the shell executor, or inside virtual machines. The details of installing and registering runners belong to system administration, but for CI/CD usage it is important to know that every job must be picked up by some runner that supports its configuration.
Jobs can request particular runners using tags. In the GitLab interface, you can assign tags to a runner. In .gitlab-ci.yml you can list those tags under the job definition.
deploy_to_aws:
stage: deploy
tags:
- aws
- linux
script:
- ./deploy_aws.sh
This job will be executed only by a runner that has both aws and linux tags. This makes it possible to separate jobs that require special tools or access credentials from generic build jobs.
Using Docker Images in Jobs
GitLab CI/CD jobs usually run inside container images when you use the Docker executor. You can specify which image to use globally for all jobs or individually for each job. This allows you to ensure that the job runs in a predictable environment with known tools installed.
To set a global image, you add the image keyword at the top level of the configuration.
image: python:3.12-slim
stages:
- test
pytest:
stage: test
script:
- pip install -r requirements.txt
- pytest
All jobs will then start inside the python:3.12-slim container. To override this per job, you add an image inside that job.
lint_js:
stage: test
image: node:20
script:
- npm install
- npm run lint
This job uses node:20, even if a different global image is defined.
You can also define services, which are additional containers linked to your job container, for example a database container. This makes it straightforward to test applications that depend on a database.
integration_tests:
stage: test
image: node:20
services:
- name: postgres:16
alias: db
variables:
POSTGRES_DB: appdb
POSTGRES_USER: appuser
POSTGRES_PASSWORD: secret
script:
- npm install
- npm run migrate
- npm test
The test code can now connect to the database using the host name db and the credentials provided by environment variables.
Pipeline Triggers
GitLab CI/CD pipelines can be triggered in several ways. The most common is a push to a branch. When you push a commit and .gitlab-ci.yml is present, GitLab automatically creates a pipeline for that commit.
You can also trigger pipelines when merge requests are created or updated. For this you define the only or rules sections. The older only and except syntax is still supported, but rules offers more flexibility.
Triggers can filter by branch names, tags, or pipeline sources. For example, you might want tests to run on every branch, but deployment only on the main branch.
stages:
- test
- deploy
test_all:
stage: test
script:
- make test
rules:
- if: '$CI_PIPELINE_SOURCE == "push"'
deploy_main:
stage: deploy
script:
- ./deploy.sh
rules:
- if: '$CI_COMMIT_BRANCH == "main"'
when: on_success
In this example, test_all runs whenever changes are pushed. The deploy_main job only runs when the commit is on the main branch and previous jobs in the pipeline have succeeded.
Pipelines can also be triggered manually from the GitLab web interface or through the GitLab API. Job level triggers such as when: manual can create jobs that require a click in the interface before running, which is useful for gated deployments.
deploy_production:
stage: deploy
script:
- ./deploy_to_production.sh
when: manual
only:
- mainHere the deployment job appears as a manual action on the pipeline page and does not start automatically.
Artifacts and Caching
Most pipelines involve files that must survive between jobs or be downloaded after the pipeline finishes. GitLab CI/CD uses artifacts and caches for this purpose.
Artifacts are files generated by a job that you want to preserve, at least for some time. For example, build results or test reports. You declare artifacts in a job with the artifacts keyword.
build_app:
stage: build
script:
- make build
artifacts:
paths:
- build/
expire_in: 1 week
The build/ directory will be uploaded at the end of the job and attached to it. Later jobs in the pipeline can download these artifacts. The expire_in setting controls how long GitLab keeps these files.
Caching is used to speed up pipelines by storing dependencies that do not change often, such as downloaded libraries. A cache is not guaranteed to persist, but when it does it avoids repeated work.
install_dependencies:
stage: build
cache:
key: "node-modules"
paths:
- node_modules/
script:
- npm install
Later jobs that share the same cache key can reuse the cached node_modules directory. You can include variables in the key to separate caches between branches or major versions.
Use artifacts for files that are part of the pipeline output, such as build results or reports. Use cache for reusable dependencies that can be regenerated if needed. Artifacts are pipeline scoped, caches are performance oriented and may be discarded.
Environments and Deployments
GitLab CI/CD understands the concept of environments such as staging or production. When a job is marked as a deployment to an environment, GitLab tracks the history of deployments and shows their status in the Environments page.
You designate a job as a deployment by using the environment keyword. At minimum you specify a name. Optionally you can define a url so that GitLab can link to the deployed application.
deploy_staging:
stage: deploy
script:
- ./deploy_to_staging.sh
environment:
name: staging
url: https://staging.example.com
When this job finishes successfully, there will be an entry in the staging environment with a link to https://staging.example.com. If you deploy again, GitLab records the new deployment and marks the previous one as outdated.
You can also define actions such as stopping an environment. For example, for review apps that are created per branch, you might want a dedicated job to tear them down when the branch is removed. For this, the environment block supports an action field.
stop_review:
stage: cleanup
script:
- ./destroy_review_app.sh
environment:
name: review/$CI_COMMIT_REF_NAME
action: stopThis job tells GitLab that it stops a specific environment associated with the branch name.
Using Variables and Secrets
GitLab CI/CD exposes many predefined variables and allows you to define your own. Variables are used to parameterize jobs and avoid hard coding values in .gitlab-ci.yml.
Predefined variables such as CI_COMMIT_SHA or CI_COMMIT_BRANCH give you information about the current pipeline and commit. You can also define custom variables in the file or in the GitLab UI under project settings.
To define variables for a job in the configuration file, use the variables keyword inside that job.
deploy_app:
stage: deploy
variables:
TARGET_HOST: "web1.example.com"
script:
- ./deploy.sh "$TARGET_HOST"
For sensitive data, you do not put values directly into the file. Instead, you add them as CI/CD variables in the GitLab interface and mark them as protected or masked. In .gitlab-ci.yml you still refer to them by name, but GitLab provides the actual values at runtime.
Environment variables are available in the job’s shell process and can be used by scripts or tools. Since scripts often run on Linux, you access variables just as you would in a normal shell using $VAR_NAME.
Working with Merge Requests
GitLab CI/CD integrates with merge requests to provide early feedback on changes. Pipelines that run for merge requests can show status directly on the merge request page.
Using rules, you can define jobs that run only for merge requests. For instance, you might want additional checks such as code quality analysis only when a merge request is created.
code_quality:
stage: test
script:
- ./run_code_quality.sh
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'GitLab can also report job results as merge request widgets. For example, test reports, coverage information, or security findings can be surfaced directly. The details of these advanced features depend on additional configuration, but the key point is that merge requests are a natural trigger for CI/CD pipelines.
You can also require that pipelines succeed before merge requests can be merged. This enforcement is done in project settings, but it relies on your .gitlab-ci.yml to define the relevant jobs.
Basic Testing and Deployment Example
To see how these concepts combine in practice, consider a small web application written in Python. You want to run tests on every push and deploy to production only on version tags.
The .gitlab-ci.yml might look like this:
image: python:3.12-slim
stages:
- test
- deploy
variables:
PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip"
cache:
key: "pip-$CI_COMMIT_REF_SLUG"
paths:
- .cache/pip
tests:
stage: test
script:
- pip install -r requirements.txt
- pytest
deploy_production:
stage: deploy
script:
- ./deploy_production.sh
environment:
name: production
url: https://example.com
rules:
- if: '$CI_COMMIT_TAG'
when: on_success
Here all jobs use the same Python image. Dependencies are cached per branch using cache. The tests job runs on every push. The deploy_production job runs only when there is a tag, because CI_COMMIT_TAG is set in that case. This pattern is common in simple GitLab CI/CD setups.
Conclusion
GitLab CI/CD brings continuous integration and delivery directly into GitLab repositories. By describing pipelines, stages, and jobs in .gitlab-ci.yml, you can automate building, testing, and deploying your projects on Linux based runners. Understanding how jobs use runners, images, variables, artifacts, and environments lets you gradually build more sophisticated workflows while keeping the configuration in version control alongside your code.