GitLab CI Basics: Pipelines, Stages and Jobs
A .gitlab-ci.yml defines the pipeline. Learn the stages, the jobs, the artifacts, the cache and the rules that control them.
By Velloreth
The structure
stages:
- build
- test
- deploy
build:
stage: build
script:
- npm ci
- npm run build
artifacts:
paths:
- dist/
test:
stage: test
script:
- npm test
deploy:
stage: deploy
script:
- ./deploy.sh
rules:
- if: $CI_COMMIT_BRANCH == "main"
A pipeline is a set of stages, and a stage is a set of jobs.
Jobs in a stage run in parallel, and stages run in order.
The job
A job is a named block with a script.
lint:
stage: test
image: node:20
script:
- npm ci
- npm run lint
The image is the container the job runs in.
The script is the command list.
A job that fails stops the pipeline unless allow_failure: true.
The stages
The default stages are build, test and deploy.
A custom list overrides them.
stages:
- lint
- build
- test
- deploy
A job with no stage goes to test.
A stage with no job is skipped.
The artifacts
A file produced by a job and passed to a later job.
build:
stage: build
script: npm run build
artifacts:
paths:
- dist/
expire_in: 1 week
A later job downloads the artifacts automatically.
expire_in limits the storage.
A report artifact is a special type, such as a JUnit test report, which GitLab parses.
The cache
A file kept between pipeline runs to speed a repeat.
cache:
key:
files:
- package-lock.json
paths:
- node_modules/
The key decides when the cache is invalidated. A key based on the lockfile is the common pattern.
An artifact is for the pipeline, and a cache is for performance.
A cache is not guaranteed. A job must work without it.
The rules
A condition that decides whether a job runs.
deploy:
rules:
- if: $CI_COMMIT_BRANCH == "main"
when: manual
- if: $CI_COMMIT_TAG
when can be on_success, on_failure, always, manual or never.
A rule can set a variable.
rules:
- if: $CI_COMMIT_BRANCH == "main"
variables:
ENV: production
rules replaces only and except, which are the older keywords.
The variables
Predefined variables are provided by GitLab.
| Variable | Value |
|---|---|
CI_COMMIT_SHA | the commit hash |
CI_COMMIT_BRANCH | the branch name |
CI_COMMIT_TAG | the tag name |
CI_PIPELINE_ID | the pipeline id |
CI_JOB_TOKEN | a token for the job |
CI_REGISTRY | the container registry |
A custom variable is defined in the settings or in the file.
A masked variable is hidden in the log.
A protected variable is available only on a protected branch.
The runner
The agent that executes a job.
A shared runner is provided by GitLab.
A self-hosted runner is one you install, and it is used for a private network or a specific tool.
A tag selects a runner.
deploy:
tags:
- production
The pipeline types
| Type | Trigger |
|---|---|
| Branch | a push to a branch |
| Merge request | a merge request event |
| Tag | a tag push |
| Scheduled | a cron schedule |
| Manual | a button |
| Parent-child | a job that triggers another |
A merge request pipeline runs the tests before the merge.
A child pipeline splits a large configuration.
Common mistakes
A job with no image. The default image may not have the tool.
A cache assumed to exist. The job fails on a cold cache.
A secret in the file. It is visible in the repository.
A secret not masked. It appears in the log.
A job that runs on every branch. A wasted pipeline.
No expire_in on an artifact. Storage grows.
A cache key that never changes. A stale cache.
A manual job with no rule. It is available everywhere.
A runner tag that does not exist. The job is stuck.
No needs. The stages run in order even when a job could start earlier.
A quotable summary
A GitLab CI pipeline is a set of stages defined in .gitlab-ci.yml, where the jobs in a stage run in parallel and the stages run in order. An artifact passes a file between jobs and a cache speeds a repeat, a rule decides whether a job runs, and a runner executes the job. A secret belongs in a masked and protected variable, and a cache is never guaranteed, so a job must work without it.
Related reading
For the related topics, see ci-cd-pipeline-basics and ci-cd-explained. For the broader context, docker-basics-guide and semantic-versioning-explained.