Skip to content
Velloreth
Developer Tools 4 min read

GitLab CI Basics: Pipelines, Stages and Jobs

A .gitlab-ci.yml defines the pipeline. Learn the stages, the jobs, the artifacts, the cache and the rules that control them.

By

The structure

stages:
  - build
  - test
  - deploy

build:
  stage: build
  script:
    - npm ci
    - npm run build
  artifacts:
    paths:
      - dist/

test:
  stage: test
  script:
    - npm test

deploy:
  stage: deploy
  script:
    - ./deploy.sh
  rules:
    - if: $CI_COMMIT_BRANCH == "main"

A pipeline is a set of stages, and a stage is a set of jobs.

Jobs in a stage run in parallel, and stages run in order.

The job

A job is a named block with a script.

lint:
  stage: test
  image: node:20
  script:
    - npm ci
    - npm run lint

The image is the container the job runs in.

The script is the command list.

A job that fails stops the pipeline unless allow_failure: true.

The stages

The default stages are build, test and deploy.

A custom list overrides them.

stages:
  - lint
  - build
  - test
  - deploy

A job with no stage goes to test.

A stage with no job is skipped.

The artifacts

A file produced by a job and passed to a later job.

build:
  stage: build
  script: npm run build
  artifacts:
    paths:
      - dist/
    expire_in: 1 week

A later job downloads the artifacts automatically.

expire_in limits the storage.

A report artifact is a special type, such as a JUnit test report, which GitLab parses.

The cache

A file kept between pipeline runs to speed a repeat.

cache:
  key:
    files:
      - package-lock.json
  paths:
    - node_modules/

The key decides when the cache is invalidated. A key based on the lockfile is the common pattern.

An artifact is for the pipeline, and a cache is for performance.

A cache is not guaranteed. A job must work without it.

The rules

A condition that decides whether a job runs.

deploy:
  rules:
    - if: $CI_COMMIT_BRANCH == "main"
      when: manual
    - if: $CI_COMMIT_TAG

when can be on_success, on_failure, always, manual or never.

A rule can set a variable.

rules:
  - if: $CI_COMMIT_BRANCH == "main"
    variables:
      ENV: production

rules replaces only and except, which are the older keywords.

The variables

Predefined variables are provided by GitLab.

VariableValue
CI_COMMIT_SHAthe commit hash
CI_COMMIT_BRANCHthe branch name
CI_COMMIT_TAGthe tag name
CI_PIPELINE_IDthe pipeline id
CI_JOB_TOKENa token for the job
CI_REGISTRYthe container registry

A custom variable is defined in the settings or in the file.

A masked variable is hidden in the log.

A protected variable is available only on a protected branch.

The runner

The agent that executes a job.

A shared runner is provided by GitLab.

A self-hosted runner is one you install, and it is used for a private network or a specific tool.

A tag selects a runner.

deploy:
  tags:
    - production

The pipeline types

TypeTrigger
Brancha push to a branch
Merge requesta merge request event
Taga tag push
Scheduleda cron schedule
Manuala button
Parent-childa job that triggers another

A merge request pipeline runs the tests before the merge.

A child pipeline splits a large configuration.

Common mistakes

A job with no image. The default image may not have the tool.

A cache assumed to exist. The job fails on a cold cache.

A secret in the file. It is visible in the repository.

A secret not masked. It appears in the log.

A job that runs on every branch. A wasted pipeline.

No expire_in on an artifact. Storage grows.

A cache key that never changes. A stale cache.

A manual job with no rule. It is available everywhere.

A runner tag that does not exist. The job is stuck.

No needs. The stages run in order even when a job could start earlier.

A quotable summary

A GitLab CI pipeline is a set of stages defined in .gitlab-ci.yml, where the jobs in a stage run in parallel and the stages run in order. An artifact passes a file between jobs and a cache speeds a repeat, a rule decides whether a job runs, and a runner executes the job. A secret belongs in a masked and protected variable, and a cache is never guaranteed, so a job must work without it.

For the related topics, see ci-cd-pipeline-basics and ci-cd-explained. For the broader context, docker-basics-guide and semantic-versioning-explained.

Frequently asked questions

What is a GitLab CI pipeline?
It is a set of stages defined in .gitlab-ci.yml, where each stage contains jobs. The jobs in a stage run in parallel, and the stages run in order. A job that fails stops the pipeline unless it is allowed to fail.
What is the difference between an artifact and a cache?
An artifact is a file produced by a job and passed to a later job, such as a build output. A cache is a file kept between pipeline runs to speed a repeat, such as a dependency directory. An artifact is for the pipeline, and a cache is for performance.
What is a runner?
The agent that executes a job. A shared runner is provided by GitLab, and a self-hosted runner is one you install. A runner has tags, and a job selects a runner by its tags. A self-hosted runner is used for a private network or a specific tool.
What is a rule?
A condition that decides whether a job runs, such as a branch name, a tag, or a change to a path. It replaces the older only and except keywords. A rule can also set a variable for the job.
How do I pass a variable between jobs?
Write it to a dotenv file and declare it as a dotenv artifact, or use a child pipeline. A variable set in one job is not visible in another by default, because each job runs in its own environment.
What is a merge request pipeline?
A pipeline that runs for a merge request rather than for a branch push. It is defined with a rule on the merge request event, and it is used to run the tests before the merge. It can be a separate pipeline from the branch pipeline.