Describing Continuous Integration

The GitLab CI/CD tool is among the suite of tools provided by GitLab for software development. It enables GitLab users to automate various stages of their CI/CD pipelines effortlessly. Unlike other CI tools like Jenkins, GitLab's tool is generally more user-friendly. This overview will give you a high-level understanding of the GitLab CI/CD tool. For more in-depth information, it's recommended to refer to the official documentation found at GitLab CI Documentation (opens in a new tab).

The .gitlab-ci.yml File

In GitLab CI/CD, a simple YAML file named .gitlab-ci.yml is used, stored in the project's root directory. This file outlines the steps for the CI/CD tool to follow in processing the pipeline. Test scripts are defined within this file, organized into jobs which can then be grouped into stages. Though pipelines can have multiple stages, starting with a simple .gitlab-ci.yml file can aid in troubleshooting.

When GitLab detects the presence of .gitlab-ci.yml in the project's root directory, it's parsed and processed by the GitLab Runner application, which runs the jobs of each stage.

The .gitlab-ci.yml file defines:

• Scripts for testing code.
• Jobs, their assigned stages, and their order of execution.
• Dependencies and caches.
• Deployment location.
• Order of pipeline stages.
• Trigger conditions for scripts, either automatic or manual.

You can find the complete reference documentation for Pipeline configuration here (opens in a new tab).

Note: The .gitlab-ci.yml file uses job names as primary keys, and each job must have a unique name. Certain reserved words cannot be used for job names. Refer to the documentation for the full list of reserved keywords, including before_script, after_script, stages, services, and image.

GitLab Runner

A GitLab Runner serves as the execution engine for GitLab CI/CD pipelines, managing the execution of jobs and reporting back to GitLab. Runners operate independently from the GitLab interface and can run on separate infrastructure, enhancing scalability. When the load surpasses a single server's capacity, additional runners distribute the workload.

GitLab Runner:

• Executes pipeline jobs.
• Reports results to GitLab.
• Scales through system runners installable on VMs.

Runners can be tagged to designate which tasks they manage, facilitating setups such as runners within a security zone or dedicated to specific projects.

Note: A laptop isn't mandatory as a runner; it's depicted for illustrative purposes only.

.gitlab-ci.yml Keywords

The script keyword is crucial within a job, specifying the commands executed by the runner. Commands can be run as a single line or as an array with multiple commands.

Example:

job1:
  script: "python run.py"
  stage: build
 
job2:
  script:
    - uname -a
    - python --version
    - python verify_script.py
  stage: test

The runner executes the main script. If it returns a non-zero exit code, the job fails, and further commands won't execute, following a sequential order.

The stages keyword defines the processing order. Runners process all tasks for a stage in parallel. If any job within a stage fails, the build fails, halting further processing beyond that stage.

Example:

stages:
  - build
  - test
  - deploy

Special .pre and .post stages always run at the beginning or end, respectively.

Before and After Scripts

GitLab CI/CD allows scripts to run before and after jobs, enhancing customization.

• Define code to execute before the script.
• The default keyword applies globally, with the ability to override using a specific before_script in a job.
• The after_script keyword always executes, even after a script failure.

Example:

default:
  before_script:
    - global before script
 
job:
  before_script:
    - exec instead of global script
  script:
    - my script
  after_script:
    - execute after my script

The before_script section concatenates defined scripts to those with the main script execution. The after_script section executes after the job, including after a failure. Global before_script and after_script can be defined using the default key.

Only and Except

Within the pipeline definition, jobs can be run on specific occasions, such as merges to particular branches.

• Use only for specific branches and except to exclude specific branches.
• Use Regex to define patterns.
• Jobs can be specified to run only on certain branches or except on specific ones.

Example:

job:
  only:
    - branches@gitlab-org/gitlab
  except:
    - master@gitlab-org/gitlab
    - /^release/.*$/@gitlab-org/gitlab

Rules are defined using only: or except: keys. For instance, to run a job only during a merge to master, the job would be specified as follows:

deploy:
  only:
    refs:
      - master

The When Keyword

Use the when keyword to define job execution based on conditions, such as cleanup after build failures.

• Options include on_success, on_failure, always, manual, and delayed.

Example:

stages:
  - build
  - cleanup_build
  - test
  - deploy
  - cleanup
 
build_job:
  stage: build
  script:
    - make build
 
cleanup_build_job:
  stage: cleanup_build
  script:
    - cleanup build on fail
  when: on_failure
 
test_job:
  stage: test
  script:
    - make test
  after_script: make cleanup_test
 
deploy_job:
  stage: deploy
  script:
    - make deploy
  when: manual
 
cleanup_job:
  stage: cleanup
  script:
    - cleanup after jobs
  when: always

Temporary files created during the build process should be removed for system hygiene.

The when keyword offers options like on_success, on_failure, always, manual, and delayed.

Putting It All Together

Below is a .gitlab-ci.yml file integrating all components:

stages:
  - build
  - cleanup_build
  - testks
  - deploy
  - cleanup
 
build_job:
  before_script: make setup
  stage: build
  script:
    - make build
 
cleanup_build_job:
  stage: cleanup_build
  script:
    - cleanup build on fail
  when: on_failure
 
test_job:
  before_script: make setup
  stage: test
  script:
    - make test
  after_script: make cleanup_test
 
deploy_job:
  stage: deploy
  script:
    - make deploy
  when: manual
 
cleanup_job:
  stage: cleanup
  script:
    - cleanup after jobs
  when: always

For more options, refer to the GitLab CI reference page when setting up a DevOps pipeline.

YAML Syntax

CI/CD YAML Syntax Reference

This document outlines the configuration options for the GitLab .gitlab-ci.yml file, where you define the CI/CD jobs for your pipeline.

If you're already familiar with basic CI/CD concepts, try creating your own .gitlab-ci.yml file following a tutorial demonstrating a simple or complex pipeline. For a variety of examples, refer to GitLab CI/CD examples (opens in a new tab). For a comprehensive example from an enterprise context, see the .gitlab-ci.yml file for GitLab itself (opens in a new tab).

When editing your .gitlab-ci.yml file, you can validate it using the CI Lint tool.

Keywords

A GitLab CI/CD pipeline configuration includes:

Global Keywords

Header Keywords

Job Keywords

Global Keywords

Some keywords are not job-specific but control pipeline behavior or import additional configuration.

default: Set global defaults for certain keywords, which are applied to jobs lacking explicit definitions.

References

• GitLab CI/CD examples: https://docs.gitlab.com/ee/ci/examples/ (opens in a new tab)
• .gitlab-ci.yml file for GitLab: https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab-ci.yml (opens in a new tab)