Scheduled Pipelines in GitLab

In general, when I use CI/CD I like to have my entire pipeline run on every commit. I tend to use Trunk Based Development (TBD) and I am very conscious of build time. In general, my projects tend to build in 10 minutes or less. Usually, my entire pipeline (unit testing, building, deploying) takes less than 20 minutes to run. Often projects are in the 5-10 minute range. I really value that quick feedback.

However, there are times when that doesn't work so well. Some steps simply take a long time, for example, VI Analyzer or just long build times. This can especially cause problems if you only have a single runner (one of the reasons I recommend launching VMs on Demand.) If you only have one runner and you have a build that takes 3 hours, things are going to queue up rather quickly.

In the spirit of getting quick feedback, what I often do in those cases is split things up. I find some steps that run quickly and can give me quick feedback. Even if it won't catch everything, if it catches some things that's still better than nothing. For example, maybe I can run some unit tests on every commit. Then I schedule the things that take longer to happen once per day at night when no one is in the office, so no one has their feedback delayed while they are waiting around for the long-running job.

In GitLab you can create a new pipeline schedule by navigating to build-> pipeline schedules. There is a new schedule button on that page that will allow you to create a new schedule. It uses a cron-type syntax. If that's new to you, just google it.

Once you have the schedule set, then just add a rule to your long-running CI/CD job like below. If you want to have multiple schedules, you can set variables inside the schedules. For example, you could have a "schedule-type" set to either "weekday" or "weekend". You can then write another rule around that.

vian:
  stage: nightly
  rules: 
    - if: '$CI_PIPELINE_SOURCE == "schedule"'
      when: always
  script:
      - echo "Running VIAN"

If you have a job that you don't want to run on the schedule, like perhaps your normal unit test and build jobs, you can add the following rule to those jobs.

  rules:
    - if: $CI_PIPELINE_SOURCE == "schedule"
      when: never

GitLab rules can be complicated, but the GitLab Help is quite good.

Need Help?

Need help setting up or optimizing GitLab Pipelines? We can help. Let's talk.