• Blog
  • New feature: Concurrent pipeline runs
New feature: Concurrent pipeline runs

October 20, 2022

New feature: Concurrent pipeline runs

This week we welcome a massive improvement for all DevOps engineers and project managers aiming to keep the CI process clear and efficient: concurrent pipeline executions. Not to confuse with parallelism considered as running several pipelines/actions at the same time, the feature allows you to run the same pipeline in non-sequential mode. In other words, instead for waiting for its turn, each commit is processed concurrently within one pipeline.

Use cases

Previously, all runs in a pipeline were executed sequentially. To test multiple branches at the same time, you had to create a separate pipeline for each branch. Thanks to this new feature, it is now possible for a single pipeline to execute multiple runs at the same time.

The two most common cases are:

  • running the same tests on every commit pushed to the repository
  • building Docker images for every branch in the repository

Configuration

To activate concurrent pipeline runs, check the option in the Runtime section of the pipeline settings:

How it works

When a pipeline is in progress and another execution is triggered, the pipeline filesystem is duplicated to serve this execution. The generated artifacts are then produced to each filesystem respectively.

The slots are labeled with the ID of the execution and can be browsed independently in the Filesystem tab of the pipeline. Here's how it looks for a pipeline run concurrently for master and staging branches:

The number of concurrent executions / filesystem slots a pipeline can be undergoing depends on the number of runners in your plan.

Summary

As the feature is currently in Beta, we strongly advise against using it for production pipelines. We do, however, ask to send any feedback that you'd like to share: from bug reports to feature requests specific for your type of workflow.

You can further optimize pipeline management by seeding pipeline parameters with YAML templates.

Buddy is 100% customer-driven. Let us know what integrations you need to take your automation game to the next level! If there’s a feature or integration you miss, talk to us on the live chat or drop us a line at support@buddy.works.

Alexander Kus

Alexander Kus

Customer Success Manager

A story-teller and conversation-lover, Alexander decided to invest his skills to help his friends at Buddy transform the cold language of patch notes into exciting narratives. Also: an avid gamer, hip-hop DJ, Liverpool FC fan, absentminded husband, and the father of two.