New feature: Load-based pipeline distribution

October 7, 2021

New feature: Load-based pipeline distribution

By default, Buddy On-Premises assigns pipelines to a single worker. This is because of how the caching mechanism works: all dependencies, Docker images, and artifacts are stored in the pipeline's filesystem so that they don't need to be downloaded or generated on every execution. However, if you are running resource-heavy builds with some extra workers in your Buddy installation, there is a chance that one of your servers will suffer excessive loads while the rest is laying idle.

To resolve this, we've added the option to automatically redistribute pipelines between workers when required.

What you need to know

  • The pipeline will be moved to a less busy worker if the average load on the current node exceeds 100% per CPU core over the last 5 minutes. For example, on a worker with 16 cores, the load must exceed 16.
  • When a pipeline is moved, its cache on the original worker is purged. All dependencies will be re-downloaded on the first execution on the new node.
  • Once moved, the new worker will remain the default node the pipeline that will remain there until required.

Configuration

To enable this option, go to the Performance in the On-Premises Admin settings. Alternatively, you can switch it on in the Buddy CLI.

Buddy is 100% customer driven. If there’s a feature or integration you miss, let us know in the comments below or directly at support@buddy.works.

Share:

Alexander Kus

Alexander Kus

Customer Success Manager

A story-teller and conversation-lover, Alexander decided to invest his skills to help his friends at Buddy transform the cold language of patch notes into exciting narratives. Also: an avid gamer, hip-hop DJ, Liverpool FC fan, absentminded husband, and the father of two.