New feature: Load-based pipeline distribution

By default, Buddy Self-Hosted assigns pipelines to a single worker. This is because of how the caching mechanism works: all dependencies, Docker images, and artifacts are stored in the pipeline's filesystem so that they don't need to be downloaded or generated on every execution. However, if you are running resource-heavy builds with some extra workers in your Buddy installation, there is a chance that one of your servers will suffer excessive loads while the rest is laying idle.

To resolve this, we've added the option to automatically redistribute pipelines between workers when required.

What you need to know

• The pipeline will be moved to a less busy worker if the average load on the current node exceeds 100% per CPU core over the last 5 minutes. For example, on a worker with 16 cores, the load must exceed 16.
• When a pipeline is moved, its cache on the original worker is purged. All dependencies will be re-downloaded on the first execution on the new node.
• Once moved, the new worker will remain the default node the pipeline that will remain there until required.

Configuration

To enable this option, go to the Performance in the On-Premises Admin settings. Alternatively, you can switch it on in the Buddy CLI.