Runners and pipelines

Introduction

• Pipelines are run on runners, i.e. nodes with installed Buddy Self-Hosted.
• The server where Buddy self-hosted is first installed is called the primary runner.
• The primary runner is treated as any other runner, i.e. can be tagged and locked.
• The pipeline will run on a particular runner if its tags match the tag on the runner.

Tag assignment mechanism

A pipeline can have no tags or any number of tags. A runner can have no tags, or one tag maximum.

A. If the pipeline doesn't have any tags > Buddy looks up all unlocked runners with no tags
B. If the pipeline has tags > Buddy looks up all unlocked runners with at least one matching tag

Pipeline assignment rules

A pipeline is assigned to a new runner if one of the following conditions occurs:

1. The pipeline is run for the first time.
2. The average load on the current node exceeds 100% per CPU core over the last 5 minutes.
3. The tag of the runner doesn't match the tags of the pipeline.
4. The current runner is not responding (is offline, is in different version, etc.).
5. The pipeline's cache is cleared.

Example use case

Let's assume you added a runner for time-consuming tasks like integration tests:

Runners tab in self-hosted Admin

To match a pipeline with that runner, you need to select its tag in the pipeline's settings – or immediately when adding a new pipeline.

Below we have a pipeline with several tags. One of them matches the runner, and so it appears on the list in the pipeline settings:

Runner tags in pipeline settings

Filesystem, static files and cache

1. A pipeline is fixed to a specific runner together with its cache and filesystem, as long as no reassignment rule is fulfilled.

2. Whenever a pipeline is switched to another runner, the contents of its cache are cleared. As a result, the first execution on the new runner can take a bit longer, as it needs to redownload the dependencies and regenerate the artifacts.

3. The exception are static files manually uploaded to the filesystem. Static files are stored in the main instance of Buddy On-Premises and are automatically downloaded on the new runner at the time of the first pipeline's run.

Draining

Draining means that the runner is finishing the current pipeline run and does not begin any new, including enqueued. Use this feature if you want to gracefully shut down your runner before update or maintenance.

You can mark the runner to drain by checking the flag in the runner settings:

Draining option in runner settings

Alternatively, you do it by running a command on the machine where the runner is installed:

bashbuddy update-runner --drain$

To restore the runner to the previous state, run:

bashbuddy update-runner --flood$

Switching the runner to the draining mode will produce a corresponding label next to it:

Runner in draining mode

Troubleshooting

In case of errors with assigning the pipeline to a runner, you can check the Prepare Environment step of the pipeline's execution logs for details.