Workers and pipelines

Introduction

  • Pipelines are run on workers, i.e. nodes with installed Buddy On-Promises.
  • The server where Buddy On-Premises is first installed is called the primary worker.
  • The primary worker is treated as any other worker, i.e. can be tagged and locked.
  • The pipeline will run on a particular worker if its tags match the tag on the worker.

Tag assignment mechanism

A pipeline can have no tags or any number of tags. A worker can have no tags, or one tag maximum.

A. If the pipeline doesn't have any tags > Buddy looks up all unlocked workers with no tags
B. If the pipeline has tags > Buddy looks up all unlocked workers with at least one matching tag

In both cases, the pipeline is assigned to the worker with the lowest load.

Pipeline assignment rules

A pipeline is assigned to a new worker if one of the following conditions occurs:

  1. The pipeline is run for the first time.
  2. The average load on the current node exceeds 100% per CPU core over the last 5 minutes.
  3. The tag of the worker doesn't match the tags of the pipeline.
  4. The current worker is not responding (is offline, is in different version, etc.).
  5. The pipeline's cache is cleared.

Example use case

Let's assume you added a worker for time-consuming tasks like integration tests:

Workers tab in On-Premises AdminWorkers tab in On-Premises Admin

To match a pipeline with that worker, you need to select its tag in the pipeline's settings – or immediately when adding a new pipeline.

Below we have a pipeline with several tags. One of them matches the worker, and so it appears on the list in the pipeline settings:

Worker tags in pipeline settingsWorker tags in pipeline settings

If you can't see the worker assignment option, your workers are probably locked and not available.

Filesystem, static files and cache

  1. A pipeline is fixed to a specific worker together with its cache and filesystem, as long as no reassignment rule is fulfilled.

  2. Whenever a pipeline is switched to another worker, the contents of its cache are cleared. As a result, the first execution on the new worker can take a bit longer, as it needs to redownload the dependencies and regenerate the artifacts.

  3. The exception are static files manually uploaded to the filesystem. Static files are stored in the main instance of Buddy On-Premises and are automatically downloaded on the new worker at the time of the first pipeline's execution

To force a pipeline to run on a specific worker, assign the worker with a unique tag and share that tag exclusively with that pipeline.

Troubleshooting

In case of errors with assigning the pipeline to a worker, you can check the Prepare Environment step of the pipeline's execution logs for details.

Get Started

Sign up for free and deploy your project in less than 10 minutes.