Preview Sandboxes Only for Visible Changes

Creating a preview for every pull request means building and running environments even for changes that do not affect the storefront.

In this guide, Jev evaluates the pull request diff and answers one question: will a shopper see the difference? Buddy creates a preview sandbox and posts the link in the pull request only when the change is visible.

This also matters at scale. With dozens of pull requests a day, every preview adds another build and sandbox, while the number of concurrently running sandboxes in a workspace is limited. Skipping previews for non-visible changes leaves that capacity for changes that need browser review.

Why path-based rules are not enough

The simplest filter is a condition on the path: build a preview when web/** changed. This filter is wrong in both directions:

• A change in src/ that shows on the page. The formatUsd function in src/money.js formats every price in the store. A new thousands separator changes every amount on the page, and not a single line in web/ changes.
• A change in web/ that nobody will see. Moving the product list from web/main.jsx to its own module touches the frontend, but the page looks the same.

A path rule cannot determine whether a code change affects the rendered application. Jev needs the same application context, which we provide in a UI map.e

Project

It is the same FrogePC store as in the Dependabot post: Vite and React, with the price and cart logic in src/.

The map of the page lives in the repository as .buddy/ui-map.md. It says which files shape the page and which never reach it:

markdown
# What the storefront renders

FrogePC is a single page built by Vite from `index.html` and `web/main.jsx`. A shopper sees a change when it alters what this page shows or how it behaves in the browser.

## Files that shape the page

- `index.html`: page title, meta description, root element.
- `web/main.jsx`: all sections of the page (navbar, hero, example builds, components cart, how it works, closing) and the product lists with names, details, and prices.
- `web/style.css`: layout, colours, typography, hover and focus states.
- `src/money.js`: `formatUsd` formats every price, subtotal, discount, and total on the page. Any change to its output is visible.
- `src/cart.js`: `cartSubtotal` and `cartTotal` compute the cart summary, including the 10% discount from $100. Any change to the numbers is visible.

## Files a shopper never sees

- `src/rates.js` and `convertUsdToEur` in `src/money.js`: not used by the page.
- `test/`: tests only.
- `.buddy/`, `.github/`, `eslint.config.js`, `.prettierrc.json`, `README.md`: tooling and docs.

Moving code between files, renaming variables, or reformatting is not visible unless the rendered text, numbers, styles, or behaviour change.

Pipeline

The pipeline starts on a PR to main and skips Dependabot branches, which have their own pipeline:

yaml
- pipeline: UI preview
  events:
  - type: PULL_REQUEST
    events:
    - opened
    - reopened
    - synchronize
    branches:
    - main
  trigger_conditions:
  - trigger_condition: VAR_NOT_CONTAINS
    trigger_variable_key: BUDDY_RUN_PR_HEAD_BRANCH
    trigger_variable_value: dependabot/

Image loading...

The diff for Jev

The first action saves the PR diff to pr-diff.md. GitHub CLI signs in through the GitHub integration in Buddy, and the GH_REPO pipeline variable points to the repository. The same action rejects pull requests from forks and saves in OUTPUT_DIFF_TRUNCATED whether the diff was cut:

yaml
- action: Collect the diff
  type: GIT_HUB_CLI
  integration: GitHub
  commands: |-
    if [ "$(gh pr view "$BUDDY_RUN_PR_NO" --json isCrossRepository -q .isCrossRepository)" = "true" ]; then
      echo "Pull requests from forks are not previewed."
      exit 1
    fi
    gh pr diff "$BUDDY_RUN_PR_NO" > pr-diff-full.md
    head -c 60000 pr-diff-full.md > pr-diff.md
    if [ "$(wc -c < pr-diff-full.md)" -gt 60000 ]; then export OUTPUT_DIFF_TRUNCATED=true; else export OUTPUT_DIFF_TRUNCATED=false; fi

One question: yes or no

yaml
- action: Is the change visible
  type: JEV
  integration: typesafe
  question: Will a shopper see any difference on the storefront after this pull request is merged?
  question_type: BOOLEAN
  true_when: The diff changes text, prices, numbers, styles, images, or behaviour that the storefront renders, including changes in src/money.js or src/cart.js that alter their output
  false_when: The diff only changes tests, tooling, docs, or code the page does not use, or moves and renames code without changing what the page renders
  state:
  - "Pull request: $BUDDY_RUN_PR_TITLE"
  - file: .buddy/ui-map.md
  - file: pr-diff.md

With BOOLEAN, Jev returns true or false in BUDDY_ACTION_JEV_RESULT, its confidence in BUDDY_ACTION_JEV_CONFIDENCE, and the probability of true in BUDDY_ACTION_JEV_PROBABILITY. true_when and false_when define the conditions for each result more precisely.

A sandbox only for visible changes

First the pipeline removes the previous preview of this PR, because a new commit needs a fresh build. This action always runs: if the next commit reverts a visible change, the old preview does not stay behind.

The sandbox is created when Jev returns true, when confidence falls below 0.7, or when the diff was truncated. The fallback conditions prevent a potentially visible change from being skipped when the classification is uncertain or based on incomplete input:

yaml
- action: Build the preview
  type: SANDBOX_CREATE
  from: SCRATCH
  trigger_conditions:
  - trigger_condition: OR
    trigger_operands:
    - trigger_condition: VAR_IS
      trigger_variable_key: BUDDY_ACTION_JEV_RESULT
      trigger_variable_value: "true"
    - trigger_condition: VAR_LESS_THAN
      trigger_variable_key: BUDDY_ACTION_JEV_CONFIDENCE
      trigger_variable_value: "0.7"
    - trigger_condition: VAR_IS
      trigger_variable_key: OUTPUT_DIFF_TRUNCATED
      trigger_variable_value: "true"
  spec:
    sandbox: "preview-pr-${BUDDY_RUN_PR_NO}"
    name: "Preview PR #${BUDDY_RUN_PR_NO}"
    os: ubuntu:24.04
    resources: 2x4
    timeout: 900
    tags:
    - preview
    app_dir: /buddy
    apps:
    - npm run preview -- --host 0.0.0.0 --port 4173
    endpoints:
    - name: www
      endpoint: 0.0.0.0:4173
      region: US
    fetch:
    - path: /buddy
      ref: "${BUDDY_RUN_PR_HEAD_BRANCH}"
      build_command: npm ci --no-audit --no-fund && npm run build

The sandbox identifier uses the PR number, not the branch name, so branches with a slash (fix/...) are not a problem. vite preview serves the built dist/. vite.config.js has to allow .buddy.app hosts in preview.allowedHosts, otherwise Vite rejects requests from the public address.

Post the preview link is another GitHub CLI action. It runs only after a successful Build the preview (ACTION_STATUS_IS) and posts the link from ${BUDDY_SANDBOX_URLS} in the PR with gh pr comment. Always take the address from this variable, because the host is shortened and salted.

Cleanup after the PR is closed

A second pipeline runs on the closed event and removes the preview-pr-${BUDDY_RUN_PR_NO} sandbox. It works the same after a merge and after closing without a merge. A preview that receives no traffic for 15 minutes is stopped automatically by timeout: 900.

Snapshots, waking a sleeping preview and protecting it with the Buddy login are covered in the guides Branch preview environments and Preview behind Buddy login.

Three PRs, one pipeline

New tagline: a visible change

"Build Faster" becomes "Build Smarter". The path rule and Jev agree: Jev returns true with probability 0.99 and confidence 0.98. The sandbox is ready in 14 seconds, after which the pipeline posts its URL in the pull request.

Image loading...

Image loading...

Thousands separator: visible, although web/ did not change

formatUsd(199_900) now returns $1,999.00 instead of $1999.00. Only src/money.js and its test change. The path rule would skip the preview, yet every price in the cart changes. Jev answers true with probability 0.92. Confidence 0.84 is lower than for the tagline but still above the threshold, so the preview is built.

Image loading...

Image loading...

In the preview, selecting the RTX 5090 shows the change in the cart: $1,999.00, a discount of -$199.90, and a total of $1,799.10.

Image loading...

Moving the catalog: a change in web/ that nobody will see

The lists of processors, graphics cards and accessories move from web/main.jsx to web/catalog.js. The path rule would build a sandbox. Jev answers false with confidence 0.92 (the probability of true is 0.04), so Build the preview and Post the preview link are skipped and the run ends after 7 seconds.

Image loading...

Image loading...

Things to watch

• The UI map is half of the result. Without the line that formatUsd formats every price, a change in src/money.js looks like a logic change. Update ui-map.md when the page starts using a new module.
• A cut diff does not lower confidence by itself. head -c 60000 cuts the diff at 60 thousand characters. If the beginning of the diff is tests and tooling, Jev can answer false with high confidence even though there are changes in web/ further down. That is why a cut diff forces a preview through OUTPUT_DIFF_TRUNCATED, whatever the answer.
• Two commits in a short time are two parallel runs. Both remove and create the preview-pr-N sandbox, so the second one can remove the preview the first one has just built. Turn off parallel runs in the pipeline settings so that the next run waits for the previous one.
• The preview is public. For client projects add the Buddy login on the endpoint.
• Code from the PR runs in your project. npm ci and vite build in the sandbox run the code from the PR branch. That is why Collect the diff rejects pull requests from forks with isCrossRepository: Buddy has no separate variable with this information.

What next

• TypeSafe integration - how to connect Jev to your workspace,
• Sandbox configuration and endpoints - the fields in spec,
• Create new sandbox and Manage sandbox - the actions that build and remove the preview,
• Conditional executions - all types of action conditions,
• Flaky Test or Bug? Let the Pipeline Decide - the same model in failed test triage.