ci: declare the Playwright image in-repo for both browser jobs #62

Merged

ullrich merged 1 commit from ci/probe-job-container into main 2026-08-18 18:22:11 +00:00

Conversation 0 Commits 1 Files changed 2 +196 -9

ullrich commented 2026-07-31 16:17:57 +00:00

Owner

Copy link

Promotes the probe to the real change, and extends it to e2e.

What the probe established

It asked one question: does act_runner still apply its global container.options to a workflow-declared container? It answered yes, on the run in this PR's history:

/dev/shm = 2147483648 bytes (2.0G)      ok
/ci-cache present and writable
host.docker.internal resolves
browsers: chromium-1228, chromium_headless_shell-1228, ffmpeg-1011, firefox-1532, webkit-2311
all three runner options survived a workflow-declared container
✓ browser (chromium) elevation-chart-draw.browser.test.tsx (6 tests) 801ms

So the tag can live in the repo at no functional cost.

Why it should

With runs-on: playwright, the tag lives in host-side runner config that nothing in this repo — Renovate included — can see. Bumping the playwright package then breaks every browser job until somebody SSHes into the runner box. #38 and #66 are the worked examples:

browserType.launch: Executable doesn't exist at
/ms-playwright/chromium_headless_shell-1234/chrome-headless-shell-linux64/chrome-headless-shell

#66 could never go green on its own, because only half the pair was visible to Renovate.

Both browser jobs, not just visual-tests

They share one lockfile, so pinning visual-tests to an in-repo tag while e2e kept the label would relocate the mismatch onto the half left behind rather than fix it. e2e's exposure is currently masked: it has needs: checks, so on #66 it shows "skipped" rather than failing — meaning its status there is unknown, not passing.

e2e is also the job that leans hardest on the runner's global options, so it asserts them before anything depends on them. Docker reachability is the one thing the probe did not cover; it is proven by the docker version call ending "Provide a docker CLI", which fails immediately and legibly rather than mid-job.

The tag stays at v1.61.1-noble

main resolves playwright 1.61.1 and the label maps to v1.61.1-noble — they match today. Bumping the tag here would create the same mismatch in the opposite direction and break main. The bump belongs in whichever PR bumps the package, i.e. #66 on rebase. This PR's job is to move the tag into the repo and couple the two.

Two guards per browser job

Both additive — step lists before/after show +2 and -0 for each job, no existing step touched.

Container options — shm ≥ 1GB, /ci-cache writable, host.docker.internal resolves. The probe proved these survive, but that is act_runner behaviour we do not control, and silently losing --shm-size becomes confusing Chromium crashes deep in a test run. Assert, don't assume.

Browser builds — ask the installed playwright which builds it expects via playwright install --dry-run (whose paths are image paths, because PLAYWRIGHT_BROWSERS_PATH is /ms-playwright) and verify the image contains them. No version string is duplicated anywhere, so there is nothing to drift.

That second guard also closes the hole renovate.json5 documents on #38: minimumReleaseAge gates the version Renovate selects, but the pnpm dedupe postUpgradeTask re-resolves afterwards and can write a newer playwright-core into the same lockfile. Grouping makes the pair move together; comparing installed-package expectations against actual image contents catches a divergence even inside one PR, because it trusts neither version number.

Renovate

mcr.microsoft.com/playwright joins the testing tooling group, so the image and the packages move in one PR instead of two that are each green alone and break main together.

One honest caveat, recorded in the config rather than glossed over: the github-actions manager documents extracting "a Docker image specified in a job's container: field" with depType docker, but it does not document the depName it assigns. mcr.microsoft.com/playwright is the conventional docker-datasource form, not a guarantee. If it is wrong the entry silently never matches — the same failure shape as the labels that were dropped in silence before 2026-07-29 — so it carries a note to confirm with a LOG_LEVEL=debug dry-run. The workflow guard is the backstop either way.

Testing

• YAML parses; every run: block in both jobs parses as bash -n.
• The browser guard exits 1 with the intended error when builds are missing and 0 when present, tested both ways by pointing PLAYWRIGHT_BROWSERS_PATH at a populated directory.
• Verified against v1.61.1-noble: the image ships exactly the five builds 1.61.1 expects, matching --dry-run 1:1.
• renovate.json5 still parses as JSON5 with the image in the group.
• Step lists before/after: visual-tests 6 → 8, e2e 16 → 18, zero removals.

This PR's own CI is the first real exercise of the converted e2e job. Both guards are designed to make any surprise legible at the top of the job rather than deep inside a browser launch.

Promotes the probe to the real change, and extends it to `e2e`. ## What the probe established It asked one question: does act_runner still apply its global `container.options` to a workflow-declared container? It answered yes, on the run in this PR's history: ``` /dev/shm = 2147483648 bytes (2.0G) ok /ci-cache present and writable host.docker.internal resolves browsers: chromium-1228, chromium_headless_shell-1228, ffmpeg-1011, firefox-1532, webkit-2311 all three runner options survived a workflow-declared container ✓ browser (chromium) elevation-chart-draw.browser.test.tsx (6 tests) 801ms ``` So the tag can live in the repo at no functional cost. ## Why it should With `runs-on: playwright`, the tag lives in host-side runner config that nothing in this repo — Renovate included — can see. Bumping the `playwright` package then breaks every browser job until somebody SSHes into the runner box. #38 and #66 are the worked examples: ``` browserType.launch: Executable doesn't exist at /ms-playwright/chromium_headless_shell-1234/chrome-headless-shell-linux64/chrome-headless-shell ``` **#66 could never go green on its own**, because only half the pair was visible to Renovate. ## Both browser jobs, not just visual-tests They share one lockfile, so pinning `visual-tests` to an in-repo tag while `e2e` kept the label would relocate the mismatch onto the half left behind rather than fix it. `e2e`'s exposure is currently *masked*: it has `needs: checks`, so on #66 it shows "skipped" rather than failing — meaning its status there is unknown, not passing. `e2e` is also the job that leans hardest on the runner's global options, so it asserts them before anything depends on them. Docker reachability is the one thing the probe did not cover; it is proven by the `docker version` call ending "Provide a docker CLI", which fails immediately and legibly rather than mid-job. ## The tag stays at v1.61.1-noble `main` resolves `playwright` **1.61.1** and the label maps to **v1.61.1-noble** — they match today. Bumping the tag here would create the same mismatch in the opposite direction and break `main`. The bump belongs in whichever PR bumps the package, i.e. **#66 on rebase**. This PR's job is to move the tag into the repo and couple the two. ## Two guards per browser job Both additive — step lists before/after show `+2` and `-0` for each job, no existing step touched. **Container options** — shm ≥ 1GB, `/ci-cache` writable, `host.docker.internal` resolves. The probe proved these survive, but that is act_runner behaviour we do not control, and silently losing `--shm-size` becomes confusing Chromium crashes deep in a test run. Assert, don't assume. **Browser builds** — ask the *installed* playwright which builds it expects via `playwright install --dry-run` (whose paths are image paths, because `PLAYWRIGHT_BROWSERS_PATH` is `/ms-playwright`) and verify the image contains them. **No version string is duplicated anywhere**, so there is nothing to drift. That second guard also closes the hole `renovate.json5` documents on #38: `minimumReleaseAge` gates the version Renovate *selects*, but the `pnpm dedupe` postUpgradeTask re-resolves afterwards and can write a newer `playwright-core` into the same lockfile. Grouping makes the pair move together; comparing installed-package expectations against actual image contents catches a divergence even inside one PR, because it trusts neither version number. ## Renovate `mcr.microsoft.com/playwright` joins the `testing tooling` group, so the image and the packages move in one PR instead of two that are each green alone and break `main` together. One honest caveat, recorded in the config rather than glossed over: the `github-actions` manager documents extracting "a Docker image specified in a job's `container:` field" with depType `docker`, but it does **not** document the depName it assigns. `mcr.microsoft.com/playwright` is the conventional docker-datasource form, not a guarantee. If it is wrong the entry silently never matches — the same failure shape as the labels that were dropped in silence before 2026-07-29 — so it carries a note to confirm with a `LOG_LEVEL=debug` dry-run. The workflow guard is the backstop either way. ## Testing - YAML parses; every `run:` block in both jobs parses as `bash -n`. - The browser guard exits **1** with the intended error when builds are missing and **0** when present, tested both ways by pointing `PLAYWRIGHT_BROWSERS_PATH` at a populated directory. - Verified against `v1.61.1-noble`: the image ships exactly the five builds 1.61.1 expects, matching `--dry-run` 1:1. - `renovate.json5` still parses as JSON5 with the image in the group. - Step lists before/after: `visual-tests` 6 → 8, `e2e` 16 → 18, zero removals. This PR's own CI is the first real exercise of the converted `e2e` job. Both guards are designed to make any surprise legible at the top of the job rather than deep inside a browser launch.

ullrich added 1 commit 2026-07-31 16:17:58 +00:00

ci: probe declaring the Playwright image in-repo instead of via runner label ... 4181f43109

Experiment, scoped to one job. `visual-tests` now names its image in the workflow;
`e2e` keeps `runs-on: playwright`, so a failure cannot take both browser jobs
down.

The tag is deliberately IDENTICAL to what the label maps to today
(mcr.microsoft.com/playwright:v1.61.1-noble), so rendering cannot shift and
snapshots cannot drift. The only variable is WHERE the tag is written.

Why: with the label, the tag lives in ~trails/forgejo-runners/config.yml, which
nothing in this repo can see. Bumping `playwright` in package.json then breaks
browser jobs until somebody edits that file and restarts the runners — exactly
what happened on #38 ("Executable doesn't exist at
/ms-playwright/chromium_headless_shell-1234 ... required: v1.62.1-noble"). ci.yml
already carried a "keep these in sync" comment, which is a coupling nothing
enforced. In-repo, the tag moves atomically with the dependency and Renovate can
see it.

Caching is not the concern, and I checked rather than assumed: the image lands in
the slot's DinD image store either way, and it is warm on both slots today
(mcr.microsoft.com/playwright:v1.61.1-noble, 2.44GB, alongside
catthehacker/ubuntu:act-22.04 at 1.55GB). Changing a tag costs one pull per slot
however the tag is declared.

The real risk is whether act_runner still applies its global container.options to
a workflow-declared container:

  --shm-size=2g                        Chromium crashes on the default 64MB /dev/shm
  -v /workspace/.ci-cache:/ci-cache    pnpm store + Turbo cache
  --add-host=host.docker.internal:host-gateway

So the job's first step asserts all three and fails with an actionable message if
any is missing, rather than letting it surface as a mystery Chromium crash or a
silently cold cache. If the probe passes, the same change can move to e2e and the
runner label can be retired.

Note on how the image cache was verified: `docker exec <dind> docker images`
returns nothing useful — the DinD daemon listens on tcp://localhost:2375, not the
unix socket, so the CLI fails to connect. With stderr suppressed that reads as
"zero images cached", which is how I initially mis-read it. Use
`docker -H tcp://localhost:2375`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X8hNxgYp777FRqYtVmQNaU

ullrich referenced this pull request from a commit 2026-08-18 15:00:16 +00:00

test: centralize vitest testTimeout so packages can't inherit the 5s default

ullrich referenced this pull request 2026-08-18 15:00:24 +00:00

test: centralize vitest testTimeout so packages can't inherit the 5s default #78

ullrich referenced this pull request from a commit 2026-08-18 15:37:57 +00:00

ci: declare the Playwright image in-repo for both browser jobs

ullrich force-pushed ci/probe-job-container from 4181f43109 to 2591c61164 2026-08-18 15:37:57 +00:00 Compare

ullrich changed title from ci: probe declaring the Playwright image in-repo instead of via runner label to ci: declare the Playwright image in-repo for both browser jobs 2026-08-18 15:38:43 +00:00

ullrich referenced this pull request 2026-08-18 16:17:15 +00:00

feat: migrate to React Router v8 #79

ullrich merged commit 3e4cacc19e into main 2026-08-18 18:22:11 +00:00

ullrich deleted branch ci/probe-job-container 2026-08-18 18:22:11 +00:00

ullrich referenced this pull request from a commit 2026-08-18 18:22:15 +00:00

Merge pull request 'ci: declare the Playwright image in-repo for both browser jobs' (#62) from ci/probe-job-container into main

ullrich referenced this pull request from a commit 2026-08-19 08:52:49 +00:00

ci: bump the Playwright image to v1.62.1-noble alongside the packages

ullrich referenced this pull request from a commit 2026-08-19 16:46:00 +00:00

chore(deps): Update testing tooling to v1.62.1

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

dependencies

Dependency updates raised by Renovate (renovate.json5 labels)

  

major

Major version bump — always needs human review, never auto-merged

  

security-pin

Touches a pnpm.overrides CVE remediation pin, not an ordinary dependency

No labels

dependencies

major

security-pin

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

trails-cool/trails!62

No description provided.