ci: publish pnpm audit findings to a vulnerability dashboard issue #167

Merged
ullrich merged 2 commits from ci/vulnerability-dashboard into main 2026-09-23 14:42:01 +00:00
Owner

Renovate's dependency dashboard covers declared dependencies. pnpm audit
walks the resolved lockfile tree. Measured on the same commit, 2026-09-23:

pnpm audit     41 advisories  (30 high, 11 moderate)
Renovate/OSV    0

Every one of the 41 is 5–10 levels deep and declared nowhere in the repo:

@xmldom/xmldom   19   .>drizzle-orm>expo-sqlite>expo>@expo/config>…>@xmldom/xmldom
brace-expansion   6   .>eslint-plugin-import-x>minimatch>brace-expansion
js-yaml           3   .>drizzle-orm>expo-sqlite>expo>@expo/cli>@expo/xcpretty>js-yaml
browserslist      2   .>@react-router/dev>babel-dead-code-elimination>…>browserslist

Neither tool is wrong — they answer different questions. Only one of them had a
reporting surface. This adds the other.

What it does

The nightly audit now renders its findings into a single issue,
"Vulnerability Dashboard (pnpm audit)", updated in place each run — the same
shape as Renovate's dashboard. Grouped by package, severity-sorted, with the
advisory link and one dependency path per finding.

It also explains where remediation goes: a pnpm.overrides entry in
pnpm-workspace.yaml, which is what the twelve existing pins already are.

Revises the failure semantics from #161

#161 made this job fail on any finding, matching disk-maintenance.yml's
"a scheduled-run failure is the notification".

That was wrong here. With 41 standing advisories the job would have gone red
on its first run tomorrow and stayed red — alert fatigue, not an alert. My
reasoning imported a pattern that fits a binary condition (disk over threshold)
into one with a permanent non-zero baseline.

Now: the issue is the notification. The run fails only when the audit cannot run
or the issue cannot be updated — when the signal is broken, not when it has
something to say.

Verification so far

The renderer was tested against synthetic audit JSON for both the populated and
empty cases. Untested: the live pnpm audit --json output shape and whether
secrets.GITHUB_TOKEN can create/update issues on this Forgejo — the
cancel-superseded-ci.yml precedent shows that token is not permitted to
cancel runs, so it may be similarly restricted here. If it is, this needs a PAT,
and the first run will say so.

Renovate's dependency dashboard covers **declared** dependencies. `pnpm audit` walks the **resolved lockfile tree**. Measured on the same commit, 2026-09-23: ``` pnpm audit 41 advisories (30 high, 11 moderate) Renovate/OSV 0 ``` Every one of the 41 is 5–10 levels deep and declared nowhere in the repo: ``` @xmldom/xmldom 19 .>drizzle-orm>expo-sqlite>expo>@expo/config>…>@xmldom/xmldom brace-expansion 6 .>eslint-plugin-import-x>minimatch>brace-expansion js-yaml 3 .>drizzle-orm>expo-sqlite>expo>@expo/cli>@expo/xcpretty>js-yaml browserslist 2 .>@react-router/dev>babel-dead-code-elimination>…>browserslist ``` Neither tool is wrong — they answer different questions. Only one of them had a reporting surface. This adds the other. ### What it does The nightly audit now renders its findings into a single issue, **"Vulnerability Dashboard (pnpm audit)"**, updated in place each run — the same shape as Renovate's dashboard. Grouped by package, severity-sorted, with the advisory link and one dependency path per finding. It also explains where remediation goes: a `pnpm.overrides` entry in `pnpm-workspace.yaml`, which is what the twelve existing pins already are. ### Revises the failure semantics from #161 #161 made this job fail on any finding, matching `disk-maintenance.yml`'s "a scheduled-run failure is the notification". **That was wrong here.** With 41 standing advisories the job would have gone red on its first run tomorrow and stayed red — alert fatigue, not an alert. My reasoning imported a pattern that fits a binary condition (disk over threshold) into one with a permanent non-zero baseline. Now: the issue is the notification. The run fails only when the audit cannot run or the issue cannot be updated — when the signal is broken, not when it has something to say. ### Verification so far The renderer was tested against synthetic audit JSON for both the populated and empty cases. **Untested:** the live `pnpm audit --json` output shape and whether `secrets.GITHUB_TOKEN` can create/update issues on this Forgejo — the `cancel-superseded-ci.yml` precedent shows that token is *not* permitted to cancel runs, so it may be similarly restricted here. If it is, this needs a PAT, and the first run will say so.
ci: publish pnpm audit findings to a vulnerability dashboard issue
Some checks failed
CI / Dockerfile Package Check (pull_request) Successful in 17s
CI / Security Scan (pull_request) Successful in 18s
CI / E2E Tests (pull_request) Has been cancelled
CI / Journal Image Smoke Test (pull_request) Has been cancelled
CI / Visual Tests (pull_request) Has been cancelled
CI / Checks (pull_request) Has been cancelled
7c36e3a00b
Renovate's dependency dashboard only covers DECLARED dependencies, so it never
sees a transitive package. pnpm audit walks the resolved lockfile tree.
Measured 2026-09-23 on the same commit:

  pnpm audit    41 advisories (30 high, 11 moderate)
  Renovate/OSV   0

Every one of the 41 is 5-10 levels deep -- @xmldom/xmldom (19),
brace-expansion (6), js-yaml (3), browserslist (2) -- and none is declared
anywhere in the repo. Neither tool is wrong; they answer different questions,
and only one of them had a reporting surface.

This adds the other one: a single issue, updated in place each night, in the
same shape as Renovate's dashboard.

Also revises the failure semantics introduced in #161. Failing on any finding
would have put this job red on its first run and kept it there, given 41
standing advisories. The issue is now the notification; the run fails only when
the audit cannot run or the issue cannot be updated.
ci: upload the full audit report as an artifact
All checks were successful
CI / Dockerfile Package Check (pull_request) Successful in 17s
CI / Security Scan (pull_request) Successful in 20s
CI / Visual Tests (pull_request) Successful in 3m42s
CI / Checks (pull_request) Successful in 3m50s
CI / Journal Image Smoke Test (pull_request) Successful in 6m29s
CI / E2E Tests (pull_request) Successful in 9m36s
CD Staging / Build & Push Docker Images (pull_request) Has been skipped
CD Staging / Build & Push Docker Images-1 (pull_request) Has been skipped
CD Staging / Deploy Staging (pull_request) Has been skipped
CD Staging / Deploy PR Preview (pull_request) Has been skipped
Cancel superseded CI / Cancel in-flight CI (pull_request) Successful in 9s
CD Staging / Tear Down PR Preview (pull_request) Successful in 20s
ae3e61525d
tail -40 into the step summary threw away most of the report, which is the one
thing this job exists to produce. The full text and JSON now go up as the
`dependency-audit` artifact; the summary keeps only the headline counts plus
pointers to the artifact and the dashboard issue.

Uses forgejo/upload-artifact@v5 rather than actions/upload-artifact, per the
note at the top of ci.yml (the latter throws GHESNotSupportedError on Forgejo).

Retention is 14 days via ARTIFACT_RETENTION_DAYS in app.ini, which is ample for
a nightly job -- the standing state lives in the issue, not the artifact.
ullrich deleted branch ci/vulnerability-dashboard 2026-09-23 14:42:01 +00:00
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
trails-cool/trails!167
No description provided.