docs(release): record browser crash and operator handoff

This commit is contained in:
cesnimda
2026-09-10 07:09:16 +02:00
parent 32adac74f3
commit fc749b1a30
4 changed files with 40 additions and 1 deletions
@@ -366,3 +366,39 @@ Logs are under `.artifacts/redesign-audit/`. The latest full Linux backend run r
Independent verification downloaded the exact `es-abstract` **1.24.2** tarball URL from the lockfile, verified its SHA-512 integrity against that lock and read the member without extracting it. The official member and locally installed file match byte-for-byte; line 19 is `var flags = typeof F === 'undefined' ? '' : ToString(F);`. This is evidence of different dependency bytes in the runner's failure, not a source lint violation or a need to edit the dependency. The corruption mechanism is unproven; the existing [runner investigation](../../../infrastructure/runner-investigation.md) documents analogous failures.
This evidence-only push makes **one further unchanged-code CI attempt** with the workflow's existing isolated install cache. No new retry loop, lint suppression or gate bypass is added. If corrupted dependency bytes or assertion-free process crashes recur, stop release retries and request runner-host diagnostics; do not generate repeated empty commits. Public `/health` still reports version **299**, `/ready` reports ready before this attempt; that is the existing deployment, not proof that `a861ca4` is live. The completed Phase 1 audit is pushed separately at `957fb57` on `codex/ui-redesign-audit`.
### Final release retry — 2026-09-10
[Run 743, test job 1621](https://git.cesnimda.uk/cesnimda/jobtrackingapp/actions/runs/743/jobs/1621), for `27657f31131cd95d42f0e1290853ec9c9b3959a4`, completed in **7m25s**; Gitea recorded the failed status at **2026-09-09 22:32:25 UTC** (September 10 locally). Repository safety/Compose, extension, parser, secret/SBOM, backend build and dependency checks passed. The full backend suite passed **916/916**, zero skipped (15s test duration); frontend passed **67/67 suites, 298/298 tests** (82.867s). Locked installation, npm audit, lint and the production frontend build all passed. The previous malformed `es-abstract` source did not recur at lint.
The browser gate failed: **6 passed, 12 failed** (1.4m). The first failure was `worker process exited unexpectedly (code=null, signal=SIGSEGV)` in the authenticated accessibility case; its existing retry timed out in `beforeEach` after 30 seconds. The remaining eleven failures report `page.goto: Target page, context or browser has been closed`, including after their existing retries. There is no reported accessibility violation or application assertion mismatch in these failures. This establishes a process/browser execution failure, but does **not** establish its underlying host, runtime or hardware cause. The local **18/18** browser result above remains separate evidence and cannot replace the failed CI gate.
[Deploy job 1622](https://git.cesnimda.uk/cesnimda/jobtrackingapp/actions/runs/743/jobs/1622) was **skipped**. At **2026-09-10 05:07:21 UTC**, read-only HTTPS requests still returned `/health` → `{"status":"ok","version":"299"}` and `/ready` → `{"status":"ready"}`. The requested commits are pushed to main, but this release has **not deployed**. No production service was replaced by this run.
The documented stop condition is now reached: **no further unchanged-code release retries or gate bypasses**. Capture runner diagnostics before another attempt. This result is recorded on `codex/ui-redesign-audit` so that documenting the block does not itself start another main deployment. Phase 1 is complete; Phase 2 and B07 have not started. **B06 implementation complete, but F04 remains open**, for the runtime/production reasons in the closure checklist above as well as the unshipped release.
#### Runner evidence to collect before another attempt
Run these read-only commands on the actual Ubuntu **runner host**, which must be identified independently of the application host. No SSH alias or credentials are assumed. Use the run's timestamps for a bounded kernel-log window; the example covers this observed attempt. Do not reset Docker, delete caches, alter limits or restart production services just to obtain evidence.
```bash
date -u
uname -srmo
free -h
df -h / /tmp /dev/shm
df -i / /tmp
docker ps -a --no-trunc --format '{{.ID}} {{.Names}} {{.Image}} {{.Status}}'
docker stats --no-stream --format '{{.Name}} memory={{.MemUsage}} cpu={{.CPUPerc}} pids={{.PIDs}}'
sudo journalctl -k --utc --since '2026-09-09 22:20:00 UTC' --until '2026-09-09 22:40:00 UTC' --no-pager \
| grep -Ei 'segfault|oom|out of memory|killed process|I/O error|machine check|hardware error|EDAC'
# If systemd-coredump is installed: list metadata only, not memory dumps.
command -v coredumpctl >/dev/null && \
coredumpctl --since '2026-09-09 22:20:00 UTC' --until '2026-09-09 22:40:00 UTC' --no-pager list
# Select a verified runner/job container ID from the inventory, if it still exists.
read -r -p 'Observed runner/job container ID: ' runner_container_id
test -n "$runner_container_id" && docker inspect --format \
'{{.Name}} image={{.Image}} state={{json .State}} memory={{.HostConfig.Memory}} memorySwap={{.HostConfig.MemorySwap}} pidsLimit={{.HostConfig.PidsLimit}} shmSize={{.HostConfig.ShmSize}} ulimits={{json .HostConfig.Ulimits}}' \
"$runner_container_id"
```
Return selected, sanitized output plus the job-container identity/image and exact failure window. **Do not provide environment dumps, runner registration/config files, tokens, core files, full Docker inspection or unrelated logs.** A removed job container or unavailable kernel/core records is an evidence gap, not a clean bill of health. Host free memory after completion does not prove the earlier job's cgroup or shared-memory capacity. OOM/cgroup failures, segfault records, or storage/hardware errors would direct the next investigation; absent records do not justify guessing a cause. Retain the existing failed-job browser trace/error-context artifacts if available, keeping synthetic test data separate from production information. A successful full CI run, actual deploy completion and live version/health verification are still required before calling the push live; F04 then additionally requires its independent ingress evidence.