From b705cbaf60864fdb4d438bc3fd848e52d7134de2 Mon Sep 17 00:00:00 2001 From: cesnimda Date: Sun, 19 Jul 2026 23:14:27 +0200 Subject: [PATCH] chore: pin shell scripts to LF via .gitattributes deploy.sh and the other scripts run on the Linux deploy host and in Docker. A CRLF checkout breaks them with "bad interpreter: bash\r". The committed blobs are already LF, but nothing guaranteed it against a host with core.autocrlf=true. `*.sh text eol=lf` makes it explicit. Verified: git ls-files --eol shows attr/text eol=lf on all .sh files; renormalize produced no index churn (already LF). Co-Authored-By: Claude Opus 4.8 --- .gitattributes | 4 ++ docs/infrastructure/runner-investigation.md | 75 ++++++++++++++++++++- docs/release-candidate-review.md | 13 +++- 3 files changed, 86 insertions(+), 6 deletions(-) create mode 100644 .gitattributes diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..d014dc3 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,4 @@ +# Shell scripts must be LF: they run on the Linux deploy host and in Docker. +# A CRLF deploy.sh fails with "bad interpreter: /usr/bin/env bash\r" or silently +# mis-parses. This guarantees LF in every checkout regardless of core.autocrlf. +*.sh text eol=lf diff --git a/docs/infrastructure/runner-investigation.md b/docs/infrastructure/runner-investigation.md index c4568c4..e4298ef 100644 --- a/docs/infrastructure/runner-investigation.md +++ b/docs/infrastructure/runner-investigation.md @@ -1,9 +1,17 @@ # Infrastructure Investigation — CI runner and deploy failures > 2026-07-18. Supersedes `docs/ci-runner-investigation.md`. -> **Conclusion: both failures are outside the repository.** Application code has been eliminated as a -> cause by direct experiment. Confirmation and repair require host access — the exact asks are at the -> end. +> **Conclusion (2026-07-18): failures A and B are outside the repository.** Application code was +> eliminated as a cause for those two by direct experiment. Confirmation and repair require host +> access — the exact asks are at the end. +> +> **Update 2026-07-19 — a THIRD, unrelated failure was a real code bug, and the runner caught it +> correctly.** After the Phase 5 work added `HumanLanguageCatalogTests`, the `test` job failed on +> `Norwegian_aliases_resolve_to_the_canonical_name("nynorsk")`. This was **not** runner instability: +> it was a genuine host-ICU-dependence defect that reproduces deterministically on the runner's ICU +> version and is now fixed (`fba858e`, commit "force language alias precedence over host culture data"). +> Full evidence in *Finding C* below. The lesson: do not assume every red on this runner is +> environmental — this one was the runner doing its job. There are **two independent infrastructure failures**: @@ -188,6 +196,67 @@ changed host key / rotated `PROD_SSH_KEY`. Confidence: **medium (~50 %)** — th --- +## C — Backend test fails on the runner's ICU version (real code bug, FIXED 2026-07-19) + +Unlike A and B, this failure **was** in the application code. The runner reported it correctly. + +### Symptom + +``` +JobTrackerApi.Tests.HumanLanguageCatalogTests.Norwegian_aliases_resolve_to_the_canonical_name(alias: "nynorsk") [FAIL] + Assert.Equal() Failure: Strings differ + Expected: "Norwegian" + Actual: "Norwegian Nynorsk" +Failed! - Failed: 1, Passed: 419 +``` + +Passed on the author's Windows machine and in Debian/Ubuntu-22.04+ containers; failed only on the +runner. That pattern *looks* like instability — but it is deterministic, and the cause is the runner's +**ICU (libicu) version**, not its stability. + +### Root cause — ruled in, not assumed + +`HumanLanguageCatalog.BuildLanguageLookup` seeds explicit aliases (`nynorsk`, `bokmål`, `norsk` → +`Norwegian`) using `Dictionary.TryAdd`, *after* enumerating `CultureInfo.GetCultures`. `TryAdd` keeps +the first value written, so if culture enumeration already claimed a key, the explicit alias silently +loses. + +The `nn` (Norwegian Nynorsk) culture's **NativeName differs by libicu version**: + +| ICU | `nn` NativeName (cleaned) | key `nynorsk` populated by enumeration? | old code result | +|---|---|---|---| +| libicu66 (Ubuntu 20.04) | `nynorsk` | **yes** → `Norwegian Nynorsk` | `nynorsk` → **`Norwegian Nynorsk`** ❌ | +| libicu70 (Ubuntu 22.04) | `norsk nynorsk` | no | `nynorsk` → `Norwegian` ✓ | +| libicu72 (Debian 12) | `norsk nynorsk` | no | `nynorsk` → `Norwegian` ✓ | +| libicu74 (Ubuntu 24.04) | `norsk nynorsk` | no | `nynorsk` → `Norwegian` ✓ | + +So the runner is on **old ICU (libicu66 / Ubuntu 20.04-class)**. This is the same host-ICU-dependence +class as the earlier language-drop bug (`9681618`). + +### Evidence (reproduced end to end) + +- **Probe on `runtime:6.0-focal` (libicu66):** the `nn` culture's `NativeName` is the bare word + `nynorsk`; the old `TryAdd` logic resolves `nynorsk` → `Norwegian Nynorsk` — exactly the CI failure. +- **Same probe with the fix logic on the same libicu66:** `nynorsk` → `Norwegian`. +- **The real net9 test DLL on Ubuntu 20.04 / libicu66, .NET 9.0.316 installed via `dotnet-install.sh` + (identical to CI):** 420/420 pass with the fix. +- Also 420/420 on libicu72 (Debian) and libicu74 (Ubuntu 24.04), and locally on Windows. + +### Fix + +`fba858e` — an `Override` helper (`map[key] = value`) applied to the alias block so the explicit +mappings win regardless of what culture enumeration inserted. Correct for any ICU version by +construction; no test weakened. + +### Why this matters for A and B + +It does not exonerate the runner — A (test host dies at ~3 s) and B (deploy SSH fails at ~3 s) remain +separately evidenced as environmental. But it is a caution: **not every red on this runner is +infrastructure.** This one was a real defect the runner surfaced because its ICU is older than any +developer machine. Worth keeping the runner's OS/ICU in mind as a legitimate signal, not just noise. + +--- + ## Required infrastructure changes **Blocking — cannot proceed without one of these:** diff --git a/docs/release-candidate-review.md b/docs/release-candidate-review.md index d3e9534..dab92d1 100644 --- a/docs/release-candidate-review.md +++ b/docs/release-candidate-review.md @@ -28,6 +28,13 @@ index that never created on MariaDB (and logged a false rollback signal on every nondeterministic timeline test. Both are recorded below. Backup → restore → app-start was re-run end to end. No open blocker remains in the code or deployment path. +**Correction 2026-07-19 (later).** The next CI run then failed on a *third*, real code bug — a +language-alias lookup that depended on the runner's older ICU version (`nynorsk` → `Norwegian Nynorsk` +instead of `Norwegian`). This was **not** runner instability; the runner caught a genuine defect. Fixed +in `fba858e` and verified on the runner's exact ICU (Ubuntu 20.04 / libicu66). So B2 below is no longer +"purely external": the most recent red was ours. See `docs/infrastructure/runner-investigation.md` +Finding C. The current status is carried by `docs/release-final-report.md`. + **Verdict: one blocker remains, and it is external — CI (B2).** Nothing in the application or the deployment path is now known to be blocking. Everything verified here is *local* verification; CI has proven none of it. @@ -102,9 +109,9 @@ precede 375 and 382. | Order | Step | Where | |---|---|---| -| 1 | `ReconcileSchema()` — pass 1, repair and reconciler-owned tables | `StartupInitializationExtensions.cs:1965` | -| 2 | `Database.Migrate()` — migration-owned tables | `:1973` | -| 3 | `ReconcileSchema()` — pass 2, everything pass 1 had to skip | `:1984` | +| 1 | `ReconcileSchema()` — pass 1, repair and reconciler-owned tables | `StartupInitializationExtensions.cs:1972` | +| 2 | `Database.Migrate()` — migration-owned tables | `:1980` | +| 3 | `ReconcileSchema()` — pass 2, everything pass 1 had to skip | `:1991` | **Migrations expected to run: none that create anything.** All seven Phase 4/5 migrations were confirmed to have a **literally empty `Up` body** (0 statements each): `AddCareerProfileRelationalChildren`,