jobtrackingapp/docs/release-final-report.md

Final release report

Historical snapshot (2026-07-19). The current release state is maintained in docs/implementation-roadmap.md, with external/operator dependencies in BLOCKERS.md.

2026-07-19. Compiled at the release-candidate stage after the CI runner failure was investigated and a real code bug behind it was fixed. Consolidates the verification done across the release-hardening work. Every claim here was verified this session unless explicitly marked as owner-only or unproven.

Release status

READY WITH DOCUMENTED RISKS

The code and deployment path have no known blocker. The most recent CI failure — which had been assumed environmental — was found to be a real code bug and is fixed and verified on the runner's exact ICU version. What remains are (a) the runner's separately-evidenced instability, which the owner must confirm is gone by re-running CI, and (b) owner-only verification that cannot be automated (sign-in, production data, production backup). None of these is a code defect.

Verification summary

Backend

• Tests: 420 passing, Release configuration, real build (not --no-build).
  • Windows (local).
  • Linux, full ICU — Debian 12 / libicu72 and Ubuntu 24.04 / libicu74.
  • Linux under DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=1.
  • Ubuntu 20.04 / libicu66 with .NET 9.0.316 installed via dotnet-install.sh — the exact CI runner environment — 420/420. This is the environment that failed before the ICU fix.
• Build: backend Docker image builds.
• No test weakened, skipped, filtered, or disabled.

Frontend

• Tests: 128 passing (36 suites).
• TypeScript: tsc --noEmit clean.
• Build: production build clean; frontend Docker image builds.

Database

All four startup scenarios run against live MariaDB 11 and SQLite, with the real backend binary:

Scenario	Result
Empty MariaDB	42 tables created, healthy, 0 exceptions, 0 skipped indexes
Populated MariaDB restart	idempotent — 42 tables, rows preserved
Partially-migrated MariaDB	healed 35 → 42, 7 Phase 4/5 tables recreated, surviving rows preserved
Fresh SQLite + restart	schema built, idempotent, 0 errors

Startup order confirmed in logs: reconcile → Database.Migrate() → reconcile → seed. All seven Phase 4/5 migrations have empty Up bodies; the reconciler owns their DDL. No duplicate ownership.

Deployment

• Backup: the real deploy.sh backup_database produces a valid gzip dump with 42 CREATE TABLE statements and the Dump completed trailer. Validation rejects empty/truncated/wrong-type files. Backup runs before build/replace (verified by line order).
• Restore: dump restored into a clean MariaDB 11 container; all 42 tables and every row count matched; the application started healthy against the restored database.
• UTF-8 / Norwegian characters: Ærøskøbing, Søknad – Bjørn Håkonsen, bjørn@dåg.no, and career-profile Norwegian text survived a full backup → restore byte-exact (HEX compared).
• Health checks: /health 200 (anonymous, no DB touch); frontend wget; compose gates frontend on backend service_healthy; 4 healthcheck blocks defined.

Security

• Authorization: every user-owned controller has class-level [Authorize]; 5 user endpoints return 401 unauthenticated; independent of the Auth:Require flag.
• Tenant isolation: 25 global query filters, all deny-on-null.
• Public endpoints: /health 200, api/auth/config 200, public CV unknown slug 404 (not 401/500, no leak). ai-service unreachable from outside its private network; authenticates its caller.

Remaining risks

Separated by kind. Nothing here is hidden.

Code issues

• None open. Fixed this release cycle: the ICU language-alias precedence bug (fba858e), the MariaDB follow-up-reminder index (95646e1), the nondeterministic timeline test (c1ff98f), the CV language ICU drop (9681618), /health version (8f6f2ba), and the deploy backup env-loading (66b02bc).

Infrastructure issues

• Runner instability (Findings A & B) — separate from the ICU bug and still unconfirmed-fixed. The test host has died at ~3 s and the deploy SSH step at ~3 s on prior runs, with host-access-only diagnostics. The ICU fix removes the known code cause of red, but does not prove the runner is stable. The owner should re-run CI; if it still dies at a fixed ~3 s mark regardless of the change, Findings A/B are confirmed and need host access (journalctl -u act_runner, runner resources).
• Runner ICU is old (libicu66 / Ubuntu 20.04-class). Legitimate as a signal — it caught a real bug — but worth upgrading so the CI environment is closer to production and to developer machines.

Manual owner verification (cannot be automated)

• Sign-in and the authenticated journey — requires a password; no tool here handles one. Use docs/deployment/manual-smoke-test.md.
• Production backup on real data — no production access from this environment; no credentials were discovered or guessed. Owner must run one real backup + scratch restore, including the æøå check. See docs/deployment/backup-restore.md.
• Production dataset scale — dump duration, disk headroom, and lock behaviour are unproven on a large real database.

Deployment recommendation

Deploy with documented risks, in this order:

1. Re-run CI with the ICU fix included. If the test job now passes, the current known red is cleared. If it still dies at a fixed early mark, Findings A/B are confirmed environmental — decide whether to deploy manually from the verified commit or fix the runner first.
2. Owner runs a production backup + scratch restore (with the æøå check) before deploying.
3. Deploy via deploy/deploy.sh — backup is automatic and gates the deploy.
4. Owner runs docs/deployment/manual-smoke-test.md immediately after.

Do not treat a green pipeline as sufficient on its own: two real defects this cycle passed local test runs and only surfaced against a real container or the runner's older ICU. The manual smoke test is not optional.