diff --git a/docs/production-readiness-review.md b/docs/production-readiness-review.md new file mode 100644 index 0000000..eeac318 --- /dev/null +++ b/docs/production-readiness-review.md @@ -0,0 +1,150 @@ +# Production readiness review + +> 2026-07-19, before the first production deployment after the Phase 4/5 architecture changes. +> Companion to `docs/phase-5-completion-report.md` and `docs/infrastructure/database-ownership.md`. +> +> **Verdict: not ready to deploy unattended.** The application itself verifies clean, but three +> operational gaps (no pre-deploy backup, no documented restore, no backend health endpoint) and one +> external blocker (CI) stand between here and a safe first deploy. All are listed below with what +> would close them. + +## Verified + +### Database + +| Item | Status | Evidence | +|---|---|---| +| Fresh empty MariaDB starts | ✅ | 42 tables created, 0 exceptions, app listening | +| Existing populated MariaDB restarts | ✅ | 42 tables, rows preserved, reconciler idempotent | +| Existing partially-migrated SQLite upgrades | ✅ | 34 → 44 tables, 13 applications and 8 companies intact | +| Migration path | ✅ | All Phase 4/5 migrations are no-ops; the reconciler owns their DDL per provider | +| Ownership model documented | ✅ | `docs/infrastructure/database-ownership.md` | +| Reconciler never destroys data | ✅ | `DropMalformedMySqlTable` checks row count and skips any table holding rows | +| Column types on MariaDB | ✅ | `int AUTO_INCREMENT` PKs, `varchar` owner keys, `datetime(6)`, indexes inside the 3072-byte limit | + +### Security + +| Item | Status | Evidence | +|---|---|---| +| Explicit authorization on every controller | ✅ | Reflection test over the whole assembly; public endpoints are an allow-list | +| No reliance on `Auth:Require` alone | ✅ | The five previously-implicit controllers now declare `[Authorize]` | +| Admin separation | ✅ | `AdminAudit`, `AdminSystem`, `Users` require `Roles = "Admin"`, asserted by test | +| Tenant isolation | ✅ | Global query filters are deny-on-null (`CurrentUserId != null && OwnerUserId == CurrentUserId`); every Phase 5 service scopes by owner and returns 404 across tenants | +| AI endpoints protected | ✅ | `AiWorkspaceController` requires auth; generation refuses another user's application | +| AI cannot mutate user content | ✅ | Suggestion-only, pinned by four tests | +| Public CV is user-controlled | ✅ | Off by default, per-variant opt-in, unguessable slug, `noindex` | +| Uploaded documents are private | ✅ | `AttachmentsController` requires auth and scopes by owner | +| Secrets are not committed | ✅ | 28 env-var references in compose; `.env` and `.env.*` gitignored; nothing sensitive tracked | +| Stack traces not leaked | ✅ | No developer exception page outside Development | + +### Application + +| Item | Status | +|---|---| +| Backend tests (Release) | ✅ 393 passed | +| Frontend tests | ✅ 128 passed, 36 suites | +| TypeScript | ✅ clean | +| Frontend production build | ✅ clean | +| Backend Docker image | ✅ builds | +| Frontend Docker image | ✅ builds | +| Backend container against empty MariaDB | ✅ starts, builds schema | + +## Remaining risks + +### 1. No pre-deploy database backup — **blocker for first deploy** + +`deploy/deploy.sh` does not dump the database before bringing the stack down. The first deploy after +these changes is exactly when a backup matters most: prod is many commits behind, and the reconciler +will create roughly a dozen tables on first boot. That path is verified on containers, **not on your +data**. + +`BackupController` exposes only `POST /api/backup/encrypted` — an application-level encrypted export, +not an operational database dump. It is not a substitute. + +**To close:** run `mariadb-dump` before `compose down`, keep the dump, and ideally add that step to +`deploy.sh` so it is not a thing to remember. + +### 2. No documented restore procedure — **blocker for first deploy** + +There is no written restore path. A backup you have never restored is a hypothesis. `deploy/README.md` +mentions backups in one line ("keep backups and volume persistence") and stops there. + +**To close:** document restore, and rehearse it once against a scratch database. + +### 3. No backend health endpoint — **operational gap** + +`docker-compose.yml` defines health checks for `ai-service` and `ollama` but **not for `backend` or +`frontend`**. Nothing automatically detects a backend that started and then became unhealthy, and +`depends_on` cannot gate on its readiness. + +**To close:** add a `/health` endpoint (anonymous, no data) and a compose health check. + +### 4. CI is red for an environmental reason — **external blocker** + +Unresolved and proven outside the repository: commit `8f73548` changed one markdown file and its test +job failed in the same duration band as every other run. Deployment is gated on CI, so nothing ships +until this is resolved. Everything in this review is **local verification**. + +Blocked on: job logs (a read-scoped Gitea token), `journalctl -u act_runner`, and the runner container +config. See `docs/infrastructure/runner-investigation.md`. + +### 5. Logging is console-only — **accepted for now** + +`builder.Logging` clears providers and adds Console and Debug. Adequate under Docker (captured by +`docker logs`), but there is no retention, no structure, and no aggregation. Acceptable for a +self-hosted single-node deployment; revisit if that changes. + +### 6. No global exception handler — **minor** + +There is no `UseExceptionHandler`, so unhandled exceptions return a bare 500. No stack leaks outside +Development, so this is not a security issue — but errors reach the client with no correlation id, +which makes support harder. + +### 7. `ClientErrorsController` is anonymous and unauthenticated — **accepted, by design** + +It accepts browser error reports and must work on pages reached before sign-in. It is size-limited +(32 KB) and field-truncated. Worth knowing it is an unauthenticated write path; it stores nothing. + +### 8. End-to-end journey unwalked — **verification gap** + +No one has driven the real authenticated flow from empty profile through to recorded outcome. All +verification is tests, builds and startup checks. Friction points in that journey remain unmapped. + +## Deployment checklist + +Run in order. Stop if any step fails. + +1. **Resolve CI**, or make a deliberate decision to deploy from a manually verified commit. +2. **Back up the production database.** + `mariadb-dump -u -p --single-transaction --routines jobtracker > jobtracker-$(date +%F).sql` +3. **Verify the dump is non-empty and contains `CREATE TABLE JobApplications`.** +4. **Note the current commit** for rollback: `git rev-parse HEAD` on the prod checkout. +5. Confirm `.env` has `AUTH_JWT_KEY`, `AI_SERVICE_TOKEN`, `JOBTRACKER_CONNECTION_STRING`, + `DATABASE_PROVIDER=mysql`, and that `Auth__Require=true` is still in `docker-compose.yml`. +6. Deploy: `deploy/deploy.sh`. +7. **Watch backend startup logs.** The reconciler logs what it creates. Any unhandled exception at + startup means stop and roll back — `Database.Migrate()` throws rather than continuing. +8. Confirm the expected tables exist: + `SELECT COUNT(*) FROM information_schema.tables WHERE table_schema='jobtracker';` — expect ~42. +9. Sign in and load one application workspace. Check the Overview, Checklist, Timeline, Analysis and + Match sections render. +10. Confirm `/cv/{slug}` still resolves for an existing public variant. + +## Rollback plan + +**If startup fails or the schema is wrong:** + +1. `docker compose down` +2. Restore the dump: `mariadb -u -p jobtracker < jobtracker-.sql` +3. `git checkout ` on the prod checkout +4. `deploy/deploy.sh` +5. Verify sign-in works + +**Why this is safe:** every Phase 4/5 migration is a no-op, so rolling the code back does not leave +migration state ahead of the schema. The reconciler only adds tables, columns and indexes — it never +drops a table holding rows — so the restored database is compatible with the older code, which simply +ignores the extra tables. + +**What rollback does not recover:** anything a user created in the new tables during the window +(checklist items, cover letter versions, interview prep, AI history). Those tables are dropped by the +restore. Keep the deploy window short and quiet.