jobtrackingapp/docs/release-candidate-review.md

Release candidate review

2026-07-19. Final validation pass before the first production deployment. No application code was changed by this review — findings only. Every claim below was checked against the implementation, not against the other documents. Where a document and the code disagreed, the code is reported.

Companions: deploy/first-production-deployment.md (how to deploy), docs/production-readiness-review.md (what was audited), docs/infrastructure/database-ownership.md (who creates which table), docs/release-checklist.md (state of the build).

Updated 2026-07-19 (final pass). Two findings from the first pass are now fixed and verified — B1 (the pre-deploy backup silently backed up the wrong thing) and N2 (/health always reported version: unknown). Both are recorded under Closed since the first pass, with their original text kept because the failure modes are worth understanding.

Verdict: one blocker remains, and it is external — CI (B2). Nothing in the application or the deployment path is now known to be blocking. Everything verified here is local verification; CI has proven none of it.

Section	Meaning
READY	Verified. Nothing further needed before deploying
BLOCKED	Requires action outside this repository. Deployment is gated on it
MANUAL	Only the owner can confirm — sign-in, real data, production smoke tests
Accepted	Known limitations, deliberately shipped as-is

---

READY — completed technical checks

Database

Item	Verified against
Startup order is connect → reconcile → Migrate() → reconcile → seed	StartupInitializationExtensions.InitializeJobTrackerAsync; matches database-ownership.md exactly
Migration ownership	Companies, JobApplications, Jobs, Correspondences, Attachments, JobEvents, RuleSettings, Identity — created only by migrations; reconciler repairs but never CREATE TABLEs them
Reconciler ownership	Every Phase 4/5 table is reconciler-owned with a paired no-op migration holding the snapshot. The seven no-op migrations named in database-ownership.md all exist on disk
Dependency guards	Reconciler tables with an FK are guarded on the parent existing, so pass 1 skips and pass 2 creates. EnsureMySqlIndex guards on table existence, not just index existence
Reconciler is non-destructive	DropMalformedMySqlTable checks row count first and skips any table holding rows
Provider selection accepts both spellings	Program.cs:68 — provider is "mysql" or "mariadb". deploy/README.md documents DATABASE_PROVIDER=mariadb; that value works
Rollback reasoning is sound	Phase 4/5 migrations are no-ops, so reverting code never leaves migration history ahead of the schema. Older code ignores the extra tables

Security

Item	Verified against
Explicit authorization	Every user-owned controller carries a class-level [Authorize(AuthenticationSchemes = "local")]. Confirmed directly on JobApplicationsController, CompaniesController, CorrespondenceController, RulesController, AttachmentsController, BackupController, ExportController
Independent of Auth:Require	The FallbackPolicy at Program.cs:380 is additional, not the only defence. /api/jobapplications returns 401 with Auth:Require unset
Anonymous surface is a deliberate allow-list	PublicCvController (class-level [AllowAnonymous]), ClientErrorsController, /health, and per-method anonymity on AuthController / TwoFactorController / the two OAuth callbacks
OAuth callbacks are not an open door	GmailController.Callback and MicrosoftGraphController.Callback are anonymous by necessity but gated on ConsumeState(state); an unknown or replayed state is rejected before any token exchange
Tenant isolation	25 global query filters, all deny-on-null (CurrentUserId != null && OwnerUserId == CurrentUserId), covering every owner-scoped root entity
Attachments are tenant-scoped and path-safe	Resolved through the parent JobApplication query (so the job-level filter applies); stored names go through Path.GetFileName + BuildStoredFileName, so a crafted upload name cannot escape the attachments root
AI service is not reachable from outside	ai-service is on ai_internal only — not on default, not on the external shared_services, no published port. Backend is the only member that can route to it
AI service authenticates its caller	X-Ai-Service-Token middleware with hmac.compare_digest; only /health is open. Compose declares the token with :? so a deploy that forgets it fails loudly
Public CV is opt-in	Off by default, per-variant, unguessable slug, noindex
Secrets not committed	.env is gitignored (.gitignore:8); nothing sensitive tracked
Backup handles the password safely	MYSQL_PWD, never on the command line, so it cannot reach the process list or the deploy log. Dump stderr is scrubbed before it is echoed

Application integrity — architecture rules

Rule	Verified
CareerProfile is the only editable career source	The two Phase 5 services that touch CareerProfiles — ApplicationChecklistService:343 and ApplicationIntelligenceService:199 — both read AsNoTracking(). Nothing downstream writes to it
CvVariant is a derived lens	ApplicationAssetsService.AttachVariantAsync writes only JobApplicationId and UpdatedAtUtc. Attaching a CV to an application never touches variant content
Application Workspace aggregates only	ApplicationIntelligenceService and ApplicationTimelineService contain zero SaveChanges calls. They are pure projections
JobEvent is the timeline source of truth	ApplicationTimelineService interprets JobEvent rows and stores nothing. Emission is centralised in JobLifecycleEvents
AI is suggestion-only	Generation appends to AiInteraction; nothing is written to a profile, variant, cover letter or prep item without an explicit user save

Deployment

Item	Verified
CI gates deployment	deploy job declares needs: test and if: push && ref == refs/heads/main
Deploy is pinned to the tested commit	The remote script git reset --hard ${{ github.sha }} and aborts if that commit is not fetchable
Container dependency ordering	frontend declares depends_on: backend: condition: service_healthy. A backend that never reports healthy makes compose up fail, and set -e aborts the deploy — a broken backend cannot present as a running stack
Health checks exist on both services	curl against /health for backend (90s start period, covering first-boot reconciliation), wget against nginx for frontend
/health does not touch the database	Deliberate: a DB-querying probe would restart a healthy backend on any database blip, and would hand out an unauthenticated way to probe database availability
Missing AI_SERVICE_TOKEN fails the stack	${AI_SERVICE_TOKEN:?…} in compose, on both backend and ai-service
Missing AUTH_JWT_KEY fails the backend	With Auth__Require=true, Program.cs:241 throws InvalidOperationException. It does not silently generate an ephemeral key
Backup rejects a useless dump	verify_backup fails the deploy on an empty file, and on a MariaDB dump lacking CREATE TABLE
Backups never overwrite	UTC-timestamped filenames; retention is explicitly manual and the script says so
Deploy script loads its own environment	deploy.sh parses the shared .env into its own shell before any decision. Values already in the environment (CI's APP_VERSION) still win. No value is echoed
DATABASE_PROVIDER is required, not defaulted	Missing or unrecognised aborts the deploy. Verified: exits 1, names the variable, leaves no backup file
Configuration validated before the stack is touched	Connection string, AI_SERVICE_TOKEN and AUTH_JWT_KEY are checked before the backup, and therefore before any build, stop or replace
Backups verified per provider	A dump needs valid gzip, CREATE TABLE and the Dump completed trailer; an archive needs jobtracker.db. Truncated and trailer-stripped dumps both rejected
SQLite volume resolved by real name	Project-prefixed, and fails if absent — the bare name would have created an empty volume and backed that up
No secret leakage in deploy output	Zero occurrences of any password, token or key across every failure-path test
/health reports the deployed version	Reads App:Version; verified end to end that App__Version=9.9.9-test surfaces as 9.9.9-test, and that an unset value falls back to the assembly version
Test suite	401 backend tests pass in Release, run for this review

---

Closed since the first pass

Both were found by this review, fixed in their own commits, and re-verified. Original findings kept.

B1. The pre-deploy database backup does not back up the database — CLOSED 2026-07-19

Original finding, kept because the failure mode is worth understanding:

Severity: critical. This invalidates the restore point that the entire deployment plan depends on.

deploy/deploy.sh symlinks /opt/job-tracker/shared/.env into the checkout so that docker compose can read it. It never sources it. There is no set -a, no . .env, no export of the database variables anywhere in the script.

So the script's own shell evaluates:

local provider="${DATABASE_PROVIDER:-sqlite}"

DATABASE_PROVIDER is not set in that shell. The CI deploy step exports only APP_VERSION, APP_COMMIT_SHA and APP_BUILD_STAMP, and a non-interactive ssh session does not read a profile. provider resolves to sqlite on a MariaDB production host.

The consequence is not a loud failure, which is what makes this serious:

1. The SQLite branch runs and tars the jobtracker_data volume.
2. That volume exists in production (the backend mounts it for /data, exports and attachments), so the tar succeeds and produces a non-empty file.
3. verify_backup is called without the CREATE TABLE expectation on this path — it only checks the file is non-empty.
4. The script prints Backup verified: … and the deploy proceeds.

The operator sees a green backup line and a new file in /opt/job-tracker/backups. There is no MariaDB dump. If the deploy then damages the schema, there is nothing to restore.

Same root cause, two further silent effects:

• APP_PUBLIC_BASE_URL is likewise unset, so the public smoke check at the end of deploy.sh never runs. deploy/first-production-deployment.md states it does.
• OLLAMA_MODEL is unset, so the post-deploy Ollama warmup never runs.

How to confirm before trusting any fix: run the deploy and check that the newest file in /opt/job-tracker/backups is named jobtracker-<dbname>-<stamp>.sql.gz, not jobtracker-sqlite-<stamp>.tar.gz. The filename alone distinguishes the two paths.

Closed. deploy/deploy.sh now:

1. Loads /opt/job-tracker/shared/.env into its own shell before any decision. Parsed line by line rather than sourced, because a compose .env is not a shell script. Variables already set in the environment win, so CI-provided APP_VERSION and friends still override the file. No value is echoed.
2. Requires DATABASE_PROVIDER explicitly. The :-sqlite default is gone. Missing means stop and say which variable is missing; an unrecognised value means stop.
3. Validates the rest of the deployment configuration before the backup, and therefore before anything is built, stopped or replaced: the connection string when the provider needs one, AI_SERVICE_TOKEN (compose declares it with :?, so missing it would otherwise kill the stack after the images are built) and AUTH_JWT_KEY (the backend throws at startup on a blank key, after the containers have been replaced). Names only in the output, never values.
4. Verifies each backup against its own format. A MariaDB dump must be valid gzip, contain CREATE TABLE, and carry the Dump completed trailer that mariadb-dump writes last — so a dump that died partway through is rejected. A SQLite archive must be valid gzip and actually contain jobtracker.db. A tar can no longer pass the dump check, which is precisely what went wrong.
5. Resolves the SQLite volume by its real, project-prefixed name and fails if it does not exist. The old code named the bare jobtracker_data, which on a real deploy would have silently created an empty volume and backed that up — a second instance of the same class of bug, found while testing the fix.

Two side effects of the same root cause are also resolved: APP_PUBLIC_BASE_URL now reaches the script, so the post-deploy public smoke check actually runs (and the script says so explicitly when it is unset rather than skipping in silence), as does OLLAMA_MODEL for the warmup.

Verified against a seeded MariaDB 11 container and real Docker volumes:

Scenario	Result
MariaDB production-style .env	✅ env loaded, MariaDB path selected, .sql.gz written containing CREATE TABLE, the seeded row, and the Dump completed trailer
DATABASE_PROVIDER missing	✅ exits 1 naming the variable; nothing built, stopped or replaced; no backup file
DATABASE_PROVIDER unrecognised	✅ exits 1
AI_SERVICE_TOKEN / AUTH_JWT_KEY missing	✅ both reported in one pass, exits 1
SQLite with a populated volume	✅ .tar.gz written and verified
SQLite volume containing no jobtracker.db	✅ rejected, file deleted
SQLite volume name not present	✅ rejected before any container ran; no stray empty volume created
Broken database credentials	✅ exits 1, no partial file left behind
Truncated dump (no CREATE TABLE)	✅ rejected
Dump with the trailer stripped	✅ rejected as truncated
Secret leakage across every test's output	✅ zero occurrences of any password, token or key

N2. /health always reports version: unknown under Docker — CLOSED 2026-07-19

Original finding: the endpoint read the APP_VERSION environment variable, but compose passes App__Version, which binds to the App:Version configuration key. No variable by that name existed in the container, so the version was always unknown.

Closed. /health now reads App:Version through IConfiguration — the approach AdminSystemController already used for the same value. The resolution rule (configured version, else assembly version) moved to a shared BuildMetadata helper rather than being written twice, so the admin page and /health cannot drift apart.

Verified against a running backend: App__Version=9.9.9-test reports 9.9.9-test; unset reports the assembly version rather than unknown. Tests pin the configuration key, including that an App__Version environment variable binds to App:Version — the original bug failed silently, so a behavioural test alone would not have caught it.

---

BLOCKED — requires external action

One item. Nothing in this repository can clear it.

B2. CI is red — deployment is gated on it

Unchanged and still external to the repository. Commit 8f73548 changed one markdown file and its test job failed in the same duration band as every other run; a change that cannot affect compilation cannot fail a test job. The deploy job declares needs: test, so nothing promotes until this clears.

Everything in this review is therefore local verification. Blocked on, and needing you:

• Job logs — the Gitea API returns 401 unauthenticated. A read-scoped token would unblock this
• journalctl -u act_runner --since '2 hours ago' on the runner host
• The act_runner container/service configuration

Evidence in docs/infrastructure/runner-investigation.md.

The decision this forces. Either fix the runner, or deploy deliberately from a locally verified commit with CI knowingly red. The second is a defensible choice for a self-hosted single-node deployment — but it should be a decision rather than a default, and it means the 401 backend tests and 128 frontend tests have only ever run on one machine.

---

Accepted for the first release (non-blocking)

Known limitations, deliberately shipped as-is. None should stop a deploy; all are worth knowing.

#	Finding	Why it is not blocking
N1	.env.example omits DATABASE_PROVIDER and JOBTRACKER_CONNECTION_STRING — CLOSED 2026-07-19	Both added to the template with the connection-string host caveat. deploy.sh now hard-fails without DATABASE_PROVIDER, so the template had to name it
N2	/health always reports "version":"unknown" under Docker — CLOSED 2026-07-19	Fixed; see Closed since the first pass above
N3	The post-deploy gate in deploy.sh checks .State == running, not health — a container can be running while starting or unhealthy	Harmless in practice: compose up already blocks on service_healthy for the frontend's dependency, so an unhealthy backend aborts the deploy before this check is reached. The check is weaker than it looks, not wrong
N4	Table-count discrepancy across documents — database-ownership.md records 40 tables on MariaDB; release-checklist.md and the runbook say ~42	Both were measured, at different points in Phase 5. Use "the tables listed in database-ownership.md all exist" as the check, not a number
N5	ai-service opens completely if AI_SERVICE_TOKEN is the empty string — the middleware is if AI_SERVICE_TOKEN and …, so a blank token disables the check rather than failing closed	Compose declares the variable with :? on both services, so the stack refuses to start without it. No defence in depth behind that, though
N6	Logging is console-only	Captured by docker logs; adequate for a single-node self-hosted deployment. No retention, structure or aggregation
N7	No global exception handler	Unhandled errors return a bare 500 with no correlation id. No stack traces leak outside Development, so this is a supportability cost, not a security one
N8	ClientErrorsController is anonymous	By design — browser error reports must work on pages reached before sign-in. Size-limited to 32 KB and field-truncated; stores nothing
N9	A code rollback does not recover rows written to the new tables	Correct and documented. A code rollback loses nothing; only a database restore discards checklist items, cover letter versions, interview prep and AI history created since the dump
N10	Prompt quality is unmeasured	Interview generation now receives analysis and match context. Whether the output is better is a judgement no test makes

---

MANUAL — only the owner can verify

Nothing in this review, and nothing in the test suite, covers any of these — every automated check stops at the authentication boundary, and none of it has touched production data.

Backup and restore readiness

The automatic backup is fixed and verified against containers (B1). What remains unproven is the one thing containers cannot prove: that it works on your database.

• Take a MariaDB dump by hand from production and restore it into a scratch database. Not a formality. Container verification says the mechanism is sound; it says nothing about your data volume, your disk space, or your MariaDB version's dump quirks.
• Confirm /opt/job-tracker/backups exists and has room. deploy.sh creates it, but a full disk fails the backup and therefore the deploy.
• Know which backup is the restore point before you start. After the deploy, confirm the newest file is jobtracker-<database>-<stamp>.sql.gz — a jobtracker-sqlite-*.tar.gz would mean the environment is wrong, though the provider check should now stop that first.

Before deploying

• Confirm /opt/job-tracker/shared/.env contains DATABASE_PROVIDER=mariadb and JOBTRACKER_CONNECTION_STRING. deploy.sh now aborts without them, so a missing value costs an aborted deploy rather than a bad backup — but check first and skip the round trip.
• Confirm the connection-string host resolves from inside the backend container — 127.0.0.1 there means the container, not the Docker host.
• Record the current commit (git rev-parse HEAD) and the current row counts for JobApplications and Companies. The row counts are the check that matters most afterwards.

After deploying

• Login with a real existing account. Requires a password; no automated step here handles one.
• Existing user data intact — application and company counts match the pre-deploy numbers exactly, and applications open with their real content.
• CV builder loads an existing variant, including via an application deep link (/career/builder/:id).
• Application workspace renders — Overview, Checklist, Timeline, Analysis, Match. New sections being empty on existing applications is correct, not a fault.
• Public CV resolves at /cv/<slug> for a variant already marked public, including on refresh and from a shared link.
• An AI generation completes end to end and writes nothing until you save it.
• Email and calendar integrations still connect, if you use them — the OAuth callbacks were reviewed statically but never exercised against live Google or Microsoft.
• /health reports the version you deployed, not 1.0.0.0. The assembly fallback means APP_VERSION did not reach the container — harmless in itself, but build metadata is then missing from the admin page too.
• The full journey once, deliberately — empty profile through to a recorded outcome. No one has walked it. Friction in that path is currently unmapped.

Why this section cannot shrink

Authenticated smoke testing is not an automation gap that more work would close. Signing in requires a password, and no automated step in this repository should ever handle one. The 401 backend tests verify that the authorization boundary exists and holds; only you can verify what is behind it, with your data.