deploy.sh symlinked /opt/job-tracker/shared/.env for docker compose but
never loaded it into its own shell. Its own decisions therefore ran
against an empty environment: DATABASE_PROVIDER fell back to sqlite on a
MariaDB host, so the deploy tarred the data volume, printed "Backup
verified" and continued with no database dump. The operator saw a green
backup line and a new file in the backups directory, and had no restore
point.
Load the shared env before any decision. Parsed line by line rather than
sourced, because a compose .env is not a shell script and an unquoted
value containing spaces would execute as a command. Values already in
the environment win, so CI-provided APP_VERSION and friends still
override the file. No value is echoed.
Remove the sqlite default. DATABASE_PROVIDER must be stated; missing or
unrecognised aborts the deploy.
Validate deployment configuration before the backup, and so before
anything is built, stopped or replaced: the connection string when the
provider needs one, AI_SERVICE_TOKEN (compose declares it with :?) and
AUTH_JWT_KEY (the backend throws on a blank key). Names in the output,
never values.
Verify each backup against its own format. A dump must be valid gzip,
contain CREATE TABLE, and carry the "Dump completed" trailer, so a dump
that died partway through is rejected. An archive must contain
jobtracker.db. A tar can no longer pass the dump check.
Also resolve the SQLite volume by its project-prefixed name and fail if
absent. The bare jobtracker_data name would have silently created an
empty volume and backed that up -- the same class of bug, found while
testing this fix.
Verified against a seeded MariaDB 11 container and real Docker volumes:
provider selection, all four validation failures, both backup formats
and their failure paths, truncated and trailer-stripped dumps, and zero
secret occurrences across every test's output.
Docs updated for the drift: deploy/README.md, deploy/first-production-
deployment.md, docs/release-candidate-review.md (B1 closed) and
.env.example, which now names the two database variables.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Closes the three operational blockers from the production readiness review.
deploy.sh now takes a database backup before it builds, stops or replaces
anything, and aborts the deploy if the backup fails — so no deploy proceeds
without a restore point. Dumps are gzipped and timestamped into
/opt/job-tracker/backups (override with BACKUP_DIR), so one deploy never
overwrites an earlier backup. Credentials come from the existing connection
string and travel via MYSQL_PWD, never on the command line, so they cannot reach
the process list or the deploy log. A dump that is empty or missing CREATE TABLE
is rejected, because a truncated file that looks like a restore point is worse
than none. SQLite deployments get their data volume tarred instead. Nothing is
ever deleted automatically; retention is documented as manual.
deploy/README.md documents backup creation, location, retention, database
restore, application rollback, and when to use which — restore and rollback kept
distinct, because a bad deploy usually needs only the rollback and restoring
would discard everything written since the dump.
Health checks now cover backend and frontend, which previously had none. GET
/health is anonymous, cheap, and deliberately does not touch the database: a
health check that queried MariaDB would restart a healthy backend whenever the
database blipped, and would hand out an unauthenticated way to probe database
availability. The backend image gains curl on the existing chromium apt layer,
since the aspnet runtime ships neither curl nor wget. frontend now waits for
backend to be healthy rather than merely started, because nginx proxies /api and
refuses to start when the upstream cannot be resolved.
Verified against real containers, no production data: backup from a seeded
MariaDB 11; restore into a clean MariaDB 11 with rows identical; bad credentials
and a missing connection string both abort non-zero and leave no partial file;
SQLite volume backup produces a readable archive; backend and frontend both
reach healthy; and a backend pointed at an unreachable database exits and is
reported unhealthy, so a broken deploy cannot present as a running stack.
Incidentally confirmed the earlier authorization work: with Auth:Require unset,
/health returns 200 while /api/jobapplications returns 401.
393 backend tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The compose file shipped its own ollama service, so 'docker compose pull'
during deploy re-downloaded the Ollama image and a deploy that starts the
AI stack would spin up a second Ollama alongside an existing/shared one.
- ollama service moved behind a 'bundled-ollama' compose profile, so it is
excluded from the default pull/up (no duplicate, no re-download)
- ai-service no longer depends_on ollama and is documented to point at a
shared instance via OLLAMA_BASE_URL (e.g. http://<host>:11435)
- deploy.sh no longer names ollama in 'compose up'
To run a self-contained Ollama: docker compose --profile bundled-ollama up
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
cesnimda
cesnimda