docs: consolidate repository guides and preserve audit history

This commit is contained in:
cesnimda
2026-09-11 14:22:41 +02:00
parent 115441a409
commit 682a3028bb
404 changed files with 3402 additions and 2621 deletions
+54 -62
View File
@@ -1,70 +1,54 @@
# Copy this file to `.env` (same folder as docker-compose.yml) and fill in values.
#
# Used by docker-compose.yml
#
# Database. deploy/deploy.sh REQUIRES DATABASE_PROVIDER to be set explicitly and
# refuses to deploy without it — it selects which backup to take, and guessing it
# wrong means backing up the wrong database. Use `mariadb` (or `mysql`) for a
# server deployment, `sqlite` for a single-file local one.
# Copy to .env beside docker-compose.yml. Never commit real secret values.
# Full descriptions: docs/configuration/environment-variables.md
# =============================================================================
# DATABASE
# =============================================================================
DATABASE_PROVIDER=sqlite
# Required when DATABASE_PROVIDER is mariadb/mysql. Ignored for sqlite, which
# stores its file in the jobtracker_data volume.
# The host resolves from INSIDE the backend container: 127.0.0.1 means the
# container, not the Docker host.
# Required for mariadb/mysql; ignored for sqlite.
JOBTRACKER_CONNECTION_STRING=
AUTH_JWT_KEY=CHANGE_ME_LONG_RANDOM_SECRET
# =============================================================================
# AUTHENTICATION, IDENTITY, BILLING
# =============================================================================
AUTH_JWT_KEY=
AUTH_ADMIN_EMAIL=admin@example.com
AUTH_ADMIN_PASSWORD=CHANGE_ME_STRONG_PASSWORD
# Public signup remains closed until explicitly enabled. Configure both Turnstile keys first.
AUTH_ADMIN_PASSWORD=
AUTH_ALLOW_REGISTRATION=false
# Require local accounts to confirm ownership of their email address before signing in.
AUTH_REQUIRE_EMAIL_VERIFICATION=true
# Destructive account deletion stays dark-launched until the retention and restore runbook is
# approved and rehearsed. The tombstone path is mounted separately by docker-compose.yml.
ACCOUNT_DELETION_ENABLED=false
TURNSTILE_SITE_KEY=
TURNSTILE_SECRET_KEY=
# Optional hosted Stripe Checkout. Configure all three values and the customer portal before enabling billing.
STRIPE_SECRET_KEY=
STRIPE_PRICE_PREMIUM=
STRIPE_WEBHOOK_SECRET=
AUTH_GOOGLE_CLIENT_ID=CHANGE_ME_GOOGLE_CLIENT_ID
# Optional: enables the "Continue with Microsoft" sign-in tab (separate from the
# MICROSOFT_CLIENT_ID below, which is for Outlook mail linking, not sign-in).
AUTH_GOOGLE_CLIENT_ID=
AUTH_MICROSOFT_CLIENT_ID=
# Microsoft application sign-in tenant policy: tenant GUID, organizations, consumers, or common.
# This is separate from MICROSOFT_TENANT_ID, which configures Graph mailbox OAuth.
AUTH_MICROSOFT_TENANT=common
GOOGLE_GMAIL_CLIENT_SECRET=CHANGE_ME_GOOGLE_OAUTH_CLIENT_SECRET
# Optional. If omitted, the backend uses https://<your-domain>/api/gmail/oauth/callback
MICROSOFT_CLIENT_ID=CHANGE_ME_MICROSOFT_CLIENT_ID
MICROSOFT_CLIENT_SECRET=CHANGE_ME_MICROSOFT_OAUTH_CLIENT_SECRET
# Optional. Defaults to "common" (personal + work/school accounts).
# =============================================================================
# MAIL PROVIDER CONNECTIONS
# =============================================================================
GOOGLE_GMAIL_CLIENT_SECRET=
MICROSOFT_CLIENT_ID=
MICROSOFT_CLIENT_SECRET=
MICROSOFT_TENANT_ID=
# Optional. If omitted, the backend uses https://<your-domain>/api/microsoft-graph/oauth/callback
# =============================================================================
# AI SERVICE AND PROVIDERS
# =============================================================================
AI_SERVICE_BASE_URL=http://ai-service:8001
# REQUIRED. Shared secret the backend sends to ai-service on every call except /health.
# The stack refuses to start without it. Generate with: openssl rand -hex 32
# Required for protected backend -> AI service calls.
AI_SERVICE_TOKEN=
# Optional: enables hybrid CV block classification in the local AI service.
OLLAMA_BASE_URL=http://ollama:11434
OLLAMA_MODEL=qwen2.5:7b
# Optional external fallback provider for heavy /cv/* calls: ollama (none) | gemini | groq.
# Local Ollama is always attempted first unless the explicitly configured mode is
# external_only. External processing still requires the administrator gate, an
# allowed task, and the authenticated Pro user's opt-in. /summarize stays local.
# Keys are read from the environment only — never commit real keys.
AI_PROVIDER=ollama
EXTERNAL_AI_ENABLED=false
AI_ROUTING_MODE=local_first
EXTERNAL_AI_ALLOWED_TASKS=cv-normalize,cv-classify,cv-rewrite
# Per-request cost/privacy ceiling. Requests above this size remain local even after local failure.
EXTERNAL_AI_MAX_PROMPT_CHARS=24000
LOCAL_AI_CIRCUIT_FAILURE_THRESHOLD=3
LOCAL_AI_CIRCUIT_OPEN_SECONDS=30
# Isolated CV document decoder budgets. Increase only after benign-file measurement.
PARSER_TIMEOUT_SECONDS=25
PARSER_CPU_SECONDS=20
PARSER_MEMORY_MIB=512
@@ -73,8 +57,6 @@ GEMINI_API_KEY=
GEMINI_MODEL=gemini-2.0-flash
GROQ_API_KEY=
GROQ_MODEL=llama-3.3-70b-versatile
# Durable AI operation worker. Keep false until handlers, monitoring and rollout gates are verified.
WORKER_AI_OPERATIONS_ENABLED=false
AI_QUEUE_WORKER_CONCURRENCY=1
AI_QUEUE_GLOBAL_CAPACITY=100
@@ -82,47 +64,57 @@ AI_QUEUE_PER_USER_CAPACITY=10
AI_QUEUE_DEADLINE_MINUTES=15
AI_QUEUE_OPERATION_TIMEOUT_SECONDS=300
# Optional: only needed if you want the UI to call a non-default API base URL.
# In production the UI defaults to `/api`.
# =============================================================================
# FRONTEND AND PUBLIC APPLICATION
# =============================================================================
# Leave empty in production so nginx proxies same-origin /api requests.
NEXT_PUBLIC_API_BASE_URL=
# Used by docker-compose.yml (email / password resets / notifications)
APP_PUBLIC_BASE_URL=https://jobs.cesnimda.uk
# Dedicated nginx-to-backend network. Confirm this CIDR does not overlap existing Docker networks.
WEB_PROXY_SUBNET=172.31.250.0/29
# Required for production: space-separated observed Traefik socket-peer CIDRs at nginx.
# Prefer individual /32 or /128 addresses. Never guess Docker gateway/proxy addresses.
# Empty fails deployment/nginx startup; development explicitly uses direct mode.
INGRESS_PROXY_CIDRS=
APP_VERSION=
APP_COMMIT_SHA=
APP_BUILD_STAMP=
# Optional. Use a narrowly scoped Gitea token that can dispatch Actions workflows.
# =============================================================================
# INGRESS AND MAINTENANCE
# =============================================================================
WEB_PROXY_SUBNET=172.31.250.0/29
# Space-separated trusted reverse-proxy peer CIDRs; required in production.
INGRESS_PROXY_CIDRS=
MAINTENANCE_GITEA_BASE_URL=https://git.cesnimda.uk
MAINTENANCE_GITEA_OWNER=cesnimda
MAINTENANCE_GITEA_REPOSITORY=jobtrackingapp
MAINTENANCE_GITEA_TOKEN=
# Complete recovery sets are created before deployment and nightly at 02:00 Europe/Oslo.
# =============================================================================
# BACKUPS
# =============================================================================
BACKUP_TIMEZONE=Europe/Oslo
BACKUP_DAILY_RETENTION_DAYS=30
BACKUP_MONTHLY_RETENTION_COUNT=12
# Optional encrypted rclone destination, for example gdrive-crypt:jobtracker.
# Optional encrypted rclone destination, e.g. gdrive-crypt:jobtracker.
BACKUP_RCLONE_REMOTE=
# =============================================================================
# SMTP AND EMAIL WORKERS
# =============================================================================
EMAIL_ENABLED=false
EMAIL_SMTP_HOST=smtp.gmail.com
EMAIL_SMTP_PORT=587
EMAIL_SMTP_USER=CHANGE_ME_GMAIL_ADDRESS
EMAIL_SMTP_PASSWORD=CHANGE_ME_GOOGLE_APP_PASSWORD
EMAIL_FROM=CHANGE_ME_GMAIL_ADDRESS
EMAIL_SMTP_USER=
EMAIL_SMTP_PASSWORD=
EMAIL_FROM=
EMAIL_FROM_NAME=Jobbjakt
EMAIL_FOLLOWUPREMINDERS_ENABLED=false
EMAIL_FOLLOWUPREMINDERS_UPCOMINGDAYS=2
EMAIL_SMTP_ENABLE_SSL=true
EMAIL_SMTP_TIMEOUT_MS=15000
# =============================================================================
# APPLICATION WORKERS AND DISCOVERY
# =============================================================================
WORKER_RULES_ENABLED=false
WORKER_FOLLOWUP_REMINDERS_ENABLED=false
WORKER_SAVED_SEARCH_ALERTS_ENABLED=true
SAVED_SEARCH_INTERVAL_HOURS=6
WORKER_DAILY_EXPORT_ENABLED=false
WORKER_JOB_ENRICHMENT_ENABLED=false
EMAIL_SMTP_ENABLE_SSL=true
EMAIL_SMTP_TIMEOUT_MS=15000
+5
View File
@@ -69,6 +69,11 @@ website_details.md
docker-compose.override.yml
docs.7z
todo.md
!docs/roadmap/todo.md
# The documentation snapshot can contain local dependency/cache README copies.
# Preserve those locally without vendoring generated environments into the archive.
docs/_archive/**/.venv*/
docs/_archive/**/.pytest_cache/
# Private local test files
vendor/
+4 -4
View File
@@ -9,7 +9,7 @@ Updated: 2026-09-01
- **Required:** Configure the Pro recurring price, enable the Stripe customer portal, register `/api/billing/webhook` for `customer.subscription.created`, `customer.subscription.updated`, and `customer.subscription.deleted`, then supply `STRIPE_SECRET_KEY`, the recurring `price_...` value in the legacy-named `STRIPE_PRICE_PREMIUM` setting, and `STRIPE_WEBHOOK_SECRET` through the deployment environment. Do not place secret values in source control or chat.
- **Recommended:** One monthly Pro price first; add annual billing only after the monthly flow is operating.
- **Current access check:** Production has test-mode secret and webhook values, but `STRIPE_PRICE_PREMIUM` currently contains a `prod_...` Product ID. Checkout requires the recurring `price_...` Price ID; the server now treats the wrong identifier type as disabled rather than calling Stripe. The publishable key is not used by hosted Checkout. Local fake-gateway coverage proves active → expired → canceled/replayed role transitions without losing non-AI data (V-185).
- **Runbook:** Follow `docs/operations/stripe-activation.md`, completing test mode before creating or installing live-mode values.
- **Runbook:** Follow `docs/infrastructure/stripe-activation.md`, completing test mode before creating or installing live-mode values.
## Public registration verification
@@ -23,9 +23,9 @@ Updated: 2026-09-01
- **Blocked:** Authenticated production smoke tests, backup restore verification against real data, OAuth-provider checks, and rollback rehearsal.
- **Why:** The authorised deployment is complete, but authenticated smoke, port/network closure, complete backup restore, provider checks and failure-injection rollback require credentials and/or separately authorised production mutations.
- **Required:** Confirm the admin-only version badge matches version 286 / commit `ea7d9f3`, then run the authenticated application workspace, Career, CV, attachment, email-verification and rollback checks from `docs/release-candidate-review.md` and `docs/release-checklist.md`.
- **Required:** Confirm the admin-only version badge matches version 286 / commit `ea7d9f3`, then run the authenticated application workspace, Career, CV, attachment, email-verification and rollback checks from `docs/infrastructure/deployment.md` and `docs/infrastructure/disaster-recovery.md`.
- **Recommended:** Verify backup/restore before deployment, then exercise login, existing application counts, Career Workspace, public CV refresh/download, AI, and attachments in order.
- **Current status:** Gitea run 705 (application version 286, commit `ea7d9f3`) completed both the complete test job and remote deployment successfully on 2026-09-01. Public `/health`, `/ready`, and `/api/auth/config` returned HTTP 200 after replacement; liveness reported version 286 and readiness reported `ready`. This proves the proxy/frontend, API and database readiness path, but not authenticated application behavior, provider delivery or restore/rollback behavior. Read-only PROD-001 inventory previously found the JobTracker Ollama and frontend published on all host interfaces, the newest observed gzip-valid MariaDB backup dated 2026-08-02, no observed scheduled JobTracker backup, and no owner-file/key/tombstone recovery bundle. Those remaining gates still require operator-controlled verification; see `docs/production/production-ai-hardware-assessment.md`.
- **Current status:** Gitea run 705 (application version 286, commit `ea7d9f3`) completed both the complete test job and remote deployment successfully on 2026-09-01. Public `/health`, `/ready`, and `/api/auth/config` returned HTTP 200 after replacement; liveness reported version 286 and readiness reported `ready`. This proves the proxy/frontend, API and database readiness path, but not authenticated application behavior, provider delivery or restore/rollback behavior. Read-only PROD-001 inventory previously found the JobTracker Ollama and frontend published on all host interfaces, the newest observed gzip-valid MariaDB backup dated 2026-08-02, no observed scheduled JobTracker backup, and no owner-file/key/tombstone recovery bundle. Those remaining gates still require operator-controlled verification; see the current [deployment](docs/infrastructure/deployment.md), [backup](docs/infrastructure/backups.md), and [disaster-recovery](docs/infrastructure/disaster-recovery.md) guides; the detailed historical inventory remains in the pre-cleanup archive.
- **Repository-side rollback status:** Core images are now tagged by commit, the prior running backend/frontend images are retained, and any failure after replacement triggers an automatic core rollback while preserving the failed CI status. Compose configuration validates locally. A real rollback rehearsal remains blocked because the local Linux Docker daemon is unavailable and production mutation is not authorised; run the failure-injection rehearsal on a disposable Linux Docker host before relying on it in production.
## Account deletion retention and restore policy
@@ -39,5 +39,5 @@ Updated: 2026-09-01
- **Blocked:** Removing the opportunity columns duplicated between `JobApplication` and `Job`.
- **Why:** The compatibility dual-write protects existing production rows and older clients. The release branch now backfills missing opportunities on startup, synchronizes both creation paths, and leaves all legacy columns intact. Dropping columns still requires production validation and an observation release.
- **Required:** After deployment, run the read-only report in `docs/operations/job-opportunity-cutover.md` against production and a restored backup, then confirm that backward API compatibility is no longer required.
- **Required:** After deployment, run the read-only report in `docs/database/migrations.md#production-cutover-validation` against production and a restored backup, then confirm that backward API compatibility is no longer required.
- **Recommended:** Use an expand/contract release: first stop legacy reads after a verified backfill, observe one release, then drop the duplicate columns in the following migration.
@@ -10,7 +10,7 @@ namespace JobTrackerApi.Tests;
// The point is that a NEW controller cannot ship unprotected by accident. Authentication used to rely
// on the Auth:Require fallback policy, which defaults to false — a deployment that lost that flag
// would have served tenant data anonymously. Every controller must now declare its intent.
// docs/production-readiness-review.md.
// docs/security/security-notes.md.
public sealed class ApiAuthorizationPostureTests
{
// Endpoints that are anonymous ON PURPOSE. Adding to this list is a deliberate security decision.
@@ -9,7 +9,7 @@ using Microsoft.EntityFrameworkCore;
namespace JobTrackerApi.Controllers;
// Phase 5 — the AI Workspace for one job application. Every module runs through ISummarizerService and
// is stored as append-only history; nothing is applied automatically. docs/architecture/ai-career-assistant.md.
// is stored as append-only history; nothing is applied automatically. docs/ai/architecture.md.
[ApiController]
[Route("api/jobapplications/{jobId:int}/ai")]
[Authorize(AuthenticationSchemes = "local")]
@@ -12,7 +12,7 @@ namespace JobTrackerApi.Controllers;
// /api/cv (CvVariantController), and documents stay on /api/attachments. These routes only cover what
// is genuinely application-scoped — which variant this application uses, what to tailor, and the
// cover letter with its history. Every route is tenant-scoped and returns 404 for another user's
// application. docs/architecture/application-workspace.md.
// application. docs/architecture/overview.md.
[ApiController]
[Route("api/jobapplications/{jobId:int}")]
[Authorize(AuthenticationSchemes = "local")]
@@ -8,7 +8,7 @@ namespace JobTrackerApi.Controllers;
// Phase 5 Milestone 2 — the application checklist. System items seed themselves on first read from the
// existing readiness signals; the user owns everything after that.
// docs/architecture/application-workspace.md.
// docs/architecture/overview.md.
[ApiController]
[Route("api/jobapplications/{jobId:int}/checklist")]
[Authorize(AuthenticationSchemes = "local")]
@@ -11,7 +11,7 @@ namespace JobTrackerApi.Controllers;
//
// None of them write anything. The AI narrative for analysis and match stays on the existing
// /api/jobapplications/{id}/ai routes, which are suggestion-only and versioned by AiInteraction.
// docs/architecture/application-workspace.md.
// docs/architecture/overview.md.
[ApiController]
[Route("api/jobapplications/{jobId:int}")]
[Authorize(AuthenticationSchemes = "local")]
@@ -8,7 +8,7 @@ namespace JobTrackerApi.Controllers;
// Phase 5 Milestone 1 — one aggregate read for the Application Workspace overview, so the page loads
// from a single call instead of fanning out. Read-only; owns no data.
// docs/architecture/application-workspace.md.
// docs/architecture/overview.md.
[ApiController]
[Route("api/jobapplications/{jobId:int}/workspace")]
[Authorize(AuthenticationSchemes = "local")]
@@ -11,7 +11,7 @@ namespace JobTrackerApi.Controllers;
// here. The relational children (via CareerProfileService) are the source of truth; the
// ApplicationUser.ProfileCvStructureJson blob is kept in sync as a derived projection so the legacy
// read paths (CV rendering, tailoring, match-score) keep working. See
// docs/architecture/career-profile-model.md.
// docs/architecture/overview.md.
[ApiController]
[Route("api/career/profile")]
[Authorize]
@@ -10,7 +10,7 @@ namespace JobTrackerApi.Controllers
[ApiController]
// Explicitly authorized. These endpoints are all tenant-scoped user data, so they must not
// depend on the Auth:Require fallback policy being switched on: a deployment that lost that flag
// would otherwise serve them anonymously. docs/production-readiness-review.md.
// would otherwise serve them anonymously. docs/security/security-notes.md.
[Route("api/companies")]
[Authorize(AuthenticationSchemes = "local")]
public class CompaniesController : ControllerBase
@@ -11,7 +11,7 @@ namespace JobTrackerApi.Controllers
[ApiController]
// Explicitly authorized. These endpoints are all tenant-scoped user data, so they must not
// depend on the Auth:Require fallback policy being switched on: a deployment that lost that flag
// would otherwise serve them anonymously. docs/production-readiness-review.md.
// would otherwise serve them anonymously. docs/security/security-notes.md.
[Route("api/correspondence")]
[Authorize(AuthenticationSchemes = "local")]
public class CorrespondenceController : ControllerBase
@@ -8,7 +8,7 @@ namespace JobTrackerApi.Controllers;
// Phase 4 — Career Workspace Builder API. Every endpoint reads the master profile through
// ICareerProfileService and renders through the data-driven theme engine; none of them store career
// data. docs/architecture/cv-builder.md.
// data. docs/architecture/overview.md.
[ApiController]
[Route("api/cv")]
[Authorize(AuthenticationSchemes = "local")]
@@ -11,7 +11,7 @@ namespace JobTrackerApi.Controllers;
// Prep content is the user's; AI suggestions come from the existing /api/jobapplications/{id}/ai
// routes and only reach here once the user accepts one (POST with source "ai"). Communication stays
// entirely on the existing Correspondence routes — there is no messaging endpoint here.
// docs/architecture/application-workspace.md.
// docs/architecture/overview.md.
[ApiController]
[Route("api/jobapplications/{jobId:int}")]
[Authorize(AuthenticationSchemes = "local")]
@@ -20,7 +20,7 @@ namespace JobTrackerApi.Controllers
[ApiController]
// Explicitly authorized. These endpoints are all tenant-scoped user data, so they must not
// depend on the Auth:Require fallback policy being switched on: a deployment that lost that flag
// would otherwise serve them anonymously. docs/production-readiness-review.md.
// would otherwise serve them anonymously. docs/security/security-notes.md.
[Route("api/jobapplications")]
[Authorize(AuthenticationSchemes = "local")]
public class JobApplicationsController : ControllerBase
@@ -1356,7 +1356,7 @@ Candidate CV/profile:
// application checklist is the one workflow surface; readiness projects it into the score /
// completed / missing / reminders health view the dialog and dashboard already consume, so
// the two can never disagree. The DTO shape is unchanged on purpose.
// docs/architecture/application-workspace.md.
// docs/architecture/overview.md.
var checklist = job.OwnerUserId is null
? null
: await _checklist.GetAsync(job.OwnerUserId, id, cancellationToken);
@@ -7,7 +7,7 @@ namespace JobTrackerApi.Controllers;
[ApiController]
// Explicitly authorized. These endpoints are all tenant-scoped user data, so they must not
// depend on the Auth:Require fallback policy being switched on: a deployment that lost that flag
// would otherwise serve them anonymously. docs/production-readiness-review.md.
// would otherwise serve them anonymously. docs/security/security-notes.md.
[Route("api/jobimport")]
[Authorize(AuthenticationSchemes = "local")]
public sealed class JobImportController : ControllerBase
@@ -29,4 +29,3 @@ public sealed class JobImportController : ControllerBase
return Ok(result);
}
}
@@ -8,7 +8,7 @@ using Microsoft.AspNetCore.RateLimiting;
namespace JobTrackerApi.Controllers;
// Anonymous read-only surface for public CVs: /cv/{slug} in the SPA fetches this. Only variants the
// owner has explicitly toggled public are served; noindex by default. docs/architecture/cv-builder.md.
// owner has explicitly toggled public are served; noindex by default. docs/architecture/overview.md.
[ApiController]
[Route("api/public-cv")]
[AllowAnonymous]
+1 -1
View File
@@ -9,7 +9,7 @@ namespace JobTrackerApi.Controllers
[ApiController]
// Explicitly authorized. These endpoints are all tenant-scoped user data, so they must not
// depend on the Auth:Require fallback policy being switched on: a deployment that lost that flag
// would otherwise serve them anonymously. docs/production-readiness-review.md.
// would otherwise serve them anonymously. docs/security/security-notes.md.
[Route("api/rules")]
[Authorize(AuthenticationSchemes = "local")]
public class RulesController : ControllerBase
+7 -7
View File
@@ -385,7 +385,7 @@ namespace JobTrackerApi.Data
modelBuilder.Entity<UserSession>()
.HasIndex(x => x.UserId);
// Career Workspace foundation (docs/career-workspace-implementation-roadmap.md Phase F1).
// Career Workspace foundation (docs/roadmap/todo.md).
// One CareerProfile per user for now -- see roadmap "Not now: multiple profiles per user".
modelBuilder.Entity<CareerProfile>()
.HasQueryFilter(x => CurrentUserId != null && x.OwnerUserId == CurrentUserId);
@@ -412,7 +412,7 @@ namespace JobTrackerApi.Data
.OnDelete(DeleteBehavior.Cascade);
// Phase 3: relational children of CareerProfile (the editable master profile).
// docs/architecture/career-profile-model.md. Same deny-on-null tenant filter as everything
// docs/architecture/overview.md. Same deny-on-null tenant filter as everything
// else; cascade-delete with the parent; indexed by (OwnerUserId, CareerProfileId, SortOrder)
// for the ordered per-profile reads the /career editor does.
ConfigureCareerChild<CareerExperience>(modelBuilder);
@@ -470,7 +470,7 @@ namespace JobTrackerApi.Data
.OnDelete(DeleteBehavior.Cascade);
// Phase 4: CV variants (a lens over the master profile) + their autosave history.
// docs/architecture/cv-builder.md. Same deny-on-null tenant filter as everything else. A
// docs/architecture/overview.md. Same deny-on-null tenant filter as everything else. A
// variant references a job application (reference-not-ownership, SetNull on delete) and never
// owns career data. PublicSlug is globally unique so /cv/{slug} can resolve it anonymously.
modelBuilder.Entity<CvVariant>()
@@ -504,7 +504,7 @@ namespace JobTrackerApi.Data
// Phase 5: append-only AI interaction history per job application. Same deny-on-null tenant
// filter; indexed for the per-job history read; cascades with the application.
// docs/architecture/ai-career-assistant.md.
// docs/ai/architecture.md.
modelBuilder.Entity<AiInteraction>()
.HasQueryFilter(x => CurrentUserId != null && x.OwnerUserId == CurrentUserId);
// varchar (not longtext) for the indexed columns — see the CvVariant note above.
@@ -532,7 +532,7 @@ namespace JobTrackerApi.Data
// Phase 5 Milestone 2: the application checklist — a workflow guidance layer over the existing
// readiness signals, not a second store of truth. Same deny-on-null tenant filter; cascades with
// the application. docs/architecture/application-workspace.md.
// the application. docs/architecture/overview.md.
modelBuilder.Entity<ApplicationChecklistItem>()
.HasQueryFilter(x => CurrentUserId != null && x.OwnerUserId == CurrentUserId);
// varchar (not longtext) for the indexed columns — see the CvVariant note above.
@@ -557,7 +557,7 @@ namespace JobTrackerApi.Data
// Phase 5.4: append-only cover letter history. JobApplication.CoverLetterText stays the
// current text; this makes every previous state recoverable.
// docs/architecture/application-workspace.md.
// docs/architecture/overview.md.
modelBuilder.Entity<CoverLetterVersion>()
.HasQueryFilter(x => CurrentUserId != null && x.OwnerUserId == CurrentUserId);
// varchar (not longtext) for the indexed columns — see the CvVariant note above.
@@ -600,7 +600,7 @@ namespace JobTrackerApi.Data
.OnDelete(DeleteBehavior.Cascade);
// Phase 5.5: user-owned interview preparation. Unlike InterviewPrepNote (an AI cache), nothing
// regenerates this. docs/architecture/application-workspace.md.
// regenerates this. docs/architecture/overview.md.
modelBuilder.Entity<InterviewPrepItem>()
.HasQueryFilter(x => CurrentUserId != null && x.OwnerUserId == CurrentUserId);
// varchar (not longtext) for the indexed columns — see the CvVariant note above.
@@ -14,7 +14,7 @@ namespace JobTrackerApi.Migrations
// The six CareerProfile child tables were initially provisioned by the reconciler. Explicit
// ownership moved to 20260830133000_AdoptCareerProfileSchema; this historical migration stays
// recorded so existing chains remain compatible.
// docs/infrastructure/database-ownership.md.
// docs/database/migrations.md.
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
@@ -14,7 +14,7 @@ namespace JobTrackerApi.Migrations
// CoverLetterVersions is reconciler-owned and provisioned by StartupInitializationExtensions,
// which carries correct DDL per provider and guards the create on JobApplications existing.
// This migration exists only to keep the model snapshot in sync.
// docs/infrastructure/database-ownership.md.
// docs/database/migrations.md.
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
@@ -13,7 +13,7 @@ namespace JobTrackerApi.Migrations
//
// InterviewPrepItems is reconciler-owned and provisioned by StartupInitializationExtensions,
// which carries correct DDL per provider and guards the create on JobApplications existing.
// docs/infrastructure/database-ownership.md.
// docs/database/migrations.md.
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
+1 -1
View File
@@ -4,7 +4,7 @@ namespace JobTrackerApi.Models;
// Unlike AiWorkspaceNote (one row per (owner, job, type), overwritten on regenerate — a cache), this
// keeps EVERY generation so the user can restore, compare, reuse, or delete past outputs. Suggestion
// only: an interaction never mutates the master profile, a CV variant, or the application itself.
// docs/architecture/ai-career-assistant.md.
// docs/ai/architecture.md.
public sealed class AiInteraction
{
public int Id { get; set; }
@@ -9,7 +9,7 @@ namespace JobTrackerApi.Models;
// System items carry a stable SystemKey and (usually) an AutoSignal: the same readiness signal the
// /readiness endpoint already computes. When the signal is satisfied the item auto-completes, so the
// checklist and readiness can never disagree — readiness became the calculation, this is the surface.
// docs/architecture/application-workspace.md.
// docs/architecture/overview.md.
public sealed class ApplicationChecklistItem
{
public int Id { get; set; }
+1 -1
View File
@@ -4,7 +4,7 @@ using System.Text.Json;
namespace JobTrackerApi.Models;
// Phase 3: relational children of CareerProfile — the editable master career profile.
// See docs/architecture/career-profile-model.md.
// See docs/architecture/overview.md.
//
// Design notes shared by all child entities:
// - OwnerUserId is denormalized so the tenant global query filter applies directly (same pattern
+1 -1
View File
@@ -12,7 +12,7 @@ public sealed class CareerProfile
// Phase 3: ProfileJson is now a DERIVED projection of the relational children + LongTailJson,
// serialized into the StructuredCvProfile shape and kept in sync on every save so the legacy
// read paths (CV rendering, tailoring, match-score) keep working unchanged. It is no longer an
// independently editable source. See docs/architecture/career-profile-model.md §4.
// independently editable source. See docs/architecture/overview.md.
public string ProfileJson { get; set; } = string.Empty;
// The editable long tail that is not worth a relational table: contact, summary, interests,
+1 -1
View File
@@ -3,7 +3,7 @@ namespace JobTrackerApi.Models;
// A theme is DATA, not code. The renderer (ThemedCvRenderer) has one render path; every visual
// difference between themes is expressed by the fields below. Adding a theme = adding one CvTheme to
// CvThemeCatalog — no renderer changes. This is the core Phase 4 architectural rule
// ("Do NOT hardcode HTML templates"). docs/architecture/cv-theme-engine.md.
// ("Do NOT hardcode HTML templates"). docs/architecture/frontend.md.
public sealed class CvTheme
{
public string Id { get; init; } = string.Empty;
+1 -1
View File
@@ -6,7 +6,7 @@ namespace JobTrackerApi.Models;
// sections show, in what order, with what per-item overrides, rendered through a chosen theme with
// layout/colour/font settings. The career data is never duplicated here — the variant stores only
// selections + overrides keyed by the master profile's stable ItemKeys, plus builder preferences.
// docs/architecture/cv-builder.md.
// docs/architecture/overview.md.
public sealed class CvVariant
{
public int Id { get; set; }
+1 -1
View File
@@ -664,7 +664,7 @@ app.MapControllers();
// this process up and serving?" and nothing else. It does NOT touch the database — a health check
// that queried MariaDB would restart a perfectly healthy backend whenever the database blipped, and
// would also be a free unauthenticated way to probe database availability.
// docs/production-readiness-review.md.
// docs/security/security-notes.md.
// Version comes from configuration (App:Version), which is how docker-compose supplies it as
// App__Version. Reading the APP_VERSION environment variable directly meant /health always
// reported "unknown" in a container. Falls back to the assembly version for local development.
@@ -14,7 +14,7 @@ namespace JobTrackerApi.Services;
//
// The flow is strictly one-directional — CareerProfile -> CvVariant -> application output. Nothing
// here writes upward: no method touches CareerProfile or its children.
// docs/architecture/application-workspace.md.
// docs/architecture/overview.md.
public sealed record ApplicationCvDto(
int? AttachedVariantId,
string? AttachedVariantName,
@@ -12,7 +12,7 @@ namespace JobTrackerApi.Services;
// /readiness endpoint computes, and are re-synced on every read, so "readiness says X is missing" and
// "the checklist says X is pending" cannot drift apart. Readiness keeps its API contract and becomes
// the calculation; the checklist is what the user actually works from and edits.
// docs/architecture/application-workspace.md.
// docs/architecture/overview.md.
public sealed record ChecklistItemDto(
int Id,
string? SystemKey,
@@ -18,7 +18,7 @@ namespace JobTrackerApi.Services;
// The AI narrative is deliberately NOT here: it stays in AiWorkspaceService's "job-analysis" and
// "career-match" modules, which are suggestion-only, append-only (AiInteraction is the version
// history), and require the user to act on them. Nothing in this file writes to the CareerProfile,
// a CvVariant, or the JobApplication. docs/architecture/application-workspace.md.
// a CvVariant, or the JobApplication. docs/architecture/overview.md.
public sealed record JobAnalysisDto(
string? Role,
string? Company,
@@ -10,7 +10,7 @@ namespace JobTrackerApi.Services;
// writes nothing, stores nothing, and adds no table. It turns rows like
// ("StatusChanged", "Applied", "Interview") into a sentence, tags each event with a category and
// whether it is a milestone, and groups the result by day so the workspace can render a real
// timeline instead of a flat list. docs/architecture/application-workspace.md.
// timeline instead of a flat list. docs/architecture/overview.md.
public sealed record TimelineEventDto(
int Id,
string Type,
@@ -9,7 +9,7 @@ namespace JobTrackerApi.Services;
// This is an AGGREGATE READ ONLY. It owns no data and duplicates none: the CV comes from the Phase 4
// CvVariant lens, documents from Attachments, activity from JobEvent, AI history from AiInteraction,
// stage semantics from JobPipeline. Nothing here writes, and no career data is copied.
// docs/architecture/application-workspace.md.
// docs/architecture/overview.md.
public sealed record WorkspaceCvDto(int? VariantId, string? VariantName, string? ThemeId, bool HasTailoredCvText, DateTimeOffset? UpdatedAtUtc);
public sealed record WorkspaceActivityDto(string Type, string? Detail, DateTime At);
public sealed record WorkspaceNextStepDto(string Key, string Label, string Reason, string? Section);
+1 -1
View File
@@ -9,7 +9,7 @@ namespace JobTrackerApi.Services;
// the anonymous /health endpoint. /health used to read the APP_VERSION *environment
// variable* directly, which compose never sets under that name — it passes App__Version,
// binding to the App:Version configuration key. So /health always reported "unknown".
// See docs/release-candidate-review.md (N2).
// See docs/infrastructure/deployment.md for the health/version verification contract.
public static class BuildMetadata
{
// Ignore unresolved shell/compose placeholders that would otherwise leak into the
@@ -6,7 +6,7 @@ namespace JobTrackerApi.Services;
// Phase 3: two-way projection between the relational CareerProfile children and the
// StructuredCvProfile shape (the derived blob + the frontend/AI edit unit).
// See docs/architecture/career-profile-model.md.
// See docs/architecture/overview.md.
//
// Relational holds: Experience, Education, Skill, Project, Certification, Language.
// LongTailJson holds the rest of StructuredCvProfile: Contact, Summary, Interests, OtherSections,
@@ -5,7 +5,7 @@ using Microsoft.EntityFrameworkCore;
namespace JobTrackerApi.Services;
// Career Workspace foundation (see docs/career-workspace-implementation-roadmap.md, Phase F1).
// Career Workspace foundation (see docs/roadmap/todo.md).
// Bounded to the profile/CV domain -- job tracking is untouched. Every existing read path still
// goes through ApplicationUser.ProfileCvStructureJson (dual-write window); this service is the
// single place stable item IDs and normalized dates get assigned, and where profile history is
@@ -3,7 +3,7 @@ namespace JobTrackerApi.Services.EmailProviders
/// <summary>
/// Provider-neutral email operations so job correspondence can be sourced from Gmail,
/// Microsoft Graph, generic IMAP, or manual/free-text entry behind a single seam.
/// See docs/remaster/PRODUCT_DIRECTION.md (multi-provider email). Gmail is the first
/// See docs/architecture/integrations.md (multi-provider email). Gmail is the first
/// implementation (<see cref="GmailProvider"/>); the controller migration and additional
/// providers land in follow-up slices.
/// </summary>
@@ -13,7 +13,7 @@ namespace JobTrackerApi.Services;
//
// Follow-ups reuse what already exists: JobApplication.FollowUpAt for the date (RulesEngine and the
// reminder hosted service already act on it) and ApplicationChecklistItem for the task. No second
// reminder system. docs/architecture/application-workspace.md.
// reminder system. docs/architecture/overview.md.
public sealed record InterviewPrepItemDto(
int Id,
string Category,
+1 -1
View File
@@ -11,7 +11,7 @@ namespace JobTrackerApi.Services;
//
// Events are derived from the TRANSITION, not from the resulting state, so one user action produces
// at most one lifecycle event and re-saving an unchanged status produces none.
// docs/architecture/application-workspace.md.
// docs/architecture/overview.md.
public static class JobLifecycleEvents
{
/// <summary>
@@ -121,7 +121,7 @@ public static class StartupInitializationExtensions
// migration-owned tables do not exist yet. Repairing an absent table is not this pass's job --
// it is skipped, Migrate() creates it, and the post-Migrate pass then finds it and reconciles.
// Without this guard a clean MariaDB install died on "Table 'CareerProfiles' doesn't exist".
// See docs/infrastructure/database-ownership.md.
// See docs/database/migrations.md.
private static void EnsureMySqlIndex(DbConnection c, string table, string indexName, string columnsSql, bool unique = false)
{
if (!HasMySqlTable(c, table)) return;
@@ -276,7 +276,7 @@ public static class StartupInitializationExtensions
// the migration-created tables and finishes compatibility repair.
//
// Every statement in here is existence-guarded, so running it twice is a no-op scan on an
// already-correct database. docs/infrastructure/database-ownership.md.
// already-correct database. docs/database/migrations.md.
void ReconcileSchema()
{
if (useSqliteBootstrap)
+1 -1
View File
@@ -15,7 +15,7 @@ public interface IThemedCvRenderer
// ONE render path for every theme. The theme (data) drives layout, palette, typography and spacing;
// the variant settings apply a few runtime overrides (accent, fonts, density, page size, photo,
// icons, page numbers). Adding a theme never touches this file. docs/architecture/cv-theme-engine.md.
// icons, page numbers). Adding a theme never touches this file. docs/architecture/frontend.md.
public sealed class ThemedCvRenderer : IThemedCvRenderer
{
public ThemedCvRenderResult Render(CvRenderModel model, CvTheme theme, CvVariantSettings settings)
+25 -288
View File
@@ -1,315 +1,52 @@
# Job Tracker
Job Tracker is a self-hosted career workspace built with a Next.js/React frontend and an ASP.NET Core API. SQLite is the local default; MariaDB/MySQL is the supported server-database option.
Job Tracker (Jobjakt) is a self-hosted workspace for managing job applications, companies, correspondence, follow-ups, attachments, career profiles, CVs, and optional local/private AI assistance.
## Features (high level)
## Stack
- Track job applications (status, applied date, notes, tags, follow-up dates, deadlines, salary, links, etc.)
- Share opt-in public CV links with recruiter-friendly PDF download
- Company management (location/source, recruiter details, pipeline stage, next contact date)
- Correspondence log per application (email/messages with subject/channel/date)
- Attachments per application (upload, list, download, rename, delete)
- Reminders + follow-up “needs attention” logic driven by configurable rules
- History/event trail per application (created, status changes, follow-up set, delete/restore)
- Export jobs to JSON/CSV + daily scheduled JSON export
- Optional “job import” preview from supported job sites (plugins) + optional translation to English
- Quick-capture bookmarklet (Settings) + installable PWA with a mobile share-target: both open `/?add=<page url>` to pre-fill Add Job from any posting
- Note: no offline service-worker cache is bundled by design (the app is deployed frequently; an aggressive cache would risk serving stale builds). The manifest provides installability and share-to-capture without it.
- Optional local AI service for short/full descriptions
- Optional Google sign-in (Google ID tokens) to protect the API
- Standard Problem Details responses with trace IDs for unhandled API errors
- Next.js 16 / React 19 / TypeScript / MUI frontend
- ASP.NET Core 9 / EF Core 9 API
- SQLite by default; MariaDB/MySQL for server deployments
- FastAPI document/AI service with optional Ollama, Gemini, or Groq CV routing
- Docker Compose + nginx; Gitea Actions deployment
## Architecture
## Quick start
- `job-tracker-ui/`: statically exported Next.js 16 / React 19 app (runs on `http://localhost:3000` in dev)
- `JobTrackerApi/`: ASP.NET Core API (defaults to `http://localhost:5202`)
- Database: SQLite defaults to `JobTrackerApi/jobtracker.db`; MariaDB/MySQL is selected with `Database:Provider`
- Attachments: stored on disk under `DataRoot/Attachments/<jobId>/...`
- Optional local AI service: `tools/summarizer/` (FastAPI) used by the API via `Ai:BaseUrl`
Prerequisites: .NET SDK 9, Node.js 24/npm, and Docker Compose for the full stack.
## Quickstart (Docker)
For Docker, copy `.env.example` to `.env`, fill required values, then run:
This runs: frontend (nginx), backend API, the local AI service, and an Ollama container for hybrid CV block classification.
1) Create a `.env` file next to `docker-compose.yml` (you can start from `.env.example`).
```bash
```powershell
docker compose -f docker-compose.yml -f docker-compose.dev.yml up --build
```
- UI: `http://localhost:3000`
- API: available via the UI container reverse proxy at `http://localhost:3000/api/...`
- Persistent data: stored in the `jobtracker_data` Docker volume (mounted at `/data` in the API container)
Open `http://localhost:3000`. The API is available at `http://localhost:5202` in local process development.
## Local development
For local processes:
### Prereqs
- .NET SDK `9.x` (API targets `net9.0`)
- Node.js 24 and npm (matching CI)
- (Optional) Python 3.12 if running the AI service without Docker
### 1) Run the API
```bash
```powershell
cd JobTrackerApi
dotnet restore --locked-mode
dotnet run
```
By default the API listens on `http://localhost:5202` (see `JobTrackerApi/Properties/launchSettings.json`).
### Local preflight before browser/UAT
Run the backend first from `JobTrackerApi/`, then use the preflight gate from the repo root:
```bash
bash scripts/s06-preflight.sh
```
The preflight assumes the dev pairing used by `job-tracker-ui/src/api.ts` and `JobTrackerApi/appsettings.Development.json`:
- UI origin: `http://localhost:3000`
- API base: `http://localhost:5202/api`
It first checks the anonymous `GET /api/auth/config` endpoint, then probes `GET /api/admin/system` for DB/Gmail/AI readiness.
If auth is enabled and `/api/admin/system` returns `401/403`, export an admin bearer token first and rerun:
```bash
export AUTH_TOKEN="<admin bearer token>"
bash scripts/s06-preflight.sh
```
To obtain a local admin token in dev, log in against the API with the seeded admin email/password from `JobTrackerApi/appsettings.Development.json` (or your environment override) via `POST /api/auth/login`, then export only the returned access token. The script never prints token values. Use `API_BASE` if your API is not on the default dev port.
### Seed acceptance-ready data
After preflight passes and you have a bearer token, seed one deterministic acceptance fixture for the `/jobs` → workspace → follow-up → dashboard/reminders rerun:
```bash
export AUTH_TOKEN="<bearer token>"
bash scripts/s06-acceptance-data.sh
```
The script reuses `scripts/s06-preflight.sh`, creates or reuses the acceptance company/job, saves tailored package material, ensures one deterministic recruiter-thread correspondence entry, schedules follow-up readiness, and prints the seeded ids/readiness summary without echoing the token.
If the placeholder development password no longer matches the local DB, use the real account for this environment or a bearer token from an already-authenticated local browser session.
### 2) Run the UI
```bash
cd job-tracker-ui
cd ..\job-tracker-ui
npm ci
npm run dev
```
The UI defaults to calling `http://localhost:5202/api` when running on localhost (see `job-tracker-ui/src/api.ts`).
## Quality checks
### 3) Run the browser smoke suite
```bash
```powershell
dotnet test JobTrackerApi.Tests/JobTrackerApi.Tests.csproj --no-restore
cd job-tracker-ui
npx playwright install chromium
npm run test:e2e
npm run lint
npm test -- --runInBand
npm run typecheck
npm run build
```
The suite uses `http://localhost:3300`, starts isolated API/SQLite and Next.js processes, then covers login, saved-job creation,
Career Workspace, and anonymous public-CV/PDF access. It is also a required CI gate.
Run browser tests with `npx playwright install chromium` once, followed by `npm run test:e2e` from `job-tracker-ui/`.
### 4) (Optional) Run the AI service
## Documentation
The API calls a local FastAPI service to generate summaries. If it’s not running, the app still works (summary generation may be empty / best-effort).
With Docker (recommended):
```bash
# One command for local Ollama startup + pull + AI-service restart
OLLAMA_MODEL=qwen2.5:7b ./scripts/start-ollama-cv.sh
# Then start the rest of the app if needed
docker compose -f docker-compose.yml -f docker-compose.dev.yml up --build -d backend frontend
```
The first Ollama startup is usually quick, but the first model pull and first generation can take a while. After the model is cached in the `ollama_data` volume, later restarts are much faster.
Or run directly from `tools/summarizer/` (see `tools/summarizer/README.md`).
## Configuration
### API settings (appsettings / env vars)
Common keys:
- `ConnectionStrings:JobTracker`: overrides SQLite location (otherwise uses `DataRoot/jobtracker.db`)
- `Data:Root`: folder for the SQLite DB + exports (defaults to API content root)
- `Data:AttachmentsRoot`: override attachments folder (defaults to `<Data:Root>/Attachments`)
- `CvExports:RetainDays`: generated PDF retention in days (default `30`, clamped to `1`–`365`)
- `Cors:Origins`: list of allowed origins (defaults to `http://localhost:3000`; wildcard origins are rejected because requests use credentials)
- `Ai:BaseUrl`: AI service base URL (default `http://127.0.0.1:8001`)
- `Workers:RulesEnabled`, `Workers:FollowUpRemindersEnabled`, `Workers:DailyExportEnabled`, `Workers:JobEnrichmentEnabled`: explicit worker kill switches, all default `false`; do not enable before the prerequisites in `docs/architecture/background-workers.md`
- `Exports:DailyEnabled`: legacy daily-export setting; both it and `Workers:DailyExportEnabled` must be true
- `Exports:DailyFolder`: export destination (relative to `Data:Root` if not absolute)
- `Exports:DailyHourLocal`: local hour (0–23) when the daily export runs
- `Backups:Enabled`: enable/disable the automated SQLite backup job (default `true`)
- `Backups:HourLocal`: local hour (0–23) when the daily database backup runs (default `3`)
- `Backups:RetainCount`: how many backup files to keep in `<Data:Root>/backups` (default `14`)
- Backups use SQLite `VACUUM INTO` (consistent snapshot, safe with WAL). A catch-up backup runs at startup when none exists from the last 24 h. For MySQL/MariaDB configure external backups instead (see `deploy/MARIADB.md`).
- `Auth:GoogleClientId`: if set, enables JWT bearer validation for Google ID tokens
- `Auth:JwtKey`: secret used to sign local JWTs for username/password login (set via env var `Auth__JwtKey`)
- `Auth:JwtIssuer`: JWT issuer (default `JobTrackerApi`)
- `Auth:JwtAudience`: JWT audience (default `job-tracker-ui`)
- `Auth:JwtExpiresMinutes`: access token lifetime in minutes (default `720`)
- `Auth:AdminEmail` / `Auth:AdminPassword`: optional seed admin user (created on startup if missing)
- `Auth:AllowRegistration`: allow self-service registration via `POST /api/auth/register` (default `false`)
- `Auth:Require`: if `true`, all endpoints require auth (except endpoints explicitly marked anonymous)
- `Stripe:SecretKey`, `Stripe:PricePremium`, `Stripe:WebhookSecret`: enable hosted Premium checkout,
the customer portal, and signed subscription webhooks when all are configured
- `Translation:Provider`: `none` (default) or `libretranslate`
- `Translation:LibreTranslate:BaseUrl`: base URL for LibreTranslate (only if provider enabled)
- `Translation:LibreTranslate:ApiKey`: optional API key for LibreTranslate
- `App:PublicBaseUrl`: the single external origin used for email links, OAuth callbacks, billing redirects, secure cookies, and production Host validation (Production requires HTTPS; local development defaults to `http://localhost:3000`)
- `Auth:MicrosoftTenant`: Microsoft application sign-in policy (`common`, `organizations`, `consumers`, or one tenant GUID); required in Production when Microsoft sign-in is enabled and distinct from Graph's `Microsoft:TenantId`
- `Email:Enabled`: enable SMTP sending (`true`/`false`)
- `Email:SmtpHost`: SMTP host (for Gmail: `smtp.gmail.com`)
- `Email:SmtpPort`: SMTP port (for Gmail: `587`)
- `Email:SmtpUser`: SMTP username (often your Gmail address)
- `Email:SmtpPassword`: SMTP password (for Gmail: use an App Password)
- `Email:From`: from address (default: `Email:SmtpUser`)
- `Email:FromName`: from name (default: `Jobbjakt`)
- `Email:FollowUpReminders:Enabled`: enable scheduled follow-up reminder emails
- `Email:FollowUpReminders:UpcomingDays`: how far ahead reminder emails look for upcoming follow-up dates (default `2`)
### UI settings
- `NEXT_PUBLIC_API_BASE_URL`: override the API base URL (example: `http://localhost:5202/api`)
The complete environment template is `.env.example`. Production requirements and restore/rollback procedures are maintained in `deploy/README.md`; current component boundaries and supported providers are documented in `docs/architecture/current.md`.
## API endpoint reference
Base URL in local dev: `http://localhost:5202` (all routes are under `/api/...`).
Authentication:
- If `Auth:GoogleClientId` is configured, most endpoints require `Authorization: Bearer <google_id_token>`.
- If it’s not configured, endpoints are effectively anonymous.
### Job applications (`/api/jobapplications`)
- `GET /api/jobapplications`
- Query: `page`, `pageSize` (15/20/25), `q`, `status`, `companyId`, `location`, `needsFollowUp`, `includeDeleted`, `deletedOnly`, `sortBy`, `sortDir`
- Returns a paged list of `JobApplicationDto` (includes computed follow-up flags and short summary).
- `GET /api/jobapplications/{id}`
- Returns a single `JobApplicationDto` (includes computed follow-up flags and a “full” summary on demand).
- `GET /api/jobapplications/board?includeDeleted=false`
- Returns all job applications (intended for a board/overview view).
- `GET /api/jobapplications/reminders?upcomingDays=7`
- Returns jobs that need follow-up / are in key statuses and have upcoming follow-up dates.
- `POST /api/jobapplications`
- Body: `CreateJobApplicationRequest` (job title, company id, status, notes/description, follow-up fields, tags, attachments flags, etc.)
- Creates a job application and a `JobEvent` of type `Created`.
- `PUT /api/jobapplications/{id}`
- Body: `UpdateJobApplicationRequest`
- Updates an application; records a `StatusChanged` event if the status changed.
- `PATCH /api/jobapplications/{id}/status`
- Body: `{ "status": "..." }`
- Updates only status; records `StatusChanged` if it changed. The status is normalized against the canonical pipeline (casing + known synonyms like `Interviewing`→`Interview`); unrecognized values are preserved as custom statuses.
- `GET /api/jobapplications/pipeline`
- Returns the canonical ordered pipeline stages (`Applied, Waiting, Interview, Offer, Rejected, Ghosted`) with display order and category (`Active`/`Success`/`Closed`). The UI renders board columns and status dropdowns from this single source of truth.
- `PATCH /api/jobapplications/{id}/followup`
- Body: `{ "followUpAt": "2026-03-13T12:00:00Z" }` (or `null`)
- Sets/clears follow-up date; records a `FollowUpSet` event.
- `GET /api/jobapplications/{id}/history`
- Returns `JobEvent` history for the job.
- `GET /api/jobapplications/{id}/timeline`
- Returns a unified timeline combining job events, correspondence, and attachments.
- `GET /api/jobapplications/stats`
- Returns totals, counts by status, applied-last-30-days, and average days since applied.
- `GET /api/jobapplications/{id}/match-score`
- Deterministic CV↔job keyword-coverage score (0–100) with matched/missing keywords and per-CV-section coverage. Missing skills also seed a checklist-backed job-specific learning path, so users can mark them learned or dismiss them. No AI calls: results are instant and reproducible. Requires profile CV text/structure and a job description. (The AI narrative equivalent is `GET /api/jobapplications/{id}/candidate-fit`.)
- `GET /api/jobapplications/{id}/status-suggestion`
- Deterministic status suggestion derived from the job's most recent inbound message (interview invite / offer / rejection). Returns a forward-only suggestion (`hasSuggestion`, `suggestedStatus`, `signal`, …) or `hasSuggestion: false`. Applying it is a normal `PATCH .../status` — always user-confirmed.
- `DELETE /api/jobapplications/{id}`
- Soft-deletes an application (`IsDeleted=true`); records a `Deleted` event.
- `POST /api/jobapplications/{id}/restore`
- Restores a soft-deleted application; records a `Restored` event.
### Companies (`/api/companies`)
- `GET /api/companies`: list companies
- `GET /api/companies/{id}`: company by id
- `POST /api/companies`: create (idempotent by name; returns existing if already present)
- Body: `{ "name": "...", "location": "...?", "source": "...?" }`
- `PUT /api/companies/{id}`: update
- Body includes recruiter details and `pipelineStage`, `lastContactedAt`, `nextContactAt`
### Correspondence (`/api/correspondence`)
- `GET /api/correspondence/{jobId}`: list messages for a job (ordered by date)
- `POST /api/correspondence`: create message
- Body: `{ "jobApplicationId": 1, "from": "...", "content": "...", "subject": "...?", "channel": "...?", "date": "..."? }`
### Attachments (`/api/attachments`)
- `GET /api/attachments/{jobId}`: list attachments for a job
- `GET /api/attachments/download/{id}`: download an attachment by attachment id
- `POST /api/attachments`: upload files (multipart/form-data)
- Form fields: `jobId` (int), `files` (one or more)
- `PATCH /api/attachments/{id}`: rename attachment
- Body: `{ "fileName": "NewName.pdf" }`
- `DELETE /api/attachments/{id}`: delete attachment record + best-effort delete file on disk
### Rules (`/api/rules`)
- `GET /api/rules`: get rule settings (auto-creates defaults on first request)
- `PUT /api/rules`: update rule settings (values are clamped to sane bounds)
### Export (`/api/export`)
- `GET /api/export/jobs?format=json|csv&includeDeleted=false`
- Downloads a file (`job-tracker-export-YYYY-MM-DD.json` or `.csv`).
### Backup (`/api/backup`)
- `POST /api/backup/encrypted`
- Returns an encrypted backup file (`.jtbackup`).
- Note: only implemented on Windows in this build (uses ASP.NET Data Protection / DPAPI).
### Job import (`/api/jobimport`)
- `POST /api/jobimport/preview`
- Body: `{ "url": "https://..." }`
- Returns a parsed preview payload (`JobImportResult`), if a matching plugin can parse it.
### Client error reporting (`/api/client-errors`)
- `POST /api/client-errors`
- Body: `{ errorId?, message?, stack?, componentStack?, url?, userAgent?, at? }`
- Logs frontend errors into the API logs (best-effort).
### Authentication (`/api/auth`)
- `GET /api/auth/config`: returns auth feature flags for the UI
- `POST /api/auth/login`: local email/password login (returns a signed JWT)
- `POST /api/auth/register`: local registration (only if enabled via `Auth:AllowRegistration=true`)
- `GET /api/auth/me`: returns current user/claims summary for the UI
- `POST /api/auth/request-password-reset`: sends reset email (requires SMTP enabled)
- `POST /api/auth/reset-password`: resets password using emailed token
### Users (`/api/users`) (admin-only)
- `GET /api/users`: list users + roles
- `POST /api/users`: create a user (and optionally roles)
- `PUT /api/users/{id}/roles`: replace roles for a user
- `DELETE /api/users/{id}`: delete user
- `POST /api/users/{id}/send-password-reset`: send a reset email to the user
## Notes for contributors
- The API applies EF Core migrations on startup for the configured SQLite database.
- The background rules engine may automatically transition jobs to `Ghosted` based on rule settings.
- In Docker, the UI proxies `/api/*` to the backend service (see `job-tracker-ui/nginx.conf`).
## Project status and planned work
This README documents released behaviour, not the roadmap. Validated findings and implementation status live in `docs/audits/audit-remediation-backlog.md` and `docs/work-programmes/master-progress.md`; archived phase documents are historical evidence and may describe superseded behaviour.
Start at [docs/README.md](docs/README.md) for architecture, configuration, frontend/backend guides, AI, database, security, infrastructure, testing, and remaining work. Production/external blockers remain in [BLOCKERS.md](BLOCKERS.md).
+3 -348
View File
@@ -1,350 +1,5 @@
# Production deployment notes
# Deployment entry point
## Gitea Actions
This repo includes `.gitea/workflows/ci-deploy.yml` for:
- backend build
- backend tests
- frontend tests
- frontend production build
- deployment to Ubuntu after successful tests on `main`
The deployment script and Gitea workflow live in this directory and `.gitea/workflows/`. The current deployment, health/readiness, rollback, and secret-handling guidance is in the repository [infrastructure guide](../docs/infrastructure/README.md), especially [deployment](../docs/infrastructure/deployment.md) and [disaster recovery](../docs/infrastructure/disaster-recovery.md).
### Required secrets in Gitea
- `PROD_HOST`
- `PROD_USER`
- `PROD_SSH_KEY`
## Ubuntu server setup
Recommended app path:
- `/opt/job-tracker/app`
Persistent runtime secrets path:
- `/opt/job-tracker/shared/.env`
Requirements:
- Docker Engine
- Docker Compose plugin
- reverse proxy in front (Nginx, Caddy, or Traefik)
- shared env file present on server in `/opt/job-tracker/shared/.env`
- network connectivity from the backend container to your `mariadb` container/service
The deploy script will automatically create a symlink from:
- `/opt/job-tracker/shared/.env`
to:
- `/opt/job-tracker/app/.env`
This keeps secrets outside the uploaded repo checkout so they are not wiped by CI deploys.
### Frontend API base URL
The production frontend already proxies `/api` to the backend container via Nginx.
Recommended default:
- leave `NEXT_PUBLIC_API_BASE_URL` unset/empty in production
Only set `NEXT_PUBLIC_API_BASE_URL` if the UI must call a different external API origin on purpose.
## Example production `.env`
```env
DATABASE_PROVIDER=mariadb
JOBTRACKER_CONNECTION_STRING=server=mariadb;port=3306;database=jobtracker;user=jobtracker;password=REPLACE_ME
AUTH_JWT_KEY=replace_with_long_random_secret
AUTH_ADMIN_EMAIL=you@example.com
AUTH_ADMIN_PASSWORD=replace_with_strong_password
AUTH_REQUIRE_EMAIL_VERIFICATION=true
APP_PUBLIC_BASE_URL=https://your-domain.example
WEB_PROXY_SUBNET=172.31.250.0/29
STRIPE_SECRET_KEY=sk_live_...
STRIPE_PRICE_PREMIUM=price_...
STRIPE_WEBHOOK_SECRET=whsec_...
AI_SERVICE_BASE_URL=http://ai-service:8001
OLLAMA_BASE_URL=http://ollama:11434
OLLAMA_MODEL=qwen2.5:7b
EMAIL_FOLLOWUPREMINDERS_ENABLED=true
EMAIL_FOLLOWUPREMINDERS_UPCOMINGDAYS=2
# Optional backward-compatible alias if older config still references the previous name:
SUMMARIZER_BASE_URL=http://ai-service:8001
```
## Supported databases
The application supports only these provider values:
- `sqlite` — local development and small single-instance deployments; persist and back up the data volume.
- `mariadb` or `mysql` — server deployments through the Pomelo EF Core provider. The production example uses MariaDB.
PostgreSQL is not implemented. Do not configure or recommend it without first adding an EF Core provider, migrations, startup validation, backup/restore support, and a provider test matrix.
## Deployment flow
Production automation always selects `docker-compose.yml` explicitly. Local development must add
`docker-compose.dev.yml`; never add that file to a production command. The effective base configuration
publishes **frontend 3000 → 80** on all host interfaces for the recorded Traefik host-port dependency.
Backend, ai-service and bundled Ollama have no host-port bindings. CORS and Docker service names are
not network isolation. The backend also joins the operator-managed `jobtracker_shared` network for
existing service dependencies; its membership and other host services need operator inspection.
The external Traefik configuration is operator-owned and is not stored here. Before deployment it
must route only the exact host from `APP_PUBLIC_BASE_URL` to the recorded frontend host port 3000,
terminate TLS, and replace `X-Forwarded-For` with one client IP and `X-Forwarded-Proto=https`.
Do not migrate to container routing or bind port 3000 to loopback without verifying where Traefik runs.
Host firewall/Docker forwarding rules must restrict ingress appropriately; their existence is unproven.
Nginx requires `INGRESS_PROXY_CIDRS` (space-separated observed Traefik peer CIDRs, preferably /32 or
/128), rejects other peers before serving application routes, rejects missing scheme/IP and forwarding
chains, and emits normalized forwarding metadata. It rejects non-canonical Hosts except the default
server's liveness-only `/health`. The backend trusts only `WEB_PROXY_SUBNET` and one hop. Never trust a
NAT/gateway address shared by both Traefik and untrusted callers. If peer identity cannot distinguish
them, stop rollout and establish a verified host binding/firewall or dedicated proxy route first.
Deployment requires Python 3 (standard library) and Docker Compose's JSON config support. The new
`scripts/validate-ingress.py` preflight checks the **rendered** production configuration before normal
deployment backup/build/replacement; backup-only runs remain independent of the new ingress setting.
CI runs `python3 scripts/test-ingress.py` without needing a Docker engine. Empty/missing peer values
fail safely; do not copy synthetic addresses from tests into production. Runtime/container verification
is still pending locally due to Docker Desktop failure. See the [B06 report and operator checks](../docs/audits/2026-09-07/validation/b06-completion.md)
before deploying this change. No production verification is implied by configuration tests.
1. push to `main`
2. Gitea Actions runs tests
3. if green, workflow uploads repo to server
4. `deploy/deploy.sh` links `/opt/job-tracker/shared/.env` into the repo checkout, then explicitly runs `docker compose -f docker-compose.yml build` and `up -d`
5. if `OLLAMA_MODEL` is set, the deploy script waits for Ollama, pulls the configured model if missing, then restarts `ai-service` so hybrid CV classification can use it
6. workflow checks service status after deployment
## Post-deploy verification you should also do manually the first time
- confirm reverse proxy routes to the frontend correctly
- confirm API auth/login works with production config
- confirm backend can connect to MariaDB
- confirm AI service container is reachable from backend
- confirm reminder and admin/system pages load
- verify follow-up reminder emails are enabled only when intended and that links open the correct job/tab
---
# Backups, restore and rollback
**Database restore and application rollback are two different operations.** A bad deploy usually needs
only the rollback. Restore the database only if the data itself is wrong or lost — it discards
everything written since the dump.
## Environment loading
`deploy/deploy.sh` **loads `/opt/job-tracker/shared/.env` into its own shell** before it decides
anything. The symlink it creates in the checkout is for docker compose, which reads `.env` itself;
the script needs the values too, to pick the right backup and to check its own configuration.
- Parsed line by line, not `source`d — a compose `.env` is not a shell script, so an unquoted value
containing spaces would execute as a command.
- **Variables already set in the environment win**, so CI-provided `APP_VERSION`, `APP_COMMIT_SHA`
and `APP_BUILD_STAMP` still override the file.
- No value is ever echoed. Error messages name variables, never their contents.
This was added on 2026-07-19. Before it, the script read an empty environment: `DATABASE_PROVIDER`
fell back to `sqlite` on a MariaDB host, so the deploy tarred the data volume, printed
`Backup verified`, and continued with no database dump at all. See
`docs/release-candidate-review.md` (B1).
## Required production variables
`validate_deploy_config` runs **before** the backup, and therefore before anything is built, stopped
or replaced. A missing variable aborts the deploy while the running stack is still untouched.
| Variable | Required | Why it is checked here |
|---|---|---|
| `DATABASE_PROVIDER` | **Always** — no default | Selects the backup. Guessing it wrong backs up the wrong database and reports success |
| `JOBTRACKER_CONNECTION_STRING` | When provider is `mariadb`/`mysql` | Without it there is no way to dump the database |
| `AI_SERVICE_TOKEN` | Always | `docker-compose.yml` declares it with `:?`; missing it kills the stack *after* the images are built |
| `AUTH_JWT_KEY` | Always | Compose sets `Auth__Require=true`, and the backend throws at startup on a blank key — after the containers have been replaced |
| `APP_PUBLIC_BASE_URL` | Always | Canonical HTTPS origin for links, OAuth callbacks, billing redirects, secure cookies, Host validation, and the public smoke check |
| `AUTH_MICROSOFT_TENANT` | When `AUTH_MICROSOFT_CLIENT_ID` is set | Exact Microsoft application sign-in account mode; distinct from the Graph mailbox tenant |
| `WEB_PROXY_SUBNET` | Always | Dedicated nginx-to-backend CIDR trusted for exactly one forwarded hop; must not overlap another Docker network |
### Microsoft sign-in migration gate
Before enabling `AUTH_MICROSOFT_CLIENT_ID` with the canonical identity release, take the normal
backup and record counts only. Do not print subjects or email addresses:
```sql
SELECT COUNT(*) AS legacy_links
FROM AspNetUsers
WHERE MicrosoftSubject IS NOT NULL OR MicrosoftEmail IS NOT NULL;
SELECT COUNT(*) AS legacy_without_alternate_credential
FROM AspNetUsers
WHERE (MicrosoftSubject IS NOT NULL OR MicrosoftEmail IS NOT NULL)
AND PasswordHash IS NULL
AND GoogleSubject IS NULL;
SELECT COUNT(*) AS duplicate_legacy_subject_groups
FROM (
SELECT MicrosoftSubject
FROM AspNetUsers
WHERE MicrosoftSubject IS NOT NULL
GROUP BY MicrosoftSubject HAVING COUNT(*) > 1
) duplicate_subjects;
SELECT COUNT(*) AS duplicate_legacy_email_groups
FROM (
SELECT MicrosoftEmail
FROM AspNetUsers
WHERE MicrosoftEmail IS NOT NULL
GROUP BY MicrosoftEmail HAVING COUNT(*) > 1
) duplicate_emails;
```
Apply `20260802212509_AddCanonicalMicrosoftIdentity` before deploying code that queries the two new
columns. The migration does not backfill legacy rows and adds a unique nullable composite index.
Keep Microsoft sign-in disabled if the inventory or migration fails. Roll back the application
binary while leaving the additive columns in place; never roll back to email auto-linking.
`DATABASE_PROVIDER` deliberately has **no default**. An unset value used to mean "sqlite"; it now
means "stop and tell me".
## Backup creation
`deploy/deploy.sh` takes a backup **before** it builds, stops or replaces anything, and **aborts the
deploy if the backup fails**. Nothing else in the deploy runs without a restore point.
- **Location:** `/opt/job-tracker/backups` — override with `BACKUP_DIR`. If an earlier root-run deploy owns the directory, the deploy script repairs its ownership through Docker before writing.
- **Selection:** driven solely by `DATABASE_PROVIDER`, which must be set.
- `mariadb` / `mysql` → SQL dump, `jobtracker-<database>-<UTC timestamp>.sql.gz`, e.g.
`jobtracker-jobtracker-20260719T153759Z.sql.gz`
- MariaDB deployments also archive `jobtracker_data`, which contains attachments, imported CVs,
generated exports and Data Protection keys.
- `sqlite` → data volume archive, `jobtracker-sqlite-<UTC timestamp>.tar.gz`
- every deployment archives the separate account-deletion tombstone volume when it exists;
restore it independently and never replace it with an older application-data snapshot
- anything else → the deploy stops
- **The filename tells you which path ran.** If you expect a MariaDB deploy and find a
`jobtracker-sqlite-*.tar.gz`, the environment is wrong — that is the exact failure this check exists
to make visible.
- **Naming:** the UTC timestamp makes every file unique, so a deploy never overwrites an earlier backup.
- **Credentials** come from `JOBTRACKER_CONNECTION_STRING` and are passed via `MYSQL_PWD`, never on the
command line, so they cannot appear in the process list or the deploy log.
- **Compression:** gzip. A small database compresses to a few KB.
- **SQLite volume resolution:** compose prefixes volume names with the project name, so the script
resolves `<project>_jobtracker_data` and **fails if that volume does not exist**. Naming the bare
volume would silently create an empty one and back *that* up.
### Verification — each provider gets the check that proves its own format
A backup that exists but is empty, truncated, or the wrong *kind* is worse than none, because it looks
like a restore point.
| Provider | Checks |
|---|---|
| MariaDB | non-empty; valid gzip; contains `CREATE TABLE`; contains the `Dump completed` trailer that `mariadb-dump` writes last, so a dump that died partway through is rejected |
| SQLite | non-empty; valid gzip; the archive actually contains `jobtracker.db` |
| Recovery volumes | non-empty valid gzip archives; a missing required application-data volume aborts the deploy |
A failed check deletes the file rather than leaving something that looks like a backup.
### Taking one by hand
```bash
BACKUP_DIR=/opt/job-tracker/backups
mkdir -p "$BACKUP_DIR"
MYSQL_PWD='<password>' mariadb-dump \
--host=127.0.0.1 --port=3306 --user=<user> \
--single-transaction --routines --events --quick \
jobtracker | gzip -c > "$BACKUP_DIR/jobtracker-manual-$(date -u +%Y%m%dT%H%M%SZ).sql.gz"
```
### Retention
Verified top-level recovery sets are retained for 30 days and the first complete set in each month is
copied into `monthly/`; the newest 12 monthly sets are retained. Configure
`BACKUP_DAILY_RETENTION_DAYS` and `BACKUP_MONTHLY_RETENTION_COUNT` to change those positive integer
limits. Paths outside the fixed backup directory and monthly naming contract are never pruned.
Set `BACKUP_RCLONE_REMOTE` to an encrypted rclone remote to copy each current recovery set off-host.
See [`docs/operations/google-drive-backups.md`](../docs/operations/google-drive-backups.md).
## Restoring the database
Tested end-to-end against a clean MariaDB 11 container: dump taken from a seeded database, restored
into an empty one, rows verified identical.
```bash
# 1. Stop the application so nothing writes during the restore.
docker compose stop backend
# 2. Restore. This REPLACES the current contents of the named database.
gzip -dc /opt/job-tracker/backups/jobtracker-jobtracker-20260719T153759Z.sql.gz \
| MYSQL_PWD='<password>' mariadb --host=127.0.0.1 --port=3306 --user=<user> jobtracker
# 3. Verify before starting anything.
MYSQL_PWD='<password>' mariadb --host=127.0.0.1 --port=3306 --user=<user> jobtracker \
-e "SELECT COUNT(*) AS applications FROM JobApplications;"
# 4. Start again.
docker compose start backend
```
Restoring a **SQLite** deployment instead:
```bash
docker compose stop backend
docker run --rm -v jobtracker_data:/data -v /opt/job-tracker/backups:/backup \
-e ARCHIVE_NAME=jobtracker-sqlite-20260719T153941Z.tar.gz \
alpine:3 sh -c 'rm -rf /data/* && tar xzf "/backup/$ARCHIVE_NAME" -C /data'
docker compose start backend
```
## Restoring application containers (rollback)
This is the usual fix for a bad deploy, and it **does not touch the database**.
Normal deployments tag backend and frontend images with `APP_COMMIT_SHA` and retain the images from
the currently running containers as `jobtracker-backend:previous` and `jobtracker-frontend:previous`.
If any step fails after core replacement—including public liveness, readiness, or auth-configuration
checks—`deploy.sh` automatically recreates both core services from those retained images and still
returns a failed status so CI reports the rejected release. The previous images are replaced only at
the start of the next deployment.
Use the manual procedure below if automatic restoration itself fails or an older release is required.
```bash
cd /opt/job-tracker/app # the deployment checkout
git log --oneline -5 # find the last good commit
git checkout <previous-commit>
deploy/deploy.sh
```
`deploy.sh` takes a fresh backup first, so rolling back is itself protected.
**Why a code rollback is safe here:** every Phase 4/5 migration is a no-op — the startup reconciler
owns those tables — so reverting the code never leaves migration history ahead of the schema. The
reconciler is additive and never drops a table holding rows, so the older code simply ignores the
newer tables.
**What a code rollback does not undo:** rows users created in the newer tables stay. That is usually
what you want. If you additionally restore the database, those rows are lost — so restore only when
the data is the problem.
## Choosing between them
| Symptom | Action |
|---|---|
| New version starts but behaves wrong | Rollback the code. Leave the database. |
| Backend will not start; schema looks wrong | Rollback the code, then restore only if it still fails. |
| Data is missing or corrupted | Restore the database from the most recent good dump. |
| Deploy aborted before starting | Nothing to undo — the backup ran before any change. |
## Health checks
`backend` and `frontend` both report container health, so `docker compose ps` shows real state rather
than merely "running".
- **Backend:** `curl -fsS http://127.0.0.1:8080/health`. Anonymous, and deliberately **does not touch
the database** — a health check that queried MariaDB would restart a healthy backend whenever the
database blipped. `start_period` is 90s to cover first-boot schema reconciliation.
- **API/database readiness:** `curl -fsS https://<host>/ready`. This checks that the API can reach its
configured database and returns only `ready` or `unavailable`; detailed diagnostics remain admin-only.
Deployment validation requires this probe, while Docker restart policy continues to use liveness.
- **Frontend:** `wget` against nginx on port 80.
- `frontend` waits for `backend` to be *healthy*, not merely started, because nginx proxies `/api` to
it and refuses to start if the upstream cannot be resolved.
A backend that cannot reach its database during startup exits and is reported `unhealthy`. A dependency
failure after startup leaves the process alive but makes `/ready` return 503, so deployment validation
cannot mistake a live-but-unready stack for a successful release.
Keep the server `.env` outside the repository. Do not run production deployment, restore, migration, or credential rotation as part of local documentation work.
+1 -1
View File
@@ -26,7 +26,7 @@ fi
# own decisions — which backup to take, which variables are missing — ran against
# an empty environment until this existed, so DATABASE_PROVIDER defaulted to
# sqlite and a MariaDB host silently got a volume tar instead of a dump, with a
# "Backup verified" line to match. See docs/release-candidate-review.md (B1).
# "Backup verified" line to match. See docs/infrastructure/deployment.md.
#
# Parsed line by line rather than sourced: a compose .env is not a shell script,
# so an unquoted value containing spaces would execute as a command under `.`.
+35
View File
@@ -0,0 +1,35 @@
# Job Tracker documentation
This is the entry point for the current system documentation. Job Tracker (also branded Jobjakt in some UI and deployment text) is a self-hosted job-application workspace: it combines job tracking, company context, correspondence, documents, follow-up automation, and privacy-conscious AI assistance.
## System at a glance
- `job-tracker-ui/` is a statically exported Next.js 16 shell containing the React 19, TypeScript, MUI, and React Router application.
- `JobTrackerApi/` is an ASP.NET Core 9 API using EF Core 9, ASP.NET Identity, JWT/cookie authentication, SQLite by default, and MariaDB/MySQL as the server option.
- `tools/summarizer/` is a FastAPI service for local summarisation and bounded document extraction/OCR. CV generation can route through local Ollama or an explicitly enabled Gemini/Groq provider.
- Docker Compose runs the API, nginx frontend, AI service, and optional bundled Ollama. Gitea Actions builds and deploys the stack.
## Documentation map
| Area | Start here |
|---|---|
| Architecture | [Architecture overview](architecture/overview.md) |
| Frontend and UI | [Frontend guide](frontend/README.md) |
| Backend and API | [Backend guide](backend/README.md) · [API reference](backend/api.md) |
| AI | [AI guide](ai/README.md) |
| Database | [Database guide](database/README.md) |
| Authentication and security | [Security guide](security/README.md) |
| Configuration | [Configuration guide](configuration/README.md) · [Environment variables](configuration/environment-variables.md) |
| Docker and deployment | [Infrastructure guide](infrastructure/README.md) |
| Tests and quality gates | [Testing guide](testing/README.md) |
| Local development | [Development guide](development/README.md) |
| Product behavior | [Product guide](product/README.md) |
| Remaining work | [Roadmap](roadmap/README.md) |
| Decisions | [Architecture decisions](decisions/README.md) |
| Documentation maintenance | [Documentation rules](documentation-rules.md) |
## Authority and archive
The current source code and the documents linked above are authoritative. `BLOCKERS.md` remains the short operator/external-dependency handoff because repository instructions refer to it directly. Historical audits, plans, reports, and the pre-cleanup snapshot are under [`_archive/`](./_archive/); they are evidence, not current behavior.
When documentation and code disagree, code wins and the relevant document should be corrected in the same change.
@@ -0,0 +1,43 @@
# Blockers
Updated: 2026-09-01
## Stripe billing
- **Blocked:** Activating roadmap item 7.5 in production.
- **Why:** Hosted Checkout, customer-portal sessions, signed subscription webhooks, persisted billing state, Pro-role provisioning and a mock lifecycle regression are implemented. The Stripe recurring price, portal, webhook registration and production credentials must be configured outside the repository.
- **Required:** Configure the Pro recurring price, enable the Stripe customer portal, register `/api/billing/webhook` for `customer.subscription.created`, `customer.subscription.updated`, and `customer.subscription.deleted`, then supply `STRIPE_SECRET_KEY`, the recurring `price_...` value in the legacy-named `STRIPE_PRICE_PREMIUM` setting, and `STRIPE_WEBHOOK_SECRET` through the deployment environment. Do not place secret values in source control or chat.
- **Recommended:** One monthly Pro price first; add annual billing only after the monthly flow is operating.
- **Current access check:** Production has test-mode secret and webhook values, but `STRIPE_PRICE_PREMIUM` currently contains a `prod_...` Product ID. Checkout requires the recurring `price_...` Price ID; the server now treats the wrong identifier type as disabled rather than calling Stripe. The publishable key is not used by hosted Checkout. Local fake-gateway coverage proves active → expired → canceled/replayed role transitions without losing non-AI data (V-185).
- **Runbook:** Follow `docs/operations/stripe-activation.md`, completing test mode before creating or installing live-mode values.
## Public registration verification
- **Blocked:** Completing a real-browser production signup check.
- **Why:** The 2026-07-31 anonymous production check confirms `allowRegistration=true`, `turnstileEnabled=true`, and Google sign-in enabled. Completing Turnstile and creating a disposable account requires an interactive production browser session.
- **Required:** Register one disposable account through Turnstile, verify email/sign-in/rate-limit behavior, then remove the account if it is not needed.
- **Recommended:** Monitor Turnstile and rate-limit failures during the first public rollout; keep email verification required.
- **Current status:** Production returns `allowRegistration=true`, `turnstileEnabled=true`, `googleEnabled=true`, and `microsoftEnabled=false`. A registration request without a Turnstile token is rejected with HTTP 400. SMTP is configured and enabled. The operator reports `AUTH_REQUIRE_EMAIL_VERIFICATION` is now enabled; the disposable interactive signup is still required to prove the deployed behavior end to end.
## Production verification
- **Blocked:** Authenticated production smoke tests, backup restore verification against real data, OAuth-provider checks, and rollback rehearsal.
- **Why:** The authorised deployment is complete, but authenticated smoke, port/network closure, complete backup restore, provider checks and failure-injection rollback require credentials and/or separately authorised production mutations.
- **Required:** Confirm the admin-only version badge matches version 286 / commit `ea7d9f3`, then run the authenticated application workspace, Career, CV, attachment, email-verification and rollback checks from `docs/release-candidate-review.md` and `docs/release-checklist.md`.
- **Recommended:** Verify backup/restore before deployment, then exercise login, existing application counts, Career Workspace, public CV refresh/download, AI, and attachments in order.
- **Current status:** Gitea run 705 (application version 286, commit `ea7d9f3`) completed both the complete test job and remote deployment successfully on 2026-09-01. Public `/health`, `/ready`, and `/api/auth/config` returned HTTP 200 after replacement; liveness reported version 286 and readiness reported `ready`. This proves the proxy/frontend, API and database readiness path, but not authenticated application behavior, provider delivery or restore/rollback behavior. Read-only PROD-001 inventory previously found the JobTracker Ollama and frontend published on all host interfaces, the newest observed gzip-valid MariaDB backup dated 2026-08-02, no observed scheduled JobTracker backup, and no owner-file/key/tombstone recovery bundle. Those remaining gates still require operator-controlled verification; see `docs/production/production-ai-hardware-assessment.md`.
- **Repository-side rollback status:** Core images are now tagged by commit, the prior running backend/frontend images are retained, and any failure after replacement triggers an automatic core rollback while preserving the failed CI status. Compose configuration validates locally. A real rollback rehearsal remains blocked because the local Linux Docker daemon is unavailable and production mutation is not authorised; run the failure-injection rehearsal on a disposable Linux Docker host before relying on it in production.
## Account deletion retention and restore policy
- **Blocked:** Enabling SEC-009 self-service deletion in production and declaring backup erasure complete.
- **Why:** The readable export and idempotent live-data deletion coordinator are implemented behind an explicit disabled gate. Repository code cannot truthfully choose legal retention periods, backup expiry, provider obligations or the tombstone lifetime needed to prevent restoration from resurrecting an erased account. Production inventory confirms current backups are database-only and no protected tombstone volume is deployed yet; the release branch now defines the separate volume and retryable authenticated sidecar-cache purge.
- **Required:** Decide retention periods for operational backups, audit/security records and deletion tombstones; identify any legal hold/export obligations; approve the restore behavior for deleted identities.
- **Recommended:** Keep production self-service and admin deletion disabled. Decide retention, deploy/protect the configured tombstone volume, build a complete DB/files/keys backup set, then use a disposable account to prove remote-provider cleanup, sidecar purge across restart and restored-backup tombstone replay before staged activation.
## Legacy job/application column cutover
- **Blocked:** Removing the opportunity columns duplicated between `JobApplication` and `Job`.
- **Why:** The compatibility dual-write protects existing production rows and older clients. The release branch now backfills missing opportunities on startup, synchronizes both creation paths, and leaves all legacy columns intact. Dropping columns still requires production validation and an observation release.
- **Required:** After deployment, run the read-only report in `docs/operations/job-opportunity-cutover.md` against production and a restored backup, then confirm that backward API compatibility is no longer required.
- **Recommended:** Use an expand/contract release: first stop legacy reads after a verified backfill, observe one release, then drop the duplicate columns in the following migration.
@@ -0,0 +1,29 @@
# Documentation cleanup inventory
Inventory date: 2026-09-11. Scope: active repository Markdown/text documentation, README files, root notes, audit/report material, and configuration documentation. The pre-existing `docs/_archive/` was treated as historical and was not re-audited as active documentation.
## Counts
| Scope | Count | Disposition |
|---|---:|---|
| Active `docs/` Markdown/text files before cleanup | 454 | Classified below; current material consolidated, dated material preserved in this backup or the existing archive |
| Existing historical archive Markdown/text files | 127 | Retained in place |
| Root/component documentation and notes copied into backup | included | Root clutter consolidated or removed; component READMEs rewritten where they are useful entry points |
| Active Markdown/text entries removed after verification | 269 | Superseded, duplicated, temporary, or merged into current guides |
## Classification policy
| Classification | Representative paths | Result |
|---|---|---|
| KEEP | `docs/redesign/**` current UI work-in-progress; `docs/decisions/ADR-*.md`; `BLOCKERS.md` | Retained because these are active implementation evidence, durable decisions, or operator handoff material |
| MERGE/REWRITE | `docs/architecture/**`, `docs/ai/**`, `docs/security/**`, `docs/technical/**`, `docs/ux/**`, `docs/career/**`, `docs/jobs/**`, root `README.md` | Current facts moved into focused guides; duplicates and stubs removed |
| ARCHIVE | `docs/audits/**`, `docs/verification/**`, release reports, research, phase reports, work-programme logs | Human-readable originals preserved under this pre-cleanup snapshot; generated audit artifacts were not copied into this backup |
| DELETE AFTER BACKUP | root TODO/progress/portfolio notes, duplicate topic stubs, completed status diaries, obsolete CRA-era setup notes | Useful TODOs, decisions, security items, and operational gates were migrated to current roadmap/security/infrastructure docs |
## Preservation checks
- Open P4-02, P2-D01, P3-O01/P3-O02, B06/F04, CV Builder follow-up, Stripe, production verification, account deletion, and schema cutover work is represented in `docs/roadmap/` or `BLOCKERS.md`.
- Current technical debt is represented in `docs/roadmap/technical-debt.md`.
- Current security implementation and unverified gates are represented in `docs/security/security-notes.md`.
- Environment variable names and local-only/obsolete names are represented in `docs/configuration/environment-variables.md`.
- This inventory and the original files are evidence of the pre-cleanup state; they are not authoritative current documentation.
@@ -0,0 +1,7 @@
# Pre-documentation cleanup backup
This directory preserves documentation from before the repository-wide documentation consolidation on 2026-09-11.
It contains active documentation and documentation-like root files that were inventoried before being rewritten, merged, moved, or removed. The archive exists to make the cleanup recoverable and to preserve historical TODOs, decisions, audit evidence, deployment notes, and other context.
Use the current documentation tree under `docs/` as the authoritative source. This backup is a snapshot of the pre-cleanup state and must not be treated as current system documentation.
@@ -0,0 +1,20 @@
# Jobjakt Quick Capture
The Manifest V3 extension opens the current page or selected link in Jobjakt's existing reviewed import flow. It does not scrape page content, store credentials, or submit an application automatically.
## Install locally
1. Open `chrome://extensions` (or the equivalent extensions page in Edge).
2. Enable developer mode.
3. Choose **Load unpacked** and select this `browser-extension` folder.
4. Open a vacancy and click the Jobjakt extension, or use **Save vacancy to Jobjakt** from the page/link context menu.
The Jobjakt add-job dialog opens with the vacancy URL and runs the same server-side preview, validation, duplicate checking, and user review used by normal URL imports.
## Test
```sh
node --test browser-extension/tests/*.test.mjs
```
No build or package installation is required. Zip the folder contents for store submission after adding store artwork and completing the browser-store privacy declarations.
@@ -0,0 +1,350 @@
# Production deployment notes
## Gitea Actions
This repo includes `.gitea/workflows/ci-deploy.yml` for:
- backend build
- backend tests
- frontend tests
- frontend production build
- deployment to Ubuntu after successful tests on `main`
### Required secrets in Gitea
- `PROD_HOST`
- `PROD_USER`
- `PROD_SSH_KEY`
## Ubuntu server setup
Recommended app path:
- `/opt/job-tracker/app`
Persistent runtime secrets path:
- `/opt/job-tracker/shared/.env`
Requirements:
- Docker Engine
- Docker Compose plugin
- reverse proxy in front (Nginx, Caddy, or Traefik)
- shared env file present on server in `/opt/job-tracker/shared/.env`
- network connectivity from the backend container to your `mariadb` container/service
The deploy script will automatically create a symlink from:
- `/opt/job-tracker/shared/.env`
to:
- `/opt/job-tracker/app/.env`
This keeps secrets outside the uploaded repo checkout so they are not wiped by CI deploys.
### Frontend API base URL
The production frontend already proxies `/api` to the backend container via Nginx.
Recommended default:
- leave `NEXT_PUBLIC_API_BASE_URL` unset/empty in production
Only set `NEXT_PUBLIC_API_BASE_URL` if the UI must call a different external API origin on purpose.
## Example production `.env`
```env
DATABASE_PROVIDER=mariadb
JOBTRACKER_CONNECTION_STRING=server=mariadb;port=3306;database=jobtracker;user=jobtracker;password=REPLACE_ME
AUTH_JWT_KEY=replace_with_long_random_secret
AUTH_ADMIN_EMAIL=you@example.com
AUTH_ADMIN_PASSWORD=replace_with_strong_password
AUTH_REQUIRE_EMAIL_VERIFICATION=true
APP_PUBLIC_BASE_URL=https://your-domain.example
WEB_PROXY_SUBNET=172.31.250.0/29
STRIPE_SECRET_KEY=sk_live_...
STRIPE_PRICE_PREMIUM=price_...
STRIPE_WEBHOOK_SECRET=whsec_...
AI_SERVICE_BASE_URL=http://ai-service:8001
OLLAMA_BASE_URL=http://ollama:11434
OLLAMA_MODEL=qwen2.5:7b
EMAIL_FOLLOWUPREMINDERS_ENABLED=true
EMAIL_FOLLOWUPREMINDERS_UPCOMINGDAYS=2
# Optional backward-compatible alias if older config still references the previous name:
SUMMARIZER_BASE_URL=http://ai-service:8001
```
## Supported databases
The application supports only these provider values:
- `sqlite` — local development and small single-instance deployments; persist and back up the data volume.
- `mariadb` or `mysql` — server deployments through the Pomelo EF Core provider. The production example uses MariaDB.
PostgreSQL is not implemented. Do not configure or recommend it without first adding an EF Core provider, migrations, startup validation, backup/restore support, and a provider test matrix.
## Deployment flow
Production automation always selects `docker-compose.yml` explicitly. Local development must add
`docker-compose.dev.yml`; never add that file to a production command. The effective base configuration
publishes **frontend 3000 → 80** on all host interfaces for the recorded Traefik host-port dependency.
Backend, ai-service and bundled Ollama have no host-port bindings. CORS and Docker service names are
not network isolation. The backend also joins the operator-managed `jobtracker_shared` network for
existing service dependencies; its membership and other host services need operator inspection.
The external Traefik configuration is operator-owned and is not stored here. Before deployment it
must route only the exact host from `APP_PUBLIC_BASE_URL` to the recorded frontend host port 3000,
terminate TLS, and replace `X-Forwarded-For` with one client IP and `X-Forwarded-Proto=https`.
Do not migrate to container routing or bind port 3000 to loopback without verifying where Traefik runs.
Host firewall/Docker forwarding rules must restrict ingress appropriately; their existence is unproven.
Nginx requires `INGRESS_PROXY_CIDRS` (space-separated observed Traefik peer CIDRs, preferably /32 or
/128), rejects other peers before serving application routes, rejects missing scheme/IP and forwarding
chains, and emits normalized forwarding metadata. It rejects non-canonical Hosts except the default
server's liveness-only `/health`. The backend trusts only `WEB_PROXY_SUBNET` and one hop. Never trust a
NAT/gateway address shared by both Traefik and untrusted callers. If peer identity cannot distinguish
them, stop rollout and establish a verified host binding/firewall or dedicated proxy route first.
Deployment requires Python 3 (standard library) and Docker Compose's JSON config support. The new
`scripts/validate-ingress.py` preflight checks the **rendered** production configuration before normal
deployment backup/build/replacement; backup-only runs remain independent of the new ingress setting.
CI runs `python3 scripts/test-ingress.py` without needing a Docker engine. Empty/missing peer values
fail safely; do not copy synthetic addresses from tests into production. Runtime/container verification
is still pending locally due to Docker Desktop failure. See the [B06 report and operator checks](../docs/audits/2026-09-07/validation/b06-completion.md)
before deploying this change. No production verification is implied by configuration tests.
1. push to `main`
2. Gitea Actions runs tests
3. if green, workflow uploads repo to server
4. `deploy/deploy.sh` links `/opt/job-tracker/shared/.env` into the repo checkout, then explicitly runs `docker compose -f docker-compose.yml build` and `up -d`
5. if `OLLAMA_MODEL` is set, the deploy script waits for Ollama, pulls the configured model if missing, then restarts `ai-service` so hybrid CV classification can use it
6. workflow checks service status after deployment
## Post-deploy verification you should also do manually the first time
- confirm reverse proxy routes to the frontend correctly
- confirm API auth/login works with production config
- confirm backend can connect to MariaDB
- confirm AI service container is reachable from backend
- confirm reminder and admin/system pages load
- verify follow-up reminder emails are enabled only when intended and that links open the correct job/tab
---
# Backups, restore and rollback
**Database restore and application rollback are two different operations.** A bad deploy usually needs
only the rollback. Restore the database only if the data itself is wrong or lost — it discards
everything written since the dump.
## Environment loading
`deploy/deploy.sh` **loads `/opt/job-tracker/shared/.env` into its own shell** before it decides
anything. The symlink it creates in the checkout is for docker compose, which reads `.env` itself;
the script needs the values too, to pick the right backup and to check its own configuration.
- Parsed line by line, not `source`d — a compose `.env` is not a shell script, so an unquoted value
containing spaces would execute as a command.
- **Variables already set in the environment win**, so CI-provided `APP_VERSION`, `APP_COMMIT_SHA`
and `APP_BUILD_STAMP` still override the file.
- No value is ever echoed. Error messages name variables, never their contents.
This was added on 2026-07-19. Before it, the script read an empty environment: `DATABASE_PROVIDER`
fell back to `sqlite` on a MariaDB host, so the deploy tarred the data volume, printed
`Backup verified`, and continued with no database dump at all. See
`docs/release-candidate-review.md` (B1).
## Required production variables
`validate_deploy_config` runs **before** the backup, and therefore before anything is built, stopped
or replaced. A missing variable aborts the deploy while the running stack is still untouched.
| Variable | Required | Why it is checked here |
|---|---|---|
| `DATABASE_PROVIDER` | **Always** — no default | Selects the backup. Guessing it wrong backs up the wrong database and reports success |
| `JOBTRACKER_CONNECTION_STRING` | When provider is `mariadb`/`mysql` | Without it there is no way to dump the database |
| `AI_SERVICE_TOKEN` | Always | `docker-compose.yml` declares it with `:?`; missing it kills the stack *after* the images are built |
| `AUTH_JWT_KEY` | Always | Compose sets `Auth__Require=true`, and the backend throws at startup on a blank key — after the containers have been replaced |
| `APP_PUBLIC_BASE_URL` | Always | Canonical HTTPS origin for links, OAuth callbacks, billing redirects, secure cookies, Host validation, and the public smoke check |
| `AUTH_MICROSOFT_TENANT` | When `AUTH_MICROSOFT_CLIENT_ID` is set | Exact Microsoft application sign-in account mode; distinct from the Graph mailbox tenant |
| `WEB_PROXY_SUBNET` | Always | Dedicated nginx-to-backend CIDR trusted for exactly one forwarded hop; must not overlap another Docker network |
### Microsoft sign-in migration gate
Before enabling `AUTH_MICROSOFT_CLIENT_ID` with the canonical identity release, take the normal
backup and record counts only. Do not print subjects or email addresses:
```sql
SELECT COUNT(*) AS legacy_links
FROM AspNetUsers
WHERE MicrosoftSubject IS NOT NULL OR MicrosoftEmail IS NOT NULL;
SELECT COUNT(*) AS legacy_without_alternate_credential
FROM AspNetUsers
WHERE (MicrosoftSubject IS NOT NULL OR MicrosoftEmail IS NOT NULL)
AND PasswordHash IS NULL
AND GoogleSubject IS NULL;
SELECT COUNT(*) AS duplicate_legacy_subject_groups
FROM (
SELECT MicrosoftSubject
FROM AspNetUsers
WHERE MicrosoftSubject IS NOT NULL
GROUP BY MicrosoftSubject HAVING COUNT(*) > 1
) duplicate_subjects;
SELECT COUNT(*) AS duplicate_legacy_email_groups
FROM (
SELECT MicrosoftEmail
FROM AspNetUsers
WHERE MicrosoftEmail IS NOT NULL
GROUP BY MicrosoftEmail HAVING COUNT(*) > 1
) duplicate_emails;
```
Apply `20260802212509_AddCanonicalMicrosoftIdentity` before deploying code that queries the two new
columns. The migration does not backfill legacy rows and adds a unique nullable composite index.
Keep Microsoft sign-in disabled if the inventory or migration fails. Roll back the application
binary while leaving the additive columns in place; never roll back to email auto-linking.
`DATABASE_PROVIDER` deliberately has **no default**. An unset value used to mean "sqlite"; it now
means "stop and tell me".
## Backup creation
`deploy/deploy.sh` takes a backup **before** it builds, stops or replaces anything, and **aborts the
deploy if the backup fails**. Nothing else in the deploy runs without a restore point.
- **Location:** `/opt/job-tracker/backups` — override with `BACKUP_DIR`. If an earlier root-run deploy owns the directory, the deploy script repairs its ownership through Docker before writing.
- **Selection:** driven solely by `DATABASE_PROVIDER`, which must be set.
- `mariadb` / `mysql` → SQL dump, `jobtracker-<database>-<UTC timestamp>.sql.gz`, e.g.
`jobtracker-jobtracker-20260719T153759Z.sql.gz`
- MariaDB deployments also archive `jobtracker_data`, which contains attachments, imported CVs,
generated exports and Data Protection keys.
- `sqlite` → data volume archive, `jobtracker-sqlite-<UTC timestamp>.tar.gz`
- every deployment archives the separate account-deletion tombstone volume when it exists;
restore it independently and never replace it with an older application-data snapshot
- anything else → the deploy stops
- **The filename tells you which path ran.** If you expect a MariaDB deploy and find a
`jobtracker-sqlite-*.tar.gz`, the environment is wrong — that is the exact failure this check exists
to make visible.
- **Naming:** the UTC timestamp makes every file unique, so a deploy never overwrites an earlier backup.
- **Credentials** come from `JOBTRACKER_CONNECTION_STRING` and are passed via `MYSQL_PWD`, never on the
command line, so they cannot appear in the process list or the deploy log.
- **Compression:** gzip. A small database compresses to a few KB.
- **SQLite volume resolution:** compose prefixes volume names with the project name, so the script
resolves `<project>_jobtracker_data` and **fails if that volume does not exist**. Naming the bare
volume would silently create an empty one and back *that* up.
### Verification — each provider gets the check that proves its own format
A backup that exists but is empty, truncated, or the wrong *kind* is worse than none, because it looks
like a restore point.
| Provider | Checks |
|---|---|
| MariaDB | non-empty; valid gzip; contains `CREATE TABLE`; contains the `Dump completed` trailer that `mariadb-dump` writes last, so a dump that died partway through is rejected |
| SQLite | non-empty; valid gzip; the archive actually contains `jobtracker.db` |
| Recovery volumes | non-empty valid gzip archives; a missing required application-data volume aborts the deploy |
A failed check deletes the file rather than leaving something that looks like a backup.
### Taking one by hand
```bash
BACKUP_DIR=/opt/job-tracker/backups
mkdir -p "$BACKUP_DIR"
MYSQL_PWD='<password>' mariadb-dump \
--host=127.0.0.1 --port=3306 --user=<user> \
--single-transaction --routines --events --quick \
jobtracker | gzip -c > "$BACKUP_DIR/jobtracker-manual-$(date -u +%Y%m%dT%H%M%SZ).sql.gz"
```
### Retention
Verified top-level recovery sets are retained for 30 days and the first complete set in each month is
copied into `monthly/`; the newest 12 monthly sets are retained. Configure
`BACKUP_DAILY_RETENTION_DAYS` and `BACKUP_MONTHLY_RETENTION_COUNT` to change those positive integer
limits. Paths outside the fixed backup directory and monthly naming contract are never pruned.
Set `BACKUP_RCLONE_REMOTE` to an encrypted rclone remote to copy each current recovery set off-host.
See [`docs/operations/google-drive-backups.md`](../docs/operations/google-drive-backups.md).
## Restoring the database
Tested end-to-end against a clean MariaDB 11 container: dump taken from a seeded database, restored
into an empty one, rows verified identical.
```bash
# 1. Stop the application so nothing writes during the restore.
docker compose stop backend
# 2. Restore. This REPLACES the current contents of the named database.
gzip -dc /opt/job-tracker/backups/jobtracker-jobtracker-20260719T153759Z.sql.gz \
| MYSQL_PWD='<password>' mariadb --host=127.0.0.1 --port=3306 --user=<user> jobtracker
# 3. Verify before starting anything.
MYSQL_PWD='<password>' mariadb --host=127.0.0.1 --port=3306 --user=<user> jobtracker \
-e "SELECT COUNT(*) AS applications FROM JobApplications;"
# 4. Start again.
docker compose start backend
```
Restoring a **SQLite** deployment instead:
```bash
docker compose stop backend
docker run --rm -v jobtracker_data:/data -v /opt/job-tracker/backups:/backup \
-e ARCHIVE_NAME=jobtracker-sqlite-20260719T153941Z.tar.gz \
alpine:3 sh -c 'rm -rf /data/* && tar xzf "/backup/$ARCHIVE_NAME" -C /data'
docker compose start backend
```
## Restoring application containers (rollback)
This is the usual fix for a bad deploy, and it **does not touch the database**.
Normal deployments tag backend and frontend images with `APP_COMMIT_SHA` and retain the images from
the currently running containers as `jobtracker-backend:previous` and `jobtracker-frontend:previous`.
If any step fails after core replacement—including public liveness, readiness, or auth-configuration
checks—`deploy.sh` automatically recreates both core services from those retained images and still
returns a failed status so CI reports the rejected release. The previous images are replaced only at
the start of the next deployment.
Use the manual procedure below if automatic restoration itself fails or an older release is required.
```bash
cd /opt/job-tracker/app # the deployment checkout
git log --oneline -5 # find the last good commit
git checkout <previous-commit>
deploy/deploy.sh
```
`deploy.sh` takes a fresh backup first, so rolling back is itself protected.
**Why a code rollback is safe here:** every Phase 4/5 migration is a no-op — the startup reconciler
owns those tables — so reverting the code never leaves migration history ahead of the schema. The
reconciler is additive and never drops a table holding rows, so the older code simply ignores the
newer tables.
**What a code rollback does not undo:** rows users created in the newer tables stay. That is usually
what you want. If you additionally restore the database, those rows are lost — so restore only when
the data is the problem.
## Choosing between them
| Symptom | Action |
|---|---|
| New version starts but behaves wrong | Rollback the code. Leave the database. |
| Backend will not start; schema looks wrong | Rollback the code, then restore only if it still fails. |
| Data is missing or corrupted | Restore the database from the most recent good dump. |
| Deploy aborted before starting | Nothing to undo — the backup ran before any change. |
## Health checks
`backend` and `frontend` both report container health, so `docker compose ps` shows real state rather
than merely "running".
- **Backend:** `curl -fsS http://127.0.0.1:8080/health`. Anonymous, and deliberately **does not touch
the database** — a health check that queried MariaDB would restart a healthy backend whenever the
database blipped. `start_period` is 90s to cover first-boot schema reconciliation.
- **API/database readiness:** `curl -fsS https://<host>/ready`. This checks that the API can reach its
configured database and returns only `ready` or `unavailable`; detailed diagnostics remain admin-only.
Deployment validation requires this probe, while Docker restart policy continues to use liveness.
- **Frontend:** `wget` against nginx on port 80.
- `frontend` waits for `backend` to be *healthy*, not merely started, because nginx proxies `/api` to
it and refuses to start if the upstream cannot be resolved.
A backend that cannot reach its database during startup exits and is reported `unhealthy`. A dependency
failure after startup leaves the process alive but makes `/ready` return 503, so deployment validation
cannot mistake a live-but-unready stack for a successful release.
@@ -0,0 +1,51 @@
# AI Prompt Management
## Purpose
Centralise prompts.
Avoid random prompts scattered throughout code.
---
# Requirements
Prompts should:
- Be version controlled.
- Be documented.
- Have tests where possible.
---
# Prompt Structure
Every prompt should define:
Purpose.
Inputs.
Expected output.
Restrictions.
---
# Example
Task:
Improve CV bullet point.
Input:
Existing bullet.
Output:
Improved bullet.
Restrictions:
Do not add unsupported achievements.
@@ -0,0 +1,66 @@
# AI Providers
## Purpose
Support multiple AI providers.
---
# Current/Future Providers
Possible:
- OpenAI.
- Anthropic.
- Google Gemini.
- Groq.
- Ollama.
- Other compatible APIs.
---
# Provider Architecture
Use abstraction:
AI Provider Interface
↓
Provider Implementation
↓
Model
---
# Requirements
Users should eventually be able to:
- Select provider.
- Select model.
- Configure API keys.
---
# Admin Controls
Admins should control:
- Available providers.
- Allowed models.
- Usage limits.
---
# SaaS Consideration
Different plans may have:
Free models.
Premium models.
BYO-key support.
@@ -0,0 +1,8 @@
# safety
AI output is suggestion-only and cannot overwrite the Career Profile automatically. Prompts delimit
untrusted source text, require preservation of factual claims, and instruct providers not to invent
employers, titles, dates, or metrics. Attachments are included only when explicitly selected.
Provider outages degrade AI features without blocking core tracking. Calls and tokens are metered and
limited by plan. See `docs/ai/security.md` and `docs/ai/privacy.md`.
@@ -0,0 +1,75 @@
# Backend Architecture
## Purpose
Define backend organisation.
---
# Preferred Structure
Example:
Application
├── Features
├── Domain
├── Infrastructure
├── Services
├── API
---
# Responsibilities
## API Layer
Handles:
- HTTP requests.
- Authentication.
- Validation.
- Responses.
Should not contain business logic.
---
## Application Layer
Handles:
- Workflows.
- Business operations.
- Coordination.
---
## Domain Layer
Contains:
- Entities.
- Rules.
- Business concepts.
---
## Infrastructure
Handles:
- Database.
- External APIs.
- File storage.
- AI providers.
---
# Rule
Controllers should remain thin.
@@ -0,0 +1,431 @@
# Jobjakt — Current Architecture
> **This document describes the system as it actually is.** Every claim was verified against code.
> Last verified: 2026-07-31. Supersedes the archived `docs/_archive/SYSTEM_OVERVIEW.md` (2026-07-02).
>
> **Rule:** if this document and the code disagree, the code wins — and this document is a bug. Fix it.
> Topic documentation is maintained alongside this file; dated completion and review reports remain
> historical snapshots. When documentation conflicts, prefer this file and the code.
---
## 1. What the product is
Jobjakt is a self-hosted, multi-user job application tracking platform with local-AI career assistance:
- Track jobs and applications end-to-end (pipeline stages, follow-ups, deadlines, salary, tags, notes).
- Company CRM (pipeline stage, contact dates, recruiter details) — company-level only, no people entities.
- Correspondence log per application: **Gmail OAuth import with a human review queue**, IMAP, Microsoft Graph.
- Attachments per application with purpose metadata and AI-inclusion toggles.
- CV platform: upload → OCR/extraction → structured parsing → per-job tailored CV drafts → templated PDF via Playwright.
- AI drafts: cover letters, recruiter messages, follow-up drafts, job summaries, match scoring, interview prep.
- Rules engine (auto-ghosting), reminder emails, daily JSON export, event trail, automated DB backup.
- Admin: user management, audit log, system readiness.
- Production: `https://jobs.cesnimda.uk` via Gitea Actions → SSH → Docker Compose.
**Product hierarchy** (from `docs/MASTER_IMPLEMENTATION_GUIDE.md` — job tracking is the core; career tools support it):
Job Tracking → Applications → Workflow → Follow-ups → Communication, then Career Profile → Master CV → CV Builder → Cover Letters → Portfolio → Interview Prep, then Job Discovery.
---
## 2. Architecture overview
```mermaid
flowchart LR
subgraph Client
UI[React 19 SPA<br/>MUI 7, react-router 6<br/>Next.js 16 CSR shell]
end
subgraph Frontend container
NGINX[nginx 1.29-alpine<br/>serves static export + proxies /api]
end
subgraph Backend container
API[ASP.NET Core net9.0<br/>JobTrackerApi host]
BG[7 hosted services:<br/>Rules, FollowUpReminder, DailyExport,<br/>JobEnrichment, SummarizerProbe,<br/>CvProcessing, DatabaseBackup]
DB[(SQLite default<br/>or MariaDB/MySQL)]
FS[/Data root:<br/>Attachments, CvArtifacts,<br/>exports, DP keys/]
end
subgraph AI stack
AISVC[FastAPI ai-service :8001<br/>distilbart summarizer,<br/>OCR, docx/pdf extraction]
PROV[Provider via AI_PROVIDER env:<br/>ollama qwen2.5:7b / gemini / groq]
end
EXT1[Google OAuth / Gmail API]
EXT2[Microsoft Graph / IMAP]
EXT3[Job sites: Finn, NAV,<br/>LinkedIn, Jobbnorge]
EXT4[SMTP]
EXT5[LibreTranslate optional]
UI --> NGINX --> API
API --> DB
API --> FS
API --> AISVC --> PROV
API --> EXT1
API --> EXT2
API --> EXT3
API --> EXT4
API --> EXT5
BG --> DB
```
### Solution layout
| Project | Role |
|---|---|
| `JobTrackerApi/` | ASP.NET Core host plus its controllers, services, EF models, `JobTrackerContext`, migrations, and Dockerfile. |
| `JobTrackerApi.Tests/` | xUnit, 36 test files incl. authorization + hostile-fixture suites. |
| `job-tracker-ui/` | React SPA inside a Next.js shell. |
| `tools/summarizer/` | FastAPI AI service (own Dockerfile, pytest tests). |
| `tools/hostile-fixture-db/` | Security test fixture generator. |
| `deploy/`, `.gitea/workflows/` | Prod deploy script + CI/CD. |
> Corrected 2026-07-31: the transitional `JobTrackerBackend` link-compilation project and root
> `Models/`/`Data/` directories were retired. Source now compiles from the project that owns it.
---
## 3. Technology stack
**Backend:** ASP.NET Core net9.0, EF Core 9 (SQLite default; Pomelo MySQL/MariaDB via `Database:Provider`), ASP.NET Identity Core, JWT bearer (smart policy scheme: local + Google), built-in RateLimiter, DataProtection (file-system keys), Playwright (PDF export).
**Frontend:** **Next.js 16** + React 19 + **TypeScript 5.9** + MUI 7 (+ x-data-grid, x-date-pickers, lab), axios, react-router-dom 7, @tanstack/react-table, i18n EN + NB (custom provider), Jest/RTL.
> Corrected 2026-07-31: the CRA migration is complete; direct Jest/Babel configuration replaced `react-scripts`.
**AI:** FastAPI + transformers (`sshleifer/distilbart-cnn-12-6`) for summaries; pytesseract/PyMuPDF/pypdf/python-docx for extraction/OCR; **one** generation provider selected by the `AI_PROVIDER` env var ∈ {`ollama` (default, `qwen2.5:7b`), `gemini`, `groq`}; TTL cache.
**Infra:** Docker Compose (backend, frontend/nginx 1.29-alpine, ai-service, ollama opt-in via `bundled-ollama` profile w/ GPU), Gitea Actions CI → SSH deploy → `deploy/deploy.sh`, external `jobtracker_shared` network.
---
## 4. Frontend architecture
**Two routing layers coexist intentionally:**
1. **Next.js 16 App Router** (`app/layout.tsx`, `app/page.tsx`) — a thin shell that mounts a client-side app. The CRA→Next migration was a **CSR lift-and-shift**: no SSR, no server components, no Next routing, no data fetching. Next is effectively a build tool here. Static export → nginx.
2. **react-router-dom v6** — does the actual routing, in **two different patterns inside one file** (`src/App.tsx`): `createBrowserRouter` for public routes (`/`, `/login`, `/forgot-password`, `/reset-password`, `/verify-email`) and a nested `<Routes>` inside a catch-all `Shell` for authenticated routes.
Development leaves static-export mode disabled and rewrites deep links to the root shell; production exports that shell and nginx falls back to `index.html` for unknown paths.
**Routes** (`src/App.tsx`): public — `/`, `/login`, `/forgot-password`, `/reset-password`, `/verify-email`. Authenticated — `/dashboard`, `/jobs`, `/reminders`, `/kanban`, `/companies`, `/correspondence`, `/correspondence/review`, `/profile`, `/career`, `/trash`, `/settings`, `/settings/connected-accounts`, `/admin/{audit,users,system}`.
> `/register` reuses the hardened auth form. Submission remains disabled until registration is enabled by production configuration (§5).
**State management: none.** No Redux/Zustand/React Query. Local `useState` + `axios` per component, with a hand-rolled `refreshToken` counter threaded through props. Two workspace-cache hooks exist (`components/job-workspace/useWorkspaceTabCache.ts`, `useJobWorkspaceBaseData.ts`). This is the root cause of the oversized components below.
**Styling:** MUI `sx` + custom `src/theme.ts` (439 lines), light/dark. Design tokens live inline in component `sx` props rather than in the theme (e.g. the same `boxShadow: "0px 8px 24px -12px rgba(15,23,42,0.12)"` is repeated across pages). There is no component primitives layer and no Storybook.
**Oversized components** (refactor targets, in order): `JobDetailsDialog.tsx` (1400), `CareerProfilePage.tsx` (1293), `JobTable.tsx` (786), `Correspondence.tsx` (732), `DashboardView.tsx` (666), `AdminSystemPage.tsx` (623), `AddJobModal.tsx` (618). (`ProfilePage.tsx` was 1368; Phase 2.2 split it — see §4a.)
---
## 4a. Profile / Career separation (Phase 2, 2026-07-17)
`/profile` and `/career` were one 1368-line component (`ProfilePage`) forked by a `careerOnly`
boolean. Phase 2 scoped their saves; Phase 2.2 split them into two dedicated components. This is
the reference model for how account identity and the master career profile relate.
### Components & routes
| Route | Component | Owns |
|---|---|---|
| `/profile` | `views/ProfilePage.tsx` (~490 lines) | Account identity + security + preferences |
| `/career` | `views/CareerWorkspacePage.tsx` → `views/CareerProfilePage.tsx` (~1293 lines) | The **master career profile** — the single editable source of truth |
`CareerWorkspacePage` is a thin shell (heading + source-of-truth notice) around `CareerProfilePage`.
The CV Builder is a separate routed workspace and consumes the Career Profile as its source of truth.
### Request flow
```
/profile → ProfilePage
load: GET /auth/me (account row only)
save: PUT /auth/profile { email, userName, firstName, lastName, displayName }
/career → CareerWorkspacePage → CareerProfilePage
load: GET /auth/me
GET /profile-cv/runs (extraction history)
GET /jobapplications?… (for per-job CV tailoring context)
save: PUT /auth/profile { profileCvText, profileCvStructureJson }
(ProfileCvController paths additionally dual-write CareerProfileService —
CareerProfiles / CareerProfileVersions — see §9 / §16)
```
### Data ownership (the invariant)
Both surfaces persist through the **same** endpoint, `PUT /auth/profile`, which does **partial
updates** (`AuthController.UpdateProfile`): a field is touched only if the request carries it —
`null`/omitted leaves it unchanged, `""` clears it, a value sets it. Email and UserName are never
cleared (login identifiers).
| Field(s) on `ApplicationUser` | Owner (only surface that writes them) |
|---|---|
| `Email`, `UserName`, `FirstName`, `LastName`, `DisplayName`, `AvatarImageDataUrl`, password, TOTP/2FA, linked OAuth accounts | **/profile** |
| `ProfileCvText`, `ProfileCvStructureJson` (the master career profile) | **/career** |
Because the endpoint is partial, `/profile` saving identity does **not** null the master profile,
and `/career` saving the profile does **not** null identity. This is enforced by tests
(`AuthAndSystemControllerTests`: identity-save-keeps-CV, career-save-keeps-identity, empty-clears,
null-leaves).
### API responsibilities
- `AuthController.UpdateProfile` (`PUT /auth/profile`) — partial update of the account row; the
single write path for both surfaces. Local accounts only.
- `AuthController` (`GET /auth/me`) — returns the whole account row; each surface reads the fields
it owns.
- `ProfileCvController` — CV ingest/parse/rewrite/export and the `CareerProfileService` dual-write.
Called from `/career`.
- `JobApplicationsController` — `/career` reads job list for tailoring context only.
### Future extension points
- **`CareerProfilePage` is the foundation for all future career outputs** (Phase 3/4): CV Builder,
tailored CVs, cover letters, portfolio, interview prep. They attach here, referencing the master
profile — never duplicating it (per `docs/MASTER_IMPLEMENTATION_GUIDE.md`).
- **Source-of-truth flip (F5):** today `ProfileCvStructureJson` is authoritative and
`CareerProfileService` mirrors it. A later phase makes `CareerProfiles`/`CareerProfileVersions`
authoritative; the `/career` save would then route through `CareerProfileService` rather than the
blob column. The partial-update endpoint and the ownership split above do not change.
- **Full decomposition (roadmap 2.2 residue):** `CareerProfilePage` is still large because it owns
the whole master-CV surface; the CV Builder work will extract sub-components from it.
---
## 5. Authentication & authorization
- **Smart policy scheme:** inspects the bearer token issuer — Google ID tokens (`accounts.google.com`) → `google` handler (validated against `Auth:GoogleClientId`); everything else → `local` JWT (symmetric `Auth:JwtKey`, issuer/audience validated, 2-min clock skew).
- **Cookie sessions:** local handler also reads `jobtracker_auth` (HttpOnly, SameSite=Lax, Secure-configurable, 30d when persistent). **CSRF double-submit** middleware enforces cookie+header match on all mutating requests when a session cookie is present (login/register/reset/csrf exempt).
- `Auth:Require=true` sets a fallback authorize-all policy (prod compose sets it). Dev without a JWT key generates an ephemeral key + warning; **fails closed** if auth is required but unconfigured.
- Local tokens **must** carry a subject claim (`LocalAuthIdentity`), enforced in `OnTokenValidated` — hardened after finding M013-2.
- **Multi-tenancy:** every tenant entity carries `OwnerUserId`; `JobTrackerContext` applies global query filters `CurrentUserId != null && OwnerUserId == CurrentUserId` (**deny-on-null**). Correspondence/JobEvent/CV entities filter through their parent's owner. Covered by `JobApplicationsAuthorizationTests`, `OwnershipGuardTests`.
- Roles via Identity: admin-only `UsersController`, `AdminAuditController`, `AdminSystemController`.
- **2FA:** TOTP (`Otp.NET`), encrypted secrets, QR enrolment (`QRCoder`), recovery codes, trusted devices (`jobtracker_td` cookie), pending-token flow.
- **Sessions:** `UserSession` entity + `SessionsController` — list/revoke active sessions.
- Password policy: min 8, digit + lowercase. Reset via emailed token (SMTP required).
- **Registration is disabled by default** — `AuthController.cs:135` reads `Auth:AllowRegistration` defaulting to `false` and returns HTTP 403. There is no CAPTCHA anywhere.
- **Rate limiting:** login, auth email, 2FA challenge, and anonymous public-CV PDF export have dedicated fixed-window policies. AI usage is bounded by per-account monthly generation and token ceilings rather than request-window throttling.
---
## 6. Database
EF Core, **11 migrations**. App DbSets + Identity tables.
```mermaid
erDiagram
ApplicationUser ||--o{ Company : owns
ApplicationUser ||--o{ Job : owns
ApplicationUser ||--o{ JobApplication : owns
ApplicationUser ||--o| UserRuleSettings : has
ApplicationUser ||--o{ GmailConnection : has
ApplicationUser ||--o{ CvUploadArtifact : owns
ApplicationUser ||--o{ CvExtractionRun : owns
ApplicationUser ||--o{ UserSession : has
ApplicationUser ||--o{ TrustedDevice : has
Company ||--o{ Job : "posts"
Company ||--o{ JobApplication : "has jobs"
Job ||--o{ JobApplication : "applied to via"
JobApplication ||--o{ Correspondence : messages
JobApplication ||--o{ Attachment : attachments
JobApplication ||--o{ JobEvent : events
JobApplication ||--o| TailoredCvDraft : "1:1 draft"
CvUploadArtifact ||--o{ CvExtractionRun : "source of"
```
**Entities:** `Company`, **`Job`**, `JobApplication`, `Correspondence`, `GmailConnection`, `GmailReviewDecision`, `MicrosoftGraphConnection`, `ImapConnection`, `Attachment`, `RuleSettings`, `UserRuleSettings`, `SystemEmailSettings`, `JobEvent`, `CvUploadArtifact`, `CvExtractionRun`, `TailoredCvDraft`, `TwoFactorRecoveryCode`, `TrustedDevice`, `UserSession`.
**Key notes:**
- `ApplicationUser` (IdentityUser) also stores `ProfileCvText`, **`ProfileCvStructureJson`** (the master career profile — a JSON blob, not relational), `AvatarImageDataUrl` (base64 in a column, on the `/auth/me` hot path), Google/Microsoft link info, TOTP secrets, current CV artifact/run pointers.
- **`Job` vs `JobApplication`** — `Job` is the opportunity (title, company, description, URL, salary, location, deadline, tags); `JobApplication` is the user's pursuit of it (status, dates, follow-ups, correspondence, attachments). `JobApplication.JobId` remains nullable for compatibility, but startup backfills missing links and persistence paths synchronize both copies. Legacy reads and columns remain until the production cutover report passes. See §16 and `docs/decisions/ADR-002-job-application-model.md`.
- Salary is **structured**: `SalaryMin`, `SalaryMax`, `SalaryCurrency`, `SalaryPeriod` (plus a legacy free-text `Salary`).
- `Tags` is a **JSON array in a string column** — not queryable; `/tags` and `/tag-trends` must scan.
- Denormalized `HasResume`/`HasCoverLetter`/`HasPortfolio`/`HasOtherAttachment` flags duplicate `Attachments`; kept honest by `AttachmentFlagsRecomputeTests`.
- CV extraction runs retain the newest 20 completed runs per user; expired runs and unreferenced upload artifacts are pruned while the current artifact is preserved.
- `Status` is free-text at the DB level; canonicalized only in the application layer by `JobPipeline.Normalize` — deliberately, so custom user values are never destroyed.
- Indexes include owner-prefixed list/board/reminder composites `(OwnerUserId, IsDeleted)`, `(OwnerUserId, IsDeleted, Status)`, and `(OwnerUserId, FollowUpAt)`, plus CV and provider-specific indexes.
- SQLite lives at `DataRoot/jobtracker.db` (WAL); migrations and legacy-schema reconciliation run at startup through `StartupInitializationExtensions`.
> Corrected 2026-07-31: the model snapshot is current and the API directly carries the EF Design package and `JobTrackerContext`; no temporary project edit is required for `dotnet ef`.
---
## 7. API surface (19 controllers, all under `/api`)
| Controller | Lines | Highlights |
|---|---|---|
| `JobApplicationsController` | **2394** | **37 endpoints.** CRUD, paging/filter/sort, board, reminders, stats, analytics, history, status/follow-up PATCH, soft delete/restore, duplicate-check, **plus** the AI surface: match-score, candidate-fit, focus-plan, generated interview-prep brief, readiness, tailored-CV draft/preview/export/generate, application-drafts, application-package, follow-up drafts + send, ai-metrics. The canonical timeline and editable interview board live in their focused controllers. |
| `ProfileCvController` | **2249** | CV upload artifacts, extraction runs, structure parsing, reprocess/rebuild/improve, rewrite-section, rewrite-preview, templates, Playwright PDF export, benchmark harness. |
| `GmailController` | **1023** | OAuth connect/callback, sync, review queue, import decisions, job matching. |
| `AuthController` | **879** | login/register/me/config, Google + Microsoft exchange and link/unlink, avatar, password change/reset, email verification, session cookie + CSRF. |
| `AdminSystemController` | 342 | System readiness (DB/Gmail/AI). |
| `TwoFactorController` | 341 | TOTP enrol/verify/disable, recovery codes. |
| `AttachmentsController` | 342 | Validated staged upload, managed-root download, metadata-only rename, quarantined delete, purpose/AI-inclusion metadata and restart reconciliation. See `attachment-storage.md`. |
| `UsersController` | 229 | Admin user/role management. |
| `AdminAuditController` | 219 | Audit trail. |
| `CorrespondenceController` | 185 | Per-job messages CRUD. |
| `CompaniesController` | 150 | CRUD, idempotent create-by-name, recruiter/pipeline fields. |
| `MicrosoftGraphController` | 150 | Outlook/M365 mail linking. |
| `SessionsController` | 104 | List/revoke sessions. |
| `ExportController` | 102 | JSON/CSV export. |
| `RulesController` | 101 | Global + per-user rule settings, clamped. |
| `ClientErrorsController` | 100 | Frontend error intake → logs. |
| `ImapController` | 96 | IMAP mail linking (SSRF-guarded). |
| `BackupController` | 89 | Manual backup trigger. |
| `JobImportController` | **27** | **One endpoint:** `POST /preview`. URL parse only — no persistence, no import history. |
**God controllers are a top debt.** `JobApplicationsController` and `ProfileCvController` mix HTTP, business logic, AI prompt construction, and persistence. `docs/MASTER_IMPLEMENTATION_GUIDE.md` forbids exactly this ("Avoid: Massive controllers"). Refactor needs test cover first — the tests exist.
**OpenAPI** is wired (`AddOpenApi` / `MapOpenApi`) but **dev-only** — guarded by `app.Environment.IsDevelopment()`, not exposed in production.
---
## 8. Background services (7 hosted services)
| Service | Function |
|---|---|
| `RulesHostedService` → `RulesEngine` | Periodic auto-transitions (e.g. → Ghosted) from rule settings |
| `FollowUpReminderHostedService` | Reminder emails for due/upcoming follow-ups (dedup via `LastReminderEmailSentAt`) |
| `DailyExportHostedService` | Daily JSON export at a configured local hour |
| `JobEnrichmentHostedService` | Backfills summaries/enrichment |
| `SummarizerProbeHostedService` | Probes AI service readiness |
| `CvProcessingHostedService` + `CvProcessingQueue` | Process-local wake-up queue for CV extraction; queued/running database work is recovered at startup |
| `DatabaseBackupHostedService` → `DatabaseBackupRunner` | Automated DB backup (`VACUUM INTO`, server-derived path) |
Rules, reminders, daily export and enrichment now enumerate owners explicitly, then re-enter normal tenant-filtered scopes. All four have deny-by-default worker switches and remain inactive until their notification/privacy/entitlement/operations prerequisites are ready; see `docs/architecture/background-workers.md`. Caches and worker coordination are process-local. CV work itself is durable and recovered after restart, but the worker remains a deliberate single-instance design without database leasing.
---
## 9. AI pipeline
**Architecture:** the backend does **not** call any LLM in-process. It HTTP-calls a FastAPI sidecar (`tools/summarizer/app.py`) exposing `/health`, `/cv/normalize`, `/cv/classify-block`, `/cv/rewrite`, `/summarize`, `/extract-text`. The sidecar picks **one** provider from the process-wide `AI_PROVIDER` env var ∈ {`ollama`, `gemini`, `groq`}.
> **Important — `docs/00-ai-context.md` is wrong about this.** It describes a provider interface fanning out to OpenAI/Gemini/Claude/Ollama, admin-controlled, with users never locked to one model. **None of that exists.** There is one env var, one provider per deployment, no OpenAI, no Claude, no admin control, no per-user selection. Product decision 2026-07-17: **the docs get fixed, the abstraction does not get built** — revisit only if a customer asks.
**Data flow:**
1. **Job import:** URL → plugin parse (Finn/NAV/LinkedIn/Jobbnorge) or universal JSON-LD parser → optional LibreTranslate → language detect + skill tagging → preview → user accepts → persisted.
2. **Summaries:** `SummarizerService` → `/summarize` (distilbart, TTL-cached, GPU if available) → persisted `ShortSummary`.
3. **CV ingest:** upload (PDF/DOCX/image) → `/extract-text` (OCR) → block classification (`CvAiClassifier`/`CvAiNormalizer` via `/cv/classify-block`) → `StructuredCvProfile` → `ProfileCvStructureJson` on the user.
4. **Tailoring:** job description + structured CV → `/cv/rewrite` → `TailoredCvDraft` (separate entity, per application) → `CvTemplateRenderer` → Playwright → PDF.
5. **Drafts:** cover letter / recruiter message / follow-up per job, attachment-aware context selection.
**Invariant that holds:** the master profile is **never** auto-overwritten. Tailored output lands in `TailoredCvDraft`, a separate entity. This is the most important documented rule and it is correctly implemented — do not break it.
**Degradation:** if the AI service or provider is down, core tracking still works (probe service; AI is not a deploy gate).
**CV templates are hardcoded** — `CvTemplateRenderer.Render` is a C# `switch` over 6 template IDs (`ats-minimal`, `harvard`, `auckland`, `edinburgh`, `monarch`, `fjord`), each a function interpolating HTML strings, with booleans like `roundedPhoto`/`curvedHeader`. **There is no theme model and nothing is user-customisable.** This is a structural dead end for the CV Builder — see `docs/application-discovery-report.md` §10.
---
## 10. Email
`SmtpEmailSender` + `EmailSettingsResolver`: config from env/appsettings **or** DB-stored `SystemEmailSettings` (admin-editable). Gmail SMTP + app password in prod. Flows: password reset, email verification, follow-up reminders. `App:PublicBaseUrl` is the canonical external origin for generated links, OAuth callbacks, billing redirects, secure cookies and production Host validation.
Inbound: `GmailOAuthService` (655), `MicrosoftGraphOAuthService` (507), `ImapService` (345, SSRF-guarded).
---
## 11. Configuration & secrets
- `.env` (git-ignored) → docker-compose env → ASP.NET config. `.env.example` documents the shape.
- `appsettings.Development.json` holds only `CHANGE_ME_*` placeholders.
- Key knobs: `Database:Provider`, `ConnectionStrings:JobTracker`, `Data:Root`, `Cors:Origins`, `Ai:BaseUrl`, `Summarizer:BaseUrl`, **`Ai:ServiceToken`**, `Auth:*` (incl. `Auth:AllowRegistration`), `Email:*`, `Exports:*`, `App:*`, `HttpsRedirection:*` (TLS terminated at the reverse proxy).
- AI service knobs (compose): `AI_PROVIDER`, **`AI_SERVICE_TOKEN`**, `OLLAMA_BASE_URL`, `OLLAMA_MODEL`, `GEMINI_API_KEY`, `GROQ_API_KEY`.
- **`AI_SERVICE_TOKEN` is mandatory.** Both `Ai__ServiceToken` (backend) and `AI_SERVICE_TOKEN` (ai-service) use `${AI_SERVICE_TOKEN:?...}`, so `docker compose up` fails loudly rather than booting an unauthenticated AI service. Generate with `python -c "import secrets; print(secrets.token_hex(32))"`. Rotating it requires recreating **both** containers together — they must agree.
- Note both those compose entries are **quoted**: the `:?` error message contains a colon-space, which YAML would otherwise parse as a map (`services.backend.environment.[20]: unexpected type map[string]interface{}`).
- `ProductionConfigTests.cs` guards prod config shape.
- Ollama is **intentionally not bundled by default** (`bundled-ollama` compose profile) so deploys reuse a shared instance. `AI_PROVIDER=gemini` exists specifically to offload a weak local GPU in prod.
---
## 12. Build, CI/CD, deployment
- **CI is Gitea, not GitHub** — `.gitea/workflows/ci-deploy.yml`. **There is no `.github/` directory.**
- On PR + push-to-main: build backend (Release) → run **all** backend tests → `npm ci` → run the **whole** frontend suite → build frontend.
> Corrected 2026-07-17: the archived overview said CI runs "an explicit whitelist of 10 frontend test files". **The whitelist is gone.** The workflow now runs `npm test -- --watchAll=false --runInBand` and carries a comment forbidding its return: the previous whitelist "silently skipped new suites and let two regressions reach main."
- The workflow is heavily defended against a flaky self-hosted runner: dotnet install retry, `npm ci` SIGSEGV retry, frontend build OOM retry.
- **Deploy** (push to main only): SSH to prod → `git reset --hard <sha>` in `/opt/job-tracker/app` → `deploy/deploy.sh` (compose build/up with retry + cache-prune fallbacks) → verify containers. AI health is non-blocking.
- **No staging environment.** Deploys go straight to prod after CI.
---
## 13. Testing
- **Backend:** xUnit integration-style via `TestHostFactory`. 36 test files. Notable: `JobApplicationsAuthorizationTests`, `OwnershipGuardTests`, `ImapServiceSsrfGuardTests`, `ProductionConfigTests`, `AttachmentFlagsRecomputeTests`, `CvCorpusHarnessTests`, `SqliteMigrationHelperTests`, `JobPipelineTests`, plus a `tools/hostile-fixture-db` project.
- **Frontend:** ~20 Jest/RTL files — **all run in CI**.
- **AI service:** pytest (`tools/summarizer/tests/`).
- **Browser smoke:** Playwright drives login/session cookies, saved-job creation, Career Workspace routing, and anonymous public-CV rendering/PDF download against isolated API/SQLite and Next.js processes. CI installs Chromium and runs all four flows.
- **Gap:** no load/performance suite. CI reports NuGet transitive vulnerabilities and production npm audit findings.
---
## 14. Logging & error handling
Development uses simple console/debug logging; production emits structured JSON. Middleware records method, path, status, duration, trace ID, and subject. Unhandled errors return Problem Details with the same trace ID. Client errors POST to `/api/client-errors`; the frontend has an `ErrorBoundary` and route error page.
Compose bounds each container's local logs to three 10 MB files. There is still no external sink (Seq/OTLP) or cross-host aggregation.
---
## 15. Security posture
**Verified strong:**
- Multi-tenancy via deny-on-null global query filters, with a dedicated authorization test suite.
- CSRF double-submit on mutating requests; HttpOnly SameSite=Lax session cookie.
- Auth fails closed when required but unconfigured; subjectless-JWT rejected (M013-2).
- SSRF on job import and IMAP fixed and retested (DNS resolution check, private/loopback/link-local rejection, redirects disabled).
- Rate-limited login/email/2FA endpoints; Identity PBKDF2 hashing.
- OpenAPI dev-only. `.env` git-ignored; DP keys and runtime exports untracked (`519c32e`).
- 2FA + recovery codes + trusted devices + session revocation.
- **AI sidecar: backend-only.** Unpublished, on a private two-member network, and token-authenticated (§16). Verified against the running stack, not just configured.
**Findings status** (detail in `docs/application-discovery-report.md` §12 and `docs/phase-0-foundation-report.md`):
| Sev | Finding | Status |
|---|---|---|
| Medium | DataProtection keys recoverable from git history (`519c32e`, `955cae6`) | **Closed 2026-08-02 — production keys rotated by the operator** |
| Medium | Wildcard credentialed CORS configuration | **Closed — startup rejects it** |
| Medium | AI cost ceiling | **Closed — monthly generation/token limits are enforced by plan** |
| Low | Public-registration abuse control | Implemented with Turnstile; production keys/configuration still required |
| Low | Unbounded storage | **Closed — attachment quotas, extraction/artifact pruning, file-backed avatars, and PDF-export retention are implemented** |
| Low | Backup / DPAPI is Windows-oriented — verify behaviour on Linux prod | Unverified |
| Low | No dependency CVE scanning in CI | **Closed — NuGet and production npm audit reporting are in CI** |
---
## 16. Phase 0 changes (2026-07-17)
Full record: `docs/phase-0-foundation-report.md`. What changed architecturally:
- **AI sidecar secured — three layers, verified against the running stack (2026-07-17):**
1. **No host port.** `ports: "8001:8001"` removed; `expose:` only.
2. **Private network.** `ai-service` sits on a new `ai_internal` bridge and **nothing else**. It was removed from `default` (which the frontend shares) and from `shared_services` — the latter is `external: true` (`jobtracker_shared`), so any other compose stack on the host could join it and reach port 8001. `ai_internal` has exactly two members: `ai-service` and `backend`. It is **not** `internal: true`, because ai-service needs egress to Gemini/Groq.
3. **Shared secret.** `X-Ai-Service-Token` required on every endpoint except `/health`, compared with `hmac.compare_digest`. Backend sends it via `Ai:ServiceToken`; sidecar reads `AI_SERVICE_TOKEN`. Unset = open (local dev/tests), but compose declares both with `:?` so the stack **refuses to start** without it.
**Only the backend can reach the AI service.** Verified live: host → connection refused; frontend container → cannot even resolve `ai-service`; unauthenticated calls to `/summarize`, `/cv/rewrite`, `/extract-text` → 401; wrong token → 401; backend (172.23.0.3) with token → **200 OK**.
> If you point `OLLAMA_BASE_URL` at an Ollama in **another** compose stack, address it by host IP (e.g. `http://<host-ip>:11435`) — `ai-service` can no longer resolve container names on `shared_services`, by design. The bundled `ollama` profile is on `ai_internal` and still works by name.
- **Pipeline expanded beyond `Applied`** — `JobPipeline` now models pre-application stages (`Saved`, `Interested`, `Preparing`) in a new `PipelineCategory.Prospect`, so a job can be tracked before it is applied to. `Saved` is the new default for wizard-created jobs; `Applied` remains the default for the legacy create path.
- **`DateApplied` is nullable** + `SavedAt` added — a saved job no longer carries a fabricated application date.
- **`Job` entity introduced** alongside `JobApplication`; linked rows are now created and backfilled, while reads remain on the legacy columns pending production cutover validation.
---
## 17. Known debt
The live, prioritized ledger is `docs/architecture/technical-debt.md`. The principal remaining items are the legacy `JobApplication` opportunity columns, single-replica worker coordination, and optional cross-host log aggregation. Large orchestration files are refactored only when a cohesive behavior change provides a safe seam.
---
## 18. Historical decisions worth knowing
Recorded nowhere else in active docs:
1. `Status` is free-text and canonicalized in the application layer **specifically so custom user values are never destroyed** (`JobPipeline.cs` docstring). Deliberate; do not "fix" it with a DB enum.
2. The CI frontend-test whitelist was removed after it "silently skipped new suites and let two regressions reach main."
3. Ollama is intentionally not bundled by default so deploys reuse a shared instance.
4. `AI_PROVIDER=gemini` exists to offload a weak local GPU in prod.
5. `TailoredCvDraft` is a separate entity specifically to guarantee the master CV is never auto-modified.
@@ -0,0 +1,64 @@
# Frontend Architecture
## Goals
The frontend should be:
- Maintainable.
- Consistent.
- Easy to extend.
---
# Structure
Organise by feature.
Example:
features/
jobs/
career/
cv-builder/
settings/
auth/
---
# Components
Prefer:
Reusable components.
Small focused components.
---
# Pages
Pages should:
- Compose components.
- Handle routing.
- Manage page-level state.
---
# State
Avoid unnecessary global state.
Use:
- Local state.
- Feature state.
- Server state.
Where appropriate.
@@ -0,0 +1,93 @@
# Application Architecture Overview
## Purpose
Define the target architecture direction for Jobjakt.
This document does not require an immediate rewrite.
The goal is incremental improvement while keeping the application functional.
---
# Product Architecture
The application consists of:
## Core Platform
Job Tracking
↓
Applications
↓
Communication
↓
Follow-ups
---
## Supporting Platform
Career Workspace
↓
Master Career Profile
↓
CV Builder
↓
Documents
---
## Intelligence Layer
AI Services
↓
Recommendations
↓
Generation
---
# Architecture Goals
The system should be:
- Maintainable.
- Testable.
- Extensible.
- Secure.
- SaaS-ready.
---
# Evolution Strategy
Prefer:
- Refactoring.
- Extraction.
- Clear boundaries.
- Incremental migrations.
Avoid:
- Large rewrites.
- Replacing working systems unnecessarily.

Some files were not shown because too many files have changed in this diff Show More