jobtrackingapp/docs/remaster/SYSTEM_AUDIT_REPORT.md

System Audit Report — Job Tracker (Jobbjakt)

Date: 2026-07-04 Auditor role: Principal Architect / Staff Eng / Product / UX / Security (single reviewer, code-grounded) Scope: Full-system critical audit + rebuild-vs-refactor assessment. Verdict (see REBUILD_DECISION.md): Incremental Refactor — a full rebuild is not justified by the evidence.

Method note: every finding below is grounded in a file/line reference or explicitly labelled [Speculative issue]. Where I could not verify behaviour, I say so. Findings are tagged [Bug] [Design flaw] [Architectural weakness] [Speculative issue] and severity-rated in BUG_REPORT.md.

---

1. Executive summary

Job Tracker is a more mature and better-engineered system than a "rethink from scratch" framing assumes. The core architecture is sound: a React/TypeScript SPA, an ASP.NET Core + EF Core API with proper multi-tenancy (global query filters on OwnerUserId), a pluggable job-ingestion pipeline with real SSRF defence, hardened cookie-based auth with CSRF, deterministic (non-AI) scoring, and a decoupled FastAPI AI service behind an HTTP contract. There are 135 backend integration tests and 23 frontend suites, and the app is live in production (jobs.cesnimda.uk).

The problems are real but localised and fixable, not systemic rot:

1. God classes. JobApplicationsController (3,271 lines) and ProfileCvController (2,265 lines) and GmailController (1,179 lines) concentrate far too much logic. This is the #1 maintainability drag. Refactorable in place (extract services/DTOs), not a reason to rebuild.
2. God entity. JobApplication has ~40 columns mixing eight concerns, with denormalised attachment booleans (HasResume…) that can drift from the real Attachments collection, and two sources of tailored-CV truth (TailoredCvText string and TailoredCvDraft navigation).
3. Prompt-injection surface. Scraped job text + user CV + free-text instruction are string-interpolated directly into LLM prompts with no delimiting. Blast radius is limited by the human-review boundary.
4. Performance debt. No hot-path indexes (only OwnerUserId), load-all-then-count analytics, and N+1 loops in Gmail import. (This was already scoped as the Phase 7 work.)
5. Planning drift vs .gsd. An active, never-executed override "use next.js" (2026-04-10) conflicts with the shipped CRA frontend; milestone numbering jumps (M001 → M005 → M011) indicate the historical GSD plan and the built system diverged.

None of these require discarding the codebase. See §7 for the systemic-vs-fixable split.

---

2. Product logic audit

Area	Finding	Tag
Job ingestion (URL)	Real pipeline: validate URL → SSRF check → fetch (4MB cap, redirect-averse) → universal parse → site-plugin fallback → language detect → NO translation. Solid.	✅
Scraping assumptions	Plugins are HTML-structure-coupled (FinnPlugin, NavPlugin, LinkedInPlugin, JobbnorgePlugin). LinkedIn/Indeed actively fight scrapers; these will silently rot and fall back to the universal parser or fail. No plugin-health telemetry.	[Architectural weakness]
Manual fallback	Exists (AddJobModal manual entry). Good — the product degrades gracefully when scraping fails.	✅
CV match logic	Deterministic keyword-coverage score (JobCvMatchService), not AI. This is the right call — reproducible, explainable, no hallucination. But keyword coverage ≈ ATS-style matching, which over-rewards literal token overlap and under-rewards semantic equivalence ("K8s" vs "Kubernetes").	[Design flaw]
CV regeneration	AI rewrite via FastAPI /cv/*. Prompt explicitly forbids fabricating experience and analysis headings (app.py:583-608) — good guardrail — but nothing enforces factuality; the model can still invent. Output is a draft for review.	[Speculative issue]
Cover letter	Generated server-side, returned as draft. Consistent with the assistive-only model (D002).	✅

Product-shape observation. .gsd/PROJECT.md and D003 define this as a single-user personal workspace. The code has since been retrofitted to multi-tenant (OwnerUserId + query filters). That retrofit looks correct on audited endpoints, but the product identity is unresolved: is this a personal tool or a SaaS? That question drives half of the remaster decisions and should be answered explicitly (see REMASTER_PROPOSAL.md §1).

---

3. Architecture audit

Full detail in ARCHITECTURE_REVIEW.md. Summary:

• Backend structure — clean layering except the controllers, which are transaction scripts holding business logic that belongs in services. The JobTrackerBackend link-compile quirk (controllers/services glob-compiled via a separate library project) is a footgun for newcomers but works.
• AI pipeline — correctly decoupled behind HTTP. A provider swap (Ollama→cloud) needs zero .NET changes. This is the single best architectural decision in the codebase.
• Ingestion — plugin pattern is right; plugin fragility and lack of health signals are the risk.
• Notifications — hosted background services (FollowUpReminderHostedService, RulesHostedService, JobEnrichmentHostedService, DailyExportHostedService, DatabaseBackupHostedService). Reasonable, but polling-based; no event bus. Fine at single-node scale.
• Service boundaries — blurred by the god controllers; the services directory is actually well-factored (JobCvMatchService, JobPipeline, StageAnalytics, EmailStatusClassifier are all small and pure).

---

4. Data model audit

Full detail in DATA_MODEL_REVIEW.md. Headlines:

• JobApplication is a god entity (~40 columns, 8 concerns).
• Denormalisation hazard: HasResume/HasCoverLetter/HasPortfolio/HasOtherAttachment booleans duplicate the truth in the Attachments collection and can silently disagree. [Bug] risk.
• Dual CV truth: TailoredCvText (string on the entity) and TailoredCvDraft (related entity, plus TailoredCvDraftJson). Which wins? Ambiguity is a correctness liability.
• CV "versioning" is not versioned. The product promises "CV version linked to job", but the entity stores a single current tailored text. There is no version history table. The stated product goal (step 8, "CV version is linked to job") is only partially supported. [Design flaw]
• Indexing is minimal (OwnerUserId only) — a performance problem, not a modelling one.

---

5. AI system audit

Full detail in AI_SYSTEM_REVIEW.md. Headlines:

• Scoring validity: deterministic, good, but keyword-literal (see §2).
• Prompt injection: scraped job text + user instruction are interpolated raw into prompts (app.py:469,527,581). A malicious job ad can steer the CV/cover-letter output. Severity Medium because output is always a human-reviewed draft and there is no tool-use/auto-send.
• Hallucination: guarded by prompt instructions only; no factuality verification against the source CV.
• JD parsing reliability: universal parser + heuristics; brittle on JS-rendered boards.

---

6. UX / product audit

Full detail in UX_REVIEW.md. Headlines: the daily-loop navigation (jobs → dashboard/ reminders → workspace, D004) is coherent; the CV-tailoring workspace persists reusable material; the biggest UX risks are (a) the import-failure experience when scraping breaks, (b) the tailored-CV save/read-back model that historically abused the free-text notes block (D006), and (c) no visible "why this match score" beyond a number.

---

7. Systemic problems vs fixable issues

Fixable in place (majority)	Systemic (design-level, but still refactorable)
God controllers → extract services	Product identity: personal tool vs SaaS is undecided
Missing indexes, N+1s, load-all analytics	JobApplication god entity → needs schema evolution
Prompt-injection hardening (delimiters)	CV "versioning" promised but not modelled
CRA transitive-vuln debt → Vite/Next migration	Unexecuted "use next.js" override — plan/impl divergence
Denormalised attachment booleans	Scraper fragility as a long-term ingestion strategy

Nothing in the right column requires a from-scratch rebuild. Each is reachable by an incremental, test-guarded refactor because the test harness (135 integration tests) locks behaviour while internals move.

---

8. Deliverables index

• BUG_REPORT.md — severity-rated defects & risks
• ARCHITECTURE_REVIEW.md
• DATA_MODEL_REVIEW.md
• AI_SYSTEM_REVIEW.md
• UX_REVIEW.md
• REMASTER_PROPOSAL.md
• MIGRATION_PLAN.md
• RESEARCH_COMPETITORS.md
• REBUILD_DECISION.md — the gate