cesnimda/jobtrackingapp
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

docs(remaster): full-system audit + rebuild-vs-refactor decision

Browse Source 
Deep, code-grounded audit of Job Tracker producing the mission deliverables
under docs/remaster/: system audit, bug report, architecture/data-model/AI/UX
reviews, remaster proposal, migration plan, competitor research, and the gated
REBUILD_DECISION.

Verdict: Incremental Refactor (no full rebuild). Evidence: no Critical defects;
hardened cookie/CSRF auth (token never in JS storage), real SSRF defence,
enforced multi-tenancy via global query filters, decoupled provider-swappable
AI service, 135 backend tests. Debt is localised (god controllers/entity,
missing hot-path indexes, prompt-injection hardening, CRA build debt) and
reachable by in-place, test-guarded refactors.

Also harden .gitignore: exclude agent tooling (.claude/, .bg-shell/, .agent.md)
and restore/broaden the runtime-secrets block (**/keys/, **/backups/, exports,
CV artifacts) so nested DataProtection keys can't be committed accidentally.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
cesnimda
2026-07-05 10:04:03 +02:00
parent 657cb95a48
commit f0f178d77e
12 changed files with 795 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/remaster/SYSTEM_AUDIT_REPORT.md
+145
View File
@@ -0,0 +1,145 @@
# System Audit Report — Job Tracker (Jobbjakt)
**Date:** 2026-07-04
**Auditor role:** Principal Architect / Staff Eng / Product / UX / Security (single reviewer, code-grounded)
**Scope:** Full-system critical audit + rebuild-vs-refactor assessment.
**Verdict (see [REBUILD_DECISION.md](REBUILD_DECISION.md)):** **Incremental Refactor** — a full rebuild is *not* justified by the evidence.
> Method note: every finding below is grounded in a file/line reference or explicitly labelled
> `[Speculative issue]`. Where I could not verify behaviour, I say so. Findings are tagged
> `[Bug] [Design flaw] [Architectural weakness] [Speculative issue]` and severity-rated in
> [BUG_REPORT.md](BUG_REPORT.md).
---
## 1. Executive summary
Job Tracker is a **more mature and better-engineered system than a "rethink from scratch" framing assumes.**
The core architecture is sound: a React/TypeScript SPA, an ASP.NET Core + EF Core API with proper
multi-tenancy (global query filters on `OwnerUserId`), a pluggable job-ingestion pipeline with real
SSRF defence, hardened cookie-based auth with CSRF, deterministic (non-AI) scoring, and a decoupled
FastAPI AI service behind an HTTP contract. There are 135 backend integration tests and 23 frontend
suites, and the app is live in production (`jobs.cesnimda.uk`).
The problems are **real but localised and fixable**, not systemic rot:
1. **God classes.** `JobApplicationsController` (3,271 lines) and `ProfileCvController` (2,265 lines)
and `GmailController` (1,179 lines) concentrate far too much logic. This is the #1 maintainability
drag. **Refactorable in place** (extract services/DTOs), not a reason to rebuild.
2. **God entity.** `JobApplication` has ~40 columns mixing eight concerns, with *denormalised*
attachment booleans (`HasResume`…) that can drift from the real `Attachments` collection, and *two*
sources of tailored-CV truth (`TailoredCvText` string **and** `TailoredCvDraft` navigation).
3. **Prompt-injection surface.** Scraped job text + user CV + free-text instruction are string-interpolated
directly into LLM prompts with no delimiting. Blast radius is limited by the human-review boundary.
4. **Performance debt.** No hot-path indexes (only `OwnerUserId`), load-all-then-count analytics, and
N+1 loops in Gmail import. (This was already scoped as the Phase 7 work.)
5. **Planning drift vs `.gsd`.** An *active, never-executed* override "use next.js" (2026-04-10) conflicts
with the shipped CRA frontend; milestone numbering jumps (M001 → M005 → M011) indicate the historical
GSD plan and the built system diverged.
None of these require discarding the codebase. See §7 for the systemic-vs-fixable split.
---
## 2. Product logic audit
| Area | Finding | Tag |
|------|---------|-----|
| Job ingestion (URL) | Real pipeline: validate URL → SSRF check → fetch (4MB cap, redirect-averse) → universal parse → site-plugin fallback → language detect → NO translation. Solid. | ✅ |
| Scraping assumptions | Plugins are HTML-structure-coupled (`FinnPlugin`, `NavPlugin`, `LinkedInPlugin`, `JobbnorgePlugin`). LinkedIn/Indeed actively fight scrapers; these **will silently rot** and fall back to the universal parser or fail. No plugin-health telemetry. | [Architectural weakness] |
| Manual fallback | Exists (`AddJobModal` manual entry). Good — the product degrades gracefully when scraping fails. | ✅ |
| CV match logic | **Deterministic** keyword-coverage score (`JobCvMatchService`), *not* AI. This is the right call — reproducible, explainable, no hallucination. But keyword coverage ≈ ATS-style matching, which over-rewards literal token overlap and under-rewards semantic equivalence ("K8s" vs "Kubernetes"). | [Design flaw] |
| CV regeneration | AI rewrite via FastAPI `/cv/*`. Prompt explicitly forbids fabricating experience and analysis headings (`app.py:583-608`) — good guardrail — but nothing *enforces* factuality; the model can still invent. Output is a draft for review. | [Speculative issue] |
| Cover letter | Generated server-side, returned as draft. Consistent with the assistive-only model (D002). | ✅ |
**Product-shape observation.** `.gsd/PROJECT.md` and D003 define this as a **single-user personal
workspace**. The code has since been retrofitted to **multi-tenant** (`OwnerUserId` + query filters).
That retrofit looks correct on audited endpoints, but the *product identity* is unresolved: is this a
personal tool or a SaaS? That question drives half of the remaster decisions and should be answered
explicitly (see [REMASTER_PROPOSAL.md](REMASTER_PROPOSAL.md) §1).
---
## 3. Architecture audit
Full detail in [ARCHITECTURE_REVIEW.md](ARCHITECTURE_REVIEW.md). Summary:
- **Backend structure** — clean layering *except* the controllers, which are transaction scripts holding
business logic that belongs in services. The `JobTrackerBackend` link-compile quirk (controllers/services
glob-compiled via a separate library project) is a footgun for newcomers but works.
- **AI pipeline** — correctly decoupled behind HTTP. A provider swap (Ollama→cloud) needs zero .NET
changes. This is the single best architectural decision in the codebase.
- **Ingestion** — plugin pattern is right; plugin fragility and lack of health signals are the risk.
- **Notifications** — hosted background services (`FollowUpReminderHostedService`, `RulesHostedService`,
`JobEnrichmentHostedService`, `DailyExportHostedService`, `DatabaseBackupHostedService`). Reasonable, but
polling-based; no event bus. Fine at single-node scale.
- **Service boundaries** — blurred by the god controllers; the *services* directory is actually well-factored
(`JobCvMatchService`, `JobPipeline`, `StageAnalytics`, `EmailStatusClassifier` are all small and pure).
---
## 4. Data model audit
Full detail in [DATA_MODEL_REVIEW.md](DATA_MODEL_REVIEW.md). Headlines:
- `JobApplication` is a **god entity** (~40 columns, 8 concerns).
- **Denormalisation hazard:** `HasResume/HasCoverLetter/HasPortfolio/HasOtherAttachment` booleans duplicate
the truth in the `Attachments` collection and can silently disagree. `[Bug]` risk.
- **Dual CV truth:** `TailoredCvText` (string on the entity) and `TailoredCvDraft` (related entity, plus
`TailoredCvDraftJson`). Which wins? Ambiguity is a correctness liability.
- **CV "versioning" is not versioned.** The product promises "CV version linked to job", but the entity
stores a single current tailored text. There is no version history table. The stated product goal
(step 8, "CV version is linked to job") is **only partially supported**. `[Design flaw]`
- Indexing is minimal (`OwnerUserId` only) — a performance problem, not a modelling one.
---
## 5. AI system audit
Full detail in [AI_SYSTEM_REVIEW.md](AI_SYSTEM_REVIEW.md). Headlines:
- **Scoring validity:** deterministic, good, but keyword-literal (see §2).
- **Prompt injection:** scraped job text + user instruction are interpolated raw into prompts
(`app.py:469,527,581`). A malicious job ad can steer the CV/cover-letter output. **Severity Medium**
because output is always a human-reviewed draft and there is no tool-use/auto-send.
- **Hallucination:** guarded by prompt instructions only; no factuality verification against the source CV.
- **JD parsing reliability:** universal parser + heuristics; brittle on JS-rendered boards.
---
## 6. UX / product audit
Full detail in [UX_REVIEW.md](UX_REVIEW.md). Headlines: the daily-loop navigation (jobs → dashboard/
reminders → workspace, D004) is coherent; the CV-tailoring workspace persists reusable material; the
biggest UX risks are (a) the import-failure experience when scraping breaks, (b) the tailored-CV
save/read-back model that historically abused the free-text `notes` block (D006), and (c) no visible
"why this match score" beyond a number.
---
## 7. Systemic problems vs fixable issues
| Fixable in place (majority) | Systemic (design-level, but still refactorable) |
|---|---|
| God controllers → extract services | Product identity: personal tool vs SaaS is undecided |
| Missing indexes, N+1s, load-all analytics | `JobApplication` god entity → needs schema evolution |
| Prompt-injection hardening (delimiters) | CV "versioning" promised but not modelled |
| CRA transitive-vuln debt → Vite/Next migration | Unexecuted "use next.js" override — plan/impl divergence |
| Denormalised attachment booleans | Scraper fragility as a long-term ingestion strategy |
**Nothing in the right column requires a from-scratch rebuild.** Each is reachable by an incremental,
test-guarded refactor because the test harness (135 integration tests) locks behaviour while internals move.
---
## 8. Deliverables index
- [BUG_REPORT.md](BUG_REPORT.md) — severity-rated defects & risks
- [ARCHITECTURE_REVIEW.md](ARCHITECTURE_REVIEW.md)
- [DATA_MODEL_REVIEW.md](DATA_MODEL_REVIEW.md)
- [AI_SYSTEM_REVIEW.md](AI_SYSTEM_REVIEW.md)
- [UX_REVIEW.md](UX_REVIEW.md)
- [REMASTER_PROPOSAL.md](REMASTER_PROPOSAL.md)
- [MIGRATION_PLAN.md](MIGRATION_PLAN.md)
- [RESEARCH_COMPETITORS.md](RESEARCH_COMPETITORS.md)
- [REBUILD_DECISION.md](REBUILD_DECISION.md) — **the gate**