cesnimda/jobtrackingapp
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

feat: complete release readiness work

Browse Source 
- consolidate API ownership and remove dead vendor code

- add Stripe billing, learning paths, and public CV hardening

- add migration, recovery, security, audit, and browser gates
This commit is contained in:
cesnimda
2026-07-31 16:54:16 +02:00
parent a23c3dfc97
commit ce76046a29
1634 changed files with 6889 additions and 135429 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/architecture/current.md
+36 -51
View File
@@ -1,10 +1,11 @@
# Jobjakt — Current Architecture
> **This document describes the system as it actually is.** Every claim was verified against code.
> Last verified: 2026-07-17 (Phase 0). Supersedes the archived `docs/_archive/SYSTEM_OVERVIEW.md` (2026-07-02).
> Last verified: 2026-07-31. Supersedes the archived `docs/_archive/SYSTEM_OVERVIEW.md` (2026-07-02).
>
> **Rule:** if this document and the code disagree, the code wins — and this document is a bug. Fix it.
> Do not trust other files under `docs/` over this one; most are stubs.
> Topic documentation is maintained alongside this file; dated completion and review reports remain
> historical snapshots. When documentation conflicts, prefer this file and the code.
---
@@ -69,22 +70,19 @@ flowchart LR
BG --> DB
```
### Solution layout (unusual — read this first)
### Solution layout
| Project | Role |
|---|---|
| `JobTrackerApi/` | Web **host only**: `Program.cs`, appsettings, migrations, Dockerfile. Its csproj **excludes** `Controllers/**` and `Services/**` from its own compilation. |
| `JobTrackerBackend/` | "Transitional shared-backend" **library** that link-compiles, via `<Compile Include>`, files physically located in `../Data`, `../Models`, `../JobTrackerApi/Controllers`, `../JobTrackerApi/Services`. Exists so tests can reference controllers/services without the web host. |
| `JobTrackerApi/` | ASP.NET Core host plus its controllers, services, EF models, `JobTrackerContext`, migrations, and Dockerfile. |
| `JobTrackerApi.Tests/` | xUnit, 36 test files incl. authorization + hostile-fixture suites. |
| `Models/`, `Data/` (repo root) | The *real* EF models and `JobTrackerContext`. |
| `job-tracker-ui/` | React SPA inside a Next.js shell. |
| `tools/summarizer/` | FastAPI AI service (own Dockerfile, pytest tests). |
| `tools/hostile-fixture-db/` | Security test fixture generator. |
| `deploy/`, `.gitea/workflows/` | Prod deploy script + CI/CD. |
**Source lives in one place and compiles from another.** Any tool assuming csproj-adjacent source will mislead you. Check `JobTrackerBackend.csproj` before adding files or projects.
> Corrected 2026-07-17: the dead root `Controller/` (singular) folder described in the archived overview **no longer exists** — removed in `519c32e`.
> Corrected 2026-07-31: the transitional `JobTrackerBackend` link-compilation project and root
> `Models/`/`Data/` directories were retired. Source now compiles from the project that owns it.
---
@@ -94,7 +92,7 @@ flowchart LR
**Frontend:** **Next.js 16** + React 19 + **TypeScript 5.9** + MUI 7 (+ x-data-grid, x-date-pickers, lab), axios, react-router-dom 6, @tanstack/react-table, i18n EN + NB (custom provider), Jest/RTL.
> Corrected 2026-07-17: the archived overview said "CRA/react-scripts 5, TypeScript 4.9". The CRA→Next.js migration has happened. See §4 for what that migration did and did not do.
> Corrected 2026-07-31: the CRA migration is complete; direct Jest/Babel configuration replaced `react-scripts`.
**AI:** FastAPI + transformers (`sshleifer/distilbart-cnn-12-6`) for summaries; pytesseract/PyMuPDF/pypdf/python-docx for extraction/OCR; **one** generation provider selected by the `AI_PROVIDER` env var ∈ {`ollama` (default, `qwen2.5:7b`), `gemini`, `groq`}; TTL cache.
@@ -104,17 +102,16 @@ flowchart LR
## 4. Frontend architecture
**Three toolchains coexist. This is the single most confusing thing about the frontend.**
**Two routing layers coexist intentionally:**
1. **Next.js 16 App Router** (`app/layout.tsx`, `app/page.tsx`) — a thin shell that mounts a client-side app. The CRA→Next migration was a **CSR lift-and-shift**: no SSR, no server components, no Next routing, no data fetching. Next is effectively a build tool here. Static export → nginx.
2. **react-router-dom v6** — does the actual routing, in **two different patterns inside one file** (`src/App.tsx`): `createBrowserRouter` for public routes (`/`, `/login`, `/forgot-password`, `/reset-password`, `/verify-email`) and a nested `<Routes>` inside a catch-all `Shell` for authenticated routes.
3. **react-scripts 5.0.1** — still a dependency, used **only** as the test runner (`"test": "react-scripts test"`).
Known consequence: a **dev-only 404 on deep links** follows directly from the Next shell + client router combination.
Development leaves static-export mode disabled so deep links reach the client router; production exports one shell and nginx falls back to `index.html` for unknown paths.
**Routes** (`src/App.tsx`): public — `/`, `/login`, `/forgot-password`, `/reset-password`, `/verify-email`. Authenticated — `/dashboard`, `/jobs`, `/reminders`, `/kanban`, `/companies`, `/correspondence`, `/correspondence/review`, `/profile`, `/career`, `/trash`, `/settings`, `/settings/connected-accounts`, `/admin/{audit,users,system}`.
> **No `/register` route exists.** Sign-up is folded into `LoginPage.tsx`, and the endpoint is disabled by default (§5).
> `/register` reuses the hardened auth form. Submission remains disabled until registration is enabled by production configuration (§5).
**State management: none.** No Redux/Zustand/React Query. Local `useState` + `axios` per component, with a hand-rolled `refreshToken` counter threaded through props. Two workspace-cache hooks exist (`components/job-workspace/useWorkspaceTabCache.ts`, `useJobWorkspaceBaseData.ts`). This is the root cause of the oversized components below.
@@ -138,7 +135,7 @@ the reference model for how account identity and the master career profile relat
| `/career` | `views/CareerWorkspacePage.tsx` → `views/CareerProfilePage.tsx` (~1293 lines) | The **master career profile** — the single editable source of truth |
`CareerWorkspacePage` is a thin shell (heading + source-of-truth notice) around `CareerProfilePage`.
The CV Builder is **not** built yet (Phase 4); `CareerProfilePage` is where it will live.
The CV Builder is a separate routed workspace and consumes the Career Profile as its source of truth.
### Request flow
@@ -209,7 +206,7 @@ null-leaves).
- **Sessions:** `UserSession` entity + `SessionsController` — list/revoke active sessions.
- Password policy: min 8, digit + lowercase. Reset via emailed token (SMTP required).
- **Registration is disabled by default** — `AuthController.cs:135` reads `Auth:AllowRegistration` defaulting to `false` and returns HTTP 403. There is no CAPTCHA anywhere.
- **Rate limiting (3 fixed-window policies):** `auth-login` 10/window, `auth-email` 5/window, `auth-2fa-challenge` 5/window. **AI and other expensive endpoints are unthrottled.**
- **Rate limiting:** login, auth email, 2FA challenge, and anonymous public-CV PDF export have dedicated fixed-window policies. AI usage is bounded by per-account monthly generation and token ceilings rather than request-window throttling.
---
@@ -246,12 +243,12 @@ erDiagram
- Salary is **structured**: `SalaryMin`, `SalaryMax`, `SalaryCurrency`, `SalaryPeriod` (plus a legacy free-text `Salary`).
- `Tags` is a **JSON array in a string column** — not queryable; `/tags` and `/tag-trends` must scan.
- Denormalized `HasResume`/`HasCoverLetter`/`HasPortfolio`/`HasOtherAttachment` flags duplicate `Attachments`; kept honest by `AttachmentFlagsRecomputeTests`.
- CV text is stored **three times** (`CvExtractionRun.RawExtractedText`, `.NormalizedText`, `.StructuredProfileJson`) plus twice on the user. Deliberate audit trail, but **no retention policy**.
- CV extraction runs retain the newest 20 completed runs per user; expired runs and unreferenced upload artifacts are pruned while the current artifact is preserved.
- `Status` is free-text at the DB level; canonicalized only in the application layer by `JobPipeline.Normalize` — deliberately, so custom user values are never destroyed.
- Indexes: `OwnerUserId` on Company/Job/JobApplication/GmailConnection; composites `(OwnerUserId, UploadedAtUtc)`, `(OwnerUserId, StartedAtUtc)`; unique `(OwnerUserId, JobApplicationId)` on draft, unique `(OwnerUserId, GmailAddress)`. EF auto-indexes FKs by convention. **Genuinely missing:** owner-prefixed composites `(OwnerUserId, IsDeleted, Status)` and `(OwnerUserId, FollowUpAt)`.
- SQLite at `DataRoot/jobtracker.db` (WAL); migrations applied at startup by `StartupInitializationExtensions` (1356 lines — also seeds admin, creates Identity tables where `dotnet ef` is unavailable, ignores `PendingModelChangesWarning`).
- Indexes include owner-prefixed list/board/reminder composites `(OwnerUserId, IsDeleted)`, `(OwnerUserId, IsDeleted, Status)`, and `(OwnerUserId, FollowUpAt)`, plus CV and provider-specific indexes.
- SQLite lives at `DataRoot/jobtracker.db` (WAL); migrations and legacy-schema reconciliation run at startup through `StartupInitializationExtensions`.
> Corrected 2026-07-17: prior session notes recorded the EF model snapshot as **broken/empty**. It was **resynced** in `20260711181039_SyncModelSnapshot` — the snapshot now covers the full model and incremental `dotnet ef migrations add` works normally. Note `dotnet ef` still needs the `Design` package temporarily added to `JobTrackerApi` (the `MigrationsAssembly`), since it lives in `JobTrackerBackend` with `PrivateAssets=all`.
> Corrected 2026-07-31: the model snapshot is current and the API directly carries the EF Design package and `JobTrackerContext`; no temporary project edit is required for `dotnet ef`.
---
@@ -294,10 +291,10 @@ erDiagram
| `DailyExportHostedService` | Daily JSON export at a configured local hour |
| `JobEnrichmentHostedService` | Backfills summaries/enrichment |
| `SummarizerProbeHostedService` | Probes AI service readiness |
| `CvProcessingHostedService` + `CvProcessingQueue` | In-memory queue for CV extraction |
| `CvProcessingHostedService` + `CvProcessingQueue` | Process-local wake-up queue for CV extraction; queued/running database work is recovered at startup |
| `DatabaseBackupHostedService` → `DatabaseBackupRunner` | Automated DB backup (`VACUUM INTO`, server-derived path) |
**All state is in-process** (`IMemoryCache`, in-memory queue) — single-instance assumption, no distributed locks, **queued CV jobs are lost on restart**.
Caches and worker coordination are process-local. CV work itself is durable and recovered after restart, but the worker remains a deliberate single-instance design without database leasing.
---
@@ -361,15 +358,15 @@ Inbound: `GmailOAuthService` (655), `MicrosoftGraphOAuthService` (507), `ImapSer
- **Backend:** xUnit integration-style via `TestHostFactory`. 36 test files. Notable: `JobApplicationsAuthorizationTests`, `OwnershipGuardTests`, `ImapServiceSsrfGuardTests`, `ProductionConfigTests`, `AttachmentFlagsRecomputeTests`, `CvCorpusHarnessTests`, `SqliteMigrationHelperTests`, `JobPipelineTests`, plus a `tools/hostile-fixture-db` project.
- **Frontend:** ~20 Jest/RTL files — **all run in CI**.
- **AI service:** pytest (`tools/summarizer/tests/`).
- **Gaps:** no true end-to-end browser tests; no load/perf tests; **no dependency CVE scanning** (CI explicitly sets `npm_config_audit: 'false'`).
- **Browser smoke:** Playwright drives login/session cookies, saved-job creation, Career Workspace routing, and anonymous public-CV rendering/PDF download against isolated API/SQLite and Next.js processes. CI installs Chromium and runs all four flows.
- **Gap:** no load/performance suite. CI reports NuGet transitive vulnerabilities and production npm audit findings.
---
## 14. Logging & error handling
Console/debug logging; middleware logs every request (method, path, status, ms, traceId, sub claim). Unhandled exceptions logged then rethrown (500). Client errors POST to `/api/client-errors`. React `ErrorBoundary` + route error page.
No structured sink (Seq/OTLP), no in-app log rotation, no ProblemDetails standardization.
Development uses simple console/debug logging; production emits structured JSON. Middleware records method, path, status, duration, trace ID, and subject. Unhandled errors return Problem Details with the same trace ID. Client errors POST to `/api/client-errors`; the frontend has an `ErrorBoundary` and route error page.
Compose bounds each container's local logs to three 10 MB files. There is still no external sink (Seq/OTLP) or cross-host aggregation.
---
@@ -385,17 +382,17 @@ No structured sink (Seq/OTLP), no in-app log rotation, no ProblemDetails standar
- 2FA + recovery codes + trusted devices + session revocation.
- **AI sidecar: backend-only.** Unpublished, on a private two-member network, and token-authenticated (§16). Verified against the running stack, not just configured.
**Open findings** (detail in `docs/application-discovery-report.md` §12 and `docs/phase-0-foundation-report.md`):
**Findings status** (detail in `docs/application-discovery-report.md` §12 and `docs/phase-0-foundation-report.md`):
| Sev | Finding | Status |
|---|---|---|
| Medium | DataProtection keys recoverable from git history (`519c32e`, `955cae6`) | **Open — rotation required, needs an operator** |
| Medium | **CORS: `Cors:Origins="*"` triggers `SetIsOriginAllowed(_ => true)` + `AllowCredentials()`** (`Program.cs:96-102`) — reflected-origin with cookies = session theft from any site. Not currently active (compose never sets `Cors__Origins`, so it defaults to `localhost:3000`), but it is one config value away. | **Open — landmine** |
| Medium | AI cost ceiling | Metering shipped in Phase 5; enforce quotas before open registration (Phase 7). |
| Low | No CAPTCHA (rate limiting only) | Open — blocks public signup |
| Low | Unbounded storage: attachments, CV artifacts, extraction runs, base64 avatars | Open |
| Medium | Wildcard credentialed CORS configuration | **Closed — startup rejects it** |
| Medium | AI cost ceiling | **Closed — monthly generation/token limits are enforced by plan** |
| Low | Public-registration abuse control | Implemented with Turnstile; production keys/configuration still required |
| Low | Unbounded storage | **Closed — attachment quotas, extraction/artifact pruning, file-backed avatars, and PDF-export retention are implemented** |
| Low | Backup / DPAPI is Windows-oriented — verify behaviour on Linux prod | Unverified |
| Low | No dependency CVE scanning in CI | Open |
| Low | No dependency CVE scanning in CI | **Closed — NuGet and production npm audit reporting are in CI** |
---
@@ -417,20 +414,9 @@ Full record: `docs/phase-0-foundation-report.md`. What changed architecturally:
---
## 17. Known debt (ranked)
## 17. Known debt
1. **`JobApplication` still carries opportunity columns** — the `Job` split is started but not completed. Reads/writes still use the legacy columns.
2. **God controllers** — `JobApplicationsController` 2313/38 endpoints, `ProfileCvController` 2249, `StartupInitializationExtensions` 1356, `GmailController` 1023, `AuthController` 879.
3. **Hardcoded CV templates** — dead end for the CV Builder.
4. **Three frontend toolchains** — Next shell + react-router (×2 patterns) + react-scripts test runner.
5. **No frontend data layer** — root cause of the 600–1400-line components.
6. **`ProfilePage` (1368 lines) serves two routes** behind a boolean; the `careerView` prop it accepts is **never read** (the "CV Builder" tab is inert).
7. **Denormalized `Has*` flags**; **`Tags` as a JSON string column**; **unbounded CV storage**; **base64 avatars in a DB column**.
8. **`JobTrackerBackend` link-compilation** — self-described "transitional".
9. **In-memory queue/cache** — single-instance coupling; restart loses queued CV jobs.
10. **No OpenAPI in prod, no ProblemDetails, no structured logging sink.**
11. **Root-level clutter**: `temp_job.json`, `temp_post_job.py`, `todo jobtracker.txt`, `test/`, `tmp/`, `vendor/`, `docs.7z`, `CV_Changes.md`, `SMART_GMAIL_PROGRESS.md`.
12. **Norway-only import plugins** (Finn/NAV/Jobbnorge). Product decision 2026-07-17: **Norway first, but no hardcoding Norway** — market must become a data dimension, not an assumption.
The live, prioritized ledger is `docs/architecture/technical-debt.md`. The principal remaining items are the legacy `JobApplication` opportunity columns, single-replica worker coordination, and optional cross-host log aggregation. Large orchestration files are refactored only when a cohesive behavior change provides a safe seam.
---
@@ -438,9 +424,8 @@ Full record: `docs/phase-0-foundation-report.md`. What changed architecturally:
Recorded nowhere else in active docs:
1. `JobTrackerBackend` link-compilation exists so tests can reach controllers without the web host — deliberate, self-described "transitional".
2. `Status` is free-text and canonicalized in the application layer **specifically so custom user values are never destroyed** (`JobPipeline.cs` docstring). Deliberate; do not "fix" it with a DB enum.
3. The CI frontend-test whitelist was removed after it "silently skipped new suites and let two regressions reach main."
4. Ollama is intentionally not bundled by default so deploys reuse a shared instance.
5. `AI_PROVIDER=gemini` exists to offload a weak local GPU in prod.
6. `TailoredCvDraft` is a separate entity specifically to guarantee the master CV is never auto-modified.
1. `Status` is free-text and canonicalized in the application layer **specifically so custom user values are never destroyed** (`JobPipeline.cs` docstring). Deliberate; do not "fix" it with a DB enum.
2. The CI frontend-test whitelist was removed after it "silently skipped new suites and let two regressions reach main."
3. Ollama is intentionally not bundled by default so deploys reuse a shared instance.
4. `AI_PROVIDER=gemini` exists to offload a weak local GPU in prod.
5. `TailoredCvDraft` is a separate entity specifically to guarantee the master CV is never auto-modified.