jobtrackingapp/docs/audits/audit-progress.md

JobTracker full-application audit progress

Audit started: 2026-08-02

Overall status: Complete to all safe/local evidence boundaries; blocked checks are explicitly recorded.

Scope: repository-wide implementation, user-journey, security, privacy, supply-chain, reliability, testing, performance, and documentation audit. Application code and configuration are read-only for this audit.

Phase 1 — Repository discovery

Status: Complete

Work completed

• Captured initial Git status.
• Located and read the repository AGENTS.md.
• Began inventorying tracked source, documentation, configuration, generated output, archived material, vendor code, and auxiliary tools.
• Read current README, architecture, roadmap, TODO/blocker, environment, deployment, backup, release, package, container, CI, and AI-sidecar material.
• Traced executable entry points, authentication/authorization setup, EF ownership model, frontend routing, hosted workers, storage, integrations, and deployment topology.
• Compared documentation with the current source and classified ignored/generated/vendored paths.
• Searched production source for unfinished-code markers.

Commands executed

• git status --short --branch
• rg --files -g AGENTS.md -g '!**/node_modules/**' -g '!**/bin/**' -g '!**/obj/**'
• Get-ChildItem -Force | Select-Object Mode,Length,LastWriteTime,Name
• rg --files -g '!**/node_modules/**' -g '!**/bin/**' -g '!**/obj/**' -g '!**/.git/**' | Measure-Object | Select-Object -ExpandProperty Count
• Get-Content -Raw -LiteralPath AGENTS.md
• Repository documentation, directory, CI-workflow, and tracked-file listings using rg, Get-ChildItem, and git ls-files.
• git status --short --ignored | Select-Object -First 250
• rg -n -i ... '(TODO|FIXME|HACK|temporary|placeholder|\\bstub\\b|not implemented|NotImplementedException)' ...
• Targeted line-numbered inspection of Program.cs, JobTrackerContext.cs, controllers, services, models, frontend routes/auth/API client, package manifests, Dockerfiles, Compose, nginx, CI, and the AI sidecar.

Evidence collected

• Initial branch: release-readiness tracking origin/release-readiness.
• Pre-existing worktree changes: deleted .agent.md; untracked AGENTS.md.
• Initial top-level component and documentation listings.
• docs/audits/evidence/repository-inventory.md.

Findings recorded

• Documentation drift identified; detailed finding IDs will be assigned after cross-phase validation.

Checks that remain

• Validate build/test/tooling baseline and confirm whether documentation claims still hold.

Blockers and limitations

• No Phase 1 blocker. Ignored local copies and generated output were excluded from handwritten-code review.

Next phase

• Phase 2 — build and verification baseline.

Phase 2 — Build and verification baseline

Status: Complete

Work completed

• Classified planned commands as non-destructive; builds/tests may update ignored build output and local package caches only.
• Restored/validated declared dependencies without changing manifests or lockfiles.
• Built the .NET solution and frontend production export.
• Ran backend, frontend, AI-sidecar, and isolated Chromium suites.
• Ran TypeScript and formatting checks without rewriting source.
• Validated Compose, migration/model state, Dockerfiles, dependencies, and tracked-secret patterns.

Commands executed

• Exact commands and results are recorded in docs/audits/verification-log.md (V-001 through V-027).

Evidence collected

• Build/test outputs above plus the verification log.
• Current tracked tree and reachable-history secret-pattern scans with values suppressed.

Findings recorded

• Standalone TypeScript check failure, formatting-baseline failure, npm advisories, Python advisory volume, tracked expired token artifact, and reproducibility gaps require cross-phase validation and finding IDs.

Checks that remain

• Container image CVE scanning was unavailable locally.
• Advisory applicability and severity need source-path review.

Blockers and limitations

• gitleaks, trivy, and hadolint unavailable.
• Production/remote CI status is outside this local audit; no production system was contacted.

Next phase

• Phase 3 — architecture, backend, frontend, and data review.

Phase 3 — Architecture and code-quality audit

Status: Complete

Work completed

• Traced controller/service/data paths for jobs, career profiles, CVs, application workspaces, correspondence, attachments, AI, identity, rules, exports, and backups.
• Reviewed frontend routing, API-client use, state/error/empty flows, forms, persistence, rendering, responsiveness, and accessibility affordances.
• Executed default-SQLite paths identified as risky by source inspection.

Commands executed

• Targeted rg, Get-Content, EF model/migration inspection, and disposable endpoint/worker checks V-031 through V-033.

Evidence collected

• evidence/runtime-evidence.md, evidence/two-user-isolation.md, and line-numbered source locations used in the main report.

Findings recorded

• Confirmed default-SQLite API failures, ambiguous routes, inert tenant-scoped workers, non-atomic attachment/file operations, and client-only notification preferences.

Checks that remain

• Manual browser-dependent UX/accessibility checks remain blocked.

Blockers and limitations

• No MariaDB server was available, so provider parity beyond source/migration inspection is unverified.

Next phase

• Phase 4 — hands-on user journeys.

Phase 4 — Hands-on user journey audit

Status: Complete to the available evidence boundary

Work completed

• Ran isolated Chromium coverage for login, manual saved-job creation, Career Workspace shell, and anonymous public-CV/PDF.
• Used two synthetic accounts for empty-account, ownership, job, correspondence, CV, workspace, attachment, settings, and admin API checks.
• Classified every discovered workflow in user-journey-audit.md.

Commands executed

• V-026 and V-030 through V-040 in verification-log.md.

Evidence collected

• evidence/browser-evidence.md, evidence/runtime-evidence.md, and evidence/two-user-isolation.md.

Findings recorded

• Core Career/Application Workspace failures and accessibility/browser-regression gaps.

Checks that remain

• Manual viewport, keyboard, console/network, slow-network, multi-tab, and failure-injection journeys.

Blockers and limitations

• Mandatory in-app browser client missing; no permitted fallback and no screenshots.
• Real email/OAuth/AI/billing services intentionally not contacted.

Next phase

• Phase 5 — threat model and security audit.

Phase 5 — Threat model and security audit

Status: Complete

Work completed

• Modelled assets, roles, entry points, trust boundaries, flows, attacker capabilities, abuse cases, mitigations, and high-risk paths.
• Reviewed authentication, authorization/IDOR, sessions, OAuth/OIDC, CSRF/XSS/SSRF, uploads, CORS/headers, secrets, rate limiting, containers, and AI boundaries.
• Performed two-user direct-ID and live logout/verification lifecycle checks.

Commands executed

• V-031 and V-035 through V-037, secret scans, source searches, and official Microsoft identity-documentation lookup.

Evidence collected

• security-threat-model.md, two-user matrix, runtime evidence, and filenames-only secret evidence.

Findings recorded

• Microsoft identity binding, host-derived recovery links, verification/session gaps, parser advisories, and lower-severity SSRF/rendering hardening.

Checks that remain

• External-provider exploit reproduction was not safe/in scope; prerequisites remain explicit.

Blockers and limitations

• No aggressive testing, production contact, real provider tokens, or real email.

Next phase

• Phase 6 — technical privacy assessment.

Phase 6 — Technical privacy assessment

Status: Complete

Work completed

• Traced identity, profile/CV, job, correspondence, attachment, provider token, AI, document, log, backup, export, and deletion lifecycles.
• Separated technical controls from legal-policy questions.

Commands executed

• Targeted owner/entity/file/export/delete/provider/AI source inspection.

Evidence collected

• Privacy sections in the main report and threat model.

Findings recorded

• Incomplete admin deletion/export and missing per-user global AI control/provider-recipient explanation.

Checks that remain

• Production retention, logs, backups, processor contracts, and legal basis require operator/legal evidence.

Blockers and limitations

• Technical assessment only; production/provider contracts not accessed.

Next phase

• Phase 7 — dependencies and supply chain.

Phase 7 — Dependencies and supply chain

Status: Complete

Work completed

• Audited advisories, deprecations, version drift, locks, Docker bases, CI actions, remote installers, and licence/SBOM controls.
• Re-read advisory descriptions against actual upload/model paths.

Commands executed

• V-020 through V-025, V-028/V-029, V-041/V-042, and Dockerfile/CI inspection.

Evidence collected

• evidence/dependency-evidence.md.

Findings recorded

• Reachable document-parser denial of service, moderate React Router advisories, and reproducibility/provenance gaps.

Checks that remain

• Container package CVEs and full licence compatibility need dedicated scanners/legal review.

Blockers and limitations

• Trivy/gitleaks/hadolint unavailable; no dependency upgraded.

Next phase

• Phase 8 — reliability, deployment, and recovery.

Phase 8 — Reliability, deployment, and recovery

Status: Complete

Work completed

• Reviewed Compose/Dockerfiles, health/startup, shutdown, resources, migrations/reconciliation, deploy/rollback, logging, metrics, workers, partial failure, and backups.
• Rehearsed SQLite database and full-data-root restoration with disposable data.

Commands executed

• V-016/V-017/V-023 through V-025/V-033/V-034 plus deployment-source inspection.

Evidence collected

• Restore results in evidence/runtime-evidence.md.

Findings recorded

• SQLite restore passes; files/keys/config are separate; MariaDB needs external backup; no RPO/RTO or routine restore proof; workers fail silently; startup reconciler is risky complexity.

Checks that remain

• MariaDB restore, production rollback, restart/resource pressure, and monitoring delivery.

Blockers and limitations

• No production deployment, registry, remote host, or MariaDB instance used.

Next phase

• Phase 9 — testing assessment.

Phase 9 — Testing assessment

Status: Complete

Work completed

• Mapped backend, frontend, Python, and browser tests to core journeys and observed defects.
• Reviewed determinism, isolation, authorization, failure paths, accessibility, and CI gates.

Commands executed

• V-010/V-012/V-014/V-026 plus test-file and CI-workflow inventories.

Evidence collected

• Test mapping in the main and journey reports.

Findings recorded

• Missing route-table, default-SQLite HTTP, worker-context, account-lifecycle, accessibility, and Python CI gates.

Checks that remain

• Remote CI execution status was not queried.

Blockers and limitations

• Raw line coverage was not used as proof of quality.

Next phase

• Phase 10 — performance assessment.

Phase 10 — Performance assessment

Status: Complete to safe-local scope

Work completed

• Measured warm local API latency and aggregate export size; inspected pagination, query patterns, upload buffering, worker sequencing, and admin N+1 behaviour.
• Separated measured results, clear inefficiencies, measurement-needed risks, and optional optimisation.

Commands executed

• V-038/V-039 and query-loop/pagination inspection.

Evidence collected

• evidence/runtime-evidence.md performance table.

Findings recorded

• Small-data timings healthy; pre-limit buffering clearly inefficient; larger-data/browser capacity unverified.

Checks that remain

• Production-like transfer, memory, query counts, AI latency, email throughput, and large datasets.

Blockers and limitations

• No load test; browser performance tooling blocked.

Next phase

• Phase 11 — documentation and developer experience.

Phase 11 — Documentation and developer experience

Status: Complete

Work completed

• Compared feature/setup/architecture/migration/test/deploy/API claims with source and runtime; evaluated clean onboarding.

Commands executed

• Phase 1 documentation inventory plus toolchain/build/runtime verification.

Evidence collected

• Repository inventory and main-report documentation section.

Findings recorded

• Obsolete CRA README, unsupported PostgreSQL advice, stale API architecture, no global.json, incomplete environment reference, and stale CI comments.

Checks that remain

• Operator-only documentation may exist outside the repository.

Blockers and limitations

• External documentation not accessed.

Next phase

• Phase 12 — sceptical validation.

Phase 12 — Sceptical validation

Status: Complete

Work completed

• Re-read every Critical/High candidate end to end, searched mitigations, repeated safe reproductions, checked prerequisites, and merged/downgraded overlap.
• Separated parser reachability from fixed-model loader advisories.
• Confirmed no cross-user disclosure in meaningful two-user results.

Commands executed

• V-031 through V-043, official Microsoft identity guidance lookup, source rereads, and final cleanup/status checks.

Evidence collected

• All deliverables and evidence under docs/audits/.

Findings recorded

• No Critical finding. High findings retain explicit prerequisites; unperformed external exploits remain labelled unverified.

Checks that remain

• Only blocked/production/external checks listed in the reports.

Blockers and limitations

• Missing browser client, no MariaDB, no container CVE scanner, no production/provider access.

Next phase

• Stop after delivery and await remediation approval.

Post-audit programme execution — POL-002

Status: Implemented; browser/production verification incomplete (2026-08-03).

Work completed

• Revalidated AI privacy/provider paths and implemented server-persisted AI opt-out plus explicit external-processing consent.
• Added independent backend/sidecar administrator gates and a deny-by-default permission header at the shared /cv/* boundary.
• Added Settings UI, additive migration, architecture/verification documentation and synthetic-only tests.

Commands and evidence

• Verification-log entries V-089 through V-095.
• docs/verification/pol-002-ai-privacy.md and docs/audits/evidence/pol-002/README.md.

Findings / limitations

• No production, paid provider, real private data or browser was used.
• Historical EF-only clean SQLite migration remains blocked before the new migration; the new SQL/defaults and model snapshot pass inspection.
• AI-001/002 must add durable policy snapshots, actual provider/reason recording and bounded local-first fallback before rollout.

Next phase

• AI-001 durable AI queue/worker admission, reusing OPS-001A/B/C.

Post-audit programme execution — AI-001

Status: Implemented; real-handler/browser/production verification incomplete (2026-08-03).

• Work completed: shared Pro/privacy admission, bounded capacity/priority, typed default-off worker, owner/policy recheck, heartbeat/timeout/retry/cancellation integration and configuration.
• Commands/evidence: verification-log V-096/V-097; docs/verification/ai-001-durable-ai-queue.md.
• Findings: no new schema/raw private queue payload; multi-replica capacity needs a future database reservation; actual 202 producers and provider controls remain AI-003/004 and AI-002.
• Blockers: browser, MariaDB and production unavailable; worker intentionally off.
• Next phase: AI-002 Ollama adapter and central local-first routing.

Post-audit programme execution — AI-002

Status: Implemented; browser/model/provider/production verification incomplete (2026-08-09).

• Work completed: revalidated every AI endpoint/caller; replaced direct configured-provider dispatch with one sequential local-first router; added task/consent/config/prompt-cost gates, bounded circuit/health state, typed sanitized failures and actual provider/model/route persistence for AI history and durable operations.
• Commands/evidence: verification-log V-098–V-100; docs/verification/ai-002-provider-routing.md; docs/audits/evidence/ai-002/README.md.
• Findings: no schema/dependency change and no provider race. New durable tasks fail safe to local until allowlisted. Per-request prompt ceiling exists, but complete monthly cross-feature accounting remains incomplete. Circuit state is process-local for the current single-sidecar design.
• Checks that remain: real AI-003/004 handlers/producers; selected-model benchmark; browser disclosure; MariaDB; controlled synthetic external fallback; production health/restart/canary/rollback.
• Blockers and limitations: no browser, production access, model benchmark or provider authority/configuration; no real/private input used.
• Next phase: AI-003 Strategy Snapshot durable-operation migration.

Post-audit programme execution — AI-003

Status: Implemented; browser/model/MariaDB/restart/production verification incomplete (2026-08-09).

• Work completed: replaced synchronous Focus Plan generation with a typed strategy.snapshot producer/handler on the shared durable queue; GET is cache-only; added owner-scoped rehydration, one bounded structured model request, full-response validation, idempotent active work, resume lookup, provenance and explicit UI queue/cancel/retry/failure/completion states.
• Commands/evidence: verification-log V-101–V-103; docs/verification/ai-003-strategy-snapshot-queue.md; docs/audits/evidence/ai-003/README.md; commit a621226.
• Findings: root code path was four sequential model calls in an HTTP GET, sometimes alongside candidate fit. Authorization/tenant filters existed. No schema/dependency change was needed. Strategy remains local-only and the worker remains default-off.
• Checks that remain: real browser responsive/theme/keyboard/refresh checks; selected-model timeout/quality; MariaDB; real restart recovery; production telemetry/canary/rollback; cross-feature usage accounting.
• Blockers and limitations: browser policy, production access and model/MariaDB environments unavailable; fake model/synthetic data only.
• Next phase: AI-004 CV-processing 504 durable-operation migration.