feat(ai): add safe benchmark harness

2026-08-15 19:24:02 +02:00
parent 842e793f69
commit f7312f4ba6
14 changed files with 793 additions and 51 deletions
@@ -206,3 +206,6 @@ Output was reduced to filenames and commit counts. The token artifact appears un
 | V-172 | Action-matrix reconciliation; full backend/frontend/sidecar/build/Compose/preflight gates; expanded authenticated and anonymous Playwright | Repository root / `job-tracker-ui` / `tools/summarizer` | Complete VER-001 local release regression without promoting mocked/provider/external checks | PASS — backend 647/647, frontend 57 suites/232 tests, sidecar 22/22, production build, Compose config, API-down/wrong-base/malformed-JSON preflight and Chromium 9/9. Browser covers admin deployment identity/normal-user absence, notifications, honest Free, jobs, Career/CV, Kanban, responsive themes and public PDF | No external provider, private data, native AT or production mutation. Optional Compose variables remain unset; existing Jest/GSI/SWIG warnings remain. Windows CRLF materialization was normalized for shell execution; indexed LF policy was already correct | VER-001 verified locally; remote CI/provider/native-AT/production cells remain |
 | V-173 | Owner-path trace; focused CV/export/controller/background tests; full backend; build and diff hygiene | Repository root | Establish attributable generated-file ownership before SEC-009 export/deletion | PASS — CV PDFs use opaque owner/date/UUID storage while preserving download names; daily exports use opaque owner directories and atomic writes; legacy/new retention paths are covered. Focused 77/77 and backend 647/647 | No existing file moved or deleted. Legacy shared-date generated files are intentionally not guessed. No migration, production path or private data used | SEC-009 owner-scoped generated-output prerequisite verified locally |
 | V-174 | Real-SQLite two-owner export fixture; manifest/checksum/file/redaction assertions; recent-session/rate-limit API tests; focused/full frontend and backend; optimized build; Chromium ZIP response | Repository root / `job-tracker-ui` | Deliver a complete user-readable export without exposing secrets or another tenant | PASS — focused backend/API 11/11, backend 650/650, frontend focused 4/4 and full 58 suites/234 tests, builds pass. Every manifest checksum/size matches; owned attachment/CV/avatar/generated/daily files are included; secret and other-owner sentinels are absent; Chromium receives HTTP 200 `application/zip` with `PK` signature | Synthetic data/files only; no production/private/provider access. Export reports backups/logs/external retention instead of claiming erasure. One initial InMemory-only test missed SQLite DateTimeOffset translation; the test moved to real SQLite and the query boundary was corrected | SEC-009 readable export verified locally; deletion/retention/restore remain |
+| V-175 | Account-deletion threat-path review; real-SQLite lifecycle/controller tests; focused/full backend/frontend; optimized build; EF parity and MariaDB script generation; disposable Chromium startup/application suite | Repository root / `job-tracker-ui` | Deliver a production-inert, owner-isolated, retryable live-account deletion lifecycle without implying backup/provider erasure | PASS — focused backend 21/21, backend 657/657, frontend focused 8/8 and full 58 suites/237 tests, optimized build, no pending EF changes, bounded MariaDB script and Chromium 9/9. Tests prove disabled gate, exact/recent confirmation, immediate lockout, idempotence, owner-isolated row/file purge, quarantine failure safety, tombstone creation and restored-account replay; `842e793` pushed | Synthetic local rows/files only. No live deletion, provider revocation, sidecar restart, backup restore, production migration or retention decision occurred. Self/admin requests remain disabled by default. Full every-provider/disposable-production rehearsal remains external | SEC-009 repository scope implemented; production activation remains blocked |
+| V-176 | Authorized sanitized read-only SSH inventory of OS/CPU/RAM/GPU/storage/Docker/Ollama/app health/networks/selected non-secret configuration and backup metadata/integrity | Production host / repository report | Replace stale guessed hardware/access assumptions and define evidence-based rollout stops without changing production | PASS/PARTIAL — confirmed Ubuntu 24.04.4, i5-8600 6C, 31 GiB RAM, GTX 1060 6GB, 1.4 TiB Docker-volume headroom, Ollama 0.31.1 with `qwen2.5:7b`, four healthy app containers and exact deployed version. Also confirmed all-interface 11434/3000 listeners, unlimited container resources, direct Gemini selection in the old sidecar, dirty deploy-script mode, and 21 gzip-valid database-only backups ending 2026-08-02 | No secret values, logs, prompts, private rows/content, provider call, inference, model pull, file read beyond metadata/integrity, service restart, backup creation, restore or mutation. Internet/NAT exposure, logical restore and complete file/key recovery remain unverified | PROD-001 inventory/report complete; safety acceptance blocked by measured network/backup/deployment gaps |
+| V-177 | Python unit tests and plan-only execution against the checked-in synthetic fixture; script/content/SSRF/report review | Repository root | Prepare reproducible privacy-safe PROD-003 evaluation without authorizing a model or network call | PASS — harness tests 4/4; plan validates four Strategy cases across 4K/8K for eight future requests. Default performs no HTTP; exact model, `--execute` and explicit output are required; model pull/delete is absent; report excludes raw fixture/prompt/output content | No Ollama request, provider/internet access, model metadata query through the harness, candidate pull, inference or production change. Candidate metadata/licenses/results and model decision remain unmeasured | PROD-003 repository harness complete; execution remains blocked |