Files
jobtrackingapp/docs/production/production-ai-validation.md
T
cesnimda f7312f4ba6
CI and Deploy / test (pull_request) Successful in 5m35s
CI and Deploy / deploy (pull_request) Has been skipped
feat(ai): add safe benchmark harness
2026-08-15 19:24:02 +02:00

44 lines
5.4 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Production AI validation
Updated: 2026-08-15
Status: `BLOCKED`. Sanitized read-only production inventory is complete. No deployment, Ollama installation, model pull, inference/benchmark, provider call, backup/restore, restart or configuration/network change has been performed by this programme.
## Required before any production change
- documented target and access method without credential guessing or discovery scans;
- backup and tested rollback inventory;
- CPU, memory, storage, architecture and current service inventory;
- private-data routing and Pro entitlement gates implemented and verified;
- durable queue/restart recovery implemented and verified;
- bounded parser/worker resources and health checks;
- synthetic benchmark corpus and explicit acceptance thresholds;
- local-only bind/network proof for Ollama;
- canary, monitoring and rollback procedure.
PROD-001 confirms the remembered 32 GiB / GTX 1060 6GB hardware, current Ollama/model/runtime and healthy application containers. It also confirms rollout stop conditions: all-interface JobTracker Ollama/frontend listeners, unlimited container resources, an old direct-Gemini sidecar, a dirty deploy-script mode and database-only backups ending 2026-08-02. See `production-ai-hardware-assessment.md` and `production-ai-rollout-and-rollback.md`. Production state was not changed.
PROD-003 now has a tested plan-only synthetic benchmark harness and an honest empty result table in `ollama-model-benchmark.md`. No candidate was pulled or called; no primary/fallback model is selected.
BG-001 tenant-safe owner execution is implemented locally, but job enrichment remains default-off. It must not be enabled until durable operations, Pro entitlement and AI privacy policy pass their own gates; see `docs/verification/bg-001-tenant-workers.md`.
OPS-001A durable operation state is implemented locally. SQLite concurrency/migration checks pass and MariaDB DDL is generated, but no handler or worker is active and MariaDB/production execution remains blocked; see `docs/verification/ops-001a-durable-operations.md`.
OPS-001B persistent terminal notifications are implemented locally. Atomic rollback, owner isolation and SQLite migration checks pass; MariaDB DDL is generated but not executed. No notification email or worker is active; see `docs/verification/ops-001b-notifications.md`.
OPS-001C owner APIs and queue UI are implemented locally. A two-user isolated HTTP matrix and frontend build/tests pass, but browser and production smoke remain blocked. No feature producer or worker is active; see `docs/verification/ops-001c-operation-ui.md`.
POL-001 now enforces Free=no-AI with a live-role Pro policy, stable locked response, frontend locked states and worker execution rechecks. Full backend/frontend regressions pass, but incomplete cross-feature usage accounting, browser checks, Stripe lifecycle verification and production deployment keep it short of verification. Existing internal `Premium` role/config identifiers remain for rollback compatibility and are not public plan names. See `docs/verification/pol-001-free-pro-entitlements.md`.
PROD-002 provides a code-derived P0P3 workload/privacy inventory and 19-case synthetic evaluation set. Validation passes without any model or provider call. Latency values remain targets—not production measurements—and model selection remains blocked on PROD-001/003. See `docs/verification/prod-002-ai-evaluation.md`.
POL-002 now persists user AI/privacy preferences and requires independent backend/sidecar administrator gates, live Pro entitlement, AI enabled and explicit consent before `/cv/*` can use a configured external provider. The default remains local and mocked routing checks pass. No external/paid provider or production egress was exercised; durable policy snapshots, actual-provider/reason recording, payload minimization, cost controls and local-first fallback triggers remain AI-001/002 rollout gates. See `docs/verification/pol-002-ai-privacy.md`.
AI-001 adds the reusable bounded database-backed admission/worker layer over OPS-001A/B/C. It defaults to one worker and remains switched off; no real handler, model or external provider was invoked. Production activation remains blocked until AI-002 provider controls, AI-003/004 typed handlers, browser verification, MariaDB execution, monitoring and rollback/canary evidence pass. See `docs/verification/ai-001-durable-ai-queue.md`.
AI-002 now enforces sequential local-first routing, explicit task/consent/config/prompt-cost gates, a bounded process-local circuit and actual provider/model/route provenance. Backend 588/588 and sidecar fake-transport 22/22 pass. This is repository evidence only: no model/provider call or production egress occurred, the worker remains off, and PROD-001/003 plus AI-003/004 remain mandatory before any rollout. See `docs/verification/ai-002-provider-routing.md`.
AI-003 moves Strategy Snapshot generation to typed durable work and keeps GET cache-only. Automated repository evidence passes, but no selected model, browser, MariaDB, restart canary or production worker was exercised. See `docs/verification/ai-003-strategy-snapshot-queue.md`.
AI-004 moves upload/reprocess/rebuild/improve to one typed `cv.process` operation and removes the separate in-memory channel. Synthetic SQLite and UI tests pass while preserving the review gate. Parser hardening SEC-006/007, browser/private-file/model/MariaDB/restart/production gates remain incomplete; the worker remains off. See `docs/verification/ai-004-cv-processing-queue.md`.