feat(ai): enforce local-first routing

Keep external providers behind server consent, task, and prompt-cost gates while persisting actual provider provenance.
This commit is contained in:
cesnimda
2026-08-09 12:30:11 +02:00
parent c3f4a57195
commit 5eb9b3cb96
29 changed files with 967 additions and 145 deletions
+10
View File
@@ -279,3 +279,13 @@
- **Consequences:** AI-003/004 only add typed handlers/producers. Current capacity serialization is process-local for the documented single-backend deployment; database reservation is required before multiple backend replicas. Provider/model semaphores and circuit/provenance remain AI-002 responsibilities.
- **User approval required:** No; this follows both programmes' explicit instruction to reuse the smallest reliable existing infrastructure.
- **Reversible:** Yes. Keep the worker switch false, remove admission/worker registrations, and retain operation rows/API history. No new schema was added in this slice.
## DEC-029 — One sequential sidecar router owns local-first fallback
- **Date:** 2026-08-09
- **Decision:** Keep provider execution behind the existing sidecar boundary, make Ollama the default primary, permit at most one sequential external fallback, and carry the backend's rechecked privacy/task decision through the AI-001 execution scope. Reuse existing operation provider/model/progress fields for provenance; add no queue/provider schema or dependency.
- **Reason/evidence:** every generative `/cv/*` path already converges on one `_provider_generate` family, while deterministic tasks and `/summarize` must remain local. Sidecar fake-transport tests prove local success, consent/config/task/cost denials, schema fallback, circuit behavior, external failure and no parallel duplicate call. Backend tests prove policy propagation and success/failure provenance.
- **Alternatives considered:** provider selection in each controller; browser-selected providers; a second provider abstraction in .NET; simultaneous local/cloud racing; a new circuit/attempt table; increasing synchronous timeouts. These scatter policy, expose authority, duplicate the established boundary, risk double charge/output, add unneeded schema, or mask the queued-operation root cause.
- **Consequences:** `AI_ROUTING_MODE` supports `local_only`, `local_first` and explicitly gated `external_only`; invalid values fail closed. New durable task IDs stay local until allowlisted. The current circuit is process-local and one AI worker is the effective single-model concurrency limit. Per-request prompt size limits external spend/exposure, but complete monthly cross-feature accounting and model selection remain rollout gates.
- **User approval required:** No; this directly implements the approved local-first programme without invoking a provider or production service.
- **Reversible:** Yes. Set `EXTERNAL_AI_ENABLED=false` or `AI_ROUTING_MODE=local_only`; the older `AI_PROVIDER`/model configuration is retained. Existing nullable operation fields and AI history remain readable.
+9 -8
View File
@@ -1,21 +1,21 @@
# JobTracker master programme progress
Updated: 2026-08-03
Updated: 2026-08-09
- **Overall programme status:** Active. Six packages are locally verified; eleven packages including POL-001/002 and AI-001 are implemented with automated/runtime evidence but blocked from later browser/provider/production gates; AI-002 is now in progress.
- **Current work package:** `AI-002`Ollama adapter and local-first provider routing (`IN PROGRESS`), extending the POL-002 boundary and AI-001 execution context.
- **Overall programme status:** Active. Six packages are locally verified; twelve packages through AI-002 are implemented with automated/runtime evidence but blocked from later browser/provider/production gates; AI-003 is now in progress.
- **Current work package:** `AI-003`Strategy Snapshot durable-operation migration (`IN PROGRESS`), reusing AI-001 operations and AI-002 routing rather than creating a Strategy-specific queue.
- **Completed work packages:** None are `DONE`; all repository security packages still have applicable browser, provider and/or production gates.
- **Locally verified work:** SEC-001, SEC-002, SEC-003, SEC-005A, CORE-001 and PROD-002 (`VERIFIED LOCALLY`).
- **Implemented, verification incomplete:** SEC-004, SEC-005B, SEC-008, CORE-002, BG-001, OPS-001A/B/C, POL-001/002 and AI-001 (`IMPLEMENTED — NOT VERIFIED`): foundations through durable bounded AI execution pass local checks; real handlers, browser, provider, accounting and production gates remain.
- **Implemented, verification incomplete:** SEC-004, SEC-005B, SEC-008, CORE-002, BG-001, OPS-001A/B/C, POL-001/002, AI-001 and AI-002 (`IMPLEMENTED — NOT VERIFIED`): foundations through bounded local-first routing pass local checks; real producers, browser, selected-model/provider, accounting and production gates remain.
- **Production-verified work:** None.
- **Blocked work:** SEC-006 requires explicit internet/package-index permission; PROD-001/003/004 and REL-001 require documented production access and unfinished dependencies. Browser access was denied by the browser admin policy check; SMTP/MariaDB environments are unavailable.
- **Deferred work:** None. Conditional large abstractions, model deletion, multi-replica coordination and unrelated production changes remain outside current packages.
- **Next five work packages:** AI-002 Ollama adapter/routing; AI-003 Strategy Snapshot queue migration; AI-004 CV processing queue migration; UX-001 authentication/theme corrections; QA-001 job-analysis/keyword quality. SEC-006/007 resume after package-index permission.
- **Status counts:** 6 `VERIFIED LOCALLY`; 11 `IMPLEMENTED — NOT VERIFIED`; 1 `IN PROGRESS`; 16 `NOT STARTED`; 5 `BLOCKED`; 0 `DONE`; 0 `DEFERRED`.
- **Test status:** backend 581/581; AI-001 focused queue/state/API 17/17; frontend 47/47 suites and 158/158 tests plus build; Python sidecar 18/18. Prior POL/PROD/OPS/BG/SEC/CORE evidence remains green.
- **Next five work packages:** AI-003 Strategy Snapshot queue migration; AI-004 CV processing queue migration; UX-001 authentication/theme corrections; QA-001 job-analysis/keyword quality; UX-002 Career Workspace/CV Builder redesign. SEC-006/007 resume after package-index permission.
- **Status counts:** 6 `VERIFIED LOCALLY`; 12 `IMPLEMENTED — NOT VERIFIED`; 1 `IN PROGRESS`; 15 `NOT STARTED`; 5 `BLOCKED`; 0 `DONE`; 0 `DEFERRED`.
- **Test status:** backend 588/588; AI-002 focused provider/privacy/queue/history 26/26; frontend baseline 47/47 suites and 158/158 tests plus build; Python sidecar 22/22. Compose and patch checks pass. Prior POL/PROD/OPS/BG/SEC/CORE evidence remains green.
- **Deployment status:** No deployment performed. No production migrations were run.
- **Production status:** Unchanged and unverified. Pre-existing Docker development services on 3000/5202 were observed and left untouched.
- **Known regressions:** No known regression from the implemented packages. POL-002 preserves local AI behaviour and defaults external consent off. Direct clean EF-only SQLite migration still hits the pre-existing historical blank-chain defect before the new migration; normal startup owns reconciliation. Complete usage/provider provenance remains a known pre-rollout gap.
- **Known regressions:** No known regression from the implemented packages. AI-002 preserves local behavior, defaults external consent off, performs no parallel provider race and records actual provenance. Direct clean EF-only SQLite migration still hits the pre-existing historical blank-chain defect before the new migration; normal startup owns reconciliation. Complete cross-feature monthly usage accounting remains a pre-rollout gap.
- **Outstanding security findings:** JT-001 repository ownership is implemented but remains High deployment risk until migration/inventory/provider checks; production portion of JT-002; JT-006, JT-009 and associated JT-011/JT-012/JT-022 prerequisites. JT-005 owner foundation is implemented but workers remain off until persistent notification, entitlement and privacy gates. JT-007/JT-008/JT-010 repository behavior is not fully browser/provider/production verified.
## Current evidence
@@ -38,5 +38,6 @@ Updated: 2026-08-03
- `docs/verification/pol-001-free-pro-entitlements.md`
- `docs/verification/pol-002-ai-privacy.md`
- `docs/verification/ai-001-durable-ai-queue.md`
- `docs/verification/ai-002-provider-routing.md`
- `docs/verification/prod-002-ai-evaluation.md`
- `docs/work-programmes/master-work-plan.md`
+13 -13
View File
@@ -16,7 +16,7 @@ Allowed statuses are `NOT STARTED`, `IN PROGRESS`, `IMPLEMENTED — NOT VERIFIED
`DONE` requires every applicable acceptance criterion, focused and regression tests, browser/accessibility/theme/mobile checks, tenant and entitlement checks, documentation, migration/rollback evidence, and production verification. Repository-only work that still requires production is at most `VERIFIED LOCALLY`.
Exactly one implementation item may be `IN PROGRESS`. As of this revision it is **AI-002**.
Exactly one implementation item may be `IN PROGRESS`. As of this revision it is **AI-003**.
## Consolidated dependency order
@@ -383,7 +383,7 @@ Ordering differences from the suggested list:
- **Blocker:** browser localhost is denied; Stripe/MariaDB/production are unavailable. Usage accounting is complete only for AI Workspace, so provider rollout remains blocked until durable execution centralizes it.
- **Evidence:** `docs/verification/pol-001-free-pro-entitlements.md`; focused backend 74/74; full backend 568/568; focused frontend 22/22; full frontend 47 suites/157 tests; production build.
- **Commit:** none.
- **Remaining work:** browser locked/Pro state checks; mocked Stripe expiry/downgrade lifecycle; central all-task usage accounting through AI-001/002; production role/config smoke. PRODUCT-001 separately removes the known landing-page price/third-tier/unlimited claims.
- **Remaining work:** browser locked/Pro state checks; mocked Stripe expiry/downgrade lifecycle; central all-task usage accounting through AI-003/004 producers; production role/config smoke. PRODUCT-001 separately removes the known landing-page price/third-tier/unlimited claims.
### POL-002 — AI privacy, consent and external-fallback policy
@@ -398,10 +398,10 @@ Ordering differences from the suggested list:
- **Required browser verification:** user/admin controls and disclosure/locked/failure states.
- **Required production verification:** external egress capture with synthetic data only; no real private CV/email.
- **Status:** `IMPLEMENTED — NOT VERIFIED`.
- **Blocker:** browser localhost is denied; MariaDB/production/external-provider verification is unavailable. Final fallback triggers and provenance depend on AI-001/002.
- **Blocker:** browser localhost is denied; MariaDB/production/external-provider verification is unavailable. Task-specific payload minimization/accounting depend on AI-003/004.
- **Evidence:** `docs/verification/pol-002-ai-privacy.md`; focused backend 72/72 and final policy 28/28; sidecar 18/18; focused frontend 8/8; full backend 576/576; full frontend 47 suites/158 tests; production build; config and migration script checks.
- **Commit:** none.
- **Remaining work:** browser user/admin disclosure checks; MariaDB and production synthetic egress proof; AI-001 policy snapshot/recheck; AI-002 provider provenance, payload minimization, cost controls and bounded local-first fallback. Background CV calls currently fail safe to local.
- **Remaining work:** browser user/admin disclosure checks; MariaDB and production synthetic egress proof; AI-003/004 task-specific payload minimization, accounting and real producer verification. AI-001/002 now carry rechecked policy/task context and record bounded local-first provenance.
### AI-001 — Durable AI queue, backpressure and operation APIs
@@ -416,10 +416,10 @@ Ordering differences from the suggested list:
- **Required browser verification:** synthetic operation status across refresh/nav/double-click/offline/retry/cancel.
- **Required production verification:** queue depth/age, one-worker canary, Ollama offline/restart and app/worker restart.
- **Status:** `IMPLEMENTED — NOT VERIFIED`.
- **Blocker:** real 202 producers/browser verification depend on AI-003/004; provider/model circuit and provenance depend on AI-002; MariaDB/production are unavailable and the worker remains off.
- **Blocker:** real 202 producers/browser verification depend on AI-003/004; MariaDB/production are unavailable and the worker remains off.
- **Evidence:** `docs/verification/ai-001-durable-ai-queue.md`; focused queue/state/API tests 17/17; full backend 581/581; Compose config and diff checks.
- **Commit:** none.
- **Remaining work:** AI-002 provider/model controls; AI-003/004 task handlers and 202 endpoints; browser refresh/double-click/cancel/retry; MariaDB and monitored single-worker production canary. Do not create a second CV- or Strategy-specific queue.
- **Remaining work:** AI-003/004 task handlers and 202 endpoints; browser refresh/double-click/cancel/retry; MariaDB and monitored single-worker production canary. AI-002 supplies local-first circuit/provenance. Do not create a second CV- or Strategy-specific queue.
### AI-002 — Ollama adapter and local-first provider routing
@@ -433,11 +433,11 @@ Ordering differences from the suggested list:
- **Required tests:** routing matrix, Ollama adapter, schema failure, local circuit, fallback allowed/prohibited/unavailable, cost limits and deduplication.
- **Required browser verification:** provider-agnostic queued states and appropriate fallback disclosure.
- **Required production verification:** actual selected local model and controlled synthetic fallback.
- **Status:** `IN PROGRESS`.
- **Blocker:** actual model/config depends on PROD-003; adapter/policy can use fakes first.
- **Evidence:** new programme explicitly supersedes ADR-004's single-provider decision for this scope.
- **Status:** `IMPLEMENTED — NOT VERIFIED`.
- **Blocker:** browser and production checks, actual local-model selection and controlled provider fallback depend on administrator browser policy plus PROD-001/003 access/benchmarks. Repository behavior is not blocked.
- **Evidence:** `docs/verification/ai-002-provider-routing.md`; V-098V-100; focused backend 26/26, full backend 588/588, sidecar fake-transport 22/22, Compose/diff checks pass.
- **Commit:** none.
- **Remaining work:** design smallest central policy; preserve old provider config for rollback.
- **Remaining work:** AI-003/004 must register typed producers/handlers and explicit external task allowlists; complete monthly cross-feature accounting; browser/MariaDB/selected-model/controlled-provider/production verification. Old provider/model configuration remains available for rollback.
### PROD-001 — Read-only production AI inventory and rollout safety
@@ -523,9 +523,9 @@ Ordering differences from the suggested list:
- **Required tests:** endpoint/handler/provider fakes, all required states, entitlement/privacy/tenant checks, E2E.
- **Required browser verification:** complete queue/status/error/retry/cancel/refresh/back-forward/mobile/theme flow.
- **Required production verification:** local model success, timeout and restart recovery.
- **Status:** `NOT STARTED`.
- **Blocker:** dependencies.
- **Evidence:** reported timeout not yet reproduced in this programme.
- **Status:** `IN PROGRESS`.
- **Blocker:** browser and production checks remain unavailable, but repository tracing, handler/API implementation and fake-provider tests can proceed.
- **Evidence:** reported timeout was code-traced as synchronous `/cv/rewrite` work; live browser/provider reproduction remains blocked and must not be inferred.
- **Commit:** none.
- **Remaining work:** do not build a Strategy-specific queue.