feat(ai): enforce local-first routing

Keep external providers behind server consent, task, and prompt-cost gates while persisting actual provider provenance.
This commit is contained in:
cesnimda
2026-08-09 12:30:11 +02:00
parent c3f4a57195
commit 5eb9b3cb96
29 changed files with 967 additions and 145 deletions
+1 -2
View File
@@ -24,8 +24,7 @@ Status: `IMPLEMENTED — NOT VERIFIED`.
## Remaining gates
- AI-003/004 must register real Strategy/CV handlers and return actual 202 responses; no generic create API was exposed because it would bypass task ownership/policy.
- AI-002 must add provider/model concurrency, circuit health, provider/reason/model recording, payload minimization and external fallback decisions.
- AI-002 added sequential local-first routing, single-model circuit health, provider/reason/model recording and task/prompt fallback gates. AI-003/004 still own producer-specific payload minimization and accounting.
- Browser refresh/double-click/cancel/retry must be repeated against each real producer.
- Worker remains off; MariaDB and production canary/restart/queue telemetry are unavailable.
- The process-local capacity gate assumes one backend replica. Add a database reservation only before multi-replica rollout.
@@ -0,0 +1,59 @@
# AI-002 verification — local-first provider routing
Updated: 2026-08-09
Status: `IMPLEMENTED — NOT VERIFIED`.
## Confirmed route matrix
| Sidecar path | Reachable application callers | Workload/privacy | Provider policy |
|---|---|---|---|
| `/summarize` | job create/detail/refresh, job-enrichment worker, health probe | `JOB-SUMMARY` / `HEALTH-PROBE`; P0P2 depending on source | local DistilBART only; never external |
| `/extract-text` | profile-CV upload and selected application attachments | `DOC-EXTRACT`; P2 | local parser/OCR only; never external |
| `/cv/normalize` | profile-CV reconstruction/normalization | `CV-NORMALIZE`; P2 | primary Ollama; permitted external fallback only after all gates |
| `/cv/classify-block` | ambiguous profile-CV block classification | `CV-CLASSIFY`; P2 | primary Ollama; permitted external fallback only after all gates |
| `/cv/rewrite` | profile/CV rewrite, CV Builder assistance, candidate fit/focus/strategy/application drafting, follow-up drafting, selected attachment context and AI Workspace modules | `PROFILE-EXTRACT`, `STRATEGY`, `CV-TAILOR`, `APPLICATION-DRAFT`, `FOLLOWUP-DRAFT`, `INTERVIEW`, `WRITING`; P2 | primary Ollama; permitted external fallback only after all gates |
Deterministic match, profile diff, keyword, email-classification and application-intelligence paths do not enter the provider router. Existing synchronous `/cv/*` calls use the endpoint task identifier. Durable handlers receive their typed operation task through `AiOperationExecutionScope`; a new operation task remains local until it is explicitly added to `EXTERNAL_AI_ALLOWED_TASKS`.
## Implemented policy
`tools/summarizer/app.py` is the one generation router. For each generative CV request it:
1. validates routing mode, task allowlist, administrator enablement, backend permission, external provider configuration and a per-request external prompt ceiling;
2. uses Ollama first in the default `local_first` mode;
3. validates non-empty text or structured JSON before accepting the local result;
4. records consecutive local failures in a bounded process-local circuit;
5. calls one external provider only after an eligible local failure/circuit-open decision and only when every gate still passes;
6. never races local and external calls; and
7. returns sanitized provider/model/route/fallback headers for persistence and diagnostics.
`local_only`, `local_first` and `external_only` are supported. Invalid modes fail closed to `local_only`. `external_only` still requires explicit backend permission and an allowed task. The default remains `local_first`, while `EXTERNAL_AI_ENABLED=false` makes it effectively local-only.
The external prompt ceiling is a per-request cost/privacy control, not a monthly spend ledger. Current plan-level monthly accounting covers AI Workspace interactions only; complete cross-feature accounting remains a POL-001/AI-003/AI-004 rollout gate.
## Backend integration
- `AiPrivacyHeaderHandler` uses live request policy for synchronous calls and the worker's rechecked immutable policy/task for durable calls.
- `SummarizerService.GenerateSectionWithMetadataAsync` preserves cancellation and returns actual provider/model/fallback metadata.
- Provider failures use a typed, sanitized `AiGenerationException`; legacy string callers retain their previous `null` behavior.
- `AiWorkspaceService` stores the actual provider and bounded model/route metadata instead of treating deployment configuration as execution evidence.
- `AiOperationWorker` persists provider, model and route stage on success and retryable/permanent provider failure. Existing operation APIs continue to hide provider internals while exposing the bounded progress stage.
- No schema migration or dependency change was needed; existing nullable `UserOperations.Provider`, `Model` and `ProgressStage` columns are reused.
## Automated evidence
- Focused backend provider/privacy/queue/history tests: 26/26.
- Full backend: 588/588.
- Sidecar: 22/22 with fake transports only.
- Compose configuration and `git diff --check`: pass; expected missing optional-environment and line-ending warnings only.
- Tests cover local success, sequential fallback, missing consent/key, invalid JSON, prompt ceiling, open circuit, external outage, unapproved durable task, external-only permission, actual metadata, sanitized failures and operation persistence.
## Remaining gates
- No Ollama model, external provider, paid API, real CV/email, production service or production egress was used.
- PROD-001/003 must identify hardware and benchmark/select the primary and optional secondary local model. No secondary local model is configured yet.
- AI-003/004 must register real Strategy/CV handlers, choose explicit task allowlists, pass cancellation through their work and verify retry/deduplication with durable results.
- The local circuit is intentionally process-local for the current single-sidecar deployment. Multi-replica or restart-persistent circuit coordination requires measured need and a separate design.
- The existing named HTTP client still has a 30-second transport timeout for synchronous callers. AI-003/004 must move long work to durable handlers and align their cancellation/transport budget; increasing the synchronous timeout is not accepted as the timeout fix.
- Browser disclosure, MariaDB execution, controlled synthetic provider fallback, production health/circuit telemetry and rollback/canary checks remain unverified.
@@ -34,5 +34,9 @@ This is the rolling action-level evidence index. `PASS (automated/runtime)` is n
| Durable AI | Pro admission, idempotent status URL and bounded capacity | PASS (real SQLite + synthetic subject IDs) | BLOCKED until real producer | NOT RUN; worker off | `ai-001-durable-ai-queue.md` |
| Durable AI | priority/task-filtered atomic claim and owner-scoped success | PASS (fake handler, real operation/notification state) | N/A | NOT RUN | `ai-001-durable-ai-queue.md` |
| Durable AI | retryable failure, downgrade recheck, lease/cancel/restart recovery | PASS (automated) | BLOCKED until real producer | NOT RUN | `ai-001-durable-ai-queue.md` |
| AI routing | local primary success and no parallel external call | PASS (fake transports) | N/A | NOT RUN | `ai-002-provider-routing.md` |
| AI routing | consent/admin/task/config/prompt-cap fallback denial | PASS (backend + sidecar policy tests) | BLOCKED | NOT RUN | `ai-002-provider-routing.md` |
| AI routing | schema/local-outage/circuit fallback and external failure | PASS (fake transports) | BLOCKED | NOT RUN | `ai-002-provider-routing.md` |
| AI routing | actual provider/model/route persistence on success/failure | PASS (real SQLite operation/history state; fake provider) | BLOCKED until real producer | NOT RUN | `ai-002-provider-routing.md` |
Remaining product actions are `NOT STARTED` in the master plan and will be added as their work packages enter verification. Browser localhost is currently denied by administrator policy; production access is not documented/configured.
+1 -2
View File
@@ -33,5 +33,4 @@ Status: `IMPLEMENTED — NOT VERIFIED`.
- No external provider, paid service, production environment or real private data was used.
- MariaDB migration execution remains unavailable.
- Direct clean `dotnet ef database update` fails in the pre-existing historical SQLite migration chain before this migration (`AddJobEntityAndProspectStages` expects a reconciler-added column). The application startup reconciler path was not exercised because the local process-launch command was blocked by execution policy.
- Background operations do not yet carry a policy snapshot; they fail safe to local. AI-001/AI-002 own durable admission/recheck, actual-provider/reason recording, cost controls, payload minimization and bounded local-first fallback triggers.
- AI-001/002 now carry admitted/rechecked policy/task context into durable calls, enforce bounded local-first fallback and record actual provider/model/route metadata. AI-003/004 still own task-specific payload minimization, complete cross-feature monthly accounting and real producer verification.