cesnimda/jobtrackingapp
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

feat(ai): enforce local-first routing

Browse Source 
Keep external providers behind server consent, task, and prompt-cost gates while persisting actual provider provenance.
This commit is contained in:
cesnimda
2026-08-09 12:30:11 +02:00
parent c3f4a57195
commit 5eb9b3cb96
29 changed files with 967 additions and 145 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/verification/ai-002-provider-routing.md
+59
View File
@@ -0,0 +1,59 @@
# AI-002 verification — local-first provider routing
Updated: 2026-08-09
Status: `IMPLEMENTED — NOT VERIFIED`.
## Confirmed route matrix
| Sidecar path | Reachable application callers | Workload/privacy | Provider policy |
|---|---|---|---|
| `/summarize` | job create/detail/refresh, job-enrichment worker, health probe | `JOB-SUMMARY` / `HEALTH-PROBE`; P0–P2 depending on source | local DistilBART only; never external |
| `/extract-text` | profile-CV upload and selected application attachments | `DOC-EXTRACT`; P2 | local parser/OCR only; never external |
| `/cv/normalize` | profile-CV reconstruction/normalization | `CV-NORMALIZE`; P2 | primary Ollama; permitted external fallback only after all gates |
| `/cv/classify-block` | ambiguous profile-CV block classification | `CV-CLASSIFY`; P2 | primary Ollama; permitted external fallback only after all gates |
| `/cv/rewrite` | profile/CV rewrite, CV Builder assistance, candidate fit/focus/strategy/application drafting, follow-up drafting, selected attachment context and AI Workspace modules | `PROFILE-EXTRACT`, `STRATEGY`, `CV-TAILOR`, `APPLICATION-DRAFT`, `FOLLOWUP-DRAFT`, `INTERVIEW`, `WRITING`; P2 | primary Ollama; permitted external fallback only after all gates |
Deterministic match, profile diff, keyword, email-classification and application-intelligence paths do not enter the provider router. Existing synchronous `/cv/*` calls use the endpoint task identifier. Durable handlers receive their typed operation task through `AiOperationExecutionScope`; a new operation task remains local until it is explicitly added to `EXTERNAL_AI_ALLOWED_TASKS`.
## Implemented policy
`tools/summarizer/app.py` is the one generation router. For each generative CV request it:
1. validates routing mode, task allowlist, administrator enablement, backend permission, external provider configuration and a per-request external prompt ceiling;
2. uses Ollama first in the default `local_first` mode;
3. validates non-empty text or structured JSON before accepting the local result;
4. records consecutive local failures in a bounded process-local circuit;
5. calls one external provider only after an eligible local failure/circuit-open decision and only when every gate still passes;
6. never races local and external calls; and
7. returns sanitized provider/model/route/fallback headers for persistence and diagnostics.
`local_only`, `local_first` and `external_only` are supported. Invalid modes fail closed to `local_only`. `external_only` still requires explicit backend permission and an allowed task. The default remains `local_first`, while `EXTERNAL_AI_ENABLED=false` makes it effectively local-only.
The external prompt ceiling is a per-request cost/privacy control, not a monthly spend ledger. Current plan-level monthly accounting covers AI Workspace interactions only; complete cross-feature accounting remains a POL-001/AI-003/AI-004 rollout gate.
## Backend integration
- `AiPrivacyHeaderHandler` uses live request policy for synchronous calls and the worker's rechecked immutable policy/task for durable calls.
- `SummarizerService.GenerateSectionWithMetadataAsync` preserves cancellation and returns actual provider/model/fallback metadata.
- Provider failures use a typed, sanitized `AiGenerationException`; legacy string callers retain their previous `null` behavior.
- `AiWorkspaceService` stores the actual provider and bounded model/route metadata instead of treating deployment configuration as execution evidence.
- `AiOperationWorker` persists provider, model and route stage on success and retryable/permanent provider failure. Existing operation APIs continue to hide provider internals while exposing the bounded progress stage.
- No schema migration or dependency change was needed; existing nullable `UserOperations.Provider`, `Model` and `ProgressStage` columns are reused.
## Automated evidence
- Focused backend provider/privacy/queue/history tests: 26/26.
- Full backend: 588/588.
- Sidecar: 22/22 with fake transports only.
- Compose configuration and `git diff --check`: pass; expected missing optional-environment and line-ending warnings only.
- Tests cover local success, sequential fallback, missing consent/key, invalid JSON, prompt ceiling, open circuit, external outage, unapproved durable task, external-only permission, actual metadata, sanitized failures and operation persistence.
## Remaining gates
- No Ollama model, external provider, paid API, real CV/email, production service or production egress was used.
- PROD-001/003 must identify hardware and benchmark/select the primary and optional secondary local model. No secondary local model is configured yet.
- AI-003/004 must register real Strategy/CV handlers, choose explicit task allowlists, pass cancellation through their work and verify retry/deduplication with durable results.
- The local circuit is intentionally process-local for the current single-sidecar deployment. Multi-replica or restart-persistent circuit coordination requires measured need and a separate design.
- The existing named HTTP client still has a 30-second transport timeout for synchronous callers. AI-003/004 must move long work to durable handlers and align their cancellation/transport budget; increasing the synchronous timeout is not accepted as the timeout fix.
- Browser disclosure, MariaDB execution, controlled synthetic provider fallback, production health/circuit telemetry and rollback/canary checks remain unverified.