cesnimda
5eb9b3cb96
Keep external providers behind server consent, task, and prompt-cost gates while persisting actual provider provenance.
60 lines
5.4 KiB
Markdown
60 lines
5.4 KiB
Markdown
# AI-002 verification — local-first provider routing
|
||
|
||
Updated: 2026-08-09
|
||
|
||
Status: `IMPLEMENTED — NOT VERIFIED`.
|
||
|
||
## Confirmed route matrix
|
||
|
||
| Sidecar path | Reachable application callers | Workload/privacy | Provider policy |
|
||
|---|---|---|---|
|
||
| `/summarize` | job create/detail/refresh, job-enrichment worker, health probe | `JOB-SUMMARY` / `HEALTH-PROBE`; P0–P2 depending on source | local DistilBART only; never external |
|
||
| `/extract-text` | profile-CV upload and selected application attachments | `DOC-EXTRACT`; P2 | local parser/OCR only; never external |
|
||
| `/cv/normalize` | profile-CV reconstruction/normalization | `CV-NORMALIZE`; P2 | primary Ollama; permitted external fallback only after all gates |
|
||
| `/cv/classify-block` | ambiguous profile-CV block classification | `CV-CLASSIFY`; P2 | primary Ollama; permitted external fallback only after all gates |
|
||
| `/cv/rewrite` | profile/CV rewrite, CV Builder assistance, candidate fit/focus/strategy/application drafting, follow-up drafting, selected attachment context and AI Workspace modules | `PROFILE-EXTRACT`, `STRATEGY`, `CV-TAILOR`, `APPLICATION-DRAFT`, `FOLLOWUP-DRAFT`, `INTERVIEW`, `WRITING`; P2 | primary Ollama; permitted external fallback only after all gates |
|
||
|
||
Deterministic match, profile diff, keyword, email-classification and application-intelligence paths do not enter the provider router. Existing synchronous `/cv/*` calls use the endpoint task identifier. Durable handlers receive their typed operation task through `AiOperationExecutionScope`; a new operation task remains local until it is explicitly added to `EXTERNAL_AI_ALLOWED_TASKS`.
|
||
|
||
## Implemented policy
|
||
|
||
`tools/summarizer/app.py` is the one generation router. For each generative CV request it:
|
||
|
||
1. validates routing mode, task allowlist, administrator enablement, backend permission, external provider configuration and a per-request external prompt ceiling;
|
||
2. uses Ollama first in the default `local_first` mode;
|
||
3. validates non-empty text or structured JSON before accepting the local result;
|
||
4. records consecutive local failures in a bounded process-local circuit;
|
||
5. calls one external provider only after an eligible local failure/circuit-open decision and only when every gate still passes;
|
||
6. never races local and external calls; and
|
||
7. returns sanitized provider/model/route/fallback headers for persistence and diagnostics.
|
||
|
||
`local_only`, `local_first` and `external_only` are supported. Invalid modes fail closed to `local_only`. `external_only` still requires explicit backend permission and an allowed task. The default remains `local_first`, while `EXTERNAL_AI_ENABLED=false` makes it effectively local-only.
|
||
|
||
The external prompt ceiling is a per-request cost/privacy control, not a monthly spend ledger. Current plan-level monthly accounting covers AI Workspace interactions only; complete cross-feature accounting remains a POL-001/AI-003/AI-004 rollout gate.
|
||
|
||
## Backend integration
|
||
|
||
- `AiPrivacyHeaderHandler` uses live request policy for synchronous calls and the worker's rechecked immutable policy/task for durable calls.
|
||
- `SummarizerService.GenerateSectionWithMetadataAsync` preserves cancellation and returns actual provider/model/fallback metadata.
|
||
- Provider failures use a typed, sanitized `AiGenerationException`; legacy string callers retain their previous `null` behavior.
|
||
- `AiWorkspaceService` stores the actual provider and bounded model/route metadata instead of treating deployment configuration as execution evidence.
|
||
- `AiOperationWorker` persists provider, model and route stage on success and retryable/permanent provider failure. Existing operation APIs continue to hide provider internals while exposing the bounded progress stage.
|
||
- No schema migration or dependency change was needed; existing nullable `UserOperations.Provider`, `Model` and `ProgressStage` columns are reused.
|
||
|
||
## Automated evidence
|
||
|
||
- Focused backend provider/privacy/queue/history tests: 26/26.
|
||
- Full backend: 588/588.
|
||
- Sidecar: 22/22 with fake transports only.
|
||
- Compose configuration and `git diff --check`: pass; expected missing optional-environment and line-ending warnings only.
|
||
- Tests cover local success, sequential fallback, missing consent/key, invalid JSON, prompt ceiling, open circuit, external outage, unapproved durable task, external-only permission, actual metadata, sanitized failures and operation persistence.
|
||
|
||
## Remaining gates
|
||
|
||
- No Ollama model, external provider, paid API, real CV/email, production service or production egress was used.
|
||
- PROD-001/003 must identify hardware and benchmark/select the primary and optional secondary local model. No secondary local model is configured yet.
|
||
- AI-003/004 must register real Strategy/CV handlers, choose explicit task allowlists, pass cancellation through their work and verify retry/deduplication with durable results.
|
||
- The local circuit is intentionally process-local for the current single-sidecar deployment. Multi-replica or restart-persistent circuit coordination requires measured need and a separate design.
|
||
- The existing named HTTP client still has a 30-second transport timeout for synchronous callers. AI-003/004 must move long work to durable handlers and align their cancellation/transport budget; increasing the synchronous timeout is not accepted as the timeout fix.
|
||
- Browser disclosure, MariaDB execution, controlled synthetic provider fallback, production health/circuit telemetry and rollback/canary checks remain unverified.
|