jobtrackingapp/docs/verification/ai-002-provider-routing.md

AI-002 verification — local-first provider routing

Updated: 2026-08-09

Status: IMPLEMENTED — NOT VERIFIED.

Confirmed route matrix

Sidecar path	Reachable application callers	Workload/privacy	Provider policy
/summarize	job create/detail/refresh, job-enrichment worker, health probe	JOB-SUMMARY / HEALTH-PROBE; P0–P2 depending on source	local DistilBART only; never external
/extract-text	profile-CV upload and selected application attachments	DOC-EXTRACT; P2	local parser/OCR only; never external
/cv/normalize	profile-CV reconstruction/normalization	CV-NORMALIZE; P2	primary Ollama; permitted external fallback only after all gates
/cv/classify-block	ambiguous profile-CV block classification	CV-CLASSIFY; P2	primary Ollama; permitted external fallback only after all gates
/cv/rewrite	profile/CV rewrite, CV Builder assistance, candidate fit/focus/strategy/application drafting, follow-up drafting, selected attachment context and AI Workspace modules	PROFILE-EXTRACT, STRATEGY, CV-TAILOR, APPLICATION-DRAFT, FOLLOWUP-DRAFT, INTERVIEW, WRITING; P2	primary Ollama; permitted external fallback only after all gates

Deterministic match, profile diff, keyword, email-classification and application-intelligence paths do not enter the provider router. Existing synchronous /cv/* calls use the endpoint task identifier. Durable handlers receive their typed operation task through AiOperationExecutionScope; a new operation task remains local until it is explicitly added to EXTERNAL_AI_ALLOWED_TASKS.

Implemented policy

tools/summarizer/app.py is the one generation router. For each generative CV request it:

1. validates routing mode, task allowlist, administrator enablement, backend permission, external provider configuration and a per-request external prompt ceiling;
2. uses Ollama first in the default local_first mode;
3. validates non-empty text or structured JSON before accepting the local result;
4. records consecutive local failures in a bounded process-local circuit;
5. calls one external provider only after an eligible local failure/circuit-open decision and only when every gate still passes;
6. never races local and external calls; and
7. returns sanitized provider/model/route/fallback headers for persistence and diagnostics.

local_only, local_first and external_only are supported. Invalid modes fail closed to local_only. external_only still requires explicit backend permission and an allowed task. The default remains local_first, while EXTERNAL_AI_ENABLED=false makes it effectively local-only.

The external prompt ceiling remains a per-request cost/privacy control. A separate content-free monthly ledger now covers AI Workspace and the durable Strategy/CV producers, with conservative reservation before work and actual Strategy/Workspace estimates on success. Older synchronous generators still need the same admission boundary before accounting is universal.

Backend integration

• AiPrivacyHeaderHandler uses live request policy for synchronous calls and the worker's rechecked immutable policy/task for durable calls.
• SummarizerService.GenerateSectionWithMetadataAsync preserves cancellation and returns actual provider/model/fallback metadata.
• Provider failures use a typed, sanitized AiGenerationException; legacy string callers retain their previous null behavior.
• AiWorkspaceService stores the actual provider and bounded model/route metadata instead of treating deployment configuration as execution evidence.
• AiOperationWorker persists provider, model and route stage on success and retryable/permanent provider failure. Existing operation APIs continue to hide provider internals while exposing the bounded progress stage.
• No schema migration or dependency change was needed; existing nullable UserOperations.Provider, Model and ProgressStage columns are reused.

Automated evidence

• Focused backend provider/privacy/queue/history tests: 26/26.
• Full backend: 588/588.
• Sidecar: 22/22 with fake transports only.
• Compose configuration and git diff --check: pass; expected missing optional-environment and line-ending warnings only.
• Tests cover local success, sequential fallback, missing consent/key, invalid JSON, prompt ceiling, open circuit, external outage, unapproved durable task, external-only permission, actual metadata, sanitized failures and operation persistence.

Remaining gates

• No Ollama model, external provider, paid API, real CV/email, production service or production egress was used.
• PROD-001/003 must identify hardware and benchmark/select the primary and optional secondary local model. No secondary local model is configured yet.
• Strategy and CV have registered typed handlers, cancellation, retry/deduplication and durable results. They remain local-only until an explicit task allowlist and production-safe validation authorize otherwise.
• The local circuit is intentionally process-local for the current single-sidecar deployment. Multi-replica or restart-persistent circuit coordination requires measured need and a separate design.
• The existing named HTTP client still has a 30-second transport timeout for synchronous callers. AI-003/004 must move long work to durable handlers and align their cancellation/transport budget; increasing the synchronous timeout is not accepted as the timeout fix.
• Browser disclosure, MariaDB execution, controlled synthetic provider fallback, production health/circuit telemetry and rollback/canary checks remain unverified.