feat(ai): enforce local-first routing

Keep external providers behind server consent, task, and prompt-cost gates while persisting actual provider provenance.
This commit is contained in:
cesnimda
2026-08-09 12:30:11 +02:00
parent c3f4a57195
commit 5eb9b3cb96
29 changed files with 967 additions and 145 deletions
+9 -11
View File
@@ -65,16 +65,14 @@ and all-time totals; the workspace displays the monthly calls and estimated toke
## Provider abstraction
Generation goes through the existing `ISummarizerService` ai-service, which routes to the active
provider (`AI_PROVIDER`: ollama | gemini | groq) — production can offload a weak local GPU to a cloud
provider. Each `AiInteraction` records the resolved provider for transparency, and `GET …/ai/modules`
returns the current provider so the UI can show it.
Generation goes through `ISummarizerService` to the ai-service. Ollama is primary; `AI_PROVIDER` names
only the optional external fallback candidate. Fallback is sequential and requires administrator
enablement, task approval, live Pro/user consent and the prompt cost/privacy ceiling. Each
`AiInteraction` records the provider returned by the sidecar, plus bounded model/route metadata in
`ResultJson.meta`; configuration alone is not treated as proof that a provider executed.
**Per-request user-selectable providers** (module 8's "users can choose provider") is a plumbing
extension, not yet wired end-to-end: it needs (a) ai-service to accept a per-request `provider`
override and (b) an API **key configured for each selectable provider**. Both are deployment/credential
concerns (a live paid key per provider), so the code path is left as a documented extension point
rather than shipped half-configured. The abstraction already isolates the change to one method.
Per-request user-selectable providers remain intentionally unsupported. The server-side privacy
policy selects a route, not the browser, and provider credentials remain deployment-only.
## Extension points
@@ -83,8 +81,8 @@ rather than shipped half-configured. The abstraction already isolates the change
- **New cover-letter tone**: add to `CoverLetterModes` + `ModeGuidance`.
- **Structured (JSON) results**: swap a module's prompt for JSON and parse into `ResultJson.meta`; the
UI already renders `result.text` as markdown and can read `meta`.
- **User-selectable provider**: thread a `provider` param through `ISummarizerService`
ai-service; gate on the provider having a configured key (see above).
- **Provider policy**: add task types to the explicit server-side allowlist only after their payload,
accounting and production checks pass; do not add browser provider overrides.
## Security
+4 -3
View File
@@ -1,6 +1,6 @@
# AI privacy and external-processing policy
Updated: 2026-08-03
Updated: 2026-08-09
The default execution mode is local-only. External processing of `/cv/*` payloads requires all of:
@@ -11,9 +11,10 @@ The default execution mode is local-only. External processing of `/cv/*` payload
5. AI enabled in the user's server-side settings; and
6. the user's explicit `ExternalAiProcessingAllowed` opt-in.
The backend adds `X-Ai-External-Allowed: true` only after that live policy check. The sidecar otherwise routes `/cv/*` to Ollama even when an external provider is configured. `/summarize` always uses the local summarization model. Provider keys remain server-side and are never returned by the settings API.
The backend adds `X-Ai-External-Allowed: true` only after that live policy check. Durable workers use the same header only from their admitted policy snapshot after a live execution-time recheck, and also send the bounded task type. The sidecar otherwise routes `/cv/*` to Ollama even when an external provider is configured. `/summarize` always uses the local summarization model. Provider keys remain server-side and are never returned by the settings API.
`GET/PUT /api/ai/settings` owns the user settings. Disabling AI takes effect on the next protected request and is also rechecked by the current enrichment and queued-CV workers. Existing users migrate with AI enabled to preserve current behaviour; external consent always defaults to false.
This is the privacy admission foundation, not the final routing system. AI-001/AI-002 must carry an immutable policy snapshot into durable operations, recheck it at execution, record the actual provider/reason, add bounded local-first fallback triggers and minimize each external payload. Background CV work currently fails safe to local because it has no HTTP user context.
AI-002 makes provider execution local-first and sequential. External fallback additionally requires an allowed task and stays below the configured per-request prompt ceiling. Actual provider/model/route metadata is returned by the sidecar and persisted by AI Workspace or durable operations. The process-local circuit and health diagnostics expose no prompt or credential data.
This is not permission to enable external processing globally. New durable task types remain local until explicitly allowlisted; AI-003/004 must minimize their exact payloads and complete cross-feature monthly accounting before rollout. `EXTERNAL_AI_ENABLED=false` or `Ai:RoutingMode=local_only` is the immediate rollback switch.
+3 -1
View File
@@ -22,4 +22,6 @@ Terminal notifications are described in `notifications.md`. Authenticated owner
`AiOperationAdmission` now provides the shared AI producer boundary: it rechecks live Pro/AI settings, snapshots `local_only` or `external_allowed`, applies per-user/global capacity, assigns a deadline and returns the stable `/api/operations/{id}` status URL. It stores only subject type/ID, never raw CV/email/prompt text. The current process-local admission semaphore is correct for the documented single-backend deployment; multi-replica rollout requires a database capacity reservation.
`AiOperationWorker` claims only registered task types by priority, enters the explicit owner scope, rechecks entitlement/privacy/cancellation, runs one inference by default, heartbeats the lease, enforces a timeout, classifies bounded retry/permanent failure and commits the existing terminal notification. `Workers:AiOperationsEnabled` defaults false and no production feature handler is registered yet. AI-003/004 add the Strategy/CV handlers and 202 producer endpoints; AI-002 adds provider/model concurrency, circuit and actual-provider provenance.
`AiOperationWorker` claims only registered task types by priority, enters the explicit owner scope, rechecks entitlement/privacy/cancellation, runs one inference by default, heartbeats the lease, enforces a timeout, classifies bounded retry/permanent failure and commits the existing terminal notification. Its execution scope carries the rechecked privacy/task decision to the shared sidecar client. Successful and failed provider attempts persist bounded provider/model/route provenance in existing operation fields.
`Workers:AiOperationsEnabled` defaults false and no production feature handler is registered yet. AI-002 supplies sequential local-first routing and a process-local single-model circuit; AI-003/004 add the Strategy/CV handlers and 202 producer endpoints. The current one-worker default is the local-model concurrency limit until PROD-003 benchmarks justify anything else.