feat(ai): enforce local-first routing
Keep external providers behind server consent, task, and prompt-cost gates while persisting actual provider provenance.
This commit is contained in:
@@ -65,16 +65,14 @@ and all-time totals; the workspace displays the monthly calls and estimated toke
|
||||
|
||||
## Provider abstraction
|
||||
|
||||
Generation goes through the existing `ISummarizerService` → ai-service, which routes to the active
|
||||
provider (`AI_PROVIDER`: ollama | gemini | groq) — production can offload a weak local GPU to a cloud
|
||||
provider. Each `AiInteraction` records the resolved provider for transparency, and `GET …/ai/modules`
|
||||
returns the current provider so the UI can show it.
|
||||
Generation goes through `ISummarizerService` to the ai-service. Ollama is primary; `AI_PROVIDER` names
|
||||
only the optional external fallback candidate. Fallback is sequential and requires administrator
|
||||
enablement, task approval, live Pro/user consent and the prompt cost/privacy ceiling. Each
|
||||
`AiInteraction` records the provider returned by the sidecar, plus bounded model/route metadata in
|
||||
`ResultJson.meta`; configuration alone is not treated as proof that a provider executed.
|
||||
|
||||
**Per-request user-selectable providers** (module 8's "users can choose provider") is a plumbing
|
||||
extension, not yet wired end-to-end: it needs (a) ai-service to accept a per-request `provider`
|
||||
override and (b) an API **key configured for each selectable provider**. Both are deployment/credential
|
||||
concerns (a live paid key per provider), so the code path is left as a documented extension point
|
||||
rather than shipped half-configured. The abstraction already isolates the change to one method.
|
||||
Per-request user-selectable providers remain intentionally unsupported. The server-side privacy
|
||||
policy selects a route, not the browser, and provider credentials remain deployment-only.
|
||||
|
||||
## Extension points
|
||||
|
||||
@@ -83,8 +81,8 @@ rather than shipped half-configured. The abstraction already isolates the change
|
||||
- **New cover-letter tone**: add to `CoverLetterModes` + `ModeGuidance`.
|
||||
- **Structured (JSON) results**: swap a module's prompt for JSON and parse into `ResultJson.meta`; the
|
||||
UI already renders `result.text` as markdown and can read `meta`.
|
||||
- **User-selectable provider**: thread a `provider` param through `ISummarizerService` →
|
||||
ai-service; gate on the provider having a configured key (see above).
|
||||
- **Provider policy**: add task types to the explicit server-side allowlist only after their payload,
|
||||
accounting and production checks pass; do not add browser provider overrides.
|
||||
|
||||
## Security
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# AI privacy and external-processing policy
|
||||
|
||||
Updated: 2026-08-03
|
||||
Updated: 2026-08-09
|
||||
|
||||
The default execution mode is local-only. External processing of `/cv/*` payloads requires all of:
|
||||
|
||||
@@ -11,9 +11,10 @@ The default execution mode is local-only. External processing of `/cv/*` payload
|
||||
5. AI enabled in the user's server-side settings; and
|
||||
6. the user's explicit `ExternalAiProcessingAllowed` opt-in.
|
||||
|
||||
The backend adds `X-Ai-External-Allowed: true` only after that live policy check. The sidecar otherwise routes `/cv/*` to Ollama even when an external provider is configured. `/summarize` always uses the local summarization model. Provider keys remain server-side and are never returned by the settings API.
|
||||
The backend adds `X-Ai-External-Allowed: true` only after that live policy check. Durable workers use the same header only from their admitted policy snapshot after a live execution-time recheck, and also send the bounded task type. The sidecar otherwise routes `/cv/*` to Ollama even when an external provider is configured. `/summarize` always uses the local summarization model. Provider keys remain server-side and are never returned by the settings API.
|
||||
|
||||
`GET/PUT /api/ai/settings` owns the user settings. Disabling AI takes effect on the next protected request and is also rechecked by the current enrichment and queued-CV workers. Existing users migrate with AI enabled to preserve current behaviour; external consent always defaults to false.
|
||||
|
||||
This is the privacy admission foundation, not the final routing system. AI-001/AI-002 must carry an immutable policy snapshot into durable operations, recheck it at execution, record the actual provider/reason, add bounded local-first fallback triggers and minimize each external payload. Background CV work currently fails safe to local because it has no HTTP user context.
|
||||
AI-002 makes provider execution local-first and sequential. External fallback additionally requires an allowed task and stays below the configured per-request prompt ceiling. Actual provider/model/route metadata is returned by the sidecar and persisted by AI Workspace or durable operations. The process-local circuit and health diagnostics expose no prompt or credential data.
|
||||
|
||||
This is not permission to enable external processing globally. New durable task types remain local until explicitly allowlisted; AI-003/004 must minimize their exact payloads and complete cross-feature monthly accounting before rollout. `EXTERNAL_AI_ENABLED=false` or `Ai:RoutingMode=local_only` is the immediate rollback switch.
|
||||
|
||||
@@ -22,4 +22,6 @@ Terminal notifications are described in `notifications.md`. Authenticated owner
|
||||
|
||||
`AiOperationAdmission` now provides the shared AI producer boundary: it rechecks live Pro/AI settings, snapshots `local_only` or `external_allowed`, applies per-user/global capacity, assigns a deadline and returns the stable `/api/operations/{id}` status URL. It stores only subject type/ID, never raw CV/email/prompt text. The current process-local admission semaphore is correct for the documented single-backend deployment; multi-replica rollout requires a database capacity reservation.
|
||||
|
||||
`AiOperationWorker` claims only registered task types by priority, enters the explicit owner scope, rechecks entitlement/privacy/cancellation, runs one inference by default, heartbeats the lease, enforces a timeout, classifies bounded retry/permanent failure and commits the existing terminal notification. `Workers:AiOperationsEnabled` defaults false and no production feature handler is registered yet. AI-003/004 add the Strategy/CV handlers and 202 producer endpoints; AI-002 adds provider/model concurrency, circuit and actual-provider provenance.
|
||||
`AiOperationWorker` claims only registered task types by priority, enters the explicit owner scope, rechecks entitlement/privacy/cancellation, runs one inference by default, heartbeats the lease, enforces a timeout, classifies bounded retry/permanent failure and commits the existing terminal notification. Its execution scope carries the rechecked privacy/task decision to the shared sidecar client. Successful and failed provider attempts persist bounded provider/model/route provenance in existing operation fields.
|
||||
|
||||
`Workers:AiOperationsEnabled` defaults false and no production feature handler is registered yet. AI-002 supplies sequential local-first routing and a process-local single-model circuit; AI-003/004 add the Strategy/CV handlers and 202 producer endpoints. The current one-worker default is the local-model concurrency limit until PROD-003 benchmarks justify anything else.
|
||||
|
||||
Reference in New Issue
Block a user