feat(ai): centralize durable usage
CI and Deploy / test (pull_request) Successful in 5m19s
CI and Deploy / deploy (pull_request) Has been skipped

Add a content-free usage ledger with legacy backfill. Reserve Workspace and durable Strategy/CV work before execution so deleted history or duplicate admission cannot reset limits.
This commit is contained in:
cesnimda
2026-08-15 20:03:06 +02:00
parent dbff0f8d49
commit 134aac7bcf
28 changed files with 3539 additions and 83 deletions
+1
View File
@@ -212,3 +212,4 @@ Output was reduced to filenames and commit counts. The token artifact appears un
| V-178 | Benchmark harness safety tests, plan-only execution and CI workflow inspection | Repository root | Prevent an approved private Ollama origin from escaping through proxy settings or redirects and make the boundary a release gate | PASS — 5/5 standard-library tests; proxy discovery is disabled, redirects are refused, plan-only output remains eight future Strategy requests, and CI now runs the suite without dependencies or network execution | No network, Ollama, provider, package or production call occurred | Benchmark request boundary corrected and CI-enforced |
| V-179 | Account-deletion real-SQLite failure/retry tests; sidecar token/cache tests; full backend; Compose validation | Repository root / `tools/summarizer` | Remove the live sidecar-cache and shared tombstone-path gaps without enabling deletion | PASS — lifecycle 6/6, backend 658/658, sidecar 23/23 and Compose config pass. Sidecar failure withholds completion/tombstone until retry; maintenance purge is token-protected; tombstones map to a separate named volume; activation defaults false | Synthetic rows/cache only; no production volume, deletion, restart, provider revocation, backup restore or retention decision | SEC-009 repository cache/storage boundary complete; production activation remains blocked |
| V-180 | CV operation/store focused real-SQLite tests and full backend | Repository root | Keep dormant CV extraction history consistent with cancellation, deadline recovery and retry before worker claim | PASS — focused 17/17 and backend 660/660. Cancel sets the run terminal immediately, retry reopens it, deadline recovery fails it, and owner/task/subject predicates prevent unrelated updates | Synthetic rows only; no parser/model/MariaDB/production process interruption | AI-004 dormant-row consistency gap closed |
| V-181 | AI usage meter/operation/workspace/export/deletion real-SQLite tests; EF model check; SQLite/MariaDB scripts; disposable SQLite backfill and fresh application startup; full backend | Repository root | Make Workspace and durable Strategy/CV usage owner-safe, idempotent and independent of deletable private history | PASS — focused 28/28 and backend 663/663; no pending model changes; both providers generate bounded additive DDL; SQLite backfills the synthetic legacy row exactly once; fresh runtime applies through `20260815175236_AddCrossFeatureAiUsage` and serves `/health` | Synthetic local rows only; no provider/model call, MariaDB server, production migration or worker activation. CV retains a conservative reservation and older synchronous AI paths are not yet universal | Main durable usage boundary implemented; remaining synchronous producers stay tracked under POL-001 |
+2 -2
View File
@@ -30,7 +30,7 @@ Deterministic match, profile diff, keyword, email-classification and application
`local_only`, `local_first` and `external_only` are supported. Invalid modes fail closed to `local_only`. `external_only` still requires explicit backend permission and an allowed task. The default remains `local_first`, while `EXTERNAL_AI_ENABLED=false` makes it effectively local-only.
The external prompt ceiling is a per-request cost/privacy control, not a monthly spend ledger. Current plan-level monthly accounting covers AI Workspace interactions only; complete cross-feature accounting remains a POL-001/AI-003/AI-004 rollout gate.
The external prompt ceiling remains a per-request cost/privacy control. A separate content-free monthly ledger now covers AI Workspace and the durable Strategy/CV producers, with conservative reservation before work and actual Strategy/Workspace estimates on success. Older synchronous generators still need the same admission boundary before accounting is universal.
## Backend integration
@@ -53,7 +53,7 @@ The external prompt ceiling is a per-request cost/privacy control, not a monthly
- No Ollama model, external provider, paid API, real CV/email, production service or production egress was used.
- PROD-001/003 must identify hardware and benchmark/select the primary and optional secondary local model. No secondary local model is configured yet.
- AI-003/004 must register real Strategy/CV handlers, choose explicit task allowlists, pass cancellation through their work and verify retry/deduplication with durable results.
- Strategy and CV have registered typed handlers, cancellation, retry/deduplication and durable results. They remain local-only until an explicit task allowlist and production-safe validation authorize otherwise.
- The local circuit is intentionally process-local for the current single-sidecar deployment. Multi-replica or restart-persistent circuit coordination requires measured need and a separate design.
- The existing named HTTP client still has a 30-second transport timeout for synchronous callers. AI-003/004 must move long work to durable handlers and align their cancellation/transport budget; increasing the synchronous timeout is not accepted as the timeout fix.
- Browser disclosure, MariaDB execution, controlled synthetic provider fallback, production health/circuit telemetry and rollback/canary checks remain unverified.
@@ -18,6 +18,7 @@ The Overview button in `JobDetailsDialog.tsx` called candidate fit and `GET /api
- The handler makes one bounded structured generation request, validates the entire JSON shape before publishing, passes worker cancellation, records actual provider/model/route metadata and rejects embedded source instructions. Job text, profile text, structured profile and extracted attachment context have explicit ceilings.
- UI states cover queued, local processing, retry wait, approved-fallback wait, completed, failed, cancelled and cancellation requested, with cancel/retry actions. A request-version guard prevents a stale resume lookup from erasing a newly queued operation.
- The existing generic terminal notification is produced transactionally by the operation store. No email is sent.
- Admission creates one content-free usage reservation in the same transaction as the operation. Duplicate clicks reuse it, and successful execution replaces the conservative 12,000-token reservation with the measured input/output character estimate.
`strategy.snapshot` is not in the external fallback allowlist, so it remains local-only even when a user has external consent. The worker switch remains off by default pending the production canary.
@@ -36,7 +37,7 @@ The Overview button in `JobDetailsDialog.tsx` called candidate fit and `GET /api
- No Ollama model, external provider, private CV, production service or paid API was called.
- Worker/model restart and lease recovery are proven generically by AI-001 tests but not run with a real Strategy model.
- MariaDB execution, production queue telemetry, selected-model timeout/quality benchmarks, notification navigation and deployment rollback remain unverified.
- Cross-feature monthly token/cost accounting remains incomplete; the durable operation row prevents duplicate work and records execution provenance, but is not a billing ledger.
- The Strategy operation now participates in the central monthly usage ledger. Older synchronous AI endpoints outside this workflow remain a separate POL-001 completion item.
## Rollback
@@ -19,6 +19,7 @@ CV upload previously saved an artifact and held the HTTP request while extractio
- Upload/reprocess reopen the stored owner artifact in the worker. Rebuild/improve pass the worker cancellation token to the metadata-capable generation call.
- Successful processing stops at `pending_review`. It does not update profile text/structure/current-version pointers until the existing accept endpoint is called. Discard remains available.
- The UI shows queued, local processing, retry wait, approved-fallback wait, failed, cancelled and cancellation-requested states with cancel/retry actions. The upload spinner now ends after admission and reports the queued run rather than false extraction success.
- Admission creates one content-free conservative usage reservation atomically with the operation. Active duplicates reuse it. CV processing intentionally retains the 16,000-token reservation because the multi-stage sidecar does not yet return complete per-stage usage telemetry.
No dependency, schema, migration, proxy timeout or production switch changed. `Workers:AiOperationsEnabled` remains false by default.
@@ -40,7 +41,7 @@ No dependency, schema, migration, proxy timeout or production switch changed. `W
- The generic lease tests cover restart recovery, but no CV parser/model process was interrupted and resumed in a runtime canary.
- MariaDB, selected Ollama model, worker telemetry, production activation and rollback canary remain unverified. The worker stays default-off.
The focused CV/operation/store regression slice is now 17/17 and the full backend is 660/660 after dormant-row cancellation/deadline/retry coverage.
The focused accounting/operation/lifecycle regression slice passes 28/28 and the full backend is 663/663 after durable usage and dormant-row lifecycle coverage.
## Rollback
@@ -11,22 +11,22 @@ Status: `IMPLEMENTED — NOT VERIFIED`. The server policy, worker rechecks, Free
- Pro and Admin use AI and Pro CV themes. The persisted Identity role remains `Premium`, and `Stripe:PricePremium` remains a compatibility key; neither is exposed as a public plan name.
- Current database roles are authoritative on every explicit HTTP AI action. A stale role claim cannot preserve access after downgrade.
- A locked explicit action returns HTTP 403 with `{ "code": "pro_required", "message": "This AI feature requires Pro." }`.
- Existing 250-call/1,000,000-token Pro ceilings remain because they are defined in the existing implementation roadmap. Free ceilings are zero. Only AI Workspace currently writes complete `AiInteraction` usage rows; this accounting gap blocks full verification and must be resolved as AI-001/AI-002 move all producers through durable operations.
- Existing 250-call/1,000,000-token Pro ceilings remain because they are defined in the existing implementation roadmap. Free ceilings are zero. A content-free `AiUsageRecord` ledger is now authoritative for AI Workspace plus durable Strategy and CV work; legacy `AiInteraction` usage is backfilled. Older synchronous AI actions still need the same admission boundary before the numeric ceilings can be described as universal.
## Entry-point inventory
| Capability | User entry / frontend | API or worker execution path | Admission and recheck | Usage accounting | Free behavior |
|---|---|---|---|---|---|
| AI Workspace modules | Job details → AI Workspace; `AiWorkspacePanel` | `POST /api/jobapplications/{jobId}/ai/generate``AiWorkspaceService``ISummarizerService` | `Pro` policy with live role lookup | `AiInteraction` call/token row; monthly check | Generate disabled; existing history/read/delete remain available |
| AI Workspace modules | Job details → AI Workspace; `AiWorkspacePanel` | `POST /api/jobapplications/{jobId}/ai/generate``AiWorkspaceService``ISummarizerService` | `Pro` policy with live role lookup | Ledger reservation before generation; actual estimate finalized on success | Generate disabled; existing history/read/delete remain available |
| Candidate fit | Job details Candidate Fit and Strategy Snapshot | `GET .../{id}/candidate-fit` → attachment/correspondence context → multiple summarizer calls | `Pro` policy | No complete shared usage row | Deterministic `match-score` remains available; AI narrative locked |
| Focus plan | Job details Focus Plan and Strategy Snapshot | `GET .../{id}/focus-plan` → summarizer | `Pro` policy | No complete shared usage row | Locked; no synthetic fallback presented as generated |
| Focus plan | Job details Focus Plan and Strategy Snapshot | Durable `strategy.snapshot` operation → summarizer | `Pro` admission plus worker recheck | Atomic operation-ledger reservation; successful input/output estimate finalized | Locked; no synthetic fallback presented as generated |
| Interview brief | Job details Interview Prep | `GET .../{id}/interview-prep/brief` → summarizer | `Pro` policy | No complete shared usage row | Editable non-AI interview board remains available; generated brief locked |
| Tailored CV generation | Add Job option and job Tailored CV tab | `POST .../{id}/generate-tailored-cv-draft` → shared generation helpers → summarizer | `Pro` policy | No complete shared usage row | Job creation and manual tailored-draft editing remain available; no operation is started |
| Application package | Job workspace drafts | `POST .../{id}/generate-application-package` → attachment/email context → summarizer | `Pro` policy | No complete shared usage row | Existing/manual package drafts remain readable and editable |
| Follow-up draft | Job Follow-up tab | `GET .../{id}/followup-draft` → context → summarizer | `Pro` policy | No complete shared usage row | Manual correspondence data remains available; AI draft is locked |
| Job summary refresh | Job overview | `POST .../{id}/refresh-ai``SummarizeAsync` | `Pro` policy | No complete shared usage row | Existing summary/tags remain visible; refresh locked |
| Automatic job summary | Job create/detail | Core `POST /jobapplications` and `GET /{id}` optional summarizer calls | Live role condition inside core action | No complete shared usage row | Core request succeeds without calling AI |
| CV import/parse | Career Profile upload/parse/reprocess | `/profile-cv/upload`, `/parse`, `/reprocess`extraction/structured parsing | `Pro` policy before admission; queued run rechecks live roles | CV-run state only | Manual profile editing and previous review runs remain available |
| CV import/parse | Career Profile upload/parse/reprocess | `/profile-cv/upload`, `/parse`, `/reprocess`durable `cv.process` operation | `Pro` policy before admission; queued run rechecks live roles | Atomic conservative operation-ledger reservation; no raw CV content | Manual profile editing and previous review runs remain available |
| CV rebuild/improve/rewrite/PDF | Career Profile AI buttons | `/rebuild`, `/improve`, `/rewrite-section`, `/rewrite-preview`, `/export-pdf` | `Pro` policy; queued rebuild/improve recheck live roles | CV-run state only | AI controls locked; manual profile data remains available |
| CV Builder writing aid | CV Builder AI Tools | `POST /api/cv/ai/assist` → summarizer | `Pro` policy | No complete shared usage row | AI buttons disabled; CV editing/history remain available |
| Pro CV themes | CV Builder Customize | `GET /api/cv/themes`; create/save validates selected theme | Live role lookup in theme catalog checks | Not applicable | Pro themes identified and unavailable; existing unchanged selection can still be saved |
@@ -42,6 +42,8 @@ Status: `IMPLEMENTED — NOT VERIFIED`. The server policy, worker rechecks, Free
- `ProfileCvControllerTests`: a queued CV run fails with `pro_required` semantics after downgrade and never reaches the model.
- `AccountPlansTests`: Free zero AI, Pro/Admin AI, and only `free`/`pro` external names.
- AI Workspace UI test: Free locked state, disabled generation and upgrade link.
- `AiUsageMeterTests`, operation integration, account export/deletion and SQLite compatibility tests cover idempotent reservation, limits, owner isolation, history-independent totals, Strategy finalization, CV conservative reservation and lifecycle handling.
- Full backend after the ledger migration: 663/663.
- Full backend: 568/568.
- Full frontend: 47/47 suites, 157/157 tests.
- Production frontend build: pass.
@@ -53,7 +55,7 @@ Status: `IMPLEMENTED — NOT VERIFIED`. The server policy, worker rechecks, Free
- Stripe webhook transitions were code-inspected and existing status tests cover active/trialing vs expired states, but no real or mocked end-to-end checkout/webhook cycle ran in this package.
- MariaDB and production were not changed or tested.
- PRODUCT-001 removed landing-page prices, the third “Bring your own key” tier, Free AI allowance and “Unlimited AI” claims. Public capability copy now comes from one two-plan catalogue; commercial terms remain in configured Stripe Checkout.
- Full cross-feature usage accounting is incomplete. It must be centralized with AI operation execution before provider rollout; current numeric ceilings must not be advertised as universal until then.
- The durable ledger now spans AI Workspace, Strategy Snapshot and CV processing, and deleting user-visible AI history no longer erases usage. Candidate Fit, Interview Prep, application-package/follow-up drafting, CV Builder assistance and automatic summary paths remain synchronous and are not yet universally admitted through this ledger; the UI must therefore avoid claiming that the displayed numeric ceiling covers every AI path.
## Rollback
+10
View File
@@ -789,3 +789,13 @@
- **Consequences:** generic operations remain independent; the one existing persisted subject projection is synchronized through a narrow task/subject check with an explicit owner predicate.
- **User approval required:** No; local consistency fix with no schema, dependency or production change.
- **Reversible:** Revert the store helper/tests. No stored format changed.
## DEC-080 — Separate durable AI usage from user-visible history
- **Date:** 2026-08-15
- **Decision:** Store content-free AI usage in an append-only owner ledger keyed by source. Reserve allowance before AI Workspace work and atomically with Strategy/CV operation creation; replace conservative reservations with actual character-based estimates where complete output metadata exists. Backfill legacy `AiInteraction` counters without copying prompts or generated text.
- **Reason/evidence:** `AiInteraction` is private, user-deletable history and covered only one feature, so deleting history reset plan usage while durable Strategy/CV calls were invisible. Real SQLite tests prove idempotence, tenant isolation, limits, history-independent totals, Strategy finalization, CV reservation, export/deletion handling and legacy backfill; fresh application startup reaches the new migration.
- **Alternatives considered:** retain `AiInteraction` as the meter; add counters to every feature table; estimate only after success; persist prompts/results in a billing record. These couple enforcement to deletable content, scatter one policy across unrelated schemas, permit unbounded concurrent admission, or duplicate private material.
- **Consequences:** monthly usage is stable across history deletion and duplicate durable admission. CV remains conservatively reserved until complete multi-stage telemetry exists. Older synchronous AI actions still require the same admission seam before limits are universal. The process-local reservation gate is sufficient only for the current single-backend topology.
- **User approval required:** Production migration/rollout only. The additive repository migration and synthetic tests do not change production.
- **Reversible:** Disable AI work, downgrade the additive migration only after preserving any required usage evidence, and restore the prior interaction-based display. Existing user content is unchanged.
+8 -8
View File
@@ -395,10 +395,10 @@ This queue records the highest-value work that can proceed without production cr
- **Required browser verification:** locked state/upgrade action/dismissal and Pro execution; mobile/theme/accessibility.
- **Required production verification:** configured Stripe/role mapping only when operator activation is approved.
- **Status:** `IMPLEMENTED — NOT VERIFIED`.
- **Blocker:** browser localhost is denied; Stripe/MariaDB/production are unavailable. Usage accounting is complete only for AI Workspace, so provider rollout remains blocked until durable execution centralizes it.
- **Evidence:** `docs/verification/pol-001-free-pro-entitlements.md`; focused backend 74/74; full backend 568/568; focused frontend 22/22; full frontend 47 suites/157 tests; production build.
- **Blocker:** Stripe/MariaDB/production verification is unavailable. The main durable producers are accounted, but older synchronous AI actions still need ledger admission before numeric limits are universal.
- **Evidence:** `docs/verification/pol-001-free-pro-entitlements.md`; V-181; focused accounting/operation/lifecycle 28/28; full backend 663/663; existing frontend/browser entitlement evidence.
- **Commit:** none.
- **Remaining work:** browser locked/Pro state checks; mocked Stripe expiry/downgrade lifecycle; central all-task usage accounting through AI-003/004 producers; production role/config smoke. PRODUCT-001 has removed the former landing-page price/third-tier/unlimited claims.
- **Remaining work:** mocked Stripe expiry/downgrade lifecycle; move remaining synchronous AI actions through ledger admission; production role/config smoke. PRODUCT-001 has removed the former landing-page price/third-tier/unlimited claims.
### POL-002 — AI privacy, consent and external-fallback policy
@@ -431,10 +431,10 @@ This queue records the highest-value work that can proceed without production cr
- **Required browser verification:** synthetic operation status across refresh/nav/double-click/offline/retry/cancel.
- **Required production verification:** queue depth/age, one-worker canary, Ollama offline/restart and app/worker restart.
- **Status:** `IMPLEMENTED — NOT VERIFIED`.
- **Blocker:** real 202 producers/browser verification depend on AI-003/004; MariaDB/production are unavailable and the worker remains off.
- **Blocker:** MariaDB/production restart verification is unavailable and the worker remains off by default.
- **Evidence:** `docs/verification/ai-001-durable-ai-queue.md`; focused queue/state/API tests 17/17; full backend 581/581; Compose config and diff checks.
- **Commit:** none.
- **Remaining work:** AI-003/004 task handlers and 202 endpoints; browser refresh/double-click/cancel/retry; MariaDB and monitored single-worker production canary. AI-002 supplies local-first circuit/provenance. Do not create a second CV- or Strategy-specific queue.
- **Remaining work:** MariaDB and monitored single-worker production canary. Strategy/CV browser refresh, double-click, cancel and retry are covered locally; AI-002 supplies local-first circuit/provenance. Do not create a second CV- or Strategy-specific queue.
### AI-002 — Ollama adapter and local-first provider routing
@@ -452,7 +452,7 @@ This queue records the highest-value work that can proceed without production cr
- **Blocker:** browser and production checks, actual local-model selection and controlled provider fallback depend on administrator browser policy plus PROD-001/003 access/benchmarks. Repository behavior is not blocked.
- **Evidence:** `docs/verification/ai-002-provider-routing.md`; V-098V-100; focused backend 26/26, full backend 588/588, sidecar fake-transport 22/22, Compose/diff checks pass.
- **Commit:** none.
- **Remaining work:** AI-003/004 must register typed producers/handlers and explicit external task allowlists; complete monthly cross-feature accounting; browser/MariaDB/selected-model/controlled-provider/production verification. Old provider/model configuration remains available for rollback.
- **Remaining work:** keep Strategy/CV local-only until explicit external task approval; move older synchronous AI actions through central usage admission; MariaDB/selected-model/controlled-provider/production verification. Old provider/model configuration remains available for rollback.
### PROD-001 — Read-only production AI inventory and rollout safety
@@ -542,7 +542,7 @@ This queue records the highest-value work that can proceed without production cr
- **Blocker:** browser localhost policy, selected local model, MariaDB, restart canary and production access remain unavailable; worker stays default-off.
- **Evidence:** `docs/verification/ai-003-strategy-snapshot-queue.md`; verification-log V-101V-103; `docs/audits/evidence/ai-003/README.md`.
- **Commit:** `a621226` (`feat(ai): queue strategy snapshots`).
- **Remaining work:** real browser/mobile/theme/refresh/back-forward checks; selected-model timeout/quality test; MariaDB and production single-worker restart/canary/rollback; complete cross-feature usage accounting. No Strategy-specific queue was created.
- **Remaining work:** selected-model timeout/quality test; MariaDB and production single-worker restart/canary/rollback. Local browser/mobile/theme/refresh/back-forward coverage exists in the wider application suite; no Strategy-specific queue was created.
### AI-004 — CV-processing 504 and durable-operation migration
@@ -558,7 +558,7 @@ This queue records the highest-value work that can proceed without production cr
- **Required production verification:** synthetic/local-only canary, no external payload, restart recovery.
- **Status:** `IMPLEMENTED — NOT VERIFIED`.
- **Blocker:** SEC-006 dependency upgrades need internet permission; browser/private-file/MariaDB/production reproduction remains unavailable. Synthetic repository work can continue.
- **Evidence:** `docs/verification/ai-004-cv-processing-queue.md`; V-104V-107/V-180; real SQLite synthetic integration proves 202/active deduplication/owner-scoped handler/retry provenance/notification/review gate and pre-claim cancellation/deadline synchronization; focused operation lifecycle 17/17; backend 660/660; frontend 161/161 and build.
- **Evidence:** `docs/verification/ai-004-cv-processing-queue.md`; V-104V-107/V-180/V-181; real SQLite synthetic integration proves 202/active deduplication/owner-scoped handler/retry provenance/notification/review gate, usage reservation and pre-claim cancellation/deadline synchronization; focused accounting/operation/lifecycle 28/28; backend 663/663; frontend 161/161 and build.
- **Commit:** `c3c5af8` (`feat(cv)!: queue durable processing`).
- **Remaining work:** SEC-006/007 parser dependency/isolation and complete parser-child cancellation; browser synthetic upload/refresh/retry/cancel/review at required widths/themes/keyboard; selected-model and worker-restart canary; MariaDB/production rollout. Do not use the private CV before safeguards.