cesnimda
35 lines
1.9 KiB
Markdown
35 lines
1.9 KiB
Markdown
# PROD-002 workload inventory and synthetic evaluation verification
|
|
|
|
Date: 2026-08-02
|
|
|
|
Status: `VERIFIED LOCALLY`. This package performs classification and fixture validation only; it does not benchmark or call a model.
|
|
|
|
## Delivered
|
|
|
|
- `docs/ai/workload-inventory.md` classifies every reachable AI or deterministic-adjacent task by input/size/output/schema/language/latency/quality/privacy/fallback/mode/determinism/current provider/Pro requirement.
|
|
- `JobTrackerApi.Tests/Fixtures/AiEvaluation/cases.json` contains 19 synthetic cases with constraint-based expected results.
|
|
- `AiEvaluationFixtureTests` proves required category/task coverage, unique IDs, bounded expanded input, `.invalid` contact domains, absence of the authorized private CV/path, strict-JSON assertions and prompt-injection refusal markers.
|
|
|
|
## Required coverage
|
|
|
|
English, Norwegian and mixed CVs; English/Norwegian/noisy/technology-heavy/sparse jobs; email classification; follow-up; Strategy Snapshot; CV tailoring; strict JSON; malformed document text; job/email prompt injection; long input; empty and invalid input.
|
|
|
|
## Results
|
|
|
|
- Fixture validation: 1/1 passed.
|
|
- Full backend regression: 569/569 passed.
|
|
- Full frontend regression: 47/47 suites, 157/157 tests.
|
|
- Frontend production build: passed.
|
|
- No provider, internet, production, paid service, personal document or email was accessed.
|
|
|
|
## Limits
|
|
|
|
- Latency bands are initial benchmark targets, not measurements.
|
|
- Provider/model values are repository defaults, not verified production state.
|
|
- Golden prose is intentionally omitted; later benchmark scoring must test factual constraints, schema, language, safety and useful content rather than exact wording.
|
|
- Model benchmarking and threshold decisions belong to PROD-003 after safe production/local hardware inventory.
|
|
|
|
## Rollback
|
|
|
|
Remove the inventory, synthetic fixture and its validator. No application, dependency, schema, provider or deployment state changed.
|