jobtrackingapp/docs/verification/prod-002-ai-evaluation.md

PROD-002 workload inventory and synthetic evaluation verification

Date: 2026-08-02

Status: VERIFIED LOCALLY. This package performs classification and fixture validation only; it does not benchmark or call a model.

Delivered

• docs/ai/workload-inventory.md classifies every reachable AI or deterministic-adjacent task by input/size/output/schema/language/latency/quality/privacy/fallback/mode/determinism/current provider/Pro requirement.
• JobTrackerApi.Tests/Fixtures/AiEvaluation/cases.json contains 19 synthetic cases with constraint-based expected results.
• AiEvaluationFixtureTests proves required category/task coverage, unique IDs, bounded expanded input, .invalid contact domains, absence of the authorized private CV/path, strict-JSON assertions and prompt-injection refusal markers.

Required coverage

English, Norwegian and mixed CVs; English/Norwegian/noisy/technology-heavy/sparse jobs; email classification; follow-up; Strategy Snapshot; CV tailoring; strict JSON; malformed document text; job/email prompt injection; long input; empty and invalid input.

Results

• Fixture validation: 1/1 passed.
• Full backend regression: 569/569 passed.
• Full frontend regression: 47/47 suites, 157/157 tests.
• Frontend production build: passed.
• No provider, internet, production, paid service, personal document or email was accessed.

Limits

• Latency bands are initial benchmark targets, not measurements.
• Provider/model values are repository defaults, not verified production state.
• Golden prose is intentionally omitted; later benchmark scoring must test factual constraints, schema, language, safety and useful content rather than exact wording.
• Model benchmarking and threshold decisions belong to PROD-003 after safe production/local hardware inventory.

Rollback

Remove the inventory, synthetic fixture and its validator. No application, dependency, schema, provider or deployment state changed.