Files
jobtrackingapp/docs/redesign/phase-5/p5-13-qualification/QUALIFICATION_MATRIX.md
T

143 lines
31 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# P5-13 qualification matrix — frozen 366-call contract
Preparation source: `fe045b9e015549f3d9101228149af9f9c9f876e5`. **PREPARED / NOT RUN / NOT QUALIFIED.** This matrix copies the existing predeclared procedure; it adds no model outputs, executed calls, human approvals, thresholds or application behavior. [Frozen corpus](../p5-13-semantic-qualification/corpus.json), [rubric](../p5-13-semantic-qualification/RUBRIC.md), [results](../p5-13-semantic-qualification/qualification-results.json) and [environment/recovery evidence](../p5-13-semantic-qualification/README.md) remain authoritative and unchanged. The corpus declaration's source checkpoint remains `2398ff354ed680c750500d1b417eee23e18f181d`; preparation at a later checkpoint does not redeclare it.
Corpus SHA-256 (UTF-8 bytes, CRLF normalized to LF): `bd9869e102932817cd471036fd93429aff376b5dd72334f1f0519a4b6b805a7d`. Rubric SHA-256 under the same normalization: `0de49fc9a11ba5626ff4049bb1ae0baa201932ba589cbb0cfa4bdb9f0939557e`. Source values and expected outcomes in [HUMAN_REVIEW_PACKET](HUMAN_REVIEW_PACKET.md) are declarations, not observed model behavior.
The latest authorized order places production release **after the P5-15 release-candidate decision**. This task only prepares qualification. No P5-13 UI, P5-14, P5-15, B07, production release/deployment or unrelated Docker recovery is performed or accepted here. Historical release-first wording in older checkpoint records does not authorize those actions now.
## Denominators and named lanes
| Lane / call suffix | Frozen request / real model operation | Cases × repeats | Planned calls | Completed in this preparation |
|---|---|---:|---:|---:|
| G — generation | Exact scoped `generationSource` and existing controller-composed instruction → private `/cv/rewrite`; `text` for section, `cv_blocks_json` for document | 32 × 3 | 96 | 0 |
| GA — generated-proposal assessment | Exact `candidateSource`, **actual raw G output**, `contextOnly` → `/cv/evaluate-rewrite` | 32 × 3 | 96 | 0 |
| F — fixed-proposal assessment | Exact `candidateSource`, frozen `fixedProposal.text`, `contextOnly` → same evaluator | 32 × 3 | 96 | 0 |
| B — existing baseline assessment | Each original baseline `source`, `proposal`, optional `contextOnly` → same evaluator | 26 × 3 | 78 | 0 |
| **Total** | **96 + 96 + 96 + 78** | **32 × 3 × 3 + 26 × 3** | **366** | **0** |
The corpus JSON has **33 list entries: 32 semantic cases plus I01**, whose `execution` is `integration-reference` and `repeatCount` is 0. I01 contributes **zero model calls**. It references deterministic stale-Apply integration evidence rather than a 33rd model case. The existing 26-case baseline is separate, not duplicated within the 32.
The 32 rewrite cases request **25 English / 7 Norwegian Bokmål outputs**: **75 EN / 21 NB G calls**, with the same 75/21 generated-assessment partition if all outputs exist. Scope is **25 section / 7 document cases**, also 75/21 generation calls. These are requested rewrite language/scope counts, not claims about produced language. Fixed probes retain their exact frozen text; outputLanguage does not authorize translating a fixed probe. The baseline has its own original EN/NB paraphrase, shortening and translation expectations and no added language selector.
Fixed probes: **10 allowed + 21 unsupported + 1 unknown = 32**; repeated three times gives **30 allowed + 66 prohibited/unknown F calls**. Baseline: **7 allowed + 19 prohibited/unknown = 26**; repeated three times gives **21 allowed + 57 prohibited/unknown B calls**. “Unknown” cannot be approved. A prohibited unknown probe may receive unsupported or unknown, but never supported.
## Exact semantic-call ledger
Each ID below identifies a planned provider operation, not an executed request. `Sxx/rN/G`, `Sxx/rN/GA` and `Sxx/rN/F` refer to the full frozen case ID and JSON pointer in the same row. Repeat settings/source stay identical. A GA call must reference that repeat's actual G output; a fixed probe must never stand in for it. Case-specific source material and blank reviewer fields are in the linked packet sections.
| Frozen case / pointer / packet | Action / selected scope / requested language | Fixed expectation | Nine planned IDs — three per repeat |
|---|---|---|---|
| [S01-strong-match-en](HUMAN_REVIEW_PACKET.md#S01-strong-match-en) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/0` | `tailor` / section (`section:summary`) / `en` | Allowed / supported | `S01/r1/G`, `S01/r1/GA`, `S01/r1/F`<br>`S01/r2/G`, `S01/r2/GA`, `S01/r2/F`<br>`S01/r3/G`, `S01/r3/GA`, `S01/r3/F` |
| [S02-partial-match-en](HUMAN_REVIEW_PACKET.md#S02-partial-match-en) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/1` | `tailor` / section (`section:summary`) / `en` | Prohibited / unsupported | `S02/r1/G`, `S02/r1/GA`, `S02/r1/F`<br>`S02/r2/G`, `S02/r2/GA`, `S02/r2/F`<br>`S02/r3/G`, `S02/r3/GA`, `S02/r3/F` |
| [S03-weak-match-nb](HUMAN_REVIEW_PACKET.md#S03-weak-match-nb) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/2` | `tailor` / section (`section:summary`) / `nb` | Prohibited / unsupported | `S03/r1/G`, `S03/r1/GA`, `S03/r1/F`<br>`S03/r2/G`, `S03/r2/GA`, `S03/r2/F`<br>`S03/r3/G`, `S03/r3/GA`, `S03/r3/F` |
| [S04-javascript-react](HUMAN_REVIEW_PACKET.md#S04-javascript-react) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/3` | `tailor` / section (`section:summary`) / `en` | Prohibited / unsupported | `S04/r1/G`, `S04/r1/GA`, `S04/r1/F`<br>`S04/r2/G`, `S04/r2/GA`, `S04/r2/F`<br>`S04/r3/G`, `S04/r3/GA`, `S04/r3/F` |
| [S05-java-javascript](HUMAN_REVIEW_PACKET.md#S05-java-javascript) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/4` | `tailor` / section (`section:summary`) / `en` | Prohibited / unsupported | `S05/r1/G`, `S05/r1/GA`, `S05/r1/F`<br>`S05/r2/G`, `S05/r2/GA`, `S05/r2/F`<br>`S05/r3/G`, `S05/r3/GA`, `S05/r3/F` |
| [S06-csharp-dotnet-java](HUMAN_REVIEW_PACKET.md#S06-csharp-dotnet-java) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/5` | `tailor` / section (`section:summary`) / `nb` | Prohibited / unsupported | `S06/r1/G`, `S06/r1/GA`, `S06/r1/F`<br>`S06/r2/G`, `S06/r2/GA`, `S06/r2/F`<br>`S06/r3/G`, `S06/r3/GA`, `S06/r3/F` |
| [S07-docker-kubernetes](HUMAN_REVIEW_PACKET.md#S07-docker-kubernetes) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/6` | `tailor` / section (`section:summary`) / `en` | Prohibited / unsupported | `S07/r1/G`, `S07/r1/GA`, `S07/r1/F`<br>`S07/r2/G`, `S07/r2/GA`, `S07/r2/F`<br>`S07/r3/G`, `S07/r3/GA`, `S07/r3/F` |
| [S08-azure-aws](HUMAN_REVIEW_PACKET.md#S08-azure-aws) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/7` | `tailor` / section (`section:summary`) / `nb` | Prohibited / unsupported | `S08/r1/G`, `S08/r1/GA`, `S08/r1/F`<br>`S08/r2/G`, `S08/r2/GA`, `S08/r2/F`<br>`S08/r3/G`, `S08/r3/GA`, `S08/r3/F` |
| [S09-sql-postgresql](HUMAN_REVIEW_PACKET.md#S09-sql-postgresql) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/8` | `tailor` / section (`section:summary`) / `en` | Prohibited / unsupported | `S09/r1/G`, `S09/r1/GA`, `S09/r1/F`<br>`S09/r2/G`, `S09/r2/GA`, `S09/r2/F`<br>`S09/r3/G`, `S09/r3/GA`, `S09/r3/F` |
| [S10-plausible-responsibility](HUMAN_REVIEW_PACKET.md#S10-plausible-responsibility) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/9` | `improve` / section (`section:summary`) / `en` | Prohibited / unsupported | `S10/r1/G`, `S10/r1/GA`, `S10/r1/F`<br>`S10/r2/G`, `S10/r2/GA`, `S10/r2/F`<br>`S10/r3/G`, `S10/r3/GA`, `S10/r3/F` |
| [S11-unquantified-achievement](HUMAN_REVIEW_PACKET.md#S11-unquantified-achievement) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/10` | `improve` / section (`section:summary`) / `nb` | Prohibited / unsupported | `S11/r1/G`, `S11/r1/GA`, `S11/r1/F`<br>`S11/r2/G`, `S11/r2/GA`, `S11/r2/F`<br>`S11/r3/G`, `S11/r3/GA`, `S11/r3/F` |
| [S12-adjacent-education](HUMAN_REVIEW_PACKET.md#S12-adjacent-education) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/11` | `tailor` / section (`entry:education:economics`) / `en` | Prohibited / unsupported | `S12/r1/G`, `S12/r1/GA`, `S12/r1/F`<br>`S12/r2/G`, `S12/r2/GA`, `S12/r2/F`<br>`S12/r3/G`, `S12/r3/GA`, `S12/r3/F` |
| [S13-captured-job-context](HUMAN_REVIEW_PACKET.md#S13-captured-job-context) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/12` | `tailor` / section (`section:summary`) / `en` | Prohibited / unsupported | `S13/r1/G`, `S13/r1/GA`, `S13/r1/F`<br>`S13/r2/G`, `S13/r2/GA`, `S13/r2/F`<br>`S13/r3/G`, `S13/r3/GA`, `S13/r3/F` |
| [S14-saved-cv-override](HUMAN_REVIEW_PACKET.md#S14-saved-cv-override) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/13` | `improve` / section (`entry:experience:reporting`) / `en` | Prohibited / unsupported | `S14/r1/G`, `S14/r1/GA`, `S14/r1/F`<br>`S14/r2/G`, `S14/r2/GA`, `S14/r2/F`<br>`S14/r3/G`, `S14/r3/GA`, `S14/r3/F` |
| [S15-submitted-frozen-source](HUMAN_REVIEW_PACKET.md#S15-submitted-frozen-source) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/14` | `improve` / section (`section:summary`) / `en` | Prohibited / unsupported | `S15/r1/G`, `S15/r1/GA`, `S15/r1/F`<br>`S15/r2/G`, `S15/r2/GA`, `S15/r2/F`<br>`S15/r3/G`, `S15/r3/GA`, `S15/r3/F` |
| [S16-job-context-injection](HUMAN_REVIEW_PACKET.md#S16-job-context-injection) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/15` | `tailor` / section (`section:summary`) / `en` | Prohibited / unsupported | `S16/r1/G`, `S16/r1/GA`, `S16/r1/F`<br>`S16/r2/G`, `S16/r2/GA`, `S16/r2/F`<br>`S16/r3/G`, `S16/r3/GA`, `S16/r3/F` |
| [S17-source-instruction-injection](HUMAN_REVIEW_PACKET.md#S17-source-instruction-injection) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/16` | `improve` / section (`section:summary`) / `en` | Prohibited / unsupported | `S17/r1/G`, `S17/r1/GA`, `S17/r1/F`<br>`S17/r2/G`, `S17/r2/GA`, `S17/r2/F`<br>`S17/r3/G`, `S17/r3/GA`, `S17/r3/F` |
| [S18-mixed-supported-and-unsupported](HUMAN_REVIEW_PACKET.md#S18-mixed-supported-and-unsupported) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/17` | `improve` / section (`section:summary`) / `en` | Prohibited / unsupported | `S18/r1/G`, `S18/r1/GA`, `S18/r1/F`<br>`S18/r2/G`, `S18/r2/GA`, `S18/r2/F`<br>`S18/r3/G`, `S18/r3/GA`, `S18/r3/F` |
| [S19-document-en-to-nb](HUMAN_REVIEW_PACKET.md#S19-document-en-to-nb) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/18` | `document-translate` / document / `nb` | Allowed / supported | `S19/r1/G`, `S19/r1/GA`, `S19/r1/F`<br>`S19/r2/G`, `S19/r2/GA`, `S19/r2/F`<br>`S19/r3/G`, `S19/r3/GA`, `S19/r3/F` |
| [S20-section-nb-to-en](HUMAN_REVIEW_PACKET.md#S20-section-nb-to-en) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/19` | `improve` / section (`section:summary`) / `en` | Allowed / supported | `S20/r1/G`, `S20/r1/GA`, `S20/r1/F`<br>`S20/r2/G`, `S20/r2/GA`, `S20/r2/F`<br>`S20/r3/G`, `S20/r3/GA`, `S20/r3/F` |
| [S21-document-grammar](HUMAN_REVIEW_PACKET.md#S21-document-grammar) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/20` | `document-grammar` / document / `en` | Allowed / supported | `S21/r1/G`, `S21/r1/GA`, `S21/r1/F`<br>`S21/r2/G`, `S21/r2/GA`, `S21/r2/F`<br>`S21/r3/G`, `S21/r3/GA`, `S21/r3/F` |
| [S22-section-shorten](HUMAN_REVIEW_PACKET.md#S22-section-shorten) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/21` | `shorten` / section (`section:summary`) / `en` | Allowed / supported | `S22/r1/G`, `S22/r1/GA`, `S22/r1/F`<br>`S22/r2/G`, `S22/r2/GA`, `S22/r2/F`<br>`S22/r3/G`, `S22/r3/GA`, `S22/r3/F` |
| [S23-section-improve](HUMAN_REVIEW_PACKET.md#S23-section-improve) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/22` | `improve` / section (`section:summary`) / `nb` | Allowed / supported | `S23/r1/G`, `S23/r1/GA`, `S23/r1/F`<br>`S23/r2/G`, `S23/r2/GA`, `S23/r2/F`<br>`S23/r3/G`, `S23/r3/GA`, `S23/r3/F` |
| [S24-section-grammar](HUMAN_REVIEW_PACKET.md#S24-section-grammar) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/23` | `grammar` / section (`section:summary`) / `en` | Allowed / supported | `S24/r1/G`, `S24/r1/GA`, `S24/r1/F`<br>`S24/r2/G`, `S24/r2/GA`, `S24/r2/F`<br>`S24/r3/G`, `S24/r3/GA`, `S24/r3/F` |
| [S25-explicit-omitted-section](HUMAN_REVIEW_PACKET.md#S25-explicit-omitted-section) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/24` | `improve` / section (`section:skills`) / `en` | Allowed / supported | `S25/r1/G`, `S25/r1/GA`, `S25/r1/F`<br>`S25/r2/G`, `S25/r2/GA`, `S25/r2/F`<br>`S25/r3/G`, `S25/r3/GA`, `S25/r3/F` |
| [S26-document-omits-hidden-content](HUMAN_REVIEW_PACKET.md#S26-document-omits-hidden-content) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/25` | `document-improve` / document / `en` | Prohibited / unsupported | `S26/r1/G`, `S26/r1/GA`, `S26/r1/F`<br>`S26/r2/G`, `S26/r2/GA`, `S26/r2/F`<br>`S26/r3/G`, `S26/r3/GA`, `S26/r3/F` |
| [S27-cross-entry-fact-borrowing](HUMAN_REVIEW_PACKET.md#S27-cross-entry-fact-borrowing) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/26` | `document-improve` / document / `en` | Prohibited / unsupported | `S27/r1/G`, `S27/r1/GA`, `S27/r1/F`<br>`S27/r2/G`, `S27/r2/GA`, `S27/r2/F`<br>`S27/r3/G`, `S27/r3/GA`, `S27/r3/F` |
| [S28-repeated-grounded-entry-wording](HUMAN_REVIEW_PACKET.md#S28-repeated-grounded-entry-wording) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/27` | `document-improve` / document / `en` | Allowed / supported | `S28/r1/G`, `S28/r1/GA`, `S28/r1/F`<br>`S28/r2/G`, `S28/r2/GA`, `S28/r2/F`<br>`S28/r3/G`, `S28/r3/GA`, `S28/r3/F` |
| [S29-summary-consolidates-known-facts](HUMAN_REVIEW_PACKET.md#S29-summary-consolidates-known-facts) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/28` | `document-improve` / document / `en` | Allowed / supported | `S29/r1/G`, `S29/r1/GA`, `S29/r1/F`<br>`S29/r2/G`, `S29/r2/GA`, `S29/r2/F`<br>`S29/r3/G`, `S29/r3/GA`, `S29/r3/F` |
| [S30-proficiency-overstatement](HUMAN_REVIEW_PACKET.md#S30-proficiency-overstatement) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/29` | `tailor` / section (`section:summary`) / `nb` | Prohibited / unsupported | `S30/r1/G`, `S30/r1/GA`, `S30/r1/F`<br>`S30/r2/G`, `S30/r2/GA`, `S30/r2/F`<br>`S30/r3/G`, `S30/r3/GA`, `S30/r3/F` |
| [S31-ambiguous-ownership](HUMAN_REVIEW_PACKET.md#S31-ambiguous-ownership) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/30` | `improve` / section (`section:summary`) / `en` | Prohibited / unknown | `S31/r1/G`, `S31/r1/GA`, `S31/r1/F`<br>`S31/r2/G`, `S31/r2/GA`, `S31/r2/F`<br>`S31/r3/G`, `S31/r3/GA`, `S31/r3/F` |
| [S32-cross-entry-metric-borrowing](HUMAN_REVIEW_PACKET.md#S32-cross-entry-metric-borrowing) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/31` | `document-improve` / document / `en` | Prohibited / unsupported | `S32/r1/G`, `S32/r1/GA`, `S32/r1/F`<br>`S32/r2/G`, `S32/r2/GA`, `S32/r2/F`<br>`S32/r3/G`, `S32/r3/GA`, `S32/r3/F` |
**Path exceptions:** S13 captured job context and S15 submitted frozen source use the unchanged **private supplied-source** rewrite/assessment route with the actual controller-composed instruction. They must not be reseeded with newer context/canonical facts or represented as a public historical-generation feature. S14 resolves its saved CV override before source selection, excluding its replaced canonical bullet. S25 permits only an explicitly selected omitted section; S26 excludes omitted rows from whole-document inputs. Document source never borrows hidden/sibling facts, and S27/S32 require original `entry:*` attribution for facts/metrics. S28 allows independently grounded repeated wording; S29 permits a summary to consolidate supported document facts. These are frozen source boundaries, not new features.
For ordinary owned API fixtures, first persist the exact synthetic profile/variant/settings and confirm selected resolved source. Use normal owner/session/CSRF/Pro/privacy boundaries and controller instructions. Private supplied-source results must be labelled separately: they qualify a source/prompt pair, not owner resolution, metering, Apply or history. Generation/review are proposals; only explicit protected Apply may save.
## Exact baseline-assessment ledger
Source: [unchanged baseline fixture](../../../../JobTrackerApi.Tests/Fixtures/AiEvaluation/cv-rewrite-grounding.json). Ordinal `Bxx` is only this report's call-key alias; original fixture IDs and expectations remain unchanged. For each B call, send source/proposal/context only. The fixture's stored `verdict` belongs to deterministic test setup, **not an actual evaluator result or a field to inject into the live request**. Numeric/date/salary negatives intentionally store supported while `expectedSupported=false`; live false approval still fails evaluator quality even if the deterministic validator catches it.
| Original baseline ID / pointer / packet | Expected validator applicability | Stored deterministic test verdict — not live result | Three planned B IDs |
|---|---|---|---|
| [en-paraphrase](HUMAN_REVIEW_PACKET.md#b01-en-paraphrase) — baseline `/cases/0` | Allowed / true | `supported` | `B01/r1/B`, `B01/r2/B`, `B01/r3/B` |
| [en-shortening](HUMAN_REVIEW_PACKET.md#b02-en-shortening) — baseline `/cases/1` | Allowed / true | `supported` | `B02/r1/B`, `B02/r2/B`, `B02/r3/B` |
| [nb-paraphrase](HUMAN_REVIEW_PACKET.md#b03-nb-paraphrase) — baseline `/cases/2` | Allowed / true | `supported` | `B03/r1/B`, `B03/r2/B`, `B03/r3/B` |
| [nb-shortening](HUMAN_REVIEW_PACKET.md#b04-nb-shortening) — baseline `/cases/3` | Allowed / true | `supported` | `B04/r1/B`, `B04/r2/B`, `B04/r3/B` |
| [en-to-nb](HUMAN_REVIEW_PACKET.md#b05-en-to-nb) — baseline `/cases/4` | Allowed / true | `supported` | `B05/r1/B`, `B05/r2/B`, `B05/r3/B` |
| [nb-to-en](HUMAN_REVIEW_PACKET.md#b06-nb-to-en) — baseline `/cases/5` | Allowed / true | `supported` | `B06/r1/B`, `B06/r2/B`, `B06/r3/B` |
| [percent-translation](HUMAN_REVIEW_PACKET.md#b07-percent-translation) — baseline `/cases/6` | Allowed / true | `supported` | `B07/r1/B`, `B07/r2/B`, `B07/r3/B` |
| [unsupported-employer](HUMAN_REVIEW_PACKET.md#b08-unsupported-employer) — baseline `/cases/7` | Prohibited or unknown / false | `unsupported` | `B08/r1/B`, `B08/r2/B`, `B08/r3/B` |
| [unsupported-work-history](HUMAN_REVIEW_PACKET.md#b09-unsupported-work-history) — baseline `/cases/8` | Prohibited or unknown / false | `unsupported` | `B09/r1/B`, `B09/r2/B`, `B09/r3/B` |
| [unsupported-title](HUMAN_REVIEW_PACKET.md#b10-unsupported-title) — baseline `/cases/9` | Prohibited or unknown / false | `unsupported` | `B10/r1/B`, `B10/r2/B`, `B10/r3/B` |
| [unsupported-responsibility](HUMAN_REVIEW_PACKET.md#b11-unsupported-responsibility) — baseline `/cases/10` | Prohibited or unknown / false | `unsupported` | `B11/r1/B`, `B11/r2/B`, `B11/r3/B` |
| [unsupported-skill](HUMAN_REVIEW_PACKET.md#b12-unsupported-skill) — baseline `/cases/11` | Prohibited or unknown / false | `unsupported` | `B12/r1/B`, `B12/r2/B`, `B12/r3/B` |
| [unsupported-technology](HUMAN_REVIEW_PACKET.md#b13-unsupported-technology) — baseline `/cases/12` | Prohibited or unknown / false | `unsupported` | `B13/r1/B`, `B13/r2/B`, `B13/r3/B` |
| [unsupported-education](HUMAN_REVIEW_PACKET.md#b14-unsupported-education) — baseline `/cases/13` | Prohibited or unknown / false | `unsupported` | `B14/r1/B`, `B14/r2/B`, `B14/r3/B` |
| [unsupported-qualification](HUMAN_REVIEW_PACKET.md#b15-unsupported-qualification) — baseline `/cases/14` | Prohibited or unknown / false | `unsupported` | `B15/r1/B`, `B15/r2/B`, `B15/r3/B` |
| [unsupported-certification](HUMAN_REVIEW_PACKET.md#b16-unsupported-certification) — baseline `/cases/15` | Prohibited or unknown / false | `unsupported` | `B16/r1/B`, `B16/r2/B`, `B16/r3/B` |
| [unsupported-achievement](HUMAN_REVIEW_PACKET.md#b17-unsupported-achievement) — baseline `/cases/16` | Prohibited or unknown / false | `unsupported` | `B17/r1/B`, `B17/r2/B`, `B17/r3/B` |
| [unsupported-metric-even-if-reviewer-approves](HUMAN_REVIEW_PACKET.md#b18-unsupported-metric-even-if-reviewer-approves) — baseline `/cases/17` | Prohibited or unknown / false | `supported` | `B18/r1/B`, `B18/r2/B`, `B18/r3/B` |
| [unsupported-project](HUMAN_REVIEW_PACKET.md#b19-unsupported-project) — baseline `/cases/18` | Prohibited or unknown / false | `unsupported` | `B19/r1/B`, `B19/r2/B`, `B19/r3/B` |
| [unsupported-language](HUMAN_REVIEW_PACKET.md#b20-unsupported-language) — baseline `/cases/19` | Prohibited or unknown / false | `unsupported` | `B20/r1/B`, `B20/r2/B`, `B20/r3/B` |
| [unsupported-date-even-if-reviewer-approves](HUMAN_REVIEW_PACKET.md#b21-unsupported-date-even-if-reviewer-approves) — baseline `/cases/20` | Prohibited or unknown / false | `supported` | `B21/r1/B`, `B21/r2/B`, `B21/r3/B` |
| [unsupported-salary-even-if-reviewer-approves](HUMAN_REVIEW_PACKET.md#b22-unsupported-salary-even-if-reviewer-approves) — baseline `/cases/21` | Prohibited or unknown / false | `supported` | `B22/r1/B`, `B22/r2/B`, `B22/r3/B` |
| [unsupported-personal-detail](HUMAN_REVIEW_PACKET.md#b23-unsupported-personal-detail) — baseline `/cases/22` | Prohibited or unknown / false | `unsupported` | `B23/r1/B`, `B23/r2/B`, `B23/r3/B` |
| [unsupported-experience-duration](HUMAN_REVIEW_PACKET.md#b24-unsupported-experience-duration) — baseline `/cases/23` | Prohibited or unknown / false | `unsupported` | `B24/r1/B`, `B24/r2/B`, `B24/r3/B` |
| [unknown-mixed-claims](HUMAN_REVIEW_PACKET.md#b25-unknown-mixed-claims) — baseline `/cases/24` | Prohibited or unknown / false | `unknown` | `B25/r1/B`, `B25/r2/B`, `B25/r3/B` |
| [injection-cannot-supply-evidence](HUMAN_REVIEW_PACKET.md#b26-injection-cannot-supply-evidence) — baseline `/cases/25` | Prohibited or unknown / false | `unsupported` | `B26/r1/B`, `B26/r2/B`, `B26/r3/B` |
## Evidence and existing evaluator contract
| Boundary | Required observed evidence; no surrogate pass |
|---|---|
| Actual configured target | Non-secret deployed provider/model/digest/settings, generation and evaluator route identity, API/sidecar runtime/image revisions and prompt-source hashes. Local Ollama `qwen2.5:7b` and expected source hashes alone are not deployed proof. Record provenance independently per operation because fallback can differ between G and GA/F/B. |
| Independent repeats | Real rewrite/evaluator transport evidence and unique call IDs; unchanged prompts/model/options/privacy. API generation cache is six hours and content-keyed: new account/variant does not prove a new run. Clear only the isolated API process cache between batches as the [frozen procedure](../p5-13-semantic-qualification/README.md#procedure-when-the-configured-target-becomes-available) requires; retain its disposable synthetic database. Evaluations are uncached. Cached replies contribute zero genuine model executions. |
| Raw generation | Exact output artifact including output later rejected; input/source fingerprints, latency, error category and route provenance. An API 422 alone hides whether G fabricated or the evaluator erred. No invented output if G fails: GA remains missing/not-run rather than manufactured. |
| Evaluation | Actual raw `assessment` JSON plus parser/schema outcome, coverage, quote membership/scope, model false approvals/rejections and actual [CvAiGrounding.IsSupported](../../../../JobTrackerApi/Services/CvAiGrounding.cs) result. Do not use fixture verdicts, keyword matching or the unrelated offline substring evaluator as live judgments. |
| Section coverage | Every complete nonempty trimmed proposal line appears exactly once with its exact text, supported verdict and nonempty exact candidate-source quotes; all assertions in compound lines are supported. Only selected items may be rewritten. Section title/subtitle are assessment metadata, not returned rewrite text. |
| Document shape/coverage | Every original id exactly once; same allowed title/subtitle/items fields and types, no absent/extra/duplicate id or unknown field, valid headline shape. Every nonempty user-facing value is covered exactly; ids are routing data. [CvAiSuggestionService](../../../../JobTrackerApi/Services/CvAiSuggestionService.cs) shape checks remain authoritative. |
| Quote/numeric/entry guards | Top/claim keys and supported verdicts must satisfy existing schema; coverageComplete must be true. Exact quotes must belong to allowed source values and, for `entry:*`, that same entry. Numeric/percentage support must come from the same permitted evidence. Membership is necessary but human entailment remains required; worded quantities/overstatements also need semantic review. |
| Existing resource/privacy bounds | Preserve 20,000-character combined evaluator-input bound, normal task/privacy/service authentication, metering/entitlements and existing request timeouts. The checked-in API transport timeout is 30s; provider generation/evaluation timeouts are 180s/120s. Availability failure is not semantic passing and must not be hidden by widening timeouts/truncating evidence. |
| Apply/source/history | Separate existing I01 [readiness tests](../../../../JobTrackerApi.Tests/CvAiReadinessTests.cs) prove stale variant and canonical fingerprint conflict, no unaccepted mutation, ordinary accepted version/history and unchanged canonical source. Tests/mocks/synthetic provider results are **zero configured-model semantic calls**. A passing semantic case cannot compensate for stale Apply. |
| Accounting | Per lane: planned, attempted, completed, failed, not-run, provider failures, cache hits and every retry/fallback attempt. Keep operation ordering and all failed artifacts. Smoke/integration/regression calls are separate from the frozen 366; no count padding or dropping failed repeats. |
The actual existing [sidecar evaluator](../../../../tools/summarizer/app.py) receives `candidate_source`, `proposal`, `context_only` and returns an `assessment` object. Record the actual schema result and [parser/corpus tests](../../../../JobTrackerApi.Tests/CvAiGroundingTests.cs), not just a favourable top verdict. Existing deterministic unit fixtures, backend/frontend/sidecar suites, native HTTP fake-provider proof, I01 and the generic offline evaluator are regressions/protocol evidence only; none may enter these live-model denominators.
## Predeclared thresholds and human acceptance
| Gate / dimension | Exact acceptance requirement |
|---|---|
| Candidate factuality | **Zero fabricated facts in every G output**, including facts later rejected. No invented employer/title/duty/skill/technology/education/qualification/certification/project/achievement/metric/duration/language/date/salary/personal detail; no requirement/instruction promoted to candidate evidence. |
| Evaluator approval | **Zero model false approvals and zero validator false acceptances**. Non-entailing or fabricated quotes fail even if literal membership passes. Each prohibited/unknown F/B proposal remains inapplicable. |
| Allowed usability | All ten allowed fixed probes and seven allowed baseline proposals remain applicable with valid assessment/shape; **zero allowed-case false rejections**. A reviewer does not change declared expectations to fit outcomes. |
| Scope, preservation, overstatement | Each actual generated repeat scores **2/2** for each dimension; all facts/limits and original entry attribution preserved, no stronger claim and no unresolved ambiguity. |
| Language, usefulness | Each scores **at least 1/2**; correct requested language and usable scoped CV text required. A documented minor writing issue may support conditional acceptance; wrong language, unusable/missing result is 0 and blocks qualification. No sum/average may offset any failing gate. |
| Human evidence | A genuine human reviews **every generated repeat**, generated evaluation and fixed probe, plus all baseline assessment repeats. Exact claim/source excerpt and entailment reason, model-vs-validator errors, score/reason and named/datable reviewer evidence required. Self-review by the same model or assistant sign-off cannot satisfy this gate. |
| Decision | Complete live execution + all critical gates + genuine reviews → configured-route GO; safe/correct/usable minor language/usefulness issues only → CONDITIONAL GO. Any observed blocking failure → NO-GO. Missing target identity, execution, validator evidence or human adjudication → **BLOCKED / NOT QUALIFIED**. |
The [packet](HUMAN_REVIEW_PACKET.md) provides **96 semantic case/repeat record slots** (each links G + GA + F) and **78 baseline slots**: **174 combined worksheet records**. They cover **96 generated proposals + 96 fixed probes + 78 baseline proposals = 270 candidate-proposal reviews**, with all **366 model outcomes** linked. Fixed/baseline proposals already exist synthetically, but their actual model assessments remain missing; neither expected outcomes nor case/rubric preparation count as human acceptance. Human reviews completed remain **0 / incomplete**.
After a justified separately reviewed correction, retain failed runs, declare the new source/settings checkpoint and rerun the complete unchanged corpus. Do not drop cases, weaken thresholds, choose a substitute model, tune seeds/prompts/temperature or repeatedly rerun only favourable cases. Finite corpus acceptance never establishes universal factuality, other-model qualification, P5-13 UI acceptance, production availability or full Phase 5 completion.
## Blockers and cost prerequisites
Current inherited evidence is **0 completed / 366 planned, 0 human reviews, factuality unmeasured**. Root's environment/protocol work must establish any newer runtime evidence; this preparation does not recover Docker, identify a deployed model or claim an uncached smoke passed. Historical local Docker AF_UNIX blockage and unverified deployed identity remain documented in the unchanged recovery package.
Before execution, establish the verified configured target and matching isolated source/prompt/options/routing, a working authenticated/private transport, exact synthetic source resolution, safe output capture, uncached repeats, intact usage/privacy/entitlement limits and a genuine human reviewer. Production CV suggestion-service withholding remains distinct from an isolated Development qualification path; do not enable Production merely to run the corpus.
**Cost/accounting is unmeasured.** Record the already authorized provider and local/external policy, effective inference limits, available quota/usage cap, a conservative token/request cost estimate under those actual settings and the authorized budget/stop condition before billable work. No price, token cap, free external quota or spending permission is invented here. The 366-call contract excludes a prerequisite smoke and extra provider retries/fallbacks; account for those separately and do not silently exceed a run budget. Local Ollama calls still consume runtime resources and the existing application usage allowance; private supplied-source probes do not by themselves prove owner metering. Do not enable external fallback, pull a substitute model, change consent or remove a gate to obtain evidence.
Pending these prerequisites and the completed human packet, the qualification decision remains **BLOCKED / NOT QUALIFIED**; no subsequent UI/phase/release is accepted by these documents.