143 lines
31 KiB
Markdown
143 lines
31 KiB
Markdown
# P5-13 qualification matrix — frozen 366-call contract
|
||
|
||
Preparation source: `fe045b9e015549f3d9101228149af9f9c9f876e5`. **PREPARED / NOT RUN / NOT QUALIFIED.** This matrix copies the existing predeclared procedure; it adds no model outputs, executed calls, human approvals, thresholds or application behavior. [Frozen corpus](../p5-13-semantic-qualification/corpus.json), [rubric](../p5-13-semantic-qualification/RUBRIC.md), [results](../p5-13-semantic-qualification/qualification-results.json) and [environment/recovery evidence](../p5-13-semantic-qualification/README.md) remain authoritative and unchanged. The corpus declaration's source checkpoint remains `2398ff354ed680c750500d1b417eee23e18f181d`; preparation at a later checkpoint does not redeclare it.
|
||
|
||
Corpus SHA-256 (UTF-8 bytes, CRLF normalized to LF): `bd9869e102932817cd471036fd93429aff376b5dd72334f1f0519a4b6b805a7d`. Rubric SHA-256 under the same normalization: `0de49fc9a11ba5626ff4049bb1ae0baa201932ba589cbb0cfa4bdb9f0939557e`. Source values and expected outcomes in [HUMAN_REVIEW_PACKET](HUMAN_REVIEW_PACKET.md) are declarations, not observed model behavior.
|
||
|
||
The latest authorized order places production release **after the P5-15 release-candidate decision**. This task only prepares qualification. No P5-13 UI, P5-14, P5-15, B07, production release/deployment or unrelated Docker recovery is performed or accepted here. Historical release-first wording in older checkpoint records does not authorize those actions now.
|
||
|
||
## Denominators and named lanes
|
||
|
||
| Lane / call suffix | Frozen request / real model operation | Cases × repeats | Planned calls | Completed in this preparation |
|
||
|---|---|---:|---:|---:|
|
||
| G — generation | Exact scoped `generationSource` and existing controller-composed instruction → private `/cv/rewrite`; `text` for section, `cv_blocks_json` for document | 32 × 3 | 96 | 0 |
|
||
| GA — generated-proposal assessment | Exact `candidateSource`, **actual raw G output**, `contextOnly` → `/cv/evaluate-rewrite` | 32 × 3 | 96 | 0 |
|
||
| F — fixed-proposal assessment | Exact `candidateSource`, frozen `fixedProposal.text`, `contextOnly` → same evaluator | 32 × 3 | 96 | 0 |
|
||
| B — existing baseline assessment | Each original baseline `source`, `proposal`, optional `contextOnly` → same evaluator | 26 × 3 | 78 | 0 |
|
||
| **Total** | **96 + 96 + 96 + 78** | **32 × 3 × 3 + 26 × 3** | **366** | **0** |
|
||
|
||
The corpus JSON has **33 list entries: 32 semantic cases plus I01**, whose `execution` is `integration-reference` and `repeatCount` is 0. I01 contributes **zero model calls**. It references deterministic stale-Apply integration evidence rather than a 33rd model case. The existing 26-case baseline is separate, not duplicated within the 32.
|
||
|
||
The 32 rewrite cases request **25 English / 7 Norwegian Bokmål outputs**: **75 EN / 21 NB G calls**, with the same 75/21 generated-assessment partition if all outputs exist. Scope is **25 section / 7 document cases**, also 75/21 generation calls. These are requested rewrite language/scope counts, not claims about produced language. Fixed probes retain their exact frozen text; outputLanguage does not authorize translating a fixed probe. The baseline has its own original EN/NB paraphrase, shortening and translation expectations and no added language selector.
|
||
|
||
Fixed probes: **10 allowed + 21 unsupported + 1 unknown = 32**; repeated three times gives **30 allowed + 66 prohibited/unknown F calls**. Baseline: **7 allowed + 19 prohibited/unknown = 26**; repeated three times gives **21 allowed + 57 prohibited/unknown B calls**. “Unknown” cannot be approved. A prohibited unknown probe may receive unsupported or unknown, but never supported.
|
||
|
||
## Exact semantic-call ledger
|
||
|
||
Each ID below identifies a planned provider operation, not an executed request. `Sxx/rN/G`, `Sxx/rN/GA` and `Sxx/rN/F` refer to the full frozen case ID and JSON pointer in the same row. Repeat settings/source stay identical. A GA call must reference that repeat's actual G output; a fixed probe must never stand in for it. Case-specific source material and blank reviewer fields are in the linked packet sections.
|
||
|
||
| Frozen case / pointer / packet | Action / selected scope / requested language | Fixed expectation | Nine planned IDs — three per repeat |
|
||
|---|---|---|---|
|
||
| [S01-strong-match-en](HUMAN_REVIEW_PACKET.md#S01-strong-match-en) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/0` | `tailor` / section (`section:summary`) / `en` | Allowed / supported | `S01/r1/G`, `S01/r1/GA`, `S01/r1/F`<br>`S01/r2/G`, `S01/r2/GA`, `S01/r2/F`<br>`S01/r3/G`, `S01/r3/GA`, `S01/r3/F` |
|
||
| [S02-partial-match-en](HUMAN_REVIEW_PACKET.md#S02-partial-match-en) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/1` | `tailor` / section (`section:summary`) / `en` | Prohibited / unsupported | `S02/r1/G`, `S02/r1/GA`, `S02/r1/F`<br>`S02/r2/G`, `S02/r2/GA`, `S02/r2/F`<br>`S02/r3/G`, `S02/r3/GA`, `S02/r3/F` |
|
||
| [S03-weak-match-nb](HUMAN_REVIEW_PACKET.md#S03-weak-match-nb) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/2` | `tailor` / section (`section:summary`) / `nb` | Prohibited / unsupported | `S03/r1/G`, `S03/r1/GA`, `S03/r1/F`<br>`S03/r2/G`, `S03/r2/GA`, `S03/r2/F`<br>`S03/r3/G`, `S03/r3/GA`, `S03/r3/F` |
|
||
| [S04-javascript-react](HUMAN_REVIEW_PACKET.md#S04-javascript-react) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/3` | `tailor` / section (`section:summary`) / `en` | Prohibited / unsupported | `S04/r1/G`, `S04/r1/GA`, `S04/r1/F`<br>`S04/r2/G`, `S04/r2/GA`, `S04/r2/F`<br>`S04/r3/G`, `S04/r3/GA`, `S04/r3/F` |
|
||
| [S05-java-javascript](HUMAN_REVIEW_PACKET.md#S05-java-javascript) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/4` | `tailor` / section (`section:summary`) / `en` | Prohibited / unsupported | `S05/r1/G`, `S05/r1/GA`, `S05/r1/F`<br>`S05/r2/G`, `S05/r2/GA`, `S05/r2/F`<br>`S05/r3/G`, `S05/r3/GA`, `S05/r3/F` |
|
||
| [S06-csharp-dotnet-java](HUMAN_REVIEW_PACKET.md#S06-csharp-dotnet-java) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/5` | `tailor` / section (`section:summary`) / `nb` | Prohibited / unsupported | `S06/r1/G`, `S06/r1/GA`, `S06/r1/F`<br>`S06/r2/G`, `S06/r2/GA`, `S06/r2/F`<br>`S06/r3/G`, `S06/r3/GA`, `S06/r3/F` |
|
||
| [S07-docker-kubernetes](HUMAN_REVIEW_PACKET.md#S07-docker-kubernetes) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/6` | `tailor` / section (`section:summary`) / `en` | Prohibited / unsupported | `S07/r1/G`, `S07/r1/GA`, `S07/r1/F`<br>`S07/r2/G`, `S07/r2/GA`, `S07/r2/F`<br>`S07/r3/G`, `S07/r3/GA`, `S07/r3/F` |
|
||
| [S08-azure-aws](HUMAN_REVIEW_PACKET.md#S08-azure-aws) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/7` | `tailor` / section (`section:summary`) / `nb` | Prohibited / unsupported | `S08/r1/G`, `S08/r1/GA`, `S08/r1/F`<br>`S08/r2/G`, `S08/r2/GA`, `S08/r2/F`<br>`S08/r3/G`, `S08/r3/GA`, `S08/r3/F` |
|
||
| [S09-sql-postgresql](HUMAN_REVIEW_PACKET.md#S09-sql-postgresql) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/8` | `tailor` / section (`section:summary`) / `en` | Prohibited / unsupported | `S09/r1/G`, `S09/r1/GA`, `S09/r1/F`<br>`S09/r2/G`, `S09/r2/GA`, `S09/r2/F`<br>`S09/r3/G`, `S09/r3/GA`, `S09/r3/F` |
|
||
| [S10-plausible-responsibility](HUMAN_REVIEW_PACKET.md#S10-plausible-responsibility) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/9` | `improve` / section (`section:summary`) / `en` | Prohibited / unsupported | `S10/r1/G`, `S10/r1/GA`, `S10/r1/F`<br>`S10/r2/G`, `S10/r2/GA`, `S10/r2/F`<br>`S10/r3/G`, `S10/r3/GA`, `S10/r3/F` |
|
||
| [S11-unquantified-achievement](HUMAN_REVIEW_PACKET.md#S11-unquantified-achievement) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/10` | `improve` / section (`section:summary`) / `nb` | Prohibited / unsupported | `S11/r1/G`, `S11/r1/GA`, `S11/r1/F`<br>`S11/r2/G`, `S11/r2/GA`, `S11/r2/F`<br>`S11/r3/G`, `S11/r3/GA`, `S11/r3/F` |
|
||
| [S12-adjacent-education](HUMAN_REVIEW_PACKET.md#S12-adjacent-education) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/11` | `tailor` / section (`entry:education:economics`) / `en` | Prohibited / unsupported | `S12/r1/G`, `S12/r1/GA`, `S12/r1/F`<br>`S12/r2/G`, `S12/r2/GA`, `S12/r2/F`<br>`S12/r3/G`, `S12/r3/GA`, `S12/r3/F` |
|
||
| [S13-captured-job-context](HUMAN_REVIEW_PACKET.md#S13-captured-job-context) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/12` | `tailor` / section (`section:summary`) / `en` | Prohibited / unsupported | `S13/r1/G`, `S13/r1/GA`, `S13/r1/F`<br>`S13/r2/G`, `S13/r2/GA`, `S13/r2/F`<br>`S13/r3/G`, `S13/r3/GA`, `S13/r3/F` |
|
||
| [S14-saved-cv-override](HUMAN_REVIEW_PACKET.md#S14-saved-cv-override) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/13` | `improve` / section (`entry:experience:reporting`) / `en` | Prohibited / unsupported | `S14/r1/G`, `S14/r1/GA`, `S14/r1/F`<br>`S14/r2/G`, `S14/r2/GA`, `S14/r2/F`<br>`S14/r3/G`, `S14/r3/GA`, `S14/r3/F` |
|
||
| [S15-submitted-frozen-source](HUMAN_REVIEW_PACKET.md#S15-submitted-frozen-source) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/14` | `improve` / section (`section:summary`) / `en` | Prohibited / unsupported | `S15/r1/G`, `S15/r1/GA`, `S15/r1/F`<br>`S15/r2/G`, `S15/r2/GA`, `S15/r2/F`<br>`S15/r3/G`, `S15/r3/GA`, `S15/r3/F` |
|
||
| [S16-job-context-injection](HUMAN_REVIEW_PACKET.md#S16-job-context-injection) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/15` | `tailor` / section (`section:summary`) / `en` | Prohibited / unsupported | `S16/r1/G`, `S16/r1/GA`, `S16/r1/F`<br>`S16/r2/G`, `S16/r2/GA`, `S16/r2/F`<br>`S16/r3/G`, `S16/r3/GA`, `S16/r3/F` |
|
||
| [S17-source-instruction-injection](HUMAN_REVIEW_PACKET.md#S17-source-instruction-injection) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/16` | `improve` / section (`section:summary`) / `en` | Prohibited / unsupported | `S17/r1/G`, `S17/r1/GA`, `S17/r1/F`<br>`S17/r2/G`, `S17/r2/GA`, `S17/r2/F`<br>`S17/r3/G`, `S17/r3/GA`, `S17/r3/F` |
|
||
| [S18-mixed-supported-and-unsupported](HUMAN_REVIEW_PACKET.md#S18-mixed-supported-and-unsupported) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/17` | `improve` / section (`section:summary`) / `en` | Prohibited / unsupported | `S18/r1/G`, `S18/r1/GA`, `S18/r1/F`<br>`S18/r2/G`, `S18/r2/GA`, `S18/r2/F`<br>`S18/r3/G`, `S18/r3/GA`, `S18/r3/F` |
|
||
| [S19-document-en-to-nb](HUMAN_REVIEW_PACKET.md#S19-document-en-to-nb) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/18` | `document-translate` / document / `nb` | Allowed / supported | `S19/r1/G`, `S19/r1/GA`, `S19/r1/F`<br>`S19/r2/G`, `S19/r2/GA`, `S19/r2/F`<br>`S19/r3/G`, `S19/r3/GA`, `S19/r3/F` |
|
||
| [S20-section-nb-to-en](HUMAN_REVIEW_PACKET.md#S20-section-nb-to-en) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/19` | `improve` / section (`section:summary`) / `en` | Allowed / supported | `S20/r1/G`, `S20/r1/GA`, `S20/r1/F`<br>`S20/r2/G`, `S20/r2/GA`, `S20/r2/F`<br>`S20/r3/G`, `S20/r3/GA`, `S20/r3/F` |
|
||
| [S21-document-grammar](HUMAN_REVIEW_PACKET.md#S21-document-grammar) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/20` | `document-grammar` / document / `en` | Allowed / supported | `S21/r1/G`, `S21/r1/GA`, `S21/r1/F`<br>`S21/r2/G`, `S21/r2/GA`, `S21/r2/F`<br>`S21/r3/G`, `S21/r3/GA`, `S21/r3/F` |
|
||
| [S22-section-shorten](HUMAN_REVIEW_PACKET.md#S22-section-shorten) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/21` | `shorten` / section (`section:summary`) / `en` | Allowed / supported | `S22/r1/G`, `S22/r1/GA`, `S22/r1/F`<br>`S22/r2/G`, `S22/r2/GA`, `S22/r2/F`<br>`S22/r3/G`, `S22/r3/GA`, `S22/r3/F` |
|
||
| [S23-section-improve](HUMAN_REVIEW_PACKET.md#S23-section-improve) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/22` | `improve` / section (`section:summary`) / `nb` | Allowed / supported | `S23/r1/G`, `S23/r1/GA`, `S23/r1/F`<br>`S23/r2/G`, `S23/r2/GA`, `S23/r2/F`<br>`S23/r3/G`, `S23/r3/GA`, `S23/r3/F` |
|
||
| [S24-section-grammar](HUMAN_REVIEW_PACKET.md#S24-section-grammar) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/23` | `grammar` / section (`section:summary`) / `en` | Allowed / supported | `S24/r1/G`, `S24/r1/GA`, `S24/r1/F`<br>`S24/r2/G`, `S24/r2/GA`, `S24/r2/F`<br>`S24/r3/G`, `S24/r3/GA`, `S24/r3/F` |
|
||
| [S25-explicit-omitted-section](HUMAN_REVIEW_PACKET.md#S25-explicit-omitted-section) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/24` | `improve` / section (`section:skills`) / `en` | Allowed / supported | `S25/r1/G`, `S25/r1/GA`, `S25/r1/F`<br>`S25/r2/G`, `S25/r2/GA`, `S25/r2/F`<br>`S25/r3/G`, `S25/r3/GA`, `S25/r3/F` |
|
||
| [S26-document-omits-hidden-content](HUMAN_REVIEW_PACKET.md#S26-document-omits-hidden-content) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/25` | `document-improve` / document / `en` | Prohibited / unsupported | `S26/r1/G`, `S26/r1/GA`, `S26/r1/F`<br>`S26/r2/G`, `S26/r2/GA`, `S26/r2/F`<br>`S26/r3/G`, `S26/r3/GA`, `S26/r3/F` |
|
||
| [S27-cross-entry-fact-borrowing](HUMAN_REVIEW_PACKET.md#S27-cross-entry-fact-borrowing) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/26` | `document-improve` / document / `en` | Prohibited / unsupported | `S27/r1/G`, `S27/r1/GA`, `S27/r1/F`<br>`S27/r2/G`, `S27/r2/GA`, `S27/r2/F`<br>`S27/r3/G`, `S27/r3/GA`, `S27/r3/F` |
|
||
| [S28-repeated-grounded-entry-wording](HUMAN_REVIEW_PACKET.md#S28-repeated-grounded-entry-wording) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/27` | `document-improve` / document / `en` | Allowed / supported | `S28/r1/G`, `S28/r1/GA`, `S28/r1/F`<br>`S28/r2/G`, `S28/r2/GA`, `S28/r2/F`<br>`S28/r3/G`, `S28/r3/GA`, `S28/r3/F` |
|
||
| [S29-summary-consolidates-known-facts](HUMAN_REVIEW_PACKET.md#S29-summary-consolidates-known-facts) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/28` | `document-improve` / document / `en` | Allowed / supported | `S29/r1/G`, `S29/r1/GA`, `S29/r1/F`<br>`S29/r2/G`, `S29/r2/GA`, `S29/r2/F`<br>`S29/r3/G`, `S29/r3/GA`, `S29/r3/F` |
|
||
| [S30-proficiency-overstatement](HUMAN_REVIEW_PACKET.md#S30-proficiency-overstatement) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/29` | `tailor` / section (`section:summary`) / `nb` | Prohibited / unsupported | `S30/r1/G`, `S30/r1/GA`, `S30/r1/F`<br>`S30/r2/G`, `S30/r2/GA`, `S30/r2/F`<br>`S30/r3/G`, `S30/r3/GA`, `S30/r3/F` |
|
||
| [S31-ambiguous-ownership](HUMAN_REVIEW_PACKET.md#S31-ambiguous-ownership) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/30` | `improve` / section (`section:summary`) / `en` | Prohibited / unknown | `S31/r1/G`, `S31/r1/GA`, `S31/r1/F`<br>`S31/r2/G`, `S31/r2/GA`, `S31/r2/F`<br>`S31/r3/G`, `S31/r3/GA`, `S31/r3/F` |
|
||
| [S32-cross-entry-metric-borrowing](HUMAN_REVIEW_PACKET.md#S32-cross-entry-metric-borrowing) — [corpus](../p5-13-semantic-qualification/corpus.json) `/cases/31` | `document-improve` / document / `en` | Prohibited / unsupported | `S32/r1/G`, `S32/r1/GA`, `S32/r1/F`<br>`S32/r2/G`, `S32/r2/GA`, `S32/r2/F`<br>`S32/r3/G`, `S32/r3/GA`, `S32/r3/F` |
|
||
|
||
**Path exceptions:** S13 captured job context and S15 submitted frozen source use the unchanged **private supplied-source** rewrite/assessment route with the actual controller-composed instruction. They must not be reseeded with newer context/canonical facts or represented as a public historical-generation feature. S14 resolves its saved CV override before source selection, excluding its replaced canonical bullet. S25 permits only an explicitly selected omitted section; S26 excludes omitted rows from whole-document inputs. Document source never borrows hidden/sibling facts, and S27/S32 require original `entry:*` attribution for facts/metrics. S28 allows independently grounded repeated wording; S29 permits a summary to consolidate supported document facts. These are frozen source boundaries, not new features.
|
||
|
||
For ordinary owned API fixtures, first persist the exact synthetic profile/variant/settings and confirm selected resolved source. Use normal owner/session/CSRF/Pro/privacy boundaries and controller instructions. Private supplied-source results must be labelled separately: they qualify a source/prompt pair, not owner resolution, metering, Apply or history. Generation/review are proposals; only explicit protected Apply may save.
|
||
|
||
## Exact baseline-assessment ledger
|
||
|
||
Source: [unchanged baseline fixture](../../../../JobTrackerApi.Tests/Fixtures/AiEvaluation/cv-rewrite-grounding.json). Ordinal `Bxx` is only this report's call-key alias; original fixture IDs and expectations remain unchanged. For each B call, send source/proposal/context only. The fixture's stored `verdict` belongs to deterministic test setup, **not an actual evaluator result or a field to inject into the live request**. Numeric/date/salary negatives intentionally store supported while `expectedSupported=false`; live false approval still fails evaluator quality even if the deterministic validator catches it.
|
||
|
||
| Original baseline ID / pointer / packet | Expected validator applicability | Stored deterministic test verdict — not live result | Three planned B IDs |
|
||
|---|---|---|---|
|
||
| [en-paraphrase](HUMAN_REVIEW_PACKET.md#b01-en-paraphrase) — baseline `/cases/0` | Allowed / true | `supported` | `B01/r1/B`, `B01/r2/B`, `B01/r3/B` |
|
||
| [en-shortening](HUMAN_REVIEW_PACKET.md#b02-en-shortening) — baseline `/cases/1` | Allowed / true | `supported` | `B02/r1/B`, `B02/r2/B`, `B02/r3/B` |
|
||
| [nb-paraphrase](HUMAN_REVIEW_PACKET.md#b03-nb-paraphrase) — baseline `/cases/2` | Allowed / true | `supported` | `B03/r1/B`, `B03/r2/B`, `B03/r3/B` |
|
||
| [nb-shortening](HUMAN_REVIEW_PACKET.md#b04-nb-shortening) — baseline `/cases/3` | Allowed / true | `supported` | `B04/r1/B`, `B04/r2/B`, `B04/r3/B` |
|
||
| [en-to-nb](HUMAN_REVIEW_PACKET.md#b05-en-to-nb) — baseline `/cases/4` | Allowed / true | `supported` | `B05/r1/B`, `B05/r2/B`, `B05/r3/B` |
|
||
| [nb-to-en](HUMAN_REVIEW_PACKET.md#b06-nb-to-en) — baseline `/cases/5` | Allowed / true | `supported` | `B06/r1/B`, `B06/r2/B`, `B06/r3/B` |
|
||
| [percent-translation](HUMAN_REVIEW_PACKET.md#b07-percent-translation) — baseline `/cases/6` | Allowed / true | `supported` | `B07/r1/B`, `B07/r2/B`, `B07/r3/B` |
|
||
| [unsupported-employer](HUMAN_REVIEW_PACKET.md#b08-unsupported-employer) — baseline `/cases/7` | Prohibited or unknown / false | `unsupported` | `B08/r1/B`, `B08/r2/B`, `B08/r3/B` |
|
||
| [unsupported-work-history](HUMAN_REVIEW_PACKET.md#b09-unsupported-work-history) — baseline `/cases/8` | Prohibited or unknown / false | `unsupported` | `B09/r1/B`, `B09/r2/B`, `B09/r3/B` |
|
||
| [unsupported-title](HUMAN_REVIEW_PACKET.md#b10-unsupported-title) — baseline `/cases/9` | Prohibited or unknown / false | `unsupported` | `B10/r1/B`, `B10/r2/B`, `B10/r3/B` |
|
||
| [unsupported-responsibility](HUMAN_REVIEW_PACKET.md#b11-unsupported-responsibility) — baseline `/cases/10` | Prohibited or unknown / false | `unsupported` | `B11/r1/B`, `B11/r2/B`, `B11/r3/B` |
|
||
| [unsupported-skill](HUMAN_REVIEW_PACKET.md#b12-unsupported-skill) — baseline `/cases/11` | Prohibited or unknown / false | `unsupported` | `B12/r1/B`, `B12/r2/B`, `B12/r3/B` |
|
||
| [unsupported-technology](HUMAN_REVIEW_PACKET.md#b13-unsupported-technology) — baseline `/cases/12` | Prohibited or unknown / false | `unsupported` | `B13/r1/B`, `B13/r2/B`, `B13/r3/B` |
|
||
| [unsupported-education](HUMAN_REVIEW_PACKET.md#b14-unsupported-education) — baseline `/cases/13` | Prohibited or unknown / false | `unsupported` | `B14/r1/B`, `B14/r2/B`, `B14/r3/B` |
|
||
| [unsupported-qualification](HUMAN_REVIEW_PACKET.md#b15-unsupported-qualification) — baseline `/cases/14` | Prohibited or unknown / false | `unsupported` | `B15/r1/B`, `B15/r2/B`, `B15/r3/B` |
|
||
| [unsupported-certification](HUMAN_REVIEW_PACKET.md#b16-unsupported-certification) — baseline `/cases/15` | Prohibited or unknown / false | `unsupported` | `B16/r1/B`, `B16/r2/B`, `B16/r3/B` |
|
||
| [unsupported-achievement](HUMAN_REVIEW_PACKET.md#b17-unsupported-achievement) — baseline `/cases/16` | Prohibited or unknown / false | `unsupported` | `B17/r1/B`, `B17/r2/B`, `B17/r3/B` |
|
||
| [unsupported-metric-even-if-reviewer-approves](HUMAN_REVIEW_PACKET.md#b18-unsupported-metric-even-if-reviewer-approves) — baseline `/cases/17` | Prohibited or unknown / false | `supported` | `B18/r1/B`, `B18/r2/B`, `B18/r3/B` |
|
||
| [unsupported-project](HUMAN_REVIEW_PACKET.md#b19-unsupported-project) — baseline `/cases/18` | Prohibited or unknown / false | `unsupported` | `B19/r1/B`, `B19/r2/B`, `B19/r3/B` |
|
||
| [unsupported-language](HUMAN_REVIEW_PACKET.md#b20-unsupported-language) — baseline `/cases/19` | Prohibited or unknown / false | `unsupported` | `B20/r1/B`, `B20/r2/B`, `B20/r3/B` |
|
||
| [unsupported-date-even-if-reviewer-approves](HUMAN_REVIEW_PACKET.md#b21-unsupported-date-even-if-reviewer-approves) — baseline `/cases/20` | Prohibited or unknown / false | `supported` | `B21/r1/B`, `B21/r2/B`, `B21/r3/B` |
|
||
| [unsupported-salary-even-if-reviewer-approves](HUMAN_REVIEW_PACKET.md#b22-unsupported-salary-even-if-reviewer-approves) — baseline `/cases/21` | Prohibited or unknown / false | `supported` | `B22/r1/B`, `B22/r2/B`, `B22/r3/B` |
|
||
| [unsupported-personal-detail](HUMAN_REVIEW_PACKET.md#b23-unsupported-personal-detail) — baseline `/cases/22` | Prohibited or unknown / false | `unsupported` | `B23/r1/B`, `B23/r2/B`, `B23/r3/B` |
|
||
| [unsupported-experience-duration](HUMAN_REVIEW_PACKET.md#b24-unsupported-experience-duration) — baseline `/cases/23` | Prohibited or unknown / false | `unsupported` | `B24/r1/B`, `B24/r2/B`, `B24/r3/B` |
|
||
| [unknown-mixed-claims](HUMAN_REVIEW_PACKET.md#b25-unknown-mixed-claims) — baseline `/cases/24` | Prohibited or unknown / false | `unknown` | `B25/r1/B`, `B25/r2/B`, `B25/r3/B` |
|
||
| [injection-cannot-supply-evidence](HUMAN_REVIEW_PACKET.md#b26-injection-cannot-supply-evidence) — baseline `/cases/25` | Prohibited or unknown / false | `unsupported` | `B26/r1/B`, `B26/r2/B`, `B26/r3/B` |
|
||
|
||
## Evidence and existing evaluator contract
|
||
|
||
| Boundary | Required observed evidence; no surrogate pass |
|
||
|---|---|
|
||
| Actual configured target | Non-secret deployed provider/model/digest/settings, generation and evaluator route identity, API/sidecar runtime/image revisions and prompt-source hashes. Local Ollama `qwen2.5:7b` and expected source hashes alone are not deployed proof. Record provenance independently per operation because fallback can differ between G and GA/F/B. |
|
||
| Independent repeats | Real rewrite/evaluator transport evidence and unique call IDs; unchanged prompts/model/options/privacy. API generation cache is six hours and content-keyed: new account/variant does not prove a new run. Clear only the isolated API process cache between batches as the [frozen procedure](../p5-13-semantic-qualification/README.md#procedure-when-the-configured-target-becomes-available) requires; retain its disposable synthetic database. Evaluations are uncached. Cached replies contribute zero genuine model executions. |
|
||
| Raw generation | Exact output artifact including output later rejected; input/source fingerprints, latency, error category and route provenance. An API 422 alone hides whether G fabricated or the evaluator erred. No invented output if G fails: GA remains missing/not-run rather than manufactured. |
|
||
| Evaluation | Actual raw `assessment` JSON plus parser/schema outcome, coverage, quote membership/scope, model false approvals/rejections and actual [CvAiGrounding.IsSupported](../../../../JobTrackerApi/Services/CvAiGrounding.cs) result. Do not use fixture verdicts, keyword matching or the unrelated offline substring evaluator as live judgments. |
|
||
| Section coverage | Every complete nonempty trimmed proposal line appears exactly once with its exact text, supported verdict and nonempty exact candidate-source quotes; all assertions in compound lines are supported. Only selected items may be rewritten. Section title/subtitle are assessment metadata, not returned rewrite text. |
|
||
| Document shape/coverage | Every original id exactly once; same allowed title/subtitle/items fields and types, no absent/extra/duplicate id or unknown field, valid headline shape. Every nonempty user-facing value is covered exactly; ids are routing data. [CvAiSuggestionService](../../../../JobTrackerApi/Services/CvAiSuggestionService.cs) shape checks remain authoritative. |
|
||
| Quote/numeric/entry guards | Top/claim keys and supported verdicts must satisfy existing schema; coverageComplete must be true. Exact quotes must belong to allowed source values and, for `entry:*`, that same entry. Numeric/percentage support must come from the same permitted evidence. Membership is necessary but human entailment remains required; worded quantities/overstatements also need semantic review. |
|
||
| Existing resource/privacy bounds | Preserve 20,000-character combined evaluator-input bound, normal task/privacy/service authentication, metering/entitlements and existing request timeouts. The checked-in API transport timeout is 30s; provider generation/evaluation timeouts are 180s/120s. Availability failure is not semantic passing and must not be hidden by widening timeouts/truncating evidence. |
|
||
| Apply/source/history | Separate existing I01 [readiness tests](../../../../JobTrackerApi.Tests/CvAiReadinessTests.cs) prove stale variant and canonical fingerprint conflict, no unaccepted mutation, ordinary accepted version/history and unchanged canonical source. Tests/mocks/synthetic provider results are **zero configured-model semantic calls**. A passing semantic case cannot compensate for stale Apply. |
|
||
| Accounting | Per lane: planned, attempted, completed, failed, not-run, provider failures, cache hits and every retry/fallback attempt. Keep operation ordering and all failed artifacts. Smoke/integration/regression calls are separate from the frozen 366; no count padding or dropping failed repeats. |
|
||
|
||
The actual existing [sidecar evaluator](../../../../tools/summarizer/app.py) receives `candidate_source`, `proposal`, `context_only` and returns an `assessment` object. Record the actual schema result and [parser/corpus tests](../../../../JobTrackerApi.Tests/CvAiGroundingTests.cs), not just a favourable top verdict. Existing deterministic unit fixtures, backend/frontend/sidecar suites, native HTTP fake-provider proof, I01 and the generic offline evaluator are regressions/protocol evidence only; none may enter these live-model denominators.
|
||
|
||
## Predeclared thresholds and human acceptance
|
||
|
||
| Gate / dimension | Exact acceptance requirement |
|
||
|---|---|
|
||
| Candidate factuality | **Zero fabricated facts in every G output**, including facts later rejected. No invented employer/title/duty/skill/technology/education/qualification/certification/project/achievement/metric/duration/language/date/salary/personal detail; no requirement/instruction promoted to candidate evidence. |
|
||
| Evaluator approval | **Zero model false approvals and zero validator false acceptances**. Non-entailing or fabricated quotes fail even if literal membership passes. Each prohibited/unknown F/B proposal remains inapplicable. |
|
||
| Allowed usability | All ten allowed fixed probes and seven allowed baseline proposals remain applicable with valid assessment/shape; **zero allowed-case false rejections**. A reviewer does not change declared expectations to fit outcomes. |
|
||
| Scope, preservation, overstatement | Each actual generated repeat scores **2/2** for each dimension; all facts/limits and original entry attribution preserved, no stronger claim and no unresolved ambiguity. |
|
||
| Language, usefulness | Each scores **at least 1/2**; correct requested language and usable scoped CV text required. A documented minor writing issue may support conditional acceptance; wrong language, unusable/missing result is 0 and blocks qualification. No sum/average may offset any failing gate. |
|
||
| Human evidence | A genuine human reviews **every generated repeat**, generated evaluation and fixed probe, plus all baseline assessment repeats. Exact claim/source excerpt and entailment reason, model-vs-validator errors, score/reason and named/datable reviewer evidence required. Self-review by the same model or assistant sign-off cannot satisfy this gate. |
|
||
| Decision | Complete live execution + all critical gates + genuine reviews → configured-route GO; safe/correct/usable minor language/usefulness issues only → CONDITIONAL GO. Any observed blocking failure → NO-GO. Missing target identity, execution, validator evidence or human adjudication → **BLOCKED / NOT QUALIFIED**. |
|
||
|
||
The [packet](HUMAN_REVIEW_PACKET.md) provides **96 semantic case/repeat record slots** (each links G + GA + F) and **78 baseline slots**: **174 combined worksheet records**. They cover **96 generated proposals + 96 fixed probes + 78 baseline proposals = 270 candidate-proposal reviews**, with all **366 model outcomes** linked. Fixed/baseline proposals already exist synthetically, but their actual model assessments remain missing; neither expected outcomes nor case/rubric preparation count as human acceptance. Human reviews completed remain **0 / incomplete**.
|
||
|
||
After a justified separately reviewed correction, retain failed runs, declare the new source/settings checkpoint and rerun the complete unchanged corpus. Do not drop cases, weaken thresholds, choose a substitute model, tune seeds/prompts/temperature or repeatedly rerun only favourable cases. Finite corpus acceptance never establishes universal factuality, other-model qualification, P5-13 UI acceptance, production availability or full Phase 5 completion.
|
||
|
||
## Blockers and cost prerequisites
|
||
|
||
Current inherited evidence is **0 completed / 366 planned, 0 human reviews, factuality unmeasured**. Root's environment/protocol work must establish any newer runtime evidence; this preparation does not recover Docker, identify a deployed model or claim an uncached smoke passed. Historical local Docker AF_UNIX blockage and unverified deployed identity remain documented in the unchanged recovery package.
|
||
|
||
Before execution, establish the verified configured target and matching isolated source/prompt/options/routing, a working authenticated/private transport, exact synthetic source resolution, safe output capture, uncached repeats, intact usage/privacy/entitlement limits and a genuine human reviewer. Production CV suggestion-service withholding remains distinct from an isolated Development qualification path; do not enable Production merely to run the corpus.
|
||
|
||
**Cost/accounting is unmeasured.** Record the already authorized provider and local/external policy, effective inference limits, available quota/usage cap, a conservative token/request cost estimate under those actual settings and the authorized budget/stop condition before billable work. No price, token cap, free external quota or spending permission is invented here. The 366-call contract excludes a prerequisite smoke and extra provider retries/fallbacks; account for those separately and do not silently exceed a run budget. Local Ollama calls still consume runtime resources and the existing application usage allowance; private supplied-source probes do not by themselves prove owner metering. Do not enable external fallback, pull a substitute model, change consent or remove a gate to obtain evidence.
|
||
|
||
Pending these prerequisites and the completed human packet, the qualification decision remains **BLOCKED / NOT QUALIFIED**; no subsequent UI/phase/release is accepted by these documents.
|