jobtrackingapp/docs/redesign/phase-5/p5-13-qualification/README.md

# P5-13 grounding qualification checkpoint

**NO-GO for P5-13 AI review UI — configured-model qualification unavailable and human review incomplete.** Source: `fe045b9e015549f3d9101228149af9f9c9f876e5`, branch `codex/ui-redesign-phase4`, reviewed 2026-10-08. Real qualification calls: **0 attempted / 0 completed / 0 failed / 366 not run**. Human reviews: **0**. Fabrication, false-approval, legitimate-rewrite rejection and model error rates are **unmeasured**, not 0%. The Production CV review block remains active in the candidate; no activation, production contact, merge or deployment occurred.

This checkpoint changes documentation only. P5-12 remains the latest completed CV UI slice. No P5-13 UI, P5-14, P5-15, B07, provider/prompt change, environment recovery or new evaluation system is introduced. The latest instruction schedules production deployment for the **Phase 5 release-candidate decision after P5-15**, superseding earlier release-first sequencing. This is not authorization to start those later slices.

## Sources and portable evidence

- [Exact qualification matrix](QUALIFICATION_MATRIX.md): every planned call ID, source pointer, expectation and acceptance gate.
- [Human review packet](HUMAN_REVIEW_PACKET.md): instructions, synthetic examples, complete source index and blank worksheets; no simulated outputs or sign-off.
- [Sanitized results](RESULTS.json): fresh availability/protocol observations, missing semantic outcomes and explicitly inherited complete baseline.
- Frozen [corpus](../p5-13-semantic-qualification/corpus.json), [rubric](../p5-13-semantic-qualification/RUBRIC.md), [original results](../p5-13-semantic-qualification/qualification-results.json) and [environment/recovery record](../p5-13-semantic-qualification/README.md) remain unchanged.
- [Authoritative grounding readiness](../p5-13-readiness/README.md), [approved AI architecture](../../../ai/architecture.md), [provider ADR](../../../decisions/ADR-004-ai-provider-system.md), [Phase 3 interaction model](../../phase-3/AI_INTERACTION_MODEL.md) and [deferred-work register](../OPEN_BLOCKERS.md#p5-13-deferred-debt-and-return-conditions--2026-10-08) retain their requirements.

Raw local protocol artifacts are ignored under `.artifacts/p5-13-qualification/`: `environment.json`, `focused/qualification-focused.trx`, `revision/qualification-revision.trx`, `http.json` and `http/`. They are test evidence, not production/model evidence. Portable counts and source hashes are retained in RESULTS.json; credentials, cookies, protected proposal capabilities, hidden prompts and real personal content are not committed.

## What the 366 calls mean

| Lane | Required real operations | Planned | Attempted / completed / failed | Not run |
|---|---|---:|---|---:|
| G | 32 scoped rewrites, three repeats | 96 | 0 / 0 / 0 | 96 |
| GA | Assess each repeat's actual generated output | 96 | 0 / 0 / 0 | 96 |
| F | Assess each frozen fixed proposal, three repeats | 96 | 0 / 0 / 0 | 96 |
| B | Existing 26 assessment cases, three repeats | 78 | 0 / 0 / 0 | 78 |
| Total | **96 + 96 + 96 + 78** | **366** | **0 / 0 / 0** | **366** |

I01 is a separate stale-source integration reference with repeatCount 0. The JSON's 33 entries therefore represent **32 semantic cases**, not 33 generation cases. Requested generation language is 25 English / 7 Bokmål cases, giving 75 / 21 planned generation calls; actual language-specific outcomes are both unmeasured. Fixed probes retain their exact frozen language/text. Fixed assessment has 10 allowed and 22 prohibited/unknown cases; baseline has 7 allowed and 19 prohibited/unknown. Across their repeats, 51 allowed and 123 negative probes provide predeclared denominators. Generated factuality denominators require actual outputs and human claim review.

The matrix covers invented employers/titles/dates, education/qualifications/certifications, technologies/skills/languages, duties/projects, achievements/numeric results and personal details; related technologies, job-context injection, mixed claims, overstatement, ambiguous attribution, cross-entry facts, saved overrides, captured frozen source and omitted sections. Legitimate paraphrase, grammar, shortening, restructuring and translation remain required positives. No case, repeat or threshold was removed or changed.

The unchanged rubric requires zero fabricated candidate facts in **raw generation**, even if a later guard rejects them; zero evaluator false approvals, validator false acceptances and allowed-case false rejections. Every generated repeat requires scope/preservation/overstatement 2/2 and language/usefulness at least 1/2. A score cannot compensate for a critical failure. Missing output/review is incomplete. A genuine human must adjudicate every output/repeat, with source quotations, entailment reasons, scores and dated identity. Expected answers and this assistant's analysis are not human acceptance.

## Configured-target and resource prerequisites

Fresh read-only observations at `2026-10-08T18:49:27Z`:

| Observation | Result / interpretation |
|---|---|
| Allowlisted local configuration | Ollama base host `ollama:11434`, model `qwen2.5:7b`; external selector `gemini`, model `gemini-2.0-flash`; Groq model `llama-3.3-70b-versatile` |
| Routing / external gate | Compose/code default `local_first`; `EXTERNAL_AI_ENABLED` absent locally, default false. No allowlisted process overrides. Dormant external selector is not proof of an active route |
| Optional host inventory / health | Loopback 11434 `/api/tags` and 8001 `/health` each timed out after a two-second availability probe; neither is a verified configured target |
| Docker read-only inventory | Exit 1; `dockerDesktopLinuxEngine` pipe unavailable. No startup, repair, destructive action or model pull attempted |
| Existing remote connection | User SSH config absent; no verified deployed/staging connection established |
| Actual deployed model/runtime | Provider, model digest, routing per operation, image revisions, effective options and runtime prompt identity **unverified** |

The historical AF_UNIX/Windows error 1920 remains in the earlier recovery record; it was **not freshly reproduced** here. Inventory/health timeouts are not inference failures and do not enter the model error-rate denominator. Local defaults, expected hashes and credential-name presence cannot establish deployed identity, usable credentials, external-processing consent or an approved spending budget. No external model, alternate provider or production host was contacted.

Code-verified options, also subject to actual target verification:

| Operation | Existing settings to preserve |
|---|---|
| Section **and document** generation | `/cv/rewrite`, temperature **0.2**, provider timeout **180s**, JSON mode **false**. `cv_blocks_json` constrains document output through the existing prompt/shape contract; it does not change generation temperature or enable provider JSON mode |
| Factual assessment | `/cv/evaluate-rewrite`, temperature **0.1**, provider timeout **120s**, JSON mode **true** |
| API transport | Named summarizer HTTP client timeout **30s**; no widening, new seed, model options or input truncation to force a pass |

The four checked-in runtime/prompt source hashes match the frozen expected hashes, recorded in RESULTS.json. This proves unchanged repository source, **not** deployed image/source identity. The existing generic offline evaluation runner has different prompts/options and substring checks; it cannot execute or qualify this contract as-is. No new runner was built while transport/identity remain blocked.

Before execution, verify the authorized configured target and equivalent isolated runtime, obtain matching existing service authentication (`X-Ai-Service-Token`) and owned API session/CSRF/context where applicable, confirm privacy/task/consent gates, quota and authorized budget/stop conditions, and complete one uncached synthetic smoke. Do not expose secrets. Pro/Admin defaults are 250 monthly AI calls and 1,000,000 estimated tokens; generation and assessment meter separately. The 366 provider operations are not necessarily 366 public API requests. Do not bypass metering, raise allowances or enable a paid fallback. No prices, token costs or new spending ceiling are invented; actual inference usage/charges were not measured because no inference was attempted.

For three genuine repeats, account for the existing six-hour content-keyed generation cache. Use the frozen procedure's restart of **only an isolated API process**, retaining its disposable database, and verify actual rewrite transport. New account/CV IDs do not bypass the cache. Evaluations are uncached. Retain every output, failure, retry/fallback and cache observation; cache hits are not model executions. If G fails, GA remains missing rather than being replaced by a fixed probe. Private supplied-source cases S13/S15 qualify captured source/prompt pairs, not a new historical-generation UI or ownership/metering behavior.

## Authoritative grounding and application boundary

The existing [CvAiSuggestionService](../../../../JobTrackerApi/Services/CvAiSuggestionService.cs) resolves the authenticated owner's **saved CV and canonical source**. Section generation receives selected items; assessment also receives that selected block's title/subtitle. Whole-document generation/assessment uses included editable blocks with original IDs. Resolved overrides replace the corresponding canonical values. Explicit omitted-section editing remains supported; whole-document requests exclude omitted content. Client text is a consistency check, not candidate evidence.

Allowed evidence is the action's resolved candidate source (canonical/effective CV/override or explicitly captured synthetic source as declared). Job descriptions and user instructions are separately marked context and **cannot prove candidate attributes**. A Kubernetes requirement cannot establish Kubernetes experience. Scope/entry attribution prevents borrowing facts from a sibling employer; captured/historical source cases exclude newer facts.

Generation goes through the existing separate factual assessment and [strict validator](../../../../JobTrackerApi/Services/CvAiGrounding.cs). All returned claims need valid schema, complete exact line/value coverage, supported verdict and nonempty exact source quotations, with entry/numeric and document-shape checks. Missing/malformed/unsupported/unknown assessment returns **422 `cv_ai_grounding_unproven`**, with no applicable proposal. A mixed supported/unsupported proposal is rejected **as a whole**; there is no new partial merge.

**Important limitation:** exact quote membership and deterministic numeric/shape checks do not independently prove semantic entailment, proficiency or factual faithfulness. A configured evaluator could incorrectly endorse a non-entailing quote. This is why the genuine model/human gate is still mandatory and Production remains blocked. The 108 controlled regressions prove the protocol's reaction to controlled provider/evaluator decisions, not universal no-fabrication or model accuracy.

A protected server capability binds owner, CV, action/target/context, expected version, canonical/settings fingerprint, original and reviewed result. Client `grounded: true` flags have no authority. Altered text is reassessed; tampered capability or wrong owner/CV fails. Explicit Apply performs a final source/version check inside the existing serializable transaction and ordinary versioned save. No alternate history model or canonical mutation is introduced. Generated, rejected and unapplied proposals create no CV content revision; a grounded explicit Apply creates the normal CV-only revision. Manually authored content is not truth-certified by this qualification.

## Fresh deterministic / native HTTP evidence

| Requirement | Fresh executed evidence | Result / limit |
|---|---|---|
| Strict grounding, provider fabrication, legitimate rewrites, mixed/context/entry/number cases | `CvAiGroundingTests`, `CvAiReadinessTests` and existing focused AI suites | **108/108**; controlled English/Bokmål provider/evaluator fixtures, not live semantic runs |
| Token tampering / unsupported client-edited result / other owner or CV | Focused readiness tests | API rejects; no client-only protection. Existing account isolation, not a newly introduced organization/tenant model |
| CV source A → newer saved B → Apply A | First real-HTTP case | **409 `cv_ai_source_changed`**; latest variant/content/history snapshot remains identical to B |
| Canonical source changes while CV Version stays unchanged | Same HTTP case + focused tests | Fingerprint mismatch returns 409; latest canonical/CV/history remain intact |
| Source changes during generation | Focused readiness tests | No stale proposal issued |
| Valid accepted proposal / frozen references / override isolation | HTTP Apply plus focused/revision suites | Ordinary CV-only save/history; canonical and sibling state unchanged; **5/5 revision safety** |
| Production guard before provider or writes | Second HTTP case, isolated API running Production environment | Section generation, document generation and Apply **503 `cv_ai_unavailable`**; zero provider calls and identical CV/profile/history snapshots |
| Unrelated AI retained | Same isolated Production case | Existing AI workspace module read and legacy profile-CV rewrite return 200; subsequent expected legacy provider call occurs independently |

HTTP result: **2/2, zero skips/retries/unexpected failures**, approximately 14.52 seconds. Both cases run loopback disposable APIs and a synthetic sidecar. They do not contact production or qualify a real model. No extra browser case was added; no automated count changed.

[Program.cs](../../../../JobTrackerApi/Program.cs) continues withholding `CvAiSuggestionService` in Production; the optional controller returns unavailable before generation/Apply processing. This is a tested **candidate configuration** guard, not evidence of what is deployed today. Even a future successful qualification requires a **separate explicit activation decision**; this checkpoint never automatically enables it. Unrelated AI is not globally disabled or semantically certified by this checkpoint.

## Human review status

The prepared packet references every frozen source and planned operation, with representative synthetic source/fixed-proposal examples. Actual generated outputs, live evaluator decisions and reviewer judgments remain absent. **No authorized human decisions were received or collected.** There is therefore no factual claim ledger, semantic acceptance or reviewer sign-off to report. The 96 semantic repeat worksheets (G/GA/F together) and 78 baseline worksheets remain unfilled; all 270 candidate-proposal reviews and 366 model outcomes need genuine evidence. This assistant did not simulate independent review.

## Preservation traceability

No route, workflow, provider, prompt, revision, renderer, dependency, test/CI or application file changes in this checkpoint. The following contracts are reviewed through the existing protections; this is not exhaustive certification of all 57 groups.

| PRES | Existing invariant / readiness requirement | Evidence / acceptance |
|---|---|---|
| PRES-C01 | Canonical structured Career Profile stays separate from CV Apply | Fresh readiness/native HTTP canonical snapshots; retained deterministic pass |
| PRES-C07 | Source resolution, CV overrides, included/omitted/custom content and order remain scoped | Existing source/shape/override tests; no editor/renderer change |
| PRES-C10 | Versioned save, conflict, history and newer content integrity | Fresh 5 revision + stale/final-transaction tests and HTTP snapshots |
| PRES-C11 | Contextual section/document AI proposes, reviews, explicitly applies/rejects with factual grounding | Deterministic boundary green; **configured-model/human/UI acceptance blocked**, Production capability explicitly withheld |
| PRES-S04 | Account entitlement, privacy, metering and provider boundary | Existing focused tests and owned authenticated HTTP fixture; no permission/quota relaxation |
| PRES-E09 | Account switch/isolation and stale state cannot cross the owner boundary | Existing foreign-owner/capability tests; no provider-tree/client lifecycle change |

Submitted/frozen references, canonical/override boundaries and PDF output remain protected by the unchanged revision and rendering implementation. The inherited native package and fidelity evidence remains valid for that same source; fresh qualification protocol tests do not replace it.

## Baseline comparison and repository checks

The full applicable suites were executed at the inherited release-safety checkpoint; [its complete validation record](../release-readiness/VALIDATION.json) remains authoritative. **No code/test/CI/configuration change** occurred here, so the full UI/build/export baseline was not rerun for documentation. Fresh targeted protocol/revision execution is identified separately above. Count changes: **zero in every suite**; subsets are not added to backend/browser totals.

| Gate | Retained baseline | This checkpoint |
|---|---|---|
| Backend / frontend / main browser | **1,023 / 455 (81 suites) / 112 actual passes** | Inherited, unchanged; P2-D01 is an ordinary passing case |
| Focused AI / real HTTP / revision | **108 / 2 / 5** | Fresh reruns, all pass; focused/revision included in backend total, HTTP separate |
| Sidecar | **49** | Inherited, unchanged |
| Standard rendering fidelity | **16/16; 36 preview = 36 PDF pages** | Inherited, unchanged renderer/fixtures |
| Settings exports | **8/8; 24 = 24 pages** | Inherited, unchanged |
| Rate-limit / ingress / PDF admission / configuration | **9 / 6 / 1 / 3** | Inherited, unchanged |
| Offline evaluation harness / supply-chain helpers / extension | **5 / 3 / 3** | Inherited; offline tests are not live qualification |
| TypeScript / lint / frontend production / API Release build | Passing at parent; API warnings-as-errors | Inherited; code unchanged |
| Frozen JSON/hash/source checks / 366-call ledger / packet anchors | Fresh documentation verification | See RESULTS.json; frozen evidence unchanged |
| Document links / secret scan / Git whitespace/scope | Fresh final checks | Zero missing targets/secret findings/whitespace errors; docs-only scope |

The parent recorded 229 automated axe scans with zero reported violations **and incomplete checks**. No new axe or manual screen-reader acceptance is claimed here; neither zero violations nor protocol tests close accessibility debt.

## Exact remaining prerequisites and stop

1. Establish verified authorized deployed/configured model identity, digest, per-operation routing/options and matching isolated API/sidecar runtime/prompt revisions. Restore a supported reachable transport without substituting a model or bypassing Production withholding.
2. Confirm existing authentication, privacy/consent, available quota, authorized cost/stop conditions and one uncached synthetic smoke. Keep service/account credentials out of evidence.
3. Execute the **unchanged 366-call procedure**, recording raw outputs, actual validator decisions, uncached repeats, failures, languages, latency/cost where available and per-lane factuality denominators.
4. Obtain genuine independent human claim/source review for **every output/repeat**, satisfying the frozen critical gates. Preserve and separately correct/rerun failures without weakening thresholds.
5. Reassess P5-13 UI readiness only after that evidence. Any Production activation is a separate explicit decision, subject to the later release-candidate gates.

**NO-GO** remains the result for P5-13 UI; deterministic safety is retained but insufficient for model qualification. F04, current backup/restore proof, production configuration verification, documented/tested rollback and protected deployment evidence remain separately open. Manual screen-reader/incomplete accessibility, production font/photo, pointer/reflow, Dashboard preference-save 500, Board scaling/native-touch and deferred `followMode` compose UI remain with their existing owners. No issue is closed merely by the Production block. Stop before P5-13 UI and all later work.