jobtrackingapp/docs/cv-integration-qa/EXTRACTION_ACCURACY.md

Extraction accuracy — 2026-10-27

Deterministic interpretation is PARTIAL. Real CV-model compatibility and accuracy remain BLOCKED. No inference was run and no synthetic model response contributes to these accuracy measurements.

The separate structural corpus contains 18 logical cases (8 development, 10 frozen), each rendered as PDF and DOCX. Annotations preceded observed outputs; original import fixtures and annotations were preserved. This is regression evidence, not population-level accuracy or an unseen-model evaluation. See benchmarks/README.md, the immutable generated manifest and benchmarks/RESULTS.json for definitions, per-field denominators and case results.

Measured layer	PDF	DOCX
Actual bounded document conversions	18/18	18/18
Literal source fragments recovered	134/134	134/134
Typed facts TP / FP / FN	100 / 0 / 22	122 / 0 / 0
Typed precision	100%	100%
Typed recall	81.97%	100%
Typed F1	90.09%	100%
Correct annotated block/page references	24/27	27/27
Required ambiguity cases correctly preserved/withheld	6/6	6/6

Literal recovery does not establish table relationships, reading order or geometry. Native PDF table/long-layout relationships account for missing typed facts and source references. Their text remains available for explicit review mapping. Source evidence transports block/page identity, not character spans or table-cell geometry. The separate actual browser and MariaDB tests establish reviewed mapping and persistence; this benchmark performs neither.

Explicit labels, header-defined table order, genuine lists and date-only lines now establish bounded relationships. Repeated employer/title strings retain their selected source occurrence. Unlabelled employer/title or institution/qualification order stays unresolved. A later ambiguous record cannot become a previous job's achievements. One legacy paragraph continuation is allowed only with an established compatible context and is marked needs_review in the review UI. Certification cell clusters cannot become several invented certificates.

Original corpus comparison

The original strict oracle was rerun against the current interpreter using the unchanged previously captured real Docling outputs, rather than claiming those conversions were freshly repeated. PDF completion remains 13/14 (the scan is an expected rejection); DOCX 13/13.

Original oracle	Current precision	Current recall	FP / FN
PDF, 14 cases	93.38%	94.63%	10 / 8
DOCX, 13 cases	97.26%	95.30%	4 / 7

These scores include unresolved custom-source classification and must not be replaced by the better typed-only scores. On the common typed scope excluding otherSections, PDF changes from TP132/FP4/FN8 to TP132/FP1/FN8; DOCX changes from TP140/FP0/FN0 to TP133/FP0/FN7. Seven headerless DOCX table facts are now withheld for user assignment rather than inferred from position. One PDF long-name association remains wrong. This recall tradeoff is a known limitation, not a full accuracy pass.

Evidence and reproducibility

• .artifacts/cv-integration-qa/structural-corpus-01/manifest.json: document/generator/annotation hashes.
• structural-extraction-01/: 36 real outputs, per-child resources, input/output hashes; generative models disabled.
• profiles-structural-*-03.json, metrics-structural-*-04.json: current interpretation and metrics.
• metrics-baseline-*-03.json, baseline-typed-comparison.json: unchanged original oracle and explicit common-scope comparison.
• Parser/mapping/merge focused successor: merge-order-green-02.log, 126 passed. Earlier failing attempts are retained; overlapping focused counts are not summed.
• Benchmark self-check: 9 passed. benchmarks/RESULTS.json retains complete machine-readable results.

Child wall time median 2.335s, maximum 5.653s; CPU median 1.930s, maximum 3.813s. Peak working set median 142,776,320 bytes, maximum 146,653,184 bytes. Total inputs 1,058,593 bytes and structured outputs 31,664 bytes across 36 sequential runs. These are local Windows observations, not server concurrency capacity; Windows child memory is observed rather than kernel-enforced by the parser argument.

Future isolated model qualification must use immutable approved artifacts, the same frozen documents and annotations, malformed-output/source-grounding tests, record/field/provenance metrics, hallucination counts, human correction burden and resource measurements. Keep model results separate from deterministic and reviewed results. No production model/cache/service change is authorised by this report.