cesnimda/jobtrackingapp
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

test(cv-qa): reconcile acceptance and hand off conditional readiness

Browse Source
This commit is contained in:
cesnimda
2026-10-27 05:25:47 +01:00
parent 2fdcb60fa9
commit 4a8c907d70
10 changed files with 1827 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/cv-integration-qa/EXTRACTION_ACCURACY.md
+11 -5
View File
@@ -15,20 +15,26 @@ The separate structural corpus contains 18 logical cases (8 development, 10 froz
| Correct annotated block/page references | 24/27 | 27/27 |
| Required ambiguity cases correctly preserved/withheld | 6/6 | 6/6 |
The disjoint **10-case frozen split**, scored separately without changing annotations, recovered 72/72 literal fragments per format. PDF typed TP47/FP0/FN13 (precision 100%, recall 78.33%, F1 87.85%); DOCX TP60/FP0/FN0 (all 100%). PDF jobs complete 9/10 and education 1/3; DOCX jobs 10/10 and education 3/3. Provenance is PDF11/13 versus DOCX13/13; required abstentions 5/5 per format. Empty project/certification/language categories supply no reconstruction evidence despite conventional empty-denominator metric defaults. These results overlap the full 18-case scores and must not be added to them. Evidence: `metrics-frozen-{pdf,docx}-01.json`.
Literal recovery does not establish table relationships, reading order or geometry. Native PDF table/long-layout relationships account for missing typed facts and source references. Their text remains available for explicit review mapping. Source evidence transports block/page identity, not character spans or table-cell geometry. The separate actual browser and MariaDB tests establish reviewed mapping and persistence; this benchmark performs neither.
Independent static review also identified the retained legacy pipe grammar: without recognized headers, column one is treated as title/qualification and column two as employer/institution. Reversed headerless pipe records therefore still need user correction/mapping. This existing assumption is not qualified as universal source semantics. Every proposal still requires user approval; it remains extraction acceptance debt.
Explicit labels, header-defined table order, genuine lists and date-only lines now establish bounded relationships. Repeated employer/title strings retain their selected source occurrence. Unlabelled employer/title or institution/qualification order stays unresolved. A later ambiguous record cannot become a previous job's achievements. One legacy paragraph continuation is allowed only with an established compatible context and is marked `needs_review` in the review UI. Certification cell clusters cannot become several invented certificates.
Review burden is measured only as emitted controls: PDF152 review fields across18 documents (median8.5, maximum16), DOCX147 (median8, maximum16). All proposals require approval. These counts include preserved unresolved information and do not measure human time, comprehension or correction quality. Duplicate behavior is verified by finite exact/revised/older/overlapping/similar-employer service scenarios and actual same-source browser reimport; no population duplicate precision/recall or calibrated matching probability is claimed. Zero typed false positives on this structural corpus is not a global zero-hallucination guarantee; the original oracle's classifications and truncated name remain visible below.
## Original corpus comparison
The original strict oracle was rerun against the current interpreter using the **unchanged previously captured real Docling outputs**, rather than claiming those conversions were freshly repeated. PDF completion remains 13/14 (the scan is an expected rejection); DOCX 13/13.
| Original oracle | Current precision | Current recall | FP / FN |
|---|---:|---:|---:|
| PDF, 14 cases | 93.38% | 94.63% | 10 / 8 |
| DOCX, 13 cases | 97.26% | 95.30% | 4 / 7 |
| Original oracle | Current precision | Current recall | Derived F1 | FP / FN |
|---|---:|---:|---:|---:|
| PDF, 14 cases | 93.38% | 94.63% | 94.00% | 10 / 8 |
| DOCX, 13 cases | 97.26% | 95.30% | 96.27% | 4 / 7 |
These scores include unresolved custom-source classification and must not be replaced by the better typed-only scores. On the common typed scope excluding `otherSections`, PDF changes from TP132/FP4/FN8 to TP132/FP1/FN8; DOCX changes from TP140/FP0/FN0 to TP133/FP0/FN7. Seven headerless DOCX table facts are now withheld for user assignment rather than inferred from position. One PDF long-name association remains wrong. This recall tradeoff is a known limitation, not a full accuracy pass.
These scores include unresolved custom-source classification and must not be replaced by the better typed-only scores. On the common typed scope excluding `otherSections`, PDF changes from TP132/FP4/FN8 to TP132/FP1/FN8; DOCX changes from TP140/FP0/FN0 to TP133/FP0/FN7. Seven headerless DOCX table facts are now withheld for user assignment rather than inferred from position. The remaining typed PDF error is a truncated contact name: the native parser splits the last word into a separate unlabelled paragraph. That word remains visible in unresolved source; automatically joining it would assume an unproved relationship. The user can correct the name before approval. It is not an invented employment or qualification. This recall tradeoff and correction burden remain limitations, not a full accuracy pass.
## Evidence and reproducibility