cesnimda/jobtrackingapp
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

test(cv-import): qualify regression and hand off conditional acceptance

Browse Source
This commit is contained in:
cesnimda
2026-10-26 19:09:43 +01:00
parent b62962c68a
commit ef425c288b
11 changed files with 438 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/cv-import/QUALITY.md
+4
View File
@@ -30,12 +30,16 @@ Actual artifact manifest: `.artifacts/cv-import/corpus-02/manifest.json`. Real p
The scan is an expected rejected PDF with no annotated facts. PDF differences are seven table-category/relationship errors and two long-name/unclassified-fragment errors; source words remain visible. There are no newly invented words in these discrepancies, but **nine unsupported or mis-associated field claims** remain a factual mapping failure and must not be reported as zero hallucination risk. DOCX's small synthetic exact match is not a population estimate. Employment categories matched 13/14 PDF and 13/13 DOCX; education 12/14 and 13/13, including empty categories. Date and skill per-case results are in the machine-readable reports. Conservative duplicate matching, partial dates, list provenance and source-grounding have separate application regressions; no statistically calibrated duplicate score is claimed.
Date/current-marker facts: PDF 30 correct / 30 predicted / 33 expected (precision 100%, recall 90.91%); DOCX 33/33/33. Original precision is part of the oracle. Skill attribution/normalization exact matches all 13 text-bearing cases per format; the scan has no approved facts. These are the fixed synthetic cases, not calibrated population results.
Review burden: 150 PDF and 149 DOCX proposed field decisions across completed cases; none automatically approved. Human review time/correction burden is NOT MEASURED. No model inference occurred, and synthetic adapter tests count only as integration/security tests.
Earlier attempts remain retained: corpus-01 wrongly styled short DOCX body sentences as headings; evaluator association tests caught swapped bullets for repeated employer/title with distinct dates. The first DOCX final scoring subset was accidentally read using the Windows default encoding; UTF-8 subset `corpus-docx-final-02.json` and `metrics-docx-final-02.json` supersede that invalid score. No extraction or frozen annotation was changed to improve the score.
## Performance and limits
Development generator environment: Windows 11 build 26200, CPython 3.12.14, ReportLab 4.4.9, python-docx 1.2.0, Pillow 12.3.0, pypdf 6.10.0 (`benchmark-runtime.json`). Conversion uses the separate exact Docling lock in SIDECAR.md. Frozen file hashes, rather than regenerated DOCX ZIP timestamps, bind benchmark comparisons.
27 sequential actual child conversions on this Windows development host: `extraction-resources-final/resources.json`, measured inside the real interpreter rather than its Windows venv launcher. Wall median 2.283 s (2.051–2.549), CPU median 1.891 s (1.688–2.125), peak working set median 144,257,024 bytes, maximum 146,079,744 bytes. Includes Python/import startup. Earlier `extraction-benchmark-final/resources.json` CPU/memory sampled the launcher only and is invalid for parser resource claims; its wall/input/output measurements remain valid.
Corpus input total 777,032 bytes; parser JSON output total 27,371 bytes. These are document/result disk measurements, **not peak temporary/cache disk usage**. Conversion options disable image/OCR/model artifacts; private work cleanup is covered by tests. Maximum temporary disk, load under concurrent users and upload network timings require broader resource qualification. Do not infer production capacity from these small fixtures.