test(cv-qa): qualify long imported PDFs and cross-platform fidelity
This commit is contained in:
cesnimda
@@ -0,0 +1,30 @@
|
||||
# PDF fidelity — 2026-10-27
|
||||
|
||||
The original CV16 + settings8 and independent letter4 oracles remain unchanged. Existing golden documents, portable fonts, physical-document geometry and acceptance records were preserved. Fresh Windows and isolated Linux results each pass **24 CVs /60 pages and4 letters /14 pages**. The Linux native renderer passes6 cases, with source/image/DLL/migration binding and complete owned-resource cleanup. `evidence-summary.json` reconciles both platforms; historical success alone was not used as a fresh pass.
|
||||
|
||||
## Additional imported documents
|
||||
|
||||
The six real-Docling browser journeys generated six actual one-page PDFs. All six were read with pypdf and rendered with Poppler. File hashes, extracted text, page counts and render paths are retained in `.artifacts/cv-integration-qa/imported-pdf-proof-01/results.json`. The four new journeys also supply independent byte-bound semantic checks in `semantic-results.json`:
|
||||
|
||||
| Imported journey | Checked exported content |
|
||||
|---|---|
|
||||
| Employer-first DOCX → minimal | CV-only Principal Engineer, Ambiguous Labs, 2019 and 2022 |
|
||||
| Title-first PDF → technical | Same approved mapped facts and CV-only title |
|
||||
| Ambiguous education DOCX → minimal | CV-only Advanced Computing, Example Learning Centre, 2015 and 2017 |
|
||||
| Repeated overlapping roles DOCX → technical | CV-only Engineer, Repeat AS, all three role details and associated year markers |
|
||||
|
||||
The two retained journeys contain the corrected `Reviewed Åse Example` name, and EN experience/skills or Bokmål education/language data. Actual rendered examples were visually inspected by the agent; this is not human/manual acceptance. Long synthetic account emails wrap in the narrow technical-template identity area without clipped role content. No claim of ideal typography for that artificial account identifier is made.
|
||||
|
||||
The short imported documents prove exercised approved facts reach searchable rendered output and CV-only customizations survive without changing canonical facts. They complement the following long connected case; neither qualifies every arbitrary layout, glyph or native printer. Scanned/image-only input is not qualified by these export tests.
|
||||
|
||||
## Long imported content
|
||||
|
||||
The additional real Docling browser case imports `long-labelled-integration.docx`, explicitly approves its groups and independently verifies **12 jobs,36 correctly associated details,60 skills,two qualifications and literal employment years/current state** in canonical readback. Three independent variants export minimal/A4, modern/Letter and technical/A4 PDFs. Returned settings are checked; canonical facts and earlier variant DTOs remain unchanged. The separate fixture generator is `benchmarks/generate_long_journey.py`; this fixture does not enter frozen accuracy metrics.
|
||||
|
||||
Each export has **four pages**, giving three documents /12 pages. The byte-bound oracle in `.artifacts/cv-integration-qa/long-pdf-proof-02/results.json` confirms all **142 markers per document**, all12 employer/title/detail associations, actual A4/Letter dimensions and every extracted word's page bounds. Text comparison normalizes NFKC, case and whitespace for typography/ligatures. Word bounds use installed pdfplumber; all12 pages are rendered with Poppler. The first proof attempt lacked the `pdftotext` executable and is retained as a harness failure; no package download or golden adjustment was used to resolve it.
|
||||
|
||||
The agent visually inspected first/last pages of all three templates. Details and dense skills remain readable and uncut in those samples. The technical template wraps the longer name and email within its narrow sidebar, sometimes inside a word; the minimal template leaves a sparse final page for languages/interests. These are noncritical typography/pagination limitations, not missing facts or human print acceptance. Human review remains required before a broad presentation-quality decision.
|
||||
|
||||
## Production and manual limits
|
||||
|
||||
PDF HTTP success/signature assertions are separate from semantic and render oracles. The inert PDF-admission lane tests admission/cancellation, not physical fidelity. Windows and disposable Linux qualification are separate from production runtime identity, actual viewer rendering and manual printing. Human PDF/print review remains OPEN, with actions in ACCESSIBILITY_AND_MANUAL_QA.md.
|
||||
@@ -0,0 +1,66 @@
|
||||
"""Generate a separate labelled integration fixture; never mutate benchmark goldens."""
|
||||
import argparse
|
||||
import hashlib
|
||||
import json
|
||||
from pathlib import Path
|
||||
from docx import Document
|
||||
|
||||
|
||||
def generate(output):
|
||||
output.mkdir(parents=True, exist_ok=False)
|
||||
document = Document()
|
||||
document.add_paragraph('Name: Imported Åse Østergård')
|
||||
document.add_paragraph('Email: ase@example.test')
|
||||
document.add_paragraph('Phone: +47 400 00 002')
|
||||
document.add_paragraph('Location: Tromsø')
|
||||
document.add_paragraph('Headline: Senior software engineer')
|
||||
document.add_heading('Professional Summary', level=1)
|
||||
document.add_paragraph('Builds accessible software and documents explicit evidence. Synthetic integration fixture containing no personal career information.')
|
||||
document.add_heading('Work Experience', level=1)
|
||||
employers, titles, details = [], [], []
|
||||
for index in range(1, 13):
|
||||
title = f'Engineer {index:02d}'
|
||||
company = f'Synthetic Organisation {index:02d}'
|
||||
document.add_paragraph('Title: ' + title)
|
||||
document.add_paragraph('Company: ' + company)
|
||||
document.add_paragraph(f'{2010 + index} - {2011 + index}' if index < 12 else '2022 - Present')
|
||||
if index % 2 == 0:
|
||||
document.add_paragraph('Location: Tromsø')
|
||||
for bullet in range(1, 4):
|
||||
text = f'Record {index:02d} evidence {bullet}: Built and verified accessible workflow components, preserving approved information and explicit review decisions across independent document revisions.'
|
||||
document.add_paragraph(text, style='List Bullet')
|
||||
details.append(text)
|
||||
employers.append(company)
|
||||
titles.append(title)
|
||||
document.add_heading('Education', level=1)
|
||||
qualifications = ['Bachelor i informatikk', 'Master i programvaresystemer']
|
||||
for index, qualification in enumerate(qualifications):
|
||||
document.add_paragraph('Qualification: ' + qualification)
|
||||
document.add_paragraph('Institution: Example Learning Centre')
|
||||
document.add_paragraph(f'{2005 + index * 3} - {2008 + index * 3}')
|
||||
document.add_paragraph('Studied accessible systems, databases and source-grounded document workflows.', style='List Bullet')
|
||||
document.add_heading('Skills', level=1)
|
||||
skills = ['Python', 'C#', 'SQL', 'React', 'TypeScript', 'Docker'] + [f'ExplicitTool{index:02d}' for index in range(1, 55)]
|
||||
document.add_paragraph('; '.join(skills))
|
||||
document.add_heading('Languages', level=1)
|
||||
document.add_paragraph('Norsk: morsmål')
|
||||
document.add_paragraph('English: advanced')
|
||||
document.add_heading('Interests', level=1)
|
||||
document.add_paragraph('Friluftsliv og tilgjengelighet')
|
||||
file = output / 'long-labelled-integration.docx'
|
||||
document.save(file)
|
||||
expected = {'name': 'Imported Åse Østergård', 'employers': employers, 'titles': titles,
|
||||
'details': details, 'qualifications': qualifications, 'skills': skills,
|
||||
'markers': ['Imported Åse Østergård', 'Tromsø', 'ase@example.test', *employers, *titles, *details,
|
||||
*qualifications, 'Example Learning Centre', *skills, 'Norsk', 'morsmål', 'Friluftsliv og tilgjengelighet']}
|
||||
assert len(employers) == 12 and len(details) == 36 and len(skills) == 60
|
||||
(output / 'expected.json').write_text(json.dumps(expected, ensure_ascii=True, indent=2), encoding='utf-8')
|
||||
(output / 'manifest.json').write_text(json.dumps({'file': file.name, 'sha256': hashlib.sha256(file.read_bytes()).hexdigest(),
|
||||
'generatorSha256': hashlib.sha256(Path(__file__).read_bytes()).hexdigest(),
|
||||
'purpose': 'Separate connected multi-page import/render fixture, not added to frozen accuracy metrics.'}, indent=2))
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument('--output', type=Path, required=True)
|
||||
generate(parser.parse_args().output)
|
||||
Reference in New Issue
Block a user