cesnimda
5.5 KiB
AI and parser qualification reconciliation
What was actually exercised
| Layer | Evidence / decision | What it does not establish |
|---|---|---|
| Real Docling parsing | Fresh14 real document-extractor tests; actual PDF/DOCX import browser lanes and live local PDF worker. Native extraction uses pinned model-free Docling | OCR/layout-model or arbitrary-document completeness |
| Deterministic interpretation | Frozen benchmark evidence remains separately attributed: PDF typed precision100%, recall81.967%; DOCX122/122 typed facts. Fresh oracle/structural helper tests validate their own logic | Population accuracy, real model quality, or every unusual layout |
| Review/correction/application | Real browser/MariaDB/RD21 lanes exercise approved facts, duplicates, rollback, stale writes and historical preservation | Automatic model semantic truth or human usability approval |
| Generic queued strategy | Fresh actual local operation succeeded with saved nonempty160-character summary; admin retained a prior provider_unavailable event | First-attempt reliability, latency guarantees or factual CV interpretation |
| Scheduled job enrichment | Fresh actual scheduler persisted summary681characters plus tags, then read back through API | Semantic quality or external provider/production acceptance |
| Optional CV model adapter | Three real isolated Qwen requests completed but all three were rejected with ValueError by the adapter. Compatibility and factual accuracy remain BLOCKED | Zero accepted suggestions is not zero hallucinations or100% precision |
| Active CV processing configuration | CV_IMPORT_AI_ENABLED=false; adapter remains disconnected from active deterministic processing | No model activation or owner security-risk acceptance |
Real adapter probe
AI_EVALUATION_PLAN.md records authority and guards before inference. The official local qwen2.5:7b Q4_K_M manifest845dbda0ea48ed749caafd9e6037047aa19acfcfd82e704d7ca97d631a0b697e and all stored blobs were hash-verified. A disposable Ollama instance reused those bytes through a read-only model-volume mount, with no published ports, internal-only network, cloud disabled, four CPUs/8GiB, one loaded model/parallel request and no daemon socket. A bounded read-only Python client exercised the unchanged adapter using its real transport, JSON schema, source grounding and120-second timeout. Existing development Ollama/services/configuration/cache were not changed. Exact generated containers and the empty network were removed, with cleanup recorded.
EN, Bokmål and instruction-in-source cases took79.009,47.308 and35.811 seconds respectively. All returned the explicit rejected/failed outcome. No raw response was retained by this first probe, so the precise invalid suggestion/schema distinction is not claimed. No accepted semantic precision/recall is computed. These three independently authored development probes are not the frozen corpus and are not PDF/DOCX end-to-end model evaluations. They supply genuine negative compatibility evidence; they do not replace the outstanding full qualification.
Raw source-bound process/results/cases/client hashes are in .artifacts/automation-handoff-01/model-probe-01/; portable evidence is indexed in TEST_EVIDENCE.json. The probe process exit0 means the measurement completed and cleanup ran, not that AI acceptance passed.
The local runtime's original model volume was found writable despite a read-only container root. This finding informed the isolated read-only mirror above; no stronger existing-cache protection is inferred. Artifact integrity and scoped known-advisory evidence are independent of whole-image/native sandbox security. A fresh read-only inventory of the exact current sidecar image found63 installed distributions;126 exact OSV/PyPI queries returned zero advisory records and one retained PyPI404 for torch2.13.0+cpu. The new raw audit remains INCOMPLETE. Python remains CONDITIONAL under the retained P9-SEC-02 disposition; neither this probe, this incomplete scan nor import-lock zero advisories closes native/whole-image/production qualifications. No package was resolved or installed by the audit.
Future owner-approved qualification must use immutable approved artifacts/runtime and identical frozen documents/annotations, source and prompt hashes; measure precision/recall, record association, dates, hallucinated/unsupported claims, provenance, duplicate behavior, multilingual semantics and review burden. Add missing/malformed output, unavailable service, timeout/cancel/retry, injection, resource and cache-integrity probes. Keep deterministic, model-assisted, user-corrected and persisted outcomes separate. Do not select a permanent general-purpose model or enable the adapter from these results.
Existing generic-AI precondition
StrategySnapshotService currently requires nonempty legacy ProfileCvText in both admission and generation. The reviewed Docling apply path persists canonical structured facts and deliberately does not overwrite that user-controlled text. Therefore the successful strategy smoke used separately authored fictional CV text as an explicit precondition; it does not prove strategy generation from a canonical-only imported profile. Career Profile → CV Editor → PDF is independently tested and does not require this legacy text. Product Review should assess this generic-AI source compatibility as a separate bounded backlog item; do not silently populate legacy text from unreviewed uploaded content or claim every AI option works for every profile/account.