test(handoff): add fail-closed evidence schema and qualification records

2026-10-29 18:30:53 +01:00
parent 8804231be4
commit 78e5296881
16 changed files with 2068 additions and 909 deletions
@@ -0,0 +1,14 @@
+# Retained failures and recovery
+
+Failed attempts are not deleted, counted as passing tests or hidden by retries.
+
+| Attempt | Actual failure | Correction / boundary |
+|---|---|---|
+| post-01 | Temporary proxy exited before any browser case; immediate port lookup raised KeyError | A separate reproduction retained nginx's exact `mkdir /var/cache/nginx/fastcgi_temp: Read-only file system` error. The one-off fixture omitted FastCGI/uwsgi/scgi paths. post-02 points all temporary paths into the existing bounded `/tmp` mount and checks startup state. Root stays read-only; no application/configuration or test assertion changed |
+| proxy-reproduction-02 | Expected diagnostic exit1 before tests | Exact logs/state and owned-resource cleanup retained; not a product failure or extra test case |
+| post-02 Linux build | The runner incorrectly supplied an output outside `.artifacts/cv-ux-01`; existing helper rejected it before build | linux-post-03 resumes only the unfinished Linux/audit work under a fresh allowed child. The helper guard was preserved; already passing import/MariaDB/Windows lanes were not rerun or double-counted |
+| Revised S01 oracle | Windows8/36 and the separately recorded Linux result; original2mm coordinate threshold unchanged | Content, name wrapping, sidebar raster and visible-ink diagnostics are distinct evidence. No golden/tolerance change or aesthetic self-approval |
+| Isolated real CV-model probe | All3 synthetic responses rejected with ValueError | No model/validator replacement or activation. Compatibility/semantic accuracy remains BLOCKED; process exit0 means the measurement completed |
+| Initial handoff-validator RED | New tests failed because implementation module was absent | Native parser/hash/approval tests implemented and rerun; final unit/integration results supersede overlapping focused runs, without adding them |
+
+Raw evidence is under `.artifacts/automation-handoff-01/`, with portable native reports/process/source artifacts in TEST_EVIDENCE.json. The original broad regression ran once to completion with all four groups passing and source unchanged; it is not the previous workstream's composite recovery result. Transient provider_unavailable history on the fresh successful strategy operation is retained in the local-worker result; it is not described as first-attempt success.