ci: retain native parser crash diagnostics
CI and Deploy / test (push) Failing after 2m54s
CI and Deploy / deploy (push) Has been skipped

This commit is contained in:
cesnimda
2026-10-21 07:13:04 +02:00
parent 0b6f4b39eb
commit 8699ae4f42
2 changed files with 52 additions and 1 deletions
+2 -1
View File
@@ -111,7 +111,8 @@ jobs:
done
test "$parser_install_succeeded" = true
PIP_CACHE_DIR="$parser_pip_cache" python3 /tmp/pip.pyz install --upgrade --target .python-ci pytest==8.3.5 httpx2==2.12.0
PYTHONPATH="$PWD/.python-ci" AI_SERVICE_SKIP_MODEL_LOAD=1 python3 -m pytest -q
python3 --version
PYTHONFAULTHANDLER=1 PYTHONPATH="$PWD/.python-ci" AI_SERVICE_SKIP_MODEL_LOAD=1 python3 -m pytest -q
- name: Restore backend
# The runner/proxy has occasionally supplied checksum-invalid NuGet cache entries (NU3008).
@@ -0,0 +1,50 @@
# Parser CI native exit investigation, 2026-10-21
## Verified evidence
CI run 834, API test job 1721, failed in `Test document parser boundary`.
The fetched log is `.artifacts/phase7/p7-09/instrumented-release-ci/20261021T050601Z/failed-job-logs.log`;
the adjacent `api-observation.json` records the GET of job 1721's logs.
The runner's internal task ID 1689 is separate from the API job ID.
At 2026-10-20 20:36:38 UTC, pytest reported `56 passed, 1 warning in 5.36s`.
The shell subsequently reported a segmentation fault and exit 139. Passing assertions
did not constitute successful process completion. Later browser steps did not run;
deployment remained blocked.
Read-only host inspection with
`coredumpctl --no-pager info 3804681` confirmed:
- Signal: 11 (SIGSEGV).
- Command: `python3 -m pytest -q`.
- Executable: `/usr/bin/python3.11`.
- Timestamp: 2026-10-20 22:36:38 CEST (20:36:38 UTC).
- Docker scope: `040c0b99c92e0a5a48c00994e10a264884ca8172f5eac94d49e3d7aa1d92fd27`,
matching the failed job's container.
- Stored core: `/var/lib/systemd/coredump/core.python3.0.7aab2b376ce94d92bf796bbeb62063a4.3804681.1792528598000000.zst`.
The core was inaccessible to SSH user `pi`; `sudo -n` required a password.
No native backtrace was obtained. Kernel journal error entries for
20:30-20:40 UTC were empty; that does not establish host health.
## Interpretation and bounded diagnostic
The timing is consistent with interpreter shutdown or cleanup, but the root cause
is unproven. `AI_SERVICE_SKIP_MODEL_LOAD=1` prevents model weight loading;
`app.py` still imports Torch and Transformers. Native extension cleanup is therefore
a candidate, not an identified cause. Previous download corruption and browser
crashes do not prove the cause of this Python crash.
The workflow now prints the Python version and enables `PYTHONFAULTHANDLER=1`
for the unchanged pytest command. It preserves the existing dependency installs,
all tests, and process exit status, without adding retries or suppressing signals.
The existing pip log already records installed distribution versions. In this run,
the test-runtime install upgraded locked AnyIO 4.14.2 to 4.15.1 and IDNA 3.19 to
3.20; this is a reproducibility observation, not a demonstrated cause, and no
dependency change is included here.
Next evidence: inspect the retained core with authorized access, or capture the
faulthandler output from a subsequent normal CI run before choosing a fix.
The diagnostic change has only received workflow text and whitespace checks;
it has not been exercised in CI. No production mutation, model qualification,
or successful release is claimed.