cesnimda/jobtrackingapp
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

docs(qa): qualify Phase 9 regression and acceptance

Browse Source
This commit is contained in:
cesnimda
2026-10-23 07:31:27 +02:00
parent 0fb108bd67
commit 8e7e3f6387
63 changed files with 64728 additions and 6835 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/redesign/phase-9/COMMANDS.md
+4 -4
View File
@@ -23,11 +23,11 @@ $env:PLAYWRIGHT_STATIC_EXPORT='true'
$env:PLAYWRIGHT_PREBUILT_EXPORT='true'
$env:CV_FIDELITY_OUTPUT='E:\Storage\Projects\Active\Job tracker\.artifacts\phase9\fidelity'
$env:CV_SETTINGS_FIDELITY_OUTPUT='E:\Storage\Projects\Active\Job tracker\.artifacts\phase9\settings-fidelity'
$env:PLAYWRIGHT_JSON_OUTPUT_NAME='E:\Storage\Projects\Active\Job tracker\.artifacts\phase9\ordinary-shard-1.json'
rtk proxy node node_modules/@playwright/test/cli.js test --shard=1/4 --retries=0 --reporter=list,json --output=../.artifacts/phase9/ordinary-shard-1-results
$env:PLAYWRIGHT_JSON_OUTPUT_NAME='E:\Storage\Projects\Active\Job tracker\.artifacts\phase9\ordinary-cta-shard-1.json'
rtk proxy node node_modules/@playwright/test/cli.js test --shard=1/4 --retries=0 --reporter=list,json --output=../.artifacts/phase9/ordinary-cta-shard-1-results
```
Repeat the ordinary command serially with `--shard=2/4`, `3/4` and `4/4`, separate `ordinary-shard-N.json` and output names, and an unchanged source/export. Playwright keeps connected files together because fullyParallel is false. Native `test --list --reporter=json` records the complete discovery set without starting servers. Final reconciliation requires the exact discovered362 IDs to appear once across the four successful shards. Interrupted attempts are retained but excluded. This preserves completed result checkpoints rather than repeatedly losing a nearly hour-long run; assertions/retries/skips are unchanged.
Repeat serially with `--shard=2/4`, `3/4` and `4/4`, separate `ordinary-cta-shard-N.json` and output names, and unchanged source/export. The `-cta` successor follows the P9-V02 repair/rebuild and retains the provider-worker isolation fixture. Browser interception alone does not constrain that server-originated worker: keep `Workers__SavedJobSearchAlertsEnabled=false` for local fixture qualification. Playwright keeps connected files together because fullyParallel is false. Native `test --list --reporter=json` records discovery without starting servers. Final reconciliation requires all 362 discovered IDs exactly once across four successful shards. Interrupted/superseded attempts remain excluded, with assertions/retries/skips unchanged. Use fresh output names for later reproduction to preserve evidence.
Run separate browser configurations sequentially, each with its own JSON/output name: `playwright.cv-ai.config.ts`, `playwright.auth-recovery.config.ts`, `playwright.rate-limits.config.ts`, `playwright.ingress.config.ts`, `playwright.pdf.config.ts`, and `../docs/redesign/phase-5/p5-15/p5-15-01/diagnostics/playwright.config.ts`. Use the same CLI `--config=... --retries=0 --reporter=list,json`. The Phase9 lane uses `../docs/redesign/phase-9/p9-05/diagnostics/playwright.config.ts` (18 cases); the separate API-only volume lane uses `../docs/redesign/phase-9/p9-05/volume/playwright.config.ts` (3 cases). Native PDF admission uses inert processes; it is distinct from native rendering fidelity.
@@ -35,6 +35,6 @@ The configs start isolated synthetic local API/storage and a static frontend. Do
From `tools/summarizer`, `SKIP_MODEL_LOAD=1`, run `rtk proxy .venv/Scripts/python.exe -m pytest -q tests --junitxml=../../.artifacts/phase9/sidecar.xml`. No inference semantics are qualified by these56 contracts.
The unchanged `scripts/verify-cv-fidelity.py` takes a corpus directory containing `measurements.json` and native PDF/preview fixtures. It requires the existing Python PDF/Pillow dependencies (bundled workspace runtime used in this run). Invoke it separately for the fidelity corpus and the combined Settings measurements. Never increase its tolerances to clear a failure.
The unchanged `scripts/verify-cv-fidelity.py` takes a corpus directory containing `measurements.json` and native PDF/preview fixtures. It requires the existing Python PDF/Pillow dependencies (bundled workspace runtime used in this run). After the final document producers finish, run `rtk proxy <python-runtime> docs/redesign/phase-9/native-documents.py`. It merges the four Settings measurements, invokes the original oracle separately for both corpora, verifies stable input hashes and records successful commands, final producer report/exit hashes and result hashes in `p9-05/evidence/native-document-provenance.json`. Never increase oracle tolerances to clear a failure.
After accepted reports exist, `rtk proxy python docs/redesign/phase-9/evidence.py` builds portable cases, route/axe/RD21/performance records and hashes raw reports. `rtk proxy python docs/redesign/phase-9/reconcile.py` checks362 ordinary cases,396 route vectors and57 unchanged preservation dispositions before generating final ledgers. Missing or failed data must not be replaced with baseline counts. Source/artifact identities are retained in `p9-00/source-manifest.json`.