cesnimda
62 lines
10 KiB
Markdown
62 lines
10 KiB
Markdown
# Test baseline and gaps
|
|
|
|
The [test-file census](TEST_FILES.md) lists every discovered tracked frontend test file (76: 67 unit/component and nine browser/HTTP specification files), 106 backend test-source files, plus extension/parser/repository safety files. Counts describe files, not executed cases or coverage percentages. The source snapshot is `97fce89`; subsequent bounded CI packaging corrections are tracked in the B06 release log, not redesign work.
|
|
|
|
## Test layers
|
|
|
|
| Layer | Setup and demonstrated scope | Limit |
|
|
|---|---|---|
|
|
| Frontend unit/component | Jest, jsdom, Testing Library; [config](../../../job-tracker-ui/jest.config.cjs). Account switching, routes, workspace/drafts, CV, mail, admin, theme, translations, dialogs and imports/exports | Babel strips TypeScript; mocks do not prove server ownership or real provider behavior |
|
|
| General browser | [Playwright config](../../../job-tracker-ui/playwright.config.ts): one Chromium project, isolated .NET API/SQLite data and local frontend; production static build supported | Chromium only; no actual Traefik/nginx; no cross-browser or screen-reader certification |
|
|
| Browser smoke | Nine cases in `e2e/smoke.spec.ts`: health/readiness, public plans, login, Free restrictions, reviewed job create, deep-linked job workspace/dirty guard, career, public CV/PDF, long searchable Code PDF | Selected journeys, not exhaustive editor states or all providers |
|
|
| Account isolation | Four `e2e/account-isolation.spec.ts` cases: original A→B replay, missing/forged context, crafted CV ownership/association, two-tab private-state disposal/Back | Retain API and database assertions when UI changes |
|
|
| Accessibility | Two axe cases over eight sampled routes | See missing state matrix below |
|
|
| Other general browser | One each discovery responsive review, dark Kanban, CSV security/UTF-8 byte preservation | Does not establish whole-app mobile/theme parity |
|
|
| PDF HTTP probe | `pdf-tests/admission.spec.ts` and dedicated config | Synthetic bounded concurrent requests; admission differs from time-window limiting |
|
|
| Rate-limit HTTP probe | `rate-limit-tests/isolation.spec.ts` and dedicated config | Nine account/session/IP/auth/forwarding scenarios; native API topology |
|
|
| Production ingress HTTP | `ingress-tests/production.spec.ts` and dedicated config | Six native production-mode origin/proxy checks; not running nginx or Traefik |
|
|
| Backend | xUnit/Moq, SQLite/InMemory and HTTP test seams; API/service ownership, auth, parser, CV/history/PDF, queue, billing, export, migration and rate/proxy tests | Provider-host tests and fault/probe tests have separate runtime requirements; SQLite is not full MariaDB proof |
|
|
| Repository/dependencies | Compose rendering + 17 hostile mutations, secret scan/SBOM, evaluation and supply-chain scripts | Config validation cannot prove host firewall or actual socket peers |
|
|
| Adjacent clients/services | Browser-extension node tests and Python parser/service tests | Preserve capture/document boundary; avoid pretending these are covered by React Jest |
|
|
|
|
The default browser suite currently contains **18 tests**. The nine special browser/HTTP files count includes six general e2e files and three separately configured suites. Special configurations must be invoked explicitly; a plain general Playwright run does not run all of them.
|
|
|
|
## Existing evidence versus fresh execution
|
|
|
|
| Evidence | Result and classification |
|
|
|---|---|
|
|
| Previous B06 unchanged application regression | 166 focused backend, 3 Compose, 6 production HTTP, 9 rate-limit HTTP passed; [exact prior commands/results](../../audits/2026-09-07/validation/b06-completion.md) |
|
|
| Previous B06 broader baseline | 915 selected backend tests, 298 frontend tests/67 suites, 18 browser tests, Release builds and targeted lint passed; external-DNS-dependent IMAP test excluded locally and explicitly documented |
|
|
| Fresh Phase 1 type reproduction | From `job-tracker-ui`: `node node_modules/typescript/bin/tsc --noEmit --incremental false` → exit 1, `src/interview-prep.test.tsx(120,10): error TS2571: Object is of type 'unknown'.` |
|
|
| Fresh release-prerequisite checks | `python scripts/test-ingress.py` → 3/3 pass; real standalone Compose path also tested; official downloaded Linux binary digest verified; tracked-secret scan and diff check pass |
|
|
| Fresh packaging regression at `23cc250` | Explicit installed .NET 9.0.317 SDK: Release build + **22/22 PDF admission tests** pass, then **915/915 selected backend tests** pass (same documented external-DNS IMAP exclusion). Results: `.artifacts/redesign-audit/backend.trx` |
|
|
| Actual CI push verification | Runs 736/737 exposed missing Compose/token prerequisites; corrected. Run 738 passed Compose/safety/extension/parser/API build, then exposed Linux PDF test-fixture packaging collision. Follow-up result belongs in [release log](../../audits/2026-09-07/validation/b06-completion.md#authorized-release-attempt-and-ci-prerequisite-correction) |
|
|
| Fresh Linux backend confirmation | Run 741, `2024c37`, passed **916/916 backend cases**, zero skipped, after test packaging/runtime-location and TLS EOF-or-no-read portability corrections. This includes the locally excluded DNS-dependent case. Application behavior and all security assertions remain protected; deployment has separate gates |
|
|
| Subsequent authorized release correction, `a861ca4` | Next 16.3.4/sharp 0.35.4/js-yaml 3.15.2 resolve three blocking dependency entries: installed npm audit **zero vulnerabilities**. Fresh **298 frontend / 18 browser** pass, plus **13 interview tests** after the test-only TS2571 correction. Standalone tsc, lint and production static build pass. Exact commands and failed-before evidence remain in the B06 release log; this is release repair, not a redesign implementation |
|
|
| Final CI retry, run 743 at `27657f3` | **916 backend / 298 frontend** pass, zero backend skips; Compose/safety/parser, dependency audits, lint and production builds pass. Browser gate: **6 passed / 12 failed**, beginning with worker SIGSEGV, then retry timeout and closed-browser navigation errors. Deploy skipped. Repeated process crashes reached the bounded stop condition; runner-host evidence is required before another release attempt. See [exact result and diagnostic handoff](../../audits/2026-09-07/validation/b06-completion.md#final-release-retry--2026-09-10). Local browser success is not a replacement for CI |
|
|
| Live/container boundary | Local Docker engine unavailable; F04 production evidence remains incomplete. No audit claim of real proxy-chain success |
|
|
|
|
The selected broader backend suite was repeated for the PDF fixture packaging change; the frontend suite was not repeated merely for documentation edits. Preserve the distinction between prior unchanged-source evidence, newly executed checks and actual CI failures. Run 739 subsequently failed earlier in the unchanged API compiler with `csc.dll` exit 139 on both existing attempts; it did not reach the corrected test-project build. No test gate was bypassed.
|
|
|
|
## Exact TypeScript root cause
|
|
|
|
[Interview test:120](../../../job-tracker-ui/src/interview-prep.test.tsx#L120) originally read `.content` from `mockedApi.post.mock.calls[0][1]`. Axios's mocked generic request-data argument is typed `unknown`; the test dereferenced it without narrowing. The runtime assertion passed because Babel removes types. The production component's emitted behavior was not shown broken by this test diagnostic.
|
|
|
|
[tsconfig](../../../job-tracker-ui/tsconfig.json) includes `src`, including tests. CI runs Jest/lint/Next build but has no explicit full standalone `tsc --noEmit` gate. The patched Next 16.3.4 release build independently failed on this exact diagnostic. Authorized release repair `a861ca4` replaced the property dereference with an object matcher, retaining both positive prompt and negative invented-content assertions. Fresh standalone tsc, 13 interview tests and Next build now pass. The original Phase 1 reproduction above remains historical evidence. Phase 2 still needs a deliberate standalone CI gate; no strictness or test coverage was weakened.
|
|
|
|
## Missing regression proof before redesign
|
|
|
|
| Priority | Add or preserve evidence |
|
|
|---|---|
|
|
| First | Stable type-check gate; B02 stale/queued mutation + cross-tab unmount as shared-client acceptance; legacy URL/overlay/full-page parity |
|
|
| First | Populated CV editor/profile/job detail plus open menus/dialogs/error/conflict states in EN/NB, light/dark, mobile/desktop; preserve underlying draft/history assertions |
|
|
| First | AI suggestions rejected/accepted distinctly; actual profile values unchanged when editing a CV/cover letter; unsupported/locked capabilities behave honestly |
|
|
| First | Typed date values, JSON import/export and B05 CSV bytes; attachments/private/public download distinctions; package history remains frozen |
|
|
| Before mail migration | Draft revisions, uncertain send/new attempt, provider reconnect/read failure, ownership and explicit send confirmation; use synthetic/sandbox providers |
|
|
| Before admin migration | Non-admin denied APIs, role revocation and last-admin guards, sensitive diagnostics excluded from personal pages |
|
|
| Before public release | Keyboard + NVDA/Narrator, preview scroll/focus, zoom/reflow, forced colors, real mobile keyboard and touch; provider/production gates separately |
|
|
|
|
Axe currently scans `/`, `/login`, `/dashboard`, `/jobs`, `/kanban`, `/career`, `/settings`, `/admin/system`, mostly after `body`/`main` appears, in English. Missing builder library/editor, job detail, discovery, offers, mail/review, public CV, account security, registration/reset/verification, operations, admin users/audit and interactive loaded states must be selected intentionally. Do not convert a green shell sample into a WCAG compliance claim.
|
|
|
|
Recommended later commands: locked dependency restore; Release backend build/test; full Jest; standalone tsc; lint; production frontend build; general Playwright static-export suite; explicit PDF/rate/ingress configurations; Compose validator and `git diff --check`. Use the environment/setup commands from the completion reports rather than guessing SDK paths or running probes against production.
|