jobtrackingapp/docs/audits/2026-09-07/validation/b03-completion.md

# B03 — Shared PDF admission and process lifecycle

Implementation date: **2026-09-08**. **B03 is complete locally; not deployed.** B01/B02 are preserved. No schema, deployment, infrastructure, billing, rate-middleware ordering, template redesign or storage-accounting change. B04 has not been started.

## Original behavior and root cause

The original [F03 finding](README.md#f03--pdf-host-exhaustion), [boundary probe](BoundaryProbes.cs) and [three-process observation](evidence/boundary-results.json) remain historical evidence. Before editing application code, the new permanent `Default_limit_refuses_second_actual_browser_until_first_exits` test failed against the actual exporter: **two concurrent calls launched two inert browser processes**. The failure and its process count are retained in `.artifacts/b03/before.trx`.

`ICvPdfExporter` was already registered as a singleton, but each call launched Chromium without an admission limit. A 60-second timeout limited duration, not simultaneous processes. Endpoint rate policies could not protect every caller. The old lifecycle read stdout/stderr only after exit (a full pipe could stall rendering), killed on cancellation without awaiting termination, and retained partially written output on failure.

The existing class name says Playwright, but this service uses **Chromium's CLI**, not a Playwright browser pool. One admitted render can create a browser process tree with several OS processes/threads. The concurrency limit counts rendering operations, not total PIDs or memory bytes.

## Every current rendering path

| Entry point | Owner/authorization and shared boundary |
|---|---|
| `POST /api/cv/variants/{id}/export-pdf` | Authenticated variant lookup by owner and ID; principal owner passed to the singleton exporter. |
| `POST /api/jobapplications/{id}/export-tailored-cv-pdf` | Existing authenticated, owner-filtered job lookup; current user passed to the same exporter. |
| `POST /api/profile-cv/export-pdf` | Existing Pro authorization and current-account profile; AI/HTML preparation precedes the same exporter. Permanent controller test uses inert AI to prove admission without external calls. |
| `GET /api/public-cv/{slug}/pdf` | Explicitly published variant only, existing `public-pdf` rate policy retained; resolved document owner passed to the same exporter, including anonymous visitors. |

Source search found one production `Process.Start`/`--print-to-pdf` rendering implementation and these four callers. No worker currently generates PDF. The package paths referenced in the original ticket prepare/store/copy existing artifacts; they do **not** launch a fifth renderer. Their existing tests remain in the affected/backend suites. Controller fallback exporters only throw when not configured; they cannot launch an unprotected process.

All callers resolve ownership before entering the exporter. B02 session preconditions and existing ID/owner authorization remain intact; capacity denial never authorizes access. Admin callers receive no special rendering pool. Public downloads consume the **document owner's** active allowance, so anonymous traffic cannot evade owner accounting by omitting a principal.

## Selected admission model and configuration

`Program.cs` registers one concrete [PdfRenderAdmission](../../../../JobTrackerApi/Services/PdfRenderAdmission.cs) singleton, injected into the singleton [PlaywrightCvPdfExporter](../../../../JobTrackerApi/Services/PlaywrightCvPdfExporter.cs). Even two exporter instances sharing that admission object use the same pool; a permanent process test proves this. No per-controller semaphore, queue, Redis, worker platform or database state was added.

| Setting/contract | Value |
|---|---|
| `CvExports:MaxConcurrentRenders` / `CvExports__MaxConcurrentRenders` | Default **1**, validated integer **1–8**. Eight is a configuration ceiling, not a proven safe production capacity. |
| Per document owner | Fixed **1 active render**, shared across variants, sessions and endpoints. Owner entries disappear when the lease ends; memory is bounded by the global active limit. |
| Waiting/queue | **None**: zero queued requests and zero admission-wait duration. Excess requests fail immediately; there is no pending work to wake up after cancellation. |
| Exhaustion | **503**, `code: pdf_capacity_exceeded`, `Retry-After: 3`, `Cache-Control: no-store`, safe ProblemDetails message. Retry-After is advisory, not a reservation. |
| `CvExports:RenderTimeoutSeconds` / `CvExports__RenderTimeoutSeconds` | Default **60**, validated integer **1–60**. Covers asynchronous export work after admission, including input/output IO. |
| Render timeout | **504**, `code: pdf_render_timeout`, safe ProblemDetails message. |
| Cancellation/shutdown | Request cancellation and `ApplicationStopping` linked to rendering; new work rejected when stopping. |
| Termination confirmation | Kill process tree and await root exit before returning capacity, with a separate **5-second** confirmation bound. If exit cannot be confirmed, disable new PDF admission and emit a Critical log until operator recovery. |

Options bind through existing .NET facilities and validate on host start; the admission constructor also validates direct construction. Zero, negative, malformed and out-of-range values cannot enable unlimited operation. Values are fixed for a process lifetime; restart after changing them. No production environment was changed. Compose currently declares one backend service without a replica count or backend CPU/memory/PID budget. The resource limits on the AI sidecar are not limits on the API's Chromium processes.

Default one was chosen because no reliable production per-render peak RSS/CPU budget has been measured. It prevents multiplication of the unmeasured cost, but does not prove one pathological CV is safe for every host. The bounded local test deliberately uses two **inert** renders to verify the configurable limit; it is not a capacity recommendation. Verify real container/PID/memory/headroom under R04 before raising the default.

One document owner cannot take multiple slots when the global limit is increased. With default one, any running user occupies the sole slot. Zero waiting does not promise FIFO fairness or prevent an aggressive caller winning repeated races after completion. It prevents an unbounded backlog and simultaneous exports of one owner's documents. A visitor requesting published documents belonging to different owners can occupy multiple owner allowances, but still cannot exceed the global bound; anonymous identity fairness is not established by this policy. Existing public rate limiting remains; time-window owner/IP policy ordering is the separate B04 contract. No claim that B03 implements a per-minute quota.

## Lifecycle and failure safety

Admission happens **before** pruning, temporary-file creation or process launch. A structured lease spans the entire exporter, including file reads and cleanup; caller code throwing after return cannot retain the lease. The existing headless CLI arguments are preserved, with Chromium managing its headless profile lifecycle. Application-created input directories are removed after completion; successful PDFs retain the existing retention behavior.

Stdout and stderr drain concurrently to `Stream.Null`, avoiding pipe deadlocks and unbounded diagnostic strings containing CV data/URLs. Cancellation, shutdown and timeout kill the process tree and await root exit before release. Natural exit racing the kill is handled. Nonzero exit, missing output, failed launch, cancellation and timeout remove partial PDF files and release capacity. Cleanup failures are logged without document contents. Filesystem cleanup remains best effort if the OS denies deletion.

If the OS cannot terminate/confirm the process, the implementation fails closed: no replacement renders start, a Critical event identifies the disabled admission state, and an operator must inspect/stop remaining processes before restarting. This deliberately differs from silently losing one slot or admitting replacement work alongside an unknown process. Ordinary tested failures/cancellations/timeouts do not trigger this exceptional state. R04 remains responsible for hard process-tree/container containment, including abrupt API/host death.

Both frontend PDF download controls now translate 503/504 into an understandable EN/Bokmål message, including Axios blob responses. They make no automatic retry. Other API errors retain existing handling, and B02's interceptors/session lifecycle were not modified.

## Regression and adversarial evidence

- **22 new backend cases** in [PdfRenderAdmissionTests](../../../../JobTrackerApi.Tests/PdfRenderAdmissionTests.cs): actual exporter/process boundary; default bound; two slots and two exporter instances; same-owner rejection with a spare slot; global rejection across owners; cancelled admission with no delayed launch; repeated failures, no output, active cancellation and timeout; all slots reusable after three failure cycles; successful subsequent PDF; failed process launch; large output-pipe drainage; shutdown; invalid host configuration; fail-closed termination state; repeated logging failures cannot strand a slot; safe API error mapping; profile-controller path after inert AI preparation. Existing retention regression remains.
- [PdfBrowserStub](../../../../JobTrackerApi.Tests/PdfBrowserStub/Program.cs) is a tiny test-only executable, built/copied with the test project. It launches no browser, executes no HTML, uses no network, and bounds its own wait to 30 seconds if the runner dies. Tests use fresh temporary roots and stop only their own fixture processes.
- **2 frontend cases** prove busy/timeout messages from blob failures and no retry. B02's original 15 session-boundary tests and the surrounding account-switch suites still pass.
- [Permanent HTTP adversarial test](../../../../job-tracker-ui/pdf-tests/admission.spec.ts), run with [playwright.pdf.config.ts](../../../../job-tracker-ui/playwright.pdf.config.ts): real local API, Identity login/session cookies, B02 context, controllers, owner checks, database and exporter; only Chromium is replaced with the inert executable.

The HTTP burst used three accounts, two sessions for A, multiple variants, and variant/job/public PDF routes. With limit **2**, **2 renders were admitted**, **10 attempts were rejected**, **0 queued**, and sampled/held actual child concurrency was **exactly 2**. Every admission rejection returned **503**, code `pdf_capacity_exceeded` and Retry-After **3**. A missing B02 context still returned **428** and a foreign variant **404** before rendering. After both processes exited, a job PDF returned **200**. The separately tested profile endpoint also could not bypass capacity. [Compact observations](evidence/b03-observations.json).

The first HTTP test run failed because the test runner and worker created different control directories. Corrected the fixture to inherit one directory; no application change was needed for that failure. Original reproduction and failed fixture output remain local; final passing output is recorded separately.

Real Chromium testing also caught an implementation regression that the inert fixture could not reveal: adding an explicit `--user-data-dir` made Chromium 151's CLI hang without producing a PDF. A one-paragraph local-file control timed out with the new argument; the original headless invocation generated a 43,578-byte PDF and exited with code 0 in 2,376 ms. The unnecessary argument was removed, preserving the existing browser invocation. No renderer replacement or sandbox relaxation was needed. The initial browser failure and diagnostic runs are retained separately from final verification; they are not counted as passing checks.

Final browser verification passed **all 17 tests**, with real public/Code PDFs taking approximately **1.3/1.4 seconds** in the exporter. Earlier development-server runs also encountered navigation timeouts and axe contrast failures on transient API-status chips; final production-static browser coverage passed without accessibility changes. The first static run passed 15/17: an account-switch test raced B02's intermediate `/login` navigation against its final `/login?sessionChanged=1` redirect, and a separate Code-PDF setup failed before rendering with SQLite Error 5 (`database is locked`) in `CareerProfileController.Put`'s existing profile/Identity projection. The account-switch test now waits for the final URL and visible login input before starting a new navigation; B02 application behavior is unchanged. The SQLite failure did not repeat and was not altered under B03; its projection/concurrency contract remains D02. The subsequent full run passed 17/17 with zero retries/flaky outcomes. This rerun does not prove the existing SQLite race is resolved.

## Verification results and exact commands

Final results: focused PDF/retention **23 passed**; affected backend **208 passed** before the last logging-fault regression was added; final broader backend **875 passed**, excluding the one pre-existing external-DNS IMAP test; focused frontend **37 passed / 5 suites**; full frontend **296 passed / 66 suites**; authenticated HTTP burst **1 passed**; full browser suite **17 passed**. Backend Release warnings-as-errors and Next production build pass. Targeted lint and `git diff --check` pass. Results and raw-output hashes are recorded in the [manifest](evidence/b03-test-results.json). Counts overlap; do not add focused runs to full-suite totals. The last logging-fault refinement is covered by the final full backend, focused and authenticated HTTP runs; frontend/browser behavior did not change after the passing frontend/browser suites.

Root, PowerShell (existing local packages; no network restore required for the dependency-free stub):

```powershell
New-Item -ItemType Directory -Force .artifacts/b03/empty-nuget | Out-Null
& C:/Users/Connor/.dotnet/dotnet.exe restore JobTrackerApi.Tests/PdfBrowserStub/PdfBrowserStub.csproj --source .artifacts/b03/empty-nuget
& C:/Users/Connor/.dotnet/dotnet.exe test JobTrackerApi.Tests/JobTrackerApi.Tests.csproj --configuration Release --no-restore --no-build --filter 'FullyQualifiedName~PdfRenderAdmissionTests|FullyQualifiedName~CvExportRetentionTests' --logger 'trx;LogFileName=focused-verified.trx' --results-directory .artifacts/b03 -- xUnit.parallelizeTestCollections=false xUnit.maxParallelThreads=1
& C:/Users/Connor/.dotnet/dotnet.exe test JobTrackerApi.Tests/JobTrackerApi.Tests.csproj --configuration Release --no-restore --filter 'FullyQualifiedName~PdfRenderAdmissionTests|FullyQualifiedName~CvExportRetentionTests|FullyQualifiedName~CvBuilderTests|FullyQualifiedName~ProfileCvControllerTests|FullyQualifiedName~PublicCvControllerTests|FullyQualifiedName~JobApplicationsApplicationPackageTests|FullyQualifiedName~SessionContextMiddlewareTests|FullyQualifiedName~JobImport' --logger 'trx;LogFileName=affected.trx' --results-directory .artifacts/b03 -- xUnit.parallelizeTestCollections=false xUnit.maxParallelThreads=1
& C:/Users/Connor/.dotnet/dotnet.exe test JobTrackerApi.Tests/JobTrackerApi.Tests.csproj --configuration Release --no-restore --filter 'FullyQualifiedName!~ImapServiceSsrfGuardTests.ConnectAsync_rejects_unresolvable_host_without_leaking_dns_detail' --logger 'trx;LogFileName=backend-verified.trx' --results-directory .artifacts/b03 -- xUnit.parallelizeTestCollections=false xUnit.maxParallelThreads=1
& C:/Users/Connor/.dotnet/dotnet.exe build JobTrackerApi.Tests/JobTrackerApi.Tests.csproj --configuration Release --no-restore -warnaserror
Copy-Item docs/audits/2026-09-07/validation/evidence/b02-browser-config.ts.txt .artifacts/b03/playwright.offline.config.ts
git diff --check
```

UI directory, PowerShell:

```powershell
node node_modules/jest/bin/jest.js src/cv-builder-page.test.tsx src/cv-builder-deep-link.test.tsx src/public-cv-page.test.tsx src/session-boundary.test.ts src/account-scoped-state.test.tsx --runInBand --watchAll=false --json --outputFile=../.artifacts/b03/frontend-focused.json
node node_modules/jest/bin/jest.js --runInBand --watchAll=false --json --outputFile=../.artifacts/b03/frontend.json
node node_modules/typescript/bin/tsc --noEmit --pretty false
node node_modules/eslint/bin/eslint.js playwright.pdf.config.ts pdf-tests/admission.spec.ts src/views/CvBuilderPage.tsx src/views/CvBuilderEditor.tsx src/cv-builder-page.test.tsx src/i18n/translations.ts e2e/account-isolation.spec.ts --max-warnings=0
$env:NEXT_TELEMETRY_DISABLED='1'
$env:NEXT_PUBLIC_API_BASE_URL='/api'
node --max-old-space-size=4096 node_modules/next/dist/bin/next build
$env:DOTNET_HOST_PATH='C:/Users/Connor/.dotnet/dotnet.exe'
node node_modules/@playwright/test/cli.js test --config=playwright.pdf.config.ts --reporter=json > ../.artifacts/b03/adversarial-verified.json
$env:PLAYWRIGHT_STATIC_EXPORT='true'
node node_modules/@playwright/test/cli.js test --config=../.artifacts/b03/playwright.offline.config.ts --reporter=json > ../.artifacts/b03/browser-verified.json
```

The passing browser rerun additionally set `PLAYWRIGHT_PREBUILT_EXPORT=true` to reuse the immediately preceding static build for `http://localhost:5302/api`. The copied local config also redirected backend logs to `.artifacts/b03/real-server.log` for lifecycle diagnostics. These are test artifacts, not deployment artifacts. No production credentials or services were used. The excluded IMAP test performs real external DNS, which was not authorized; all selected backend tests ran without skips.

**Existing unrelated failure:** standalone TypeScript exits **2** with only `src/interview-prep.test.tsx(120,10): error TS2571: Object is of type 'unknown'.` This is the previously validated R01a diagnostic. Neither Jest nor the successful Next build closes that separate gate. No type suppression or unrelated fix was added. Lint emits only the existing Browserslist stale-data advisory; no dependency upgrade was performed.

## Observability, limitations and rollout

Existing logging is sufficient for this change: Debug admission events record active count/limit/rejection/disabled state without owner IDs; Information completion events record duration/outcome; failures record exception type; termination uncertainty emits Critical. Waiting is always zero. No metrics platform was introduced. Production dashboards/alerts for active/rejected/timeout trends and especially disabled admission should follow the existing operations work. No CV contents, raw browser stdout/stderr, or new sensitive labels are logged by the added events.

**This limit is process-local; if Jobjakt runs N rendering instances, effective global concurrency becomes N × configured limit.** Current Compose describes one API container, not proof of the running production topology. Additional API replicas, another rendering worker or a second DI service root sharing a host require re-budgeting total capacity or a deliberately shared rendering boundary before rollout. Do not assume the per-account limit is distributed either.

The gate bounds Chromium execution and the export lifecycle, not preceding authentication/DB/HTML/AI work or arbitrary single-document CPU/memory consumption. No new page count, input/output byte quota or artifact retention policy was imposed. Those require separate product/storage/containment work; browser sandbox flags, container root execution and hard resource limits remain R04. Application temporary inputs and partial outputs are cleaned; Chromium-managed profile/cache cleanup after a forced kill still needs platform/container evidence under R04. The verified real-browser smoke establishes compatibility, not safe production throughput, OOM immunity or a PID ceiling. Public links can occupy their document owner's allowance; admission is not an anti-abuse identity or time-window quota.

No application migration or data rollback is required. Deploy with the conservative default; expose the documented configuration keys only with measured headroom. Rolling back the admission change reopens concurrent PDF exhaustion, so disable PDF entry points or retain equivalent bounds if rollback is required. Do not remove B01/B02 protections for compatibility. Historical `BoundaryProbes.cs` is tied to the original baseline constructor; the permanent tests are the current implementation's executable regression gate.

## Files changed and next ticket

Application: `Services/PdfRenderAdmission.cs` and `Services/PdfRenderExceptionHandler.cs` (new); `Services/PlaywrightCvPdfExporter.cs`; `Program.cs` (options/singleton/exception-handler registration only); `appsettings.json`. Tests: `JobTrackerApi.Tests/PdfRenderAdmissionTests.cs` and `PdfBrowserStub/{PdfBrowserStub.csproj,Program.cs}` (new), test project fixture build/copy and `CvExportRetentionTests.cs` constructor injection. UI: PDF-error handling in `CvBuilderPage.tsx` and `CvBuilderEditor.tsx`, EN/NB translations and dashboard tests; `playwright.pdf.config.ts`, `pdf-tests/admission.spec.ts`, `package.json` test command; `e2e/account-isolation.spec.ts` final-redirect wait only. Documentation: this report, evidence/manifest, validated README/queue/evidence and TODO. Pre-existing uncommitted B01/B02 and audit work remains present; the complete working-tree diff is not exclusively B03.

**Acceptance: all ten B03 criteria pass locally.** All four real generation paths enter one singleton pool before expensive work; configured process/owner bounds hold across endpoints, sessions and accounts. Immediate refusal eliminates waiting/backlog cancellation problems. Permanent process tests prove ordinary exceptions, cancellation and timeout release after exit, with repeated recovery; unconfirmed termination disables replacement work. Invalid configuration fails host startup. Final broader regression/build results are healthy subject to the explicitly pre-existing TypeScript gate and observed SQLite fixture race. Process-local scaling and deployment containment limits are documented above. No schema or architectural stop condition was encountered, and no later ticket was implemented.

**Next recommended ticket: B04 — resolve principal before owner limiting. Not started.** B03 admission completion alone does not close F03's separate R04 deployment/containment evidence or approve public launch.