0954876295
Implement B01-B06 with regression coverage and retained validation evidence. Preserve the production ingress preflight; F04 runtime verification remains open.
85 lines
11 KiB
Markdown
85 lines
11 KiB
Markdown
# B01 — HTTP import destination enforcement
|
|
|
|
**Completed locally on 2026-09-08.** B01's acceptance criteria pass. The HTTP portion of F11 and the mapped-address bypass V02 are resolved in this working tree. **F11 remains open for B07 (IMAP) and deployment egress assumptions.** B02/B07 were not started. No deployment, migration, dependency upgrade or unrelated application change was made.
|
|
|
|
Baseline application commit: `7d641edc7bca866ccf06012897ba1d107c07e59f`. The original [boundary observations](evidence/boundary-results.json), [probe source](BoundaryProbes.cs) and audit conclusions remain as historical evidence. This document records the subsequent authorized implementation.
|
|
|
|
## Files changed
|
|
|
|
- Application: `JobTrackerApi/Program.cs`, `JobTrackerApi/Services/JobImport/JobImportService.cs`, new `JobTrackerApi/Services/JobImport/JobImportHttpTransport.cs`.
|
|
- Tests: `JobTrackerApi.Tests/JobImportServiceTests.cs`, new `JobTrackerApi.Tests/JobImportHttpTransportTests.cs`.
|
|
- Status/evidence: `docs/todo/work.md`; this validation directory's `README.md`, `implementation-queue.md`, `evidence.md`, new `b01-completion.md` and new `evidence/b01-test-results.json`.
|
|
|
|
Original audit documents and TODO changes were already present before B01. They were preserved; the list above identifies this ticket's changes only.
|
|
|
|
## Root cause and change
|
|
|
|
The import URL validator treated IPv4-mapped IPv6 as ordinary IPv6. Thus `10.1.2.3` was blocked while `::ffff:10.1.2.3` passed. Separately, hostname preflight and the original HTTP handler performed different resolutions; passing preflight did not constrain the socket destination.
|
|
|
|
[JobImportHttpTransport](../../../../JobTrackerApi/Services/JobImport/JobImportHttpTransport.cs) contains the shared address classification and a narrow `SocketsHttpHandler.ConnectCallback`. Both [preflight](../../../../JobTrackerApi/Services/JobImport/JobImportService.cs) and connection setup use that classification. The callback resolves once for that connection, rejects empty/mixed/denied answers before dialing, normalizes mapped IPv4, and passes an **IPEndPoint**, never a hostname, to `Socket.ConnectAsync`. Public-address fallback uses only the already checked answer. A new connection repeats validation; an existing pooled TCP connection remains bound to its approved peer.
|
|
|
|
[Program.cs](../../../../JobTrackerApi/Program.cs) installs this handler only for the existing `jobimport` client. It returns a plain TCP stream; .NET still performs HTTPS, SNI and certificate/hostname verification using the original request URI. There is no permissive certificate callback, hostname rewrite or custom HTTP implementation.
|
|
|
|
Redirects remain disabled and all 3xx responses remain import failures. Automatic decompression, request headers, parser behavior, the streaming 4,000,000-byte limit and cancellation remain intact. The original client's 100-second timeout is unchanged.
|
|
|
|
The address policy retains the existing loopback, private, link-local, carrier-grade NAT, benchmark, IPv6 site-local/unique-local/multicast/Teredo exclusions. It also rejects IPv4 multicast/reserved high addresses and scoped IPv6. IPv4-compatible IPv6, well-known NAT64 (`64:ff9b::/96`, `64:ff9b:1::/48`) and 6to4 are rejected explicitly so translation forms do not hide a prohibited IPv4 destination. Ordinary public IPv4, native public IPv6 and public mapped IPv4 remain supported. This is a destination policy, not a complete global IP allocation registry.
|
|
|
|
**Proxy behavior:** `UseProxy = false` is deliberate for this client. Otherwise a proxy can resolve/connect the origin outside this callback. No other HttpClient or deployment configuration changed. Imports now require direct outbound connectivity; a proxy-only environment must not restore automatic proxy use without an equivalent enforced origin policy. Such an environment was not evidenced in this repository and was not contacted.
|
|
|
|
## Permanent regression evidence
|
|
|
|
[JobImportServiceTests](../../../../JobTrackerApi.Tests/JobImportServiceTests.cs) adds 12 literal/encoded-address cases. Running these before changing application code produced **4 failures / 8 passes**: mapped private, link-local, metadata and hexadecimal-mapped forms reached the fake HTTP send. The same tests now all pass.
|
|
|
|
[JobImportHttpTransportTests](../../../../JobTrackerApi.Tests/JobImportHttpTransportTests.cs) adds 76 cases at the actual .NET HTTP handler/socket boundary:
|
|
|
|
| Requirement | Evidence |
|
|
|---|---|
|
|
| Private/local IPv4 and IPv6 | 38 DNS-answer cases cover loopback, RFC1918 boundaries, link-local, ULA/site-local, unspecified, CGNAT, benchmark, multicast/reserved and translation/scoped forms; no dial attempts. |
|
|
| Equivalent literal forms | Handler tests reject decimal, hexadecimal, octal, mapped and native IPv6 literals without DNS or dialing; service tests also cover shortened IPv4. |
|
|
| Real default dial path | A denied DNS answer cannot reach an inert loopback listener using the production handler without an injected dialer. |
|
|
| Resolution failures/mixed answers | Empty, public+private and failed DNS answers fail closed; no fallback to the OS hostname connector. |
|
|
| Rebinding window | Preflight returns public, connection resolution returns private/mapped/ULA: no dial. A third hypothetical private answer is never queried after an approved connection-time answer. |
|
|
| Public imports | Real HTTP exchange and JobPosting parsing for hostname/literal/public IPv4/native IPv6/mapped IPv4 decisions; original Host/path retained. Public-address socket failure falls back only to another checked address. |
|
|
| Connection lifecycle | A second connection on the same handler revalidates DNS and rejects its changed private answer. |
|
|
| Proxy escape | A configured proxy trap receives no connection; the approved direct endpoint is used. |
|
|
| Redirects | 301/302/303/307/308 pointing to mapped metadata return fetch failure, with exactly one approved dial and no target resolution. |
|
|
| TLS identity | Local TLS server observes original-host SNI; trusted matching certificate succeeds; trusted wrong hostname and untrusted matching certificate fail before HTTP data. No certificate-validation bypass. |
|
|
| Timeout/cancellation | Short test-only client timeout cancels stalled connection DNS, dial and response headers. Caller cancellation interrupts a stalled streamed response body. Production 100-second timeout is asserted unchanged. |
|
|
| Size/decompression | Oversized declared content and gzip expansion over the byte cap fail. The existing chunked-stream test still proves reading stops at the cap plus one sentinel byte. |
|
|
|
|
All new tests use fake DNS and controlled loopback TCP/TLS fixtures. An internal optional dial delegate records the approved IP then routes only the fixture transport locally; production uses the concrete IP socket method. Tests exercise real .NET HTTP/TLS over that stream and do not claim to have contacted real public/private services. The test CA is trusted only by its test handler, not installed into OS trust. No public DNS, website, metadata endpoint, Stripe or other external service was contacted.
|
|
|
|
## Commands and exact results
|
|
|
|
Run from repository root in PowerShell, with existing restored packages:
|
|
|
|
```powershell
|
|
# Before implementation: 12 cases, 4 failed / 8 passed / 0 skipped (expected reproduction).
|
|
& C:/Users/Connor/.dotnet/dotnet.exe test JobTrackerApi.Tests/JobTrackerApi.Tests.csproj --configuration Release --no-restore --filter 'FullyQualifiedName~Preview_rejects_equivalent_private_literals_before_http_send' --logger 'trx;LogFileName=b01-before.trx' --results-directory .artifacts/b01 -- xUnit.parallelizeTestCollections=false xUnit.maxParallelThreads=1
|
|
|
|
# Final focused run: 98 passed / 0 failed / 0 skipped (92 import + 6 existing IMAP literal guards).
|
|
& C:/Users/Connor/.dotnet/dotnet.exe test JobTrackerApi.Tests/JobTrackerApi.Tests.csproj --configuration Release --no-restore --filter 'FullyQualifiedName~JobImport|FullyQualifiedName~ImapServiceSsrfGuardTests.ConnectAsync_rejects_internal_and_metadata_hosts' --logger 'trx;LogFileName=b01-focused.trx' --results-directory .artifacts/b01 -- xUnit.parallelizeTestCollections=false xUnit.maxParallelThreads=1
|
|
|
|
# Broader backend: 840 passed / 0 failed / 0 skipped; one existing DNS-dependent case excluded.
|
|
& C:/Users/Connor/.dotnet/dotnet.exe test JobTrackerApi.Tests/JobTrackerApi.Tests.csproj --configuration Release --no-restore --no-build --filter 'FullyQualifiedName!~ImapServiceSsrfGuardTests.ConnectAsync_rejects_unresolvable_host_without_leaking_dns_detail' --logger 'trx;LogFileName=b01-backend.trx' --results-directory .artifacts/b01 -- xUnit.parallelizeTestCollections=false xUnit.maxParallelThreads=1
|
|
|
|
# API and test project compilation/analyzers: passed, 0 warnings / 0 errors.
|
|
& C:/Users/Connor/.dotnet/dotnet.exe build JobTrackerApi.Tests/JobTrackerApi.Tests.csproj --configuration Release --no-restore -warnaserror
|
|
|
|
git diff --check
|
|
```
|
|
|
|
The excluded existing IMAP test performs real DNS resolution, which this task did not authorize. It is an explicit filter exclusion, not a reported test skip or pass. The broader run includes the focused tests, so their counts must not be added. No frontend files changed; frontend TypeScript/build checks were not rerun. Results/hashes are retained in [b01-test-results.json](evidence/b01-test-results.json); full TRX files remain local under `.artifacts/b01/`.
|
|
|
|
## Security self-review and residual limitations
|
|
|
|
- Equivalent mapped representations go through the same IPv4 rule; native IPv6 private/local and known translation forms fail closed. Numeric parsing is delegated to `Uri`/`IPAddress`, with encoded-form regressions.
|
|
- The exact validated IP is passed to the socket. There is no DNS lookup between classification and that connect and no unchecked proxy/redirect path. DNS/socket failures do not switch to a permissive hostname transport. Cancellation or connection failure disposes the socket.
|
|
- TLS remains the framework's original-host validation. The test-only custom CA policy changes trust for its fixture without disabling hostname checks; production does not set it.
|
|
- OS routing, transparent interception, operator-defined NAT64 prefixes and services internally routed under otherwise public IPs are outside an application IP classifier's visibility. Appropriate egress policy remains defense in depth; this change does not certify production routing or every special-use address allocation.
|
|
- As before, `HttpClient.Timeout` with `ResponseHeadersRead` covers sending/receiving headers, not the entire body. The body has its byte cap and caller cancellation, but **no independent total-body deadline**. Preflight DNS likewise uses the supplied request cancellation token. No new whole-import duration guarantee is claimed; resource admission/deadline policy can be a separate bounded follow-up.
|
|
- Public alternatives are attempted sequentially after socket failures, within the existing client timeout. No parallel connection racing or new network subsystem was added. Linux/container and real external endpoint connectivity were not exercised; local tests ran on Windows/.NET 9.
|
|
- Reverting this handler reopens B01. Do not roll it back on an exposed import route without disabling that route or equivalent verified egress enforcement. No database rollback is involved.
|
|
|
|
**Acceptance verdict: B01 complete.** The reproduced bypass now fails before send, connection-time tests close the validated DNS race, legitimate imports and existing protections pass regression checks, and the wider backend/build gates pass. F11 is only partially remediated because B07/deployment evidence remain outside B01. Stop here; do not begin B02 automatically.
|