ci(security): add secret scan and SBOM
This commit is contained in:
@@ -545,6 +545,8 @@ SEC-008 implements the same durable state machine with `<final>.uploading` and `
|
||||
|
||||
### P2-4 — Harden build provenance and secret scanning
|
||||
|
||||
**Status (2026-08-30): partially complete.** Exact .NET SDK selection, NuGet locks, hash-locked Python dependencies, npm lock enforcement, advisory gates, a no-value tracked-secret scanner with synthetic canaries, and deterministic multi-ecosystem CycloneDX generation now run in CI. Immutable action/image/installer identifiers, retained SBOM artifacts, container/licence scanning and the unresolved model-stack advisories still require verified upstream metadata or unavailable scanner/Docker runtime. See `docs/verification/jt-017-dotnet-provenance.md`, `docs/security/supply-chain-policy.md` and V-192/V-210.
|
||||
|
||||
- **Findings/scope:** JT-017/JT-020; action/image/installer pinning, SDK/locks/hashes, SBOM, container/secret scan, archive fixtures.
|
||||
- **Dependencies:** approved update cadence and scanner availability.
|
||||
- **Acceptance criteria:** immutable CI dependencies; reproducible documented toolchain; scans block policy-defined severity; no live credential patterns.
|
||||
|
||||
@@ -241,3 +241,4 @@ Output was reduced to filenames and commit counts. The token artifact appears un
|
||||
| V-207 | Ownership/migration chain; populated aggregate adoption/downgrade/retry; JSON/text/FK/index assertions; MariaDB script; full backend | Repository root / disposable local SQLite | Move the canonical Career Profile, append-only history, and six relational child types to migration ownership without losing career data | PASS — focused ownership/migration 19/19; canonical and long-tail JSON, Norwegian text, version history, experience, education, skill, project, certification and language rows survive adoption, downgrade and re-upgrade; all 14 aggregate indexes exist; deleting the profile cascades through history and children; generated MariaDB SQL uses bounded indexed keys and provider-safe types; startup creates none of the eight tables; full backend 735/735 | MariaDB SQL generated only; no production migration. Guarded historical LongTailJson/index/auto-increment repairs remain temporarily. Seven reconciler-owned Identity tables remain | JT-019 feature and career transfers complete; isolate the Identity ownership boundary next |
|
||||
| V-208 | Ownership/migration chain; populated Identity adoption/downgrade/retry; credential/preference/FK/index assertions; MariaDB script; full backend; fresh application startup | Repository root / disposable local SQLite | Complete JT-019 by moving the seven ASP.NET Identity tables to migration ownership without invalidating authentication data | PASS — focused ownership/migration 20/20; password hash, Bokmål preference, 2FA state, role assignment, role/user claims, external login and token survive adoption, downgrade and re-upgrade; all eight Identity indexes exist; user and role deletion cascades remain effective; generated MariaDB SQL is provider-safe; startup creates no model tables; full backend 736/736; a fresh application applies the complete chain, reaches Identity role initialization and listens normally | MariaDB SQL generated only; no production migration. The historical AspNetUsers compatibility bootstrap and guarded column/index repairs remain for chain traversal and legacy upgrades. Launch settings supplied an existing policy-invalid development admin password, producing a non-fatal seed warning | JT-019 complete; future work may retire individual repair statements only after provider-backed historical fixtures prove them redundant |
|
||||
| V-209 | Google Identity initialization regression; focused login Jest; ESLint; TypeScript; complete Playwright | `job-tracker-ui` / disposable local application | Remove repeated global Google SDK initialization without changing sign-in or account-link behavior | PASS — SDK initialization is stable per loaded Identity API/client ID while the mounted card owns the current credential handler; login 13/13, lint and TypeScript pass; Playwright 10/10 covers authentication, job creation/workspace, Career Workspace and searchable PDF exports with no repeated-initialize warning | Synthetic Google callback only; no real provider credential, account link or production call | Repeatable browser-console defect closed; real-provider verification remains externally gated |
|
||||
| V-210 | Standard-library supply-chain tests; tracked-tree credential scan; deterministic multi-lock CycloneDX generation; CI/policy review | Repository root | Add locally provable JT-017 SBOM and secret gates without trusting new mutable tooling | PASS/PARTIAL — 3/3 tests detect seven synthetic credential classes without returning values, ignore binary/oversized inputs and prove deterministic npm/NuGet/PyPI coverage; the tracked tree is clean; generated CycloneDX 1.5 contains 964 unique locked components; CI now blocks findings and generates the SBOM | Scanner is intentionally high-confidence/current-tree only. SBOM is not yet retained by CI. Docker daemon/scanner and verified upstream action/image/installer identifiers are unavailable; Torch/Transformers findings remain unaccepted | Local secret/SBOM controls complete; immutable input, container/licence and model-stack gates remain JT-017 |
|
||||
|
||||
@@ -0,0 +1,40 @@
|
||||
# Supply-chain and credential scanning policy
|
||||
|
||||
## Required repository gates
|
||||
|
||||
Pull requests and `main` builds must pass all of the following without silently excluding a project:
|
||||
|
||||
- locked NuGet restore for the API and test project;
|
||||
- `npm ci` plus the configured high-severity audit gate;
|
||||
- hash-required installation of the Linux Python lock;
|
||||
- the package-manager advisory checks recorded in CI;
|
||||
- `python3 scripts/supply-chain.py scan-secrets` over every Git-tracked file; and
|
||||
- `python3 scripts/supply-chain.py generate-sbom --output .artifacts/jobjakt.cdx.json`.
|
||||
|
||||
The generated CycloneDX 1.5 SBOM combines the committed npm, NuGet and Python lock files. It is a build artifact, not a replacement for the locks or an image scan. Its lock digest and deterministic component ordering make dependency changes reviewable without adding generated files to source control.
|
||||
|
||||
The built-in tracked-secret scanner is deliberately high confidence. It detects private-key headers and recognizable JWT, AWS, GitHub, Google, Slack and live Stripe credentials, skips binary files and files over 2 MiB, and reports only the credential class and file location. It never prints a matched value. This gate complements—not replaces—host secret management, history review and a dedicated scanner when one is approved.
|
||||
|
||||
## Dependency and advisory exceptions
|
||||
|
||||
High or critical reachable advisories block release. An exception must be a reviewed repository document containing the package and affected path, advisory identifiers, reachability evidence, compensating control, accountable owner, expiry date and removal condition. Expired or ownerless exceptions fail closed. A version pin alone is not an exception.
|
||||
|
||||
The outstanding Torch/Transformers findings are not accepted exceptions. Their model-loading path and production image remain blocked pending a compatible resolution or a bounded, reviewed isolation decision.
|
||||
|
||||
## Immutable inputs
|
||||
|
||||
Third-party workflow actions, container bases and downloaded installers must move only to identifiers verified from their authoritative upstream source. Do not guess a commit SHA, image digest or installer checksum. Updates are reviewed commits and retain the prior known-good identifier for rollback.
|
||||
|
||||
The current mutable action/image references and `dotnet-install.sh` download remain open until network-backed upstream metadata and a container scanner are available. The local SBOM and tracked-secret gates do not claim to close those items.
|
||||
|
||||
## Credential response
|
||||
|
||||
If the tracked scan or a reviewer finds a real credential:
|
||||
|
||||
1. stop the affected build and suppress the value from chat, tickets and logs;
|
||||
2. revoke or rotate it at the issuing system before treating source cleanup as sufficient;
|
||||
3. remove it from the current tree and add the narrow prevention rule or ignore boundary needed;
|
||||
4. assess reachable history and CI artifacts using filenames/commit IDs only; and
|
||||
5. coordinate any history rewrite separately because it disrupts clones and does not revoke a credential.
|
||||
|
||||
Never weaken a detector merely to make an unexplained finding pass.
|
||||
@@ -1,4 +1,4 @@
|
||||
# JT-017 .NET provenance verification
|
||||
# JT-017 supply-chain provenance verification
|
||||
|
||||
## Implemented
|
||||
|
||||
@@ -7,6 +7,10 @@
|
||||
- API and test projects opt into NuGet lock files.
|
||||
- Both direct and transitive package versions carry NuGet content hashes in committed `packages.lock.json` files.
|
||||
- CI restores both projects with `--locked-mode`, so unreviewed graph drift fails before build or test.
|
||||
- A standard-library tracked-file scanner blocks high-confidence credential patterns without printing values.
|
||||
- A deterministic CycloneDX 1.5 generator combines the npm, NuGet and hash-locked Python dependency graphs.
|
||||
- CI tests both controls, scans the tracked tree and generates a 964-component SBOM under ignored build artifacts.
|
||||
- The review/exception/credential-response rules are documented in `docs/security/supply-chain-policy.md`.
|
||||
|
||||
## Proof
|
||||
|
||||
@@ -14,12 +18,16 @@
|
||||
- A subsequent locked restore passed.
|
||||
- Full backend suite: 719/719 passed.
|
||||
- `dotnet list JobTrackerApi/JobTrackerApi.csproj package --vulnerable --include-transitive` reports no known vulnerable packages from the current NuGet source.
|
||||
- Supply-chain unit tests: 3/3 passed, including seven synthetic credential canaries and deterministic npm/NuGet/PyPI coverage.
|
||||
- The current tracked tree has no high-confidence credential-pattern findings.
|
||||
- Repeated SBOM generation is byte-stable for unchanged lock inputs and includes React, Entity Framework Core and FastAPI representatives.
|
||||
|
||||
## Remaining JT-017 scope
|
||||
|
||||
- Pin third-party CI actions and Docker images by reviewed immutable SHA/digest.
|
||||
- Hash or replace the downloaded `dotnet-install.sh` bootstrap path.
|
||||
- Add policy-owned SBOM, container and tracked-secret gates.
|
||||
- Retain the SBOM as a CI artifact once an immutable artifact action or approved runner store is available.
|
||||
- Add a policy-owned container CVE/licence scan with a tested canary once the Docker daemon/scanner is available.
|
||||
- Resolve or formally isolate the remaining Torch/Transformers advisories under the model-loading threat path.
|
||||
|
||||
Those items require approved registry/action metadata or a separately reviewed scanner/update process; this increment does not invent immutable identifiers without verifying them.
|
||||
Those items require approved registry/action metadata, network-backed upstream verification or a separately reviewed scanner/update process; this increment does not invent immutable identifiers without verifying them.
|
||||
|
||||
@@ -42,6 +42,7 @@ Updated: 2026-08-30
|
||||
- Localized the remaining active helper defaults in the admin SMTP test form, pasted-email importer, and image-crop alternative text.
|
||||
- Restored the local Python 3.12 toolchain and both virtual environments; the AI sidecar now passes 26/26 tests. Added a Next.js-compatible ESLint 9 flat configuration, strict zero-warning scripts, and a patched CommonJS-compatible `brace-expansion` override. The lint gate passes with zero findings and npm audit reports zero vulnerabilities.
|
||||
- Pinned the repository to .NET SDK 9.0.317, generated content-hashed transitive NuGet locks for the API and test project, and made CI restores fail on lock drift. Locked restore and backend 719/719 pass; the current NuGet graph has no known vulnerable packages.
|
||||
- Added standard-library JT-017 supply-chain gates: seven-class no-value tracked-secret scanning, deterministic CycloneDX generation across npm/NuGet/Python locks, synthetic canary tests, CI integration and an explicit advisory/credential-response policy. The current tree is clean and the SBOM contains 964 unique locked components.
|
||||
- Rebuilt the active developer/operator documentation around the actual Next.js 16/.NET 9 application, replaced CRA and `npm start` guidance, separated normal and Playwright ports, corrected React Router 7 and the SQLite/MariaDB provider matrix, removed the obsolete npm peer override, and verified the documented clean install, lint, test, build and locked-restore commands.
|
||||
- Began the JT-019 schema-ownership retirement with an executable 49-table ownership partition and transferred the leaf `SystemEmailSettings` table from MariaDB-only startup DDL to an additive provider-aware migration. Fresh SQLite now receives the table; legacy rows are preserved and startup no longer creates it.
|
||||
- Transferred the independent `UserRuleSettings` table from both provider startup paths to its own provider-aware migration; owner-keyed settings survive adoption, downgrade and retry.
|
||||
@@ -127,7 +128,7 @@ Updated: 2026-08-30
|
||||
- Manual desktop browser review: webpack development server rendered the new Career navigation and Overview correctly in dark mode; API-dependent profile status remained unavailable because the backend was not running for that isolated UI review.
|
||||
|
||||
- **Overall programme status:** Active but externally blocked. Eight packages are locally verified and twenty-seven are implemented with verification incomplete. The prioritized admin-only version indicator, every immediate repository/browser item, SEC-006/SEC-007 repository boundaries, SEC-009, the PROD-001 read-only inventory, and the PROD-003 safe benchmark harness are complete on the feature branch.
|
||||
- **Current work package:** JT-017 remaining immutable CI/image/scanner provenance and JT-019 schema-ownership inventory while external runtime gates remain blocked. JT-018 is complete in repository scope.
|
||||
- **Current work package:** JT-017 remaining immutable CI action/image/installer provenance, retained SBOM, container/licence scan and model-stack advisory resolution. JT-018 and JT-019 are complete in repository scope; the remaining JT-017 gates require verified upstream metadata or the unavailable Docker/scanner runtime.
|
||||
- **Completed work packages:** None are `DONE`; all repository security/AI packages still have applicable browser, provider and/or production gates.
|
||||
- **Locally verified work:** SEC-001, SEC-002, SEC-003, SEC-005A, CORE-001, PROD-002, DEP-001 and VER-001 (`VERIFIED LOCALLY`).
|
||||
- **Implemented, verification incomplete:** SEC-004, SEC-005B, SEC-006, SEC-007, SEC-008, SEC-009, CORE-002, BG-001, OPS-001A/B/C, POL-001/002, AI-001/002/003/004, UX-001/002/003, QA-001, CAREER-001/002, MAIL-001, JOBS-001/002 and PRODUCT-001 (`IMPLEMENTED — NOT VERIFIED`). Their safe repository/browser scope is implemented; production/native-device/provider/retention gates remain where recorded.
|
||||
@@ -136,7 +137,7 @@ Updated: 2026-08-30
|
||||
- **Deferred work:** None. Conditional multi-replica coordination, model deletion, realtime operation delivery and unrelated production changes remain outside current packages.
|
||||
- **Immediate order:** all sixteen immediate repository items are complete locally, including the original UI/release queue plus SEC-009 cache/tombstone safety, worker restart clocks, universal AI accounting, email-token/Stripe lifecycle tests, exhaustive Job email selectors, the repaired migration chain, CV/public-edge hardening and measured admin/mail scaling. PROD-001 read-only evidence and the PROD-003 plan-only harness are also complete. The final audit is checking tooling/documentation before declaring only external blockers remain.
|
||||
- **Status counts:** 8 `VERIFIED LOCALLY`; 27 `IMPLEMENTED — NOT VERIFIED`; 0 `IN PROGRESS`; 0 `NOT STARTED`; 4 `BLOCKED`; 0 `DONE`; 0 `DEFERRED`.
|
||||
- **Test status:** backend 723/723; frontend 64/64 suites and 272/272 tests; ESLint zero findings; AI sidecar 37/37 without warnings; Ollama benchmark harness 5/5 plus safe dry-run; optimized production build/TypeScript; EF model parity; SQLite/MariaDB migration scripts; blank/idempotent/populated SQLite migration-chain tests; fresh SQLite application startup through the SystemEmailSettings ownership migration; disposable fresh/restarted MariaDB 11.8 application startup; Docker Compose config; safe-failure deployment preflight; and Playwright 9/9. npm audit remains at zero. The new ownership migrations have generated MariaDB SQL but still need a MariaDB runtime rehearsal; the parser image build is pending because Docker Desktop's daemon is offline. Jest's slow/open-handle behavior remains recorded.
|
||||
- **Test status:** backend 736/736; frontend 64/64 suites and 272/272 tests; ESLint and TypeScript zero findings; AI sidecar 37/37 without warnings; Ollama benchmark harness 5/5 plus safe dry-run; optimized production build; EF model parity; SQLite/MariaDB migration scripts; blank/idempotent/populated SQLite migration-chain tests; fresh complete-chain SQLite application startup; disposable earlier-chain MariaDB 11.8 application startup; Docker Compose config; safe-failure deployment preflight; and Playwright 10/10. Supply-chain controls pass 3/3, the tracked tree is clean, and the generated CycloneDX SBOM contains 964 components. npm audit remains at zero. The final ownership migrations have generated MariaDB SQL but still need a current MariaDB runtime rehearsal; the parser image and container scan are pending because Docker Desktop's daemon is offline. Jest's slow/open-handle behavior remains recorded.
|
||||
- **Deployment status:** Gitea pull-request run 609 passed the complete CI job in 4m20s. Deploy was intentionally skipped because the workflow deploys only a `push` to `main`; live remains unchanged. No merge/deployment was performed directly, no production migrations were run and the AI operation worker remains disabled by default.
|
||||
- **Production status:** State unchanged. Sanitized read-only SSH inventory was performed; no logs, prompts, private rows/content or secret values were read, and no provider/model call, model pull, service restart, file/config change, backup, restore, migration or deployment occurred. It confirmed all-interface Ollama/frontend listeners and stale database-only backups as rollout blockers.
|
||||
- **Known regressions:** None found by automated/local browser checks. Jest still needs `--forceExit` and reports its existing open-handle notice. Email-provider/send tests are fake/local only; real delivery is not claimed. Current MAIL browser evidence is 1280×720 only because the browser surface could not resize or perform native Tab traversal. Interrupted attempts are aged after 15 minutes and notified without retry; the five-minute scan is unmeasured on a large ledger. The direct clean EF-only SQLite defect and synchronous AI accounting gap are closed; migration/reconciler dual ownership remains architectural debt.
|
||||
|
||||
Reference in New Issue
Block a user