ci(security): add secret scan and SBOM

This commit is contained in:
cesnimda
2026-08-30 22:22:19 +02:00
parent 9ad812e9ba
commit 8c3822c440
9 changed files with 330 additions and 6 deletions
+8 -1
View File
@@ -58,7 +58,14 @@ jobs:
- name: Test repository safety scripts - name: Test repository safety scripts
# Standard-library only and plan-only: this validates the synthetic benchmark harness # Standard-library only and plan-only: this validates the synthetic benchmark harness
# without contacting Ollama, pulling a model, or requiring package installation. # without contacting Ollama, pulling a model, or requiring package installation.
run: python3 scripts/test-ollama-evaluation.py run: |
python3 scripts/test-ollama-evaluation.py
python3 scripts/test-supply-chain.py
- name: Scan tracked files and generate dependency SBOM
run: |
python3 scripts/supply-chain.py scan-secrets
python3 scripts/supply-chain.py generate-sbom --output .artifacts/jobjakt.cdx.json
- name: Test document parser boundary - name: Test document parser boundary
working-directory: tools/summarizer working-directory: tools/summarizer
+1
View File
@@ -38,6 +38,7 @@ dist/
coverage/ coverage/
playwright-report/ playwright-report/
test-results/ test-results/
.artifacts/
.next/ .next/
.cache/ .cache/
+2
View File
@@ -545,6 +545,8 @@ SEC-008 implements the same durable state machine with `<final>.uploading` and `
### P2-4 — Harden build provenance and secret scanning ### P2-4 — Harden build provenance and secret scanning
**Status (2026-08-30): partially complete.** Exact .NET SDK selection, NuGet locks, hash-locked Python dependencies, npm lock enforcement, advisory gates, a no-value tracked-secret scanner with synthetic canaries, and deterministic multi-ecosystem CycloneDX generation now run in CI. Immutable action/image/installer identifiers, retained SBOM artifacts, container/licence scanning and the unresolved model-stack advisories still require verified upstream metadata or unavailable scanner/Docker runtime. See `docs/verification/jt-017-dotnet-provenance.md`, `docs/security/supply-chain-policy.md` and V-192/V-210.
- **Findings/scope:** JT-017/JT-020; action/image/installer pinning, SDK/locks/hashes, SBOM, container/secret scan, archive fixtures. - **Findings/scope:** JT-017/JT-020; action/image/installer pinning, SDK/locks/hashes, SBOM, container/secret scan, archive fixtures.
- **Dependencies:** approved update cadence and scanner availability. - **Dependencies:** approved update cadence and scanner availability.
- **Acceptance criteria:** immutable CI dependencies; reproducible documented toolchain; scans block policy-defined severity; no live credential patterns. - **Acceptance criteria:** immutable CI dependencies; reproducible documented toolchain; scans block policy-defined severity; no live credential patterns.
+1
View File
@@ -241,3 +241,4 @@ Output was reduced to filenames and commit counts. The token artifact appears un
| V-207 | Ownership/migration chain; populated aggregate adoption/downgrade/retry; JSON/text/FK/index assertions; MariaDB script; full backend | Repository root / disposable local SQLite | Move the canonical Career Profile, append-only history, and six relational child types to migration ownership without losing career data | PASS — focused ownership/migration 19/19; canonical and long-tail JSON, Norwegian text, version history, experience, education, skill, project, certification and language rows survive adoption, downgrade and re-upgrade; all 14 aggregate indexes exist; deleting the profile cascades through history and children; generated MariaDB SQL uses bounded indexed keys and provider-safe types; startup creates none of the eight tables; full backend 735/735 | MariaDB SQL generated only; no production migration. Guarded historical LongTailJson/index/auto-increment repairs remain temporarily. Seven reconciler-owned Identity tables remain | JT-019 feature and career transfers complete; isolate the Identity ownership boundary next | | V-207 | Ownership/migration chain; populated aggregate adoption/downgrade/retry; JSON/text/FK/index assertions; MariaDB script; full backend | Repository root / disposable local SQLite | Move the canonical Career Profile, append-only history, and six relational child types to migration ownership without losing career data | PASS — focused ownership/migration 19/19; canonical and long-tail JSON, Norwegian text, version history, experience, education, skill, project, certification and language rows survive adoption, downgrade and re-upgrade; all 14 aggregate indexes exist; deleting the profile cascades through history and children; generated MariaDB SQL uses bounded indexed keys and provider-safe types; startup creates none of the eight tables; full backend 735/735 | MariaDB SQL generated only; no production migration. Guarded historical LongTailJson/index/auto-increment repairs remain temporarily. Seven reconciler-owned Identity tables remain | JT-019 feature and career transfers complete; isolate the Identity ownership boundary next |
| V-208 | Ownership/migration chain; populated Identity adoption/downgrade/retry; credential/preference/FK/index assertions; MariaDB script; full backend; fresh application startup | Repository root / disposable local SQLite | Complete JT-019 by moving the seven ASP.NET Identity tables to migration ownership without invalidating authentication data | PASS — focused ownership/migration 20/20; password hash, Bokmål preference, 2FA state, role assignment, role/user claims, external login and token survive adoption, downgrade and re-upgrade; all eight Identity indexes exist; user and role deletion cascades remain effective; generated MariaDB SQL is provider-safe; startup creates no model tables; full backend 736/736; a fresh application applies the complete chain, reaches Identity role initialization and listens normally | MariaDB SQL generated only; no production migration. The historical AspNetUsers compatibility bootstrap and guarded column/index repairs remain for chain traversal and legacy upgrades. Launch settings supplied an existing policy-invalid development admin password, producing a non-fatal seed warning | JT-019 complete; future work may retire individual repair statements only after provider-backed historical fixtures prove them redundant | | V-208 | Ownership/migration chain; populated Identity adoption/downgrade/retry; credential/preference/FK/index assertions; MariaDB script; full backend; fresh application startup | Repository root / disposable local SQLite | Complete JT-019 by moving the seven ASP.NET Identity tables to migration ownership without invalidating authentication data | PASS — focused ownership/migration 20/20; password hash, Bokmål preference, 2FA state, role assignment, role/user claims, external login and token survive adoption, downgrade and re-upgrade; all eight Identity indexes exist; user and role deletion cascades remain effective; generated MariaDB SQL is provider-safe; startup creates no model tables; full backend 736/736; a fresh application applies the complete chain, reaches Identity role initialization and listens normally | MariaDB SQL generated only; no production migration. The historical AspNetUsers compatibility bootstrap and guarded column/index repairs remain for chain traversal and legacy upgrades. Launch settings supplied an existing policy-invalid development admin password, producing a non-fatal seed warning | JT-019 complete; future work may retire individual repair statements only after provider-backed historical fixtures prove them redundant |
| V-209 | Google Identity initialization regression; focused login Jest; ESLint; TypeScript; complete Playwright | `job-tracker-ui` / disposable local application | Remove repeated global Google SDK initialization without changing sign-in or account-link behavior | PASS — SDK initialization is stable per loaded Identity API/client ID while the mounted card owns the current credential handler; login 13/13, lint and TypeScript pass; Playwright 10/10 covers authentication, job creation/workspace, Career Workspace and searchable PDF exports with no repeated-initialize warning | Synthetic Google callback only; no real provider credential, account link or production call | Repeatable browser-console defect closed; real-provider verification remains externally gated | | V-209 | Google Identity initialization regression; focused login Jest; ESLint; TypeScript; complete Playwright | `job-tracker-ui` / disposable local application | Remove repeated global Google SDK initialization without changing sign-in or account-link behavior | PASS — SDK initialization is stable per loaded Identity API/client ID while the mounted card owns the current credential handler; login 13/13, lint and TypeScript pass; Playwright 10/10 covers authentication, job creation/workspace, Career Workspace and searchable PDF exports with no repeated-initialize warning | Synthetic Google callback only; no real provider credential, account link or production call | Repeatable browser-console defect closed; real-provider verification remains externally gated |
| V-210 | Standard-library supply-chain tests; tracked-tree credential scan; deterministic multi-lock CycloneDX generation; CI/policy review | Repository root | Add locally provable JT-017 SBOM and secret gates without trusting new mutable tooling | PASS/PARTIAL — 3/3 tests detect seven synthetic credential classes without returning values, ignore binary/oversized inputs and prove deterministic npm/NuGet/PyPI coverage; the tracked tree is clean; generated CycloneDX 1.5 contains 964 unique locked components; CI now blocks findings and generates the SBOM | Scanner is intentionally high-confidence/current-tree only. SBOM is not yet retained by CI. Docker daemon/scanner and verified upstream action/image/installer identifiers are unavailable; Torch/Transformers findings remain unaccepted | Local secret/SBOM controls complete; immutable input, container/licence and model-stack gates remain JT-017 |
+40
View File
@@ -0,0 +1,40 @@
# Supply-chain and credential scanning policy
## Required repository gates
Pull requests and `main` builds must pass all of the following without silently excluding a project:
- locked NuGet restore for the API and test project;
- `npm ci` plus the configured high-severity audit gate;
- hash-required installation of the Linux Python lock;
- the package-manager advisory checks recorded in CI;
- `python3 scripts/supply-chain.py scan-secrets` over every Git-tracked file; and
- `python3 scripts/supply-chain.py generate-sbom --output .artifacts/jobjakt.cdx.json`.
The generated CycloneDX 1.5 SBOM combines the committed npm, NuGet and Python lock files. It is a build artifact, not a replacement for the locks or an image scan. Its lock digest and deterministic component ordering make dependency changes reviewable without adding generated files to source control.
The built-in tracked-secret scanner is deliberately high confidence. It detects private-key headers and recognizable JWT, AWS, GitHub, Google, Slack and live Stripe credentials, skips binary files and files over 2 MiB, and reports only the credential class and file location. It never prints a matched value. This gate complements—not replaces—host secret management, history review and a dedicated scanner when one is approved.
## Dependency and advisory exceptions
High or critical reachable advisories block release. An exception must be a reviewed repository document containing the package and affected path, advisory identifiers, reachability evidence, compensating control, accountable owner, expiry date and removal condition. Expired or ownerless exceptions fail closed. A version pin alone is not an exception.
The outstanding Torch/Transformers findings are not accepted exceptions. Their model-loading path and production image remain blocked pending a compatible resolution or a bounded, reviewed isolation decision.
## Immutable inputs
Third-party workflow actions, container bases and downloaded installers must move only to identifiers verified from their authoritative upstream source. Do not guess a commit SHA, image digest or installer checksum. Updates are reviewed commits and retain the prior known-good identifier for rollback.
The current mutable action/image references and `dotnet-install.sh` download remain open until network-backed upstream metadata and a container scanner are available. The local SBOM and tracked-secret gates do not claim to close those items.
## Credential response
If the tracked scan or a reviewer finds a real credential:
1. stop the affected build and suppress the value from chat, tickets and logs;
2. revoke or rotate it at the issuing system before treating source cleanup as sufficient;
3. remove it from the current tree and add the narrow prevention rule or ignore boundary needed;
4. assess reachable history and CI artifacts using filenames/commit IDs only; and
5. coordinate any history rewrite separately because it disrupts clones and does not revoke a credential.
Never weaken a detector merely to make an unexplained finding pass.
+11 -3
View File
@@ -1,4 +1,4 @@
# JT-017 .NET provenance verification # JT-017 supply-chain provenance verification
## Implemented ## Implemented
@@ -7,6 +7,10 @@
- API and test projects opt into NuGet lock files. - API and test projects opt into NuGet lock files.
- Both direct and transitive package versions carry NuGet content hashes in committed `packages.lock.json` files. - Both direct and transitive package versions carry NuGet content hashes in committed `packages.lock.json` files.
- CI restores both projects with `--locked-mode`, so unreviewed graph drift fails before build or test. - CI restores both projects with `--locked-mode`, so unreviewed graph drift fails before build or test.
- A standard-library tracked-file scanner blocks high-confidence credential patterns without printing values.
- A deterministic CycloneDX 1.5 generator combines the npm, NuGet and hash-locked Python dependency graphs.
- CI tests both controls, scans the tracked tree and generates a 964-component SBOM under ignored build artifacts.
- The review/exception/credential-response rules are documented in `docs/security/supply-chain-policy.md`.
## Proof ## Proof
@@ -14,12 +18,16 @@
- A subsequent locked restore passed. - A subsequent locked restore passed.
- Full backend suite: 719/719 passed. - Full backend suite: 719/719 passed.
- `dotnet list JobTrackerApi/JobTrackerApi.csproj package --vulnerable --include-transitive` reports no known vulnerable packages from the current NuGet source. - `dotnet list JobTrackerApi/JobTrackerApi.csproj package --vulnerable --include-transitive` reports no known vulnerable packages from the current NuGet source.
- Supply-chain unit tests: 3/3 passed, including seven synthetic credential canaries and deterministic npm/NuGet/PyPI coverage.
- The current tracked tree has no high-confidence credential-pattern findings.
- Repeated SBOM generation is byte-stable for unchanged lock inputs and includes React, Entity Framework Core and FastAPI representatives.
## Remaining JT-017 scope ## Remaining JT-017 scope
- Pin third-party CI actions and Docker images by reviewed immutable SHA/digest. - Pin third-party CI actions and Docker images by reviewed immutable SHA/digest.
- Hash or replace the downloaded `dotnet-install.sh` bootstrap path. - Hash or replace the downloaded `dotnet-install.sh` bootstrap path.
- Add policy-owned SBOM, container and tracked-secret gates. - Retain the SBOM as a CI artifact once an immutable artifact action or approved runner store is available.
- Add a policy-owned container CVE/licence scan with a tested canary once the Docker daemon/scanner is available.
- Resolve or formally isolate the remaining Torch/Transformers advisories under the model-loading threat path. - Resolve or formally isolate the remaining Torch/Transformers advisories under the model-loading threat path.
Those items require approved registry/action metadata or a separately reviewed scanner/update process; this increment does not invent immutable identifiers without verifying them. Those items require approved registry/action metadata, network-backed upstream verification or a separately reviewed scanner/update process; this increment does not invent immutable identifiers without verifying them.
+3 -2
View File
@@ -42,6 +42,7 @@ Updated: 2026-08-30
- Localized the remaining active helper defaults in the admin SMTP test form, pasted-email importer, and image-crop alternative text. - Localized the remaining active helper defaults in the admin SMTP test form, pasted-email importer, and image-crop alternative text.
- Restored the local Python 3.12 toolchain and both virtual environments; the AI sidecar now passes 26/26 tests. Added a Next.js-compatible ESLint 9 flat configuration, strict zero-warning scripts, and a patched CommonJS-compatible `brace-expansion` override. The lint gate passes with zero findings and npm audit reports zero vulnerabilities. - Restored the local Python 3.12 toolchain and both virtual environments; the AI sidecar now passes 26/26 tests. Added a Next.js-compatible ESLint 9 flat configuration, strict zero-warning scripts, and a patched CommonJS-compatible `brace-expansion` override. The lint gate passes with zero findings and npm audit reports zero vulnerabilities.
- Pinned the repository to .NET SDK 9.0.317, generated content-hashed transitive NuGet locks for the API and test project, and made CI restores fail on lock drift. Locked restore and backend 719/719 pass; the current NuGet graph has no known vulnerable packages. - Pinned the repository to .NET SDK 9.0.317, generated content-hashed transitive NuGet locks for the API and test project, and made CI restores fail on lock drift. Locked restore and backend 719/719 pass; the current NuGet graph has no known vulnerable packages.
- Added standard-library JT-017 supply-chain gates: seven-class no-value tracked-secret scanning, deterministic CycloneDX generation across npm/NuGet/Python locks, synthetic canary tests, CI integration and an explicit advisory/credential-response policy. The current tree is clean and the SBOM contains 964 unique locked components.
- Rebuilt the active developer/operator documentation around the actual Next.js 16/.NET 9 application, replaced CRA and `npm start` guidance, separated normal and Playwright ports, corrected React Router 7 and the SQLite/MariaDB provider matrix, removed the obsolete npm peer override, and verified the documented clean install, lint, test, build and locked-restore commands. - Rebuilt the active developer/operator documentation around the actual Next.js 16/.NET 9 application, replaced CRA and `npm start` guidance, separated normal and Playwright ports, corrected React Router 7 and the SQLite/MariaDB provider matrix, removed the obsolete npm peer override, and verified the documented clean install, lint, test, build and locked-restore commands.
- Began the JT-019 schema-ownership retirement with an executable 49-table ownership partition and transferred the leaf `SystemEmailSettings` table from MariaDB-only startup DDL to an additive provider-aware migration. Fresh SQLite now receives the table; legacy rows are preserved and startup no longer creates it. - Began the JT-019 schema-ownership retirement with an executable 49-table ownership partition and transferred the leaf `SystemEmailSettings` table from MariaDB-only startup DDL to an additive provider-aware migration. Fresh SQLite now receives the table; legacy rows are preserved and startup no longer creates it.
- Transferred the independent `UserRuleSettings` table from both provider startup paths to its own provider-aware migration; owner-keyed settings survive adoption, downgrade and retry. - Transferred the independent `UserRuleSettings` table from both provider startup paths to its own provider-aware migration; owner-keyed settings survive adoption, downgrade and retry.
@@ -127,7 +128,7 @@ Updated: 2026-08-30
- Manual desktop browser review: webpack development server rendered the new Career navigation and Overview correctly in dark mode; API-dependent profile status remained unavailable because the backend was not running for that isolated UI review. - Manual desktop browser review: webpack development server rendered the new Career navigation and Overview correctly in dark mode; API-dependent profile status remained unavailable because the backend was not running for that isolated UI review.
- **Overall programme status:** Active but externally blocked. Eight packages are locally verified and twenty-seven are implemented with verification incomplete. The prioritized admin-only version indicator, every immediate repository/browser item, SEC-006/SEC-007 repository boundaries, SEC-009, the PROD-001 read-only inventory, and the PROD-003 safe benchmark harness are complete on the feature branch. - **Overall programme status:** Active but externally blocked. Eight packages are locally verified and twenty-seven are implemented with verification incomplete. The prioritized admin-only version indicator, every immediate repository/browser item, SEC-006/SEC-007 repository boundaries, SEC-009, the PROD-001 read-only inventory, and the PROD-003 safe benchmark harness are complete on the feature branch.
- **Current work package:** JT-017 remaining immutable CI/image/scanner provenance and JT-019 schema-ownership inventory while external runtime gates remain blocked. JT-018 is complete in repository scope. - **Current work package:** JT-017 remaining immutable CI action/image/installer provenance, retained SBOM, container/licence scan and model-stack advisory resolution. JT-018 and JT-019 are complete in repository scope; the remaining JT-017 gates require verified upstream metadata or the unavailable Docker/scanner runtime.
- **Completed work packages:** None are `DONE`; all repository security/AI packages still have applicable browser, provider and/or production gates. - **Completed work packages:** None are `DONE`; all repository security/AI packages still have applicable browser, provider and/or production gates.
- **Locally verified work:** SEC-001, SEC-002, SEC-003, SEC-005A, CORE-001, PROD-002, DEP-001 and VER-001 (`VERIFIED LOCALLY`). - **Locally verified work:** SEC-001, SEC-002, SEC-003, SEC-005A, CORE-001, PROD-002, DEP-001 and VER-001 (`VERIFIED LOCALLY`).
- **Implemented, verification incomplete:** SEC-004, SEC-005B, SEC-006, SEC-007, SEC-008, SEC-009, CORE-002, BG-001, OPS-001A/B/C, POL-001/002, AI-001/002/003/004, UX-001/002/003, QA-001, CAREER-001/002, MAIL-001, JOBS-001/002 and PRODUCT-001 (`IMPLEMENTED — NOT VERIFIED`). Their safe repository/browser scope is implemented; production/native-device/provider/retention gates remain where recorded. - **Implemented, verification incomplete:** SEC-004, SEC-005B, SEC-006, SEC-007, SEC-008, SEC-009, CORE-002, BG-001, OPS-001A/B/C, POL-001/002, AI-001/002/003/004, UX-001/002/003, QA-001, CAREER-001/002, MAIL-001, JOBS-001/002 and PRODUCT-001 (`IMPLEMENTED — NOT VERIFIED`). Their safe repository/browser scope is implemented; production/native-device/provider/retention gates remain where recorded.
@@ -136,7 +137,7 @@ Updated: 2026-08-30
- **Deferred work:** None. Conditional multi-replica coordination, model deletion, realtime operation delivery and unrelated production changes remain outside current packages. - **Deferred work:** None. Conditional multi-replica coordination, model deletion, realtime operation delivery and unrelated production changes remain outside current packages.
- **Immediate order:** all sixteen immediate repository items are complete locally, including the original UI/release queue plus SEC-009 cache/tombstone safety, worker restart clocks, universal AI accounting, email-token/Stripe lifecycle tests, exhaustive Job email selectors, the repaired migration chain, CV/public-edge hardening and measured admin/mail scaling. PROD-001 read-only evidence and the PROD-003 plan-only harness are also complete. The final audit is checking tooling/documentation before declaring only external blockers remain. - **Immediate order:** all sixteen immediate repository items are complete locally, including the original UI/release queue plus SEC-009 cache/tombstone safety, worker restart clocks, universal AI accounting, email-token/Stripe lifecycle tests, exhaustive Job email selectors, the repaired migration chain, CV/public-edge hardening and measured admin/mail scaling. PROD-001 read-only evidence and the PROD-003 plan-only harness are also complete. The final audit is checking tooling/documentation before declaring only external blockers remain.
- **Status counts:** 8 `VERIFIED LOCALLY`; 27 `IMPLEMENTED — NOT VERIFIED`; 0 `IN PROGRESS`; 0 `NOT STARTED`; 4 `BLOCKED`; 0 `DONE`; 0 `DEFERRED`. - **Status counts:** 8 `VERIFIED LOCALLY`; 27 `IMPLEMENTED — NOT VERIFIED`; 0 `IN PROGRESS`; 0 `NOT STARTED`; 4 `BLOCKED`; 0 `DONE`; 0 `DEFERRED`.
- **Test status:** backend 723/723; frontend 64/64 suites and 272/272 tests; ESLint zero findings; AI sidecar 37/37 without warnings; Ollama benchmark harness 5/5 plus safe dry-run; optimized production build/TypeScript; EF model parity; SQLite/MariaDB migration scripts; blank/idempotent/populated SQLite migration-chain tests; fresh SQLite application startup through the SystemEmailSettings ownership migration; disposable fresh/restarted MariaDB 11.8 application startup; Docker Compose config; safe-failure deployment preflight; and Playwright 9/9. npm audit remains at zero. The new ownership migrations have generated MariaDB SQL but still need a MariaDB runtime rehearsal; the parser image build is pending because Docker Desktop's daemon is offline. Jest's slow/open-handle behavior remains recorded. - **Test status:** backend 736/736; frontend 64/64 suites and 272/272 tests; ESLint and TypeScript zero findings; AI sidecar 37/37 without warnings; Ollama benchmark harness 5/5 plus safe dry-run; optimized production build; EF model parity; SQLite/MariaDB migration scripts; blank/idempotent/populated SQLite migration-chain tests; fresh complete-chain SQLite application startup; disposable earlier-chain MariaDB 11.8 application startup; Docker Compose config; safe-failure deployment preflight; and Playwright 10/10. Supply-chain controls pass 3/3, the tracked tree is clean, and the generated CycloneDX SBOM contains 964 components. npm audit remains at zero. The final ownership migrations have generated MariaDB SQL but still need a current MariaDB runtime rehearsal; the parser image and container scan are pending because Docker Desktop's daemon is offline. Jest's slow/open-handle behavior remains recorded.
- **Deployment status:** Gitea pull-request run 609 passed the complete CI job in 4m20s. Deploy was intentionally skipped because the workflow deploys only a `push` to `main`; live remains unchanged. No merge/deployment was performed directly, no production migrations were run and the AI operation worker remains disabled by default. - **Deployment status:** Gitea pull-request run 609 passed the complete CI job in 4m20s. Deploy was intentionally skipped because the workflow deploys only a `push` to `main`; live remains unchanged. No merge/deployment was performed directly, no production migrations were run and the AI operation worker remains disabled by default.
- **Production status:** State unchanged. Sanitized read-only SSH inventory was performed; no logs, prompts, private rows/content or secret values were read, and no provider/model call, model pull, service restart, file/config change, backup, restore, migration or deployment occurred. It confirmed all-interface Ollama/frontend listeners and stale database-only backups as rollout blockers. - **Production status:** State unchanged. Sanitized read-only SSH inventory was performed; no logs, prompts, private rows/content or secret values were read, and no provider/model call, model pull, service restart, file/config change, backup, restore, migration or deployment occurred. It confirmed all-interface Ollama/frontend listeners and stale database-only backups as rollout blockers.
- **Known regressions:** None found by automated/local browser checks. Jest still needs `--forceExit` and reports its existing open-handle notice. Email-provider/send tests are fake/local only; real delivery is not claimed. Current MAIL browser evidence is 1280×720 only because the browser surface could not resize or perform native Tab traversal. Interrupted attempts are aged after 15 minutes and notified without retry; the five-minute scan is unmeasured on a large ledger. The direct clean EF-only SQLite defect and synchronous AI accounting gap are closed; migration/reconciler dual ownership remains architectural debt. - **Known regressions:** None found by automated/local browser checks. Jest still needs `--forceExit` and reports its existing open-handle notice. Email-provider/send tests are fake/local only; real delivery is not claimed. Current MAIL browser evidence is 1280×720 only because the browser surface could not resize or perform native Tab traversal. Interrupted attempts are aged after 15 minutes and notified without retry; the five-minute scan is unmeasured on a large ledger. The direct clean EF-only SQLite defect and synchronous AI accounting gap are closed; migration/reconciler dual ownership remains architectural debt.
+199
View File
@@ -0,0 +1,199 @@
#!/usr/bin/env python3
"""Repository-local supply-chain checks with no third-party runtime dependency."""
from __future__ import annotations
import argparse
import base64
import hashlib
import json
import re
import subprocess
import sys
from pathlib import Path
from typing import Iterable
from urllib.parse import quote
SECRET_PATTERNS = {
"private-key": re.compile(rb"-----BEGIN (?:RSA |EC |OPENSSH |DSA )?PRIVATE KEY-----"),
"jwt": re.compile(rb"\beyJ[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}\b"),
"aws-access-key": re.compile(rb"\b(?:AKIA|ASIA)[A-Z0-9]{16}\b"),
"github-token": re.compile(rb"\b(?:gh[pousr]_[A-Za-z0-9]{36,255}|github_pat_[A-Za-z0-9_]{82,255})\b"),
"google-api-key": re.compile(rb"\bAIza[A-Za-z0-9_-]{35}\b"),
"slack-token": re.compile(rb"\bxox[baprs]-[A-Za-z0-9-]{20,}\b"),
"stripe-live-key": re.compile(rb"\bsk_live_[A-Za-z0-9]{16,}\b"),
}
MAX_SCANNED_FILE_BYTES = 2 * 1024 * 1024
def tracked_files(root: Path) -> list[str]:
result = subprocess.run(
["git", "ls-files", "-z"],
cwd=root,
check=True,
capture_output=True,
)
return [item.decode("utf-8", errors="surrogateescape") for item in result.stdout.split(b"\0") if item]
def scan_files(root: Path, paths: Iterable[str]) -> list[tuple[str, str, int]]:
findings: list[tuple[str, str, int]] = []
for relative in paths:
path = root / relative
if not path.is_file() or path.stat().st_size > MAX_SCANNED_FILE_BYTES:
continue
data = path.read_bytes()
if b"\0" in data:
continue
for line_number, line in enumerate(data.splitlines(), start=1):
for secret_type, pattern in SECRET_PATTERNS.items():
if pattern.search(line):
findings.append((secret_type, relative.replace("\\", "/"), line_number))
return findings
def _purl(ecosystem: str, name: str, version: str) -> str:
if ecosystem == "npm" and name.startswith("@"):
namespace, package = name[1:].split("/", 1)
return f"pkg:npm/{quote(namespace, safe='')}/{quote(package, safe='')}@{quote(version, safe='')}"
return f"pkg:{ecosystem}/{quote(name, safe='')}@{quote(version, safe='')}"
def _component(ecosystem: str, name: str, version: str, manifests: set[str], scope: str = "required", hashes: list[dict[str, str]] | None = None) -> dict[str, object]:
component: dict[str, object] = {
"type": "library",
"bom-ref": _purl(ecosystem, name, version),
"name": name,
"version": version,
"scope": scope,
"purl": _purl(ecosystem, name, version),
"properties": [{"name": "jobjakt:source-manifest", "value": manifest} for manifest in sorted(manifests)],
}
if hashes:
component["hashes"] = hashes
return component
def _npm_components(root: Path) -> list[dict[str, object]]:
relative = "job-tracker-ui/package-lock.json"
lock = json.loads((root / relative).read_text(encoding="utf-8"))
components: dict[tuple[str, str], dict[str, object]] = {}
for package_path, package in lock.get("packages", {}).items():
if not package_path or not package.get("version"):
continue
name = package.get("name") or package_path.rsplit("node_modules/", 1)[-1]
version = str(package["version"])
integrity = str(package.get("integrity", ""))
hashes: list[dict[str, str]] = []
if integrity.startswith("sha512-"):
try:
hashes.append({"alg": "SHA-512", "content": base64.b64decode(integrity[7:]).hex()})
except ValueError:
pass
components[(name, version)] = _component(
"npm",
name,
version,
{relative},
"optional" if package.get("dev") else "required",
hashes,
)
return list(components.values())
def _nuget_components(root: Path) -> list[dict[str, object]]:
manifests = ["JobTrackerApi/packages.lock.json", "JobTrackerApi.Tests/packages.lock.json"]
packages: dict[tuple[str, str], set[str]] = {}
hashes: dict[tuple[str, str], str] = {}
for relative in manifests:
lock = json.loads((root / relative).read_text(encoding="utf-8"))
for target in lock.get("dependencies", {}).values():
for name, package in target.items():
version = str(package.get("resolved", ""))
if not version:
continue
key = (name, version)
packages.setdefault(key, set()).add(relative)
if package.get("contentHash"):
hashes[key] = str(package["contentHash"])
result = []
for (name, version), sources in packages.items():
package_hashes = []
if (content_hash := hashes.get((name, version))):
try:
package_hashes.append({"alg": "SHA-512", "content": base64.b64decode(content_hash).hex()})
except ValueError:
pass
result.append(_component("nuget", name, version, sources, hashes=package_hashes))
return result
def _python_components(root: Path) -> list[dict[str, object]]:
relative = "tools/summarizer/requirements-linux.lock"
pattern = re.compile(r"^([A-Za-z0-9_.-]+)==([^\s\\]+)")
packages: dict[tuple[str, str], set[str]] = {}
for line in (root / relative).read_text(encoding="utf-8").splitlines():
if match := pattern.match(line):
packages.setdefault((match.group(1), match.group(2)), set()).add(relative)
return [_component("pypi", name, version, sources) for (name, version), sources in packages.items()]
def generate_sbom(root: Path) -> dict[str, object]:
lock_paths = [
"job-tracker-ui/package-lock.json",
"JobTrackerApi/packages.lock.json",
"JobTrackerApi.Tests/packages.lock.json",
"tools/summarizer/requirements-linux.lock",
]
digest = hashlib.sha256()
for relative in lock_paths:
digest.update(relative.encode("utf-8"))
digest.update((root / relative).read_bytes())
components = _npm_components(root) + _nuget_components(root) + _python_components(root)
components.sort(key=lambda item: (str(item["purl"]).lower(), str(item["version"])))
return {
"bomFormat": "CycloneDX",
"specVersion": "1.5",
"serialNumber": f"urn:uuid:{digest.hexdigest()[:8]}-{digest.hexdigest()[8:12]}-5{digest.hexdigest()[13:16]}-a{digest.hexdigest()[17:20]}-{digest.hexdigest()[20:32]}",
"version": 1,
"metadata": {
"component": {"type": "application", "bom-ref": "pkg:generic/jobjakt", "name": "Jobjakt"},
"tools": {"components": [{"type": "application", "name": "scripts/supply-chain.py"}]},
"properties": [{"name": "jobjakt:lock-digest-sha256", "value": digest.hexdigest()}],
},
"components": components,
}
def main() -> int:
parser = argparse.ArgumentParser()
parser.add_argument("command", choices=("scan-secrets", "generate-sbom"))
parser.add_argument("--root", type=Path, default=Path(__file__).resolve().parents[1])
parser.add_argument("--output", type=Path)
args = parser.parse_args()
root = args.root.resolve()
if args.command == "scan-secrets":
findings = scan_files(root, tracked_files(root))
for secret_type, relative, line_number in findings:
print(f"{secret_type}: {relative}:{line_number}")
if findings:
print(f"Tracked-secret scan failed with {len(findings)} high-confidence finding(s); values were suppressed.", file=sys.stderr)
return 1
print("Tracked-secret scan passed; no high-confidence credential patterns found.")
return 0
if not args.output:
parser.error("generate-sbom requires --output")
output = args.output if args.output.is_absolute() else root / args.output
output.parent.mkdir(parents=True, exist_ok=True)
output.write_text(json.dumps(generate_sbom(root), indent=2, ensure_ascii=False) + "\n", encoding="utf-8")
print(f"CycloneDX SBOM written to {output}.")
return 0
if __name__ == "__main__":
raise SystemExit(main())
+65
View File
@@ -0,0 +1,65 @@
#!/usr/bin/env python3
import importlib.util
import json
import tempfile
import unittest
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
SPEC = importlib.util.spec_from_file_location("supply_chain", ROOT / "scripts" / "supply-chain.py")
assert SPEC and SPEC.loader
SUPPLY_CHAIN = importlib.util.module_from_spec(SPEC)
SPEC.loader.exec_module(SUPPLY_CHAIN)
class SecretScanTests(unittest.TestCase):
def test_detects_canaries_without_returning_secret_values(self):
canaries = [
"-----BEGIN " + "PRIVATE KEY-----",
"eyJ" + "a" * 12 + "." + "b" * 12 + "." + "c" * 12,
"AKIA" + "A" * 16,
"ghp_" + "a" * 36,
"AIza" + "a" * 35,
"xoxb-" + "a" * 24,
"sk_" + "live_" + "a" * 20,
]
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
(root / "fixture.txt").write_text("\n".join(canaries), encoding="utf-8")
findings = SUPPLY_CHAIN.scan_files(root, ["fixture.txt"])
self.assertEqual(len(SUPPLY_CHAIN.SECRET_PATTERNS), len(findings))
rendered = "\n".join(f"{kind} {path}:{line}" for kind, path, line in findings)
for canary in canaries:
self.assertNotIn(canary, rendered)
def test_ignores_binary_and_oversized_files(self):
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
binary_canary = b"\0-----BEGIN " + b"PRIVATE KEY-----"
(root / "binary.dat").write_bytes(binary_canary)
(root / "large.txt").write_bytes(b"a" * (SUPPLY_CHAIN.MAX_SCANNED_FILE_BYTES + 1))
self.assertEqual([], SUPPLY_CHAIN.scan_files(root, ["binary.dat", "large.txt"]))
class SbomTests(unittest.TestCase):
def test_generates_deterministic_multi_ecosystem_cyclonedx(self):
first = SUPPLY_CHAIN.generate_sbom(ROOT)
second = SUPPLY_CHAIN.generate_sbom(ROOT)
self.assertEqual(first, second)
self.assertEqual("CycloneDX", first["bomFormat"])
self.assertEqual("1.5", first["specVersion"])
components = first["components"]
purls = {component["purl"] for component in components}
self.assertTrue(any(purl.startswith("pkg:npm/react@") for purl in purls))
self.assertTrue(any(purl.startswith("pkg:nuget/Microsoft.EntityFrameworkCore@") for purl in purls))
self.assertTrue(any(purl.startswith("pkg:pypi/fastapi@") for purl in purls))
self.assertEqual(len(purls), len(components))
json.dumps(first)
if __name__ == "__main__":
unittest.main()