feat(export): add readable account archive
CI and Deploy / test (pull_request) Successful in 5m13s
CI and Deploy / deploy (pull_request) Has been skipped

This commit is contained in:
cesnimda
2026-08-15 18:36:40 +02:00
parent cdcc7163fa
commit 1ec9dd037e
15 changed files with 864 additions and 35 deletions
+1
View File
@@ -205,3 +205,4 @@ Output was reduced to filenames and commit counts. The token artifact appears un
| V-171 | Public/product claim inventory; plan/notice/usage focused Jest; current entitlement/billing-policy backend slice; full frontend; optimized build; full Playwright | Repository root / `job-tracker-ui` | Replace contradictory plan/commercial claims and prove respectful Free/Pro promotion without changing billing or enforcement | PASS — exactly Free/Pro comes from one catalogue; focused frontend 7 suites/30 tests, policy/billing backend 30/30, full frontend 57 suites/232 tests, build and Playwright 8/8. Chromium proves retired claims absent, explicit Light/Dark, 375/768/1440 no overflow and keyboard Free/Pro actions | Local synthetic account only; no Stripe checkout/webhook/portal, native AT or production. Commercial terms remain intentionally absent until configured Checkout; existing Jest/GSI warnings remain | PRODUCT-001 repository/browser scope verified; configured billing lifecycle and production remain |
| V-172 | Action-matrix reconciliation; full backend/frontend/sidecar/build/Compose/preflight gates; expanded authenticated and anonymous Playwright | Repository root / `job-tracker-ui` / `tools/summarizer` | Complete VER-001 local release regression without promoting mocked/provider/external checks | PASS — backend 647/647, frontend 57 suites/232 tests, sidecar 22/22, production build, Compose config, API-down/wrong-base/malformed-JSON preflight and Chromium 9/9. Browser covers admin deployment identity/normal-user absence, notifications, honest Free, jobs, Career/CV, Kanban, responsive themes and public PDF | No external provider, private data, native AT or production mutation. Optional Compose variables remain unset; existing Jest/GSI/SWIG warnings remain. Windows CRLF materialization was normalized for shell execution; indexed LF policy was already correct | VER-001 verified locally; remote CI/provider/native-AT/production cells remain |
| V-173 | Owner-path trace; focused CV/export/controller/background tests; full backend; build and diff hygiene | Repository root | Establish attributable generated-file ownership before SEC-009 export/deletion | PASS — CV PDFs use opaque owner/date/UUID storage while preserving download names; daily exports use opaque owner directories and atomic writes; legacy/new retention paths are covered. Focused 77/77 and backend 647/647 | No existing file moved or deleted. Legacy shared-date generated files are intentionally not guessed. No migration, production path or private data used | SEC-009 owner-scoped generated-output prerequisite verified locally |
| V-174 | Real-SQLite two-owner export fixture; manifest/checksum/file/redaction assertions; recent-session/rate-limit API tests; focused/full frontend and backend; optimized build; Chromium ZIP response | Repository root / `job-tracker-ui` | Deliver a complete user-readable export without exposing secrets or another tenant | PASS — focused backend/API 11/11, backend 650/650, frontend focused 4/4 and full 58 suites/234 tests, builds pass. Every manifest checksum/size matches; owned attachment/CV/avatar/generated/daily files are included; secret and other-owner sentinels are absent; Chromium receives HTTP 200 `application/zip` with `PK` signature | Synthetic data/files only; no production/private/provider access. Export reports backups/logs/external retention instead of claiming erasure. One initial InMemory-only test missed SQLite DateTimeOffset translation; the test moved to real SQLite and the query boundary was corrected | SEC-009 readable export verified locally; deletion/retention/restore remain |
@@ -50,6 +50,7 @@ This is the rolling action-level evidence index. `PASS (automated/runtime)` is n
| Public CV | responsive A4/multi-page framing without inner or outer overflow | PASS (component) | PASS — 375px Chromium | NOT RUN | `accessibility-evidence.md`, V-170 |
| Public plans | exactly Free/Pro; no invented tier, price, interval, trial or unlimited claim | PASS (catalogue + landing components) | PASS — Light/Dark at 375/768/1440 | NOT RUN | `product-001-honest-plans.md`, V-171 |
| Public plans | Free registration and Pro sign-in-to-Settings actions | PASS (component) | PASS — keyboard activation | NOT RUN | `product-001-honest-plans.md`, V-171 |
| Account export | recent-sign-in/rate-limit gate; complete redacted owner ZIP, files, warnings and checksums | PASS (real SQLite + components) | PASS — fresh synthetic Free download response and success state | NOT RUN with production data | `sec-009-account-lifecycle.md`, V-174 |
| Pro promotion | benefit-specific locked notice, preserved-data copy and session dismissal | PASS (components) | NOT RUN on every contextual surface | NOT RUN | `product-001-honest-plans.md`, V-171 |
| Billing presentation | checkout/portal only when server status permits; unconfigured deployment disclosed | PASS (components + policy slice) | NOT RUN with configured Stripe | NOT RUN | `product-001-honest-plans.md`, V-171 |
| Admin safety | self/other Admin demotion confirmation and final-admin API protection | PASS (controller + components) | NOT RUN | NOT RUN | V-161 |
+22 -8
View File
@@ -2,7 +2,7 @@
Updated: 2026-08-15
Status: `IN PROGRESS`. Generated-output ownership is now explicit. Readable export and the disabled deletion lifecycle remain to be implemented.
Status: `IN PROGRESS`. Generated-output ownership and the readable export are implemented. The disabled deletion lifecycle remains to be implemented.
## Owner inventory boundary
@@ -18,19 +18,33 @@ The authoritative inventory must include Identity-safe account/profile fields an
No existing generated file is moved or guessed. Legacy shared-date outputs stay a separately reviewed rollout concern because they cannot be attributed safely.
## Checkpoint 2 — complete readable export
- Authenticated `POST /api/export/account` requires the current local session to have been created within the last 15 minutes and is limited to two requests per user per hour.
- One service owns both the authoritative row inventory and file inventory. It queries with explicit owner predicates and `IgnoreQueryFilters`, so soft-deleted applications remain portable and an absent/requestless tenant scope cannot silently empty the export.
- The ZIP contains readable account, company, opportunity, application, correspondence, event, attachment, Career, CV, workspace, AI operation, notification, settings/provider and security-metadata JSON categories.
- Owned attachment, CV upload, avatar, generated-CV and daily-export bytes are included only after managed-root/reparse-point checks. Missing or unsafe files produce manifest warnings rather than cross-root reads.
- `manifest.json` records schema version, generated time, category/item counts, byte sizes and SHA-256 checksums for every included entry. `README.txt` explains formats, exclusions and retention limits.
- Password/security/concurrency hashes, TOTP secrets, recovery/trusted-device hashes, session IDs, provider access/refresh tokens, IMAP passwords, operation leases, email payload hashes, global settings and data-protection keys are never serialized.
- The Settings Backup tab presents the readable export separately from the application-key-encrypted operational backup and explains recent sign-in without weakening the API rule.
- Temporary ZIPs live under an opaque owner root and are opened with delete-on-close when returned by the controller.
## Verification
- Focused CV/export/controller/background tests: 77/77.
- Full backend: 647/647.
- Owner-storage focused CV/export/controller/background tests: 77/77.
- Readable-export focused backend/API tests: 11/11, including real SQLite, two-owner isolation, file inclusion, every checksum and secret-redaction sentinels.
- Full backend: 650/650.
- Frontend export/Settings tests: 4/4; full frontend 58 suites/234 tests.
- Backend build: pass, zero warnings/errors.
- Optimized frontend build/TypeScript: pass.
- Chromium: fresh Free account receives a real ZIP response with a `PK` signature and readable-export success state.
- `git diff --check`: pass aside from line-ending notices.
## Remaining repository work
1. Implement one owner inventory used by both readable ZIP export and deletion.
2. Add manifest/checksums/warnings and include safely owned binary files without exposing storage paths.
3. Add the additive deletion state/request/file schema and disabled coordinator.
4. Add pending-account authentication/mutation gates, session/queue cancellation, provider cleanup and idempotent file quarantine/database purge.
5. Add separate tombstone storage/replay and settings/admin UX while keeping production activation disabled.
1. Reuse the completed owner inventory in the deletion coordinator.
2. Add the additive deletion state/request/file schema and disabled coordinator.
3. Add pending-account authentication/mutation gates, session/queue cancellation, provider cleanup and idempotent file quarantine/database purge.
4. Add separate tombstone storage/replay and settings/admin UX while keeping production activation disabled.
Production retention, legal hold and restored-backup decisions remain recorded in `BLOCKERS.md`.
+10
View File
@@ -739,3 +739,13 @@
- **Consequences:** all new generated outputs have an exact owner boundary and collision-resistant storage path. Existing legacy files age out under retention and remain excluded from user deletion unless independently attributed.
- **User approval required:** No; additive storage hardening within the requested account lifecycle, with no existing data mutation.
- **Reversible:** Restore shared date paths for future files. Existing owner-scoped files remain valid retention artifacts and must not be bulk-moved or deleted during rollback.
## DEC-075 — Separate readable portability export from operational backup
- **Date:** 2026-08-15
- **Decision:** Add a recent-authenticated, per-user-rate-limited readable ZIP export alongside—not in place of—the existing application-key-encrypted backup. Build one explicit redacted owner inventory with checksum manifest and reuse it as the future deletion inventory boundary.
- **Reason/evidence:** the encrypted backup is useful for application recovery but unreadable without the deployment key and omits many owned categories. A portability export must be readable, complete, tenant-isolated and secret-free; it must not be mislabeled as backup erasure.
- **Alternatives considered:** expose the encrypted backup as user export; serialize the whole EF graph; reuse the jobs-only export; include provider/token/security rows verbatim. These are unreadable, partial, cycle-prone or credential disclosures.
- **Consequences:** users can download JSON and owned files with independently verifiable SHA-256 checksums. Missing/legacy/external/backup categories are disclosed truthfully. The service becomes the authoritative inventory seam for deletion without coupling export to deletion activation.
- **User approval required:** No; this is the requested repository-side data lifecycle, using synthetic tests and no production data.
- **Reversible:** Remove the endpoint/UI and service. Existing downloaded ZIPs remain user-owned files; no stored schema or data changed.
+2 -2
View File
@@ -3,7 +3,7 @@
Updated: 2026-08-15
- **Overall programme status:** Active. Eight packages are locally verified; twenty-four are implemented with verification incomplete; SEC-009 is in progress. The prioritized admin-only version indicator and every immediate repository/browser item are implemented on the release branch; remote and production verification remain.
- **Current work package:** `SEC-009` — complete readable export and account deletion lifecycle (`IN PROGRESS`). Generated CV/daily outputs now use opaque owner directories; proceed with the shared owner inventory/readable ZIP, then the disabled deletion lifecycle while retention/restore policy blocks production activation.
- **Current work package:** `SEC-009` — complete readable export and account deletion lifecycle (`IN PROGRESS`). Owner-scoped generated paths and the real-SQLite-verified readable ZIP are complete; proceed with the disabled deletion lifecycle while retention/restore policy blocks production activation.
- **Completed work packages:** None are `DONE`; all repository security/AI packages still have applicable browser, provider and/or production gates.
- **Locally verified work:** SEC-001, SEC-002, SEC-003, SEC-005A, CORE-001, PROD-002, DEP-001 and VER-001 (`VERIFIED LOCALLY`).
- **Implemented, verification incomplete:** SEC-004, SEC-005B, SEC-008, CORE-002, BG-001, OPS-001A/B/C, POL-001/002, AI-001/002/003/004, UX-001/002/003, QA-001, CAREER-001/002, MAIL-001, JOBS-001/002 and PRODUCT-001 (`IMPLEMENTED — NOT VERIFIED`). Their safe repository/browser scope is implemented; production/native-device/provider gates remain where recorded.
@@ -12,7 +12,7 @@ Updated: 2026-08-15
- **Deferred work:** None. Conditional multi-replica coordination, model deletion, realtime operation delivery and unrelated production changes remain outside current packages.
- **Immediate order:** SEC-009 owner inventory/export first, then its disabled deletion lifecycle. The eight-item immediate queue is complete locally: admin version (`a6cffe0`), Career persistence (`f0b9b22`), CV contrast (`3b86ea2`), JOBS-002 (`deed948`), accessibility (`a7c2549`), PRODUCT-001 (`a25c31b`), VER-001 and tracking reconciliation. External-only work remains skipped, not allowed to stall repository progress.
- **Status counts:** 8 `VERIFIED LOCALLY`; 24 `IMPLEMENTED — NOT VERIFIED`; 1 `IN PROGRESS`; 1 `NOT STARTED`; 5 `BLOCKED`; 0 `DONE`; 0 `DEFERRED`.
- **Test status:** backend 647/647; frontend 57/57 suites and 232/232 tests; AI sidecar 22/22; optimized production build/TypeScript; Docker Compose config; safe-failure deployment preflight; and Playwright 9/9 pass. Chromium covers the admin deployment badge/normal-user absence, notification popover, honest Free behavior, explicit light/dark at 375/768/1440, application workspace, Career/CV, discovery, Kanban and public CV/PDF. npm audit 0 evidence remains current because the lockfile did not change. Historical JT-019 and Jest force-exit/open-handle behavior remain recorded.
- **Test status:** backend 650/650; frontend 58/58 suites and 234/234 tests; AI sidecar 22/22; optimized production build/TypeScript; Docker Compose config; safe-failure deployment preflight; and the preceding Playwright 9/9 pass. A fresh Chromium Free account now also downloads the real readable ZIP response. npm audit 0 evidence remains current because the lockfile did not change. Historical JT-019 and Jest force-exit/open-handle behavior remain recorded.
- **Deployment status:** Gitea pull-request run 609 passed the complete CI job in 4m20s. Deploy was intentionally skipped because the workflow deploys only a `push` to `main`; live remains unchanged. No merge/deployment was performed directly, no production migrations were run and the AI operation worker remains disabled by default.
- **Production status:** Unchanged and unverified. No provider/model call, model pull, external request or paid API occurred.
- **Known regressions:** None found by automated/local browser checks. Jest still needs `--forceExit` and reports its existing open-handle notice. Email-provider/send tests are fake/local only; real delivery is not claimed. Current MAIL browser evidence is 1280×720 only because the browser surface could not resize or perform native Tab traversal. Interrupted attempts are aged after 15 minutes and notified without retry; the five-minute scan is unmeasured on a large ledger. Direct clean EF-only SQLite migration still hits the pre-existing historical blank-chain defect before later migrations; normal startup owns reconciliation. Cross-feature monthly AI usage accounting remains a rollout gap.
+2 -2
View File
@@ -270,9 +270,9 @@ This queue records the highest-value work that can proceed without production cr
- **Required production verification:** backup retention/tombstone rehearsal before self-service enablement.
- **Status:** `IN PROGRESS`.
- **Blocker:** legal/operator retention and production restore decisions block activation, not the repository-side disabled/dark launch.
- **Evidence:** audit JT-009 inventory/design; `docs/verification/sec-009-account-lifecycle.md`; V-173 owner-scoped generated-output checkpoint, focused 77/77 and backend 647/647.
- **Evidence:** audit JT-009 inventory/design; `docs/verification/sec-009-account-lifecycle.md`; V-173/V-174. Owner-scoped generated storage and complete redacted readable ZIP pass real-SQLite two-owner, focused API/UI, full backend/frontend, build and Chromium checks.
- **Commit:** none.
- **Remaining work:** owner inventory/readable ZIP export next; then additive disabled deletion coordinator, tombstone replay, UI and failure/restart verification. Production activation remains blocked by retention/restore policy.
- **Remaining work:** reuse the completed inventory for the additive disabled deletion coordinator, tombstone replay, UI and failure/restart verification. Production activation remains blocked by retention/restore policy.
### CORE-001 — Restore default SQLite/MariaDB behavior parity
+5 -5
View File
@@ -3,16 +3,16 @@
Updated: 2026-08-15
- **Exact current task:** begin SEC-009 with the owner inventory/readable export, then implement the deletion lifecycle behind a disabled production gate.
- **Last completed step:** established owner-scoped storage for every newly generated CV PDF and daily export without moving unattributable legacy files.
- **Files currently modified:** `AppPaths`, CV PDF exporter/controller callers, daily export worker, focused tests and SEC-009 evidence.
- **Commands already run:** SEC-009 storage slice 77/77; full backend 647/647; backend build; diff hygiene. The preceding VER-001 frontend/sidecar/build/Compose/preflight/Playwright 9/9 evidence remains current.
- **Last completed step:** implemented a recent-authenticated, rate-limited, redacted readable account ZIP with complete safe row/file inventory, checksums, warnings and Settings UX.
- **Files currently modified:** account export service/API/rate policy, Backup Settings UI/translations, real-SQLite/backend/frontend/browser tests and SEC-009 evidence.
- **Commands already run:** readable-export backend/API 11/11, backend 650/650, frontend focused 4/4 and full 234/234, backend/frontend builds and targeted Chromium ZIP response. The preceding VER-001 sidecar/Compose/preflight/full Playwright evidence remains current.
- **Test results:** all listed local gates pass. Provider/native-AT/production cells remain explicitly partial, not run or blocked. Jest retains the documented force-exit/open-handle notice.
- **Services currently running:** none on task-owned ports 3000/5202. Playwright stopped its disposable API/Next servers. Pre-existing Docker services were not changed.
- **Temporary files or processes:** no task-owned process is running and the failed disposable migration database was removed. Existing synthetic browser evidence/account and startup-created local backup remain documented. No provider account, real email, private content, paid service or production service was accessed.
- **Production changes currently active:** none. No deployment, migration, provider connection/sync/send or production payload occurred.
- **Rollback status:** downgrade `20260810080858_AddEmailDraftClientRequestId`, then `20260810075206_AddEmailDrafts`, before reverting draft commits; then follow the existing MAIL rollback order (`ee5ef7e`, `449faeb`, `123fc55`/`e9937ac`, ledger downgrade before `653f011`). No production migration/deploy/provider grant occurred.
- **Uncommitted changes:** V-173 owner-storage code/tests/docs; no dependency, schema or production configuration change. V-172 is pushed as `0d48712`.
- **Uncommitted changes:** V-174 readable-export code/tests/docs; no dependency, schema or production configuration change. V-173 is pushed as `cdcc716`.
- **Known failures:** live deployment is not verified because PR deploy is intentionally skipped and the active branch is not approved for merge. Draft export/API/UI, full thread/category actions and non-Gmail review remain; existing accounts need re-consent and IMAP stays read-only. A clean full-chain SQLite apply fails in the pre-existing JT-019 migration before the new draft migration. Browser/provider/MariaDB/production unavailable or unverified; recovery scan performance is unmeasured at large ledger scale; Jest open handles; SEC-006 parser dependency work is still separately gated; parser isolation remains SEC-007.
- **Exact next action:** commit/push V-173, then implement one redacted owner inventory and readable ZIP export with manifest/checksums/missing-file warnings.
- **Exact next action:** complete full Chromium after the export addition, commit/push V-174, then add the disabled additive deletion state and coordinator using the same owner inventory.
- **Work that can continue independently:** SEC-009 repository-side owner inventory/export and disabled deletion lifecycle. UX/JOBS/PRODUCT production, MAIL provider mutations, SEC-006/007 and PROD packages retain their recorded external gates.
- **Decisions still required from the user:** none for synthetic/code-inspected repository work. Any provider connection or send test, internet/package upgrades, private data, external/paid providers and production actions retain explicit approval/safety gates; SEC-009 retention/legal policy remains unresolved.