docs(rd21): record reproduced data loss and remediation scope

This commit is contained in:
cesnimda
2026-10-23 07:56:04 +02:00
parent 8e7e3f6387
commit 8f2cad67f7
4 changed files with 507 additions and 0 deletions
@@ -0,0 +1,86 @@
# RD21 remediation plan
Spec: [REQUEST.md](REQUEST.md). Baseline: `8e7e3f6387544e24082ebe2c107c0b868897959e`.
## Global constraints
Keep historical Phase 9 evidence unchanged. Use synthetic local databases, existing providers,
EF concurrency tokens and existing client error handling. No production actions, main merge,
new dependencies, silent stale-write retries or historical recovery claims.
The reproduced failure is persisted full-snapshot replacement without a caller precondition.
Career Profile additionally commits canonical state before its Identity projection. The repair
must compare the revision the editor actually read and atomically guard the eventual database
write. Missing preconditions fail closed. Schema format `profile.version` is not a revision.
## Task 1: Career backend and persistence tests
Own CareerProfileService, CareerProfileController, ProfileCvController partials, and dedicated
RD21 Career tests. Enumerate canonical import/parse/review/restore callers before changing them.
Add deterministic RED tests before implementation. Require expected canonical revision on
editor replacement, restore, parse and reviewed import acceptance. Return a persistence revision
with the coherent source snapshot/diff. Guard EF updates with existing CareerProfile.Version;
serialize competing first saves using the existing unique owner constraint plus safe conflict
translation. Keep children, history, legacy projection and import application metadata in one
transaction. Failed Identity/DB updates roll back everything. No failed operation can be saved
later through a dirty tracker. Preserve read-only public loads, tenant filters, stable IDs,
CV inheritance and frozen outputs. Test independent contexts, stale restore, first saves,
transaction rollback, and canonical/projection consistency. Notify coordinator of shared
DbContext configuration required; do not edit DbContext or migrations (Task 2 owns those).
## Task 2: Job revision infrastructure and consumers
Own JobApplication model, DbContext, migrations, generic job PUT/DTO and affected job writers,
API client and job UI callers, plus dedicated job regression tests. First write a failing
SQLite persistence test reproducing stale acknowledged job replacement. Add an application
managed integer revision; advance it for every tracked JobApplication mutation, including
background/legacy writers, with EF atomic original-revision comparison. Generic full-snapshot
PUT requires the caller revision. Preserve 204 compatibility while returning the committed
revision in a response header for sequential callers. All production snapshot callers carry
the revision of their actual source; never fetch a new revision to bless an old snapshot.
Surface conflicts and retain drafts. Inspect cover letter, package drafts, timeline and other
overlapping writes; guard directly related destructive stale replacements and document distinct
unrelated findings. Preserve company ownership validation, opportunity dual writes and events.
Generate migration only against local development infrastructure; test upgrade and rollback
without touching real user data. Integrate Task 1's Career concurrency configuration here.
## Task 3: Frontend Career consumers and deterministic UI tests
After Task 1 contract is known, update Career editor/import/restore/parse call sites and test
fixtures. Hold the source revision with the draft. Disable overlapping conflicting actions or
serialize accepted saves without losing edits made during the request. Out-of-order reads and
responses cannot replace newer local edits. A rejected/failed save retains the editable draft;
recovery requires deliberate reload/review, never blind automatic overwrite. Add focused tests
for conflict/network failure, in-flight editing, and delayed response handling.
## Task 4: Integration, qualification and delivery
Run unchanged historical characterization and separate stronger browser safety tests: exact
original ordering, immediate independent database read, refresh, navigation, new authenticated
session, rapid saves, concurrent requests and recoverable rejection. Run targeted native/Jest
tests before full backend, frontend, browser, contract, sidecar, diagnostics, type/lint/build
gates. Serialize build/browser lanes and freeze source identity for final evidence. Probe local
MariaDB/disposable Docker availability without external calls or production mutations; record
unavailable qualifications honestly. Review tenant/security regressions and preservation matrix.
Obtain independent code review, fix actionable findings, write dated acceptance addendum with
separate RD21/Phase 9/Phase 10/production decisions and exact non-overlapping totals. Commit
logical checkpoints, push the dedicated branch and verify remote SHA and clean status.
## Preflight and rulings
| Tasks | Shared interface / consistency check | Ruling |
| --- | --- | --- |
| 1 / 2 | Career.Version EF configuration belongs to DbContext | Task 1 specifies; Task 2 edits shared file and generates migration. |
| 1 / 3 | Career DTO and import precondition | Task 3 waits for explicit contract; no guessed auto-refresh fallback. |
| 2 / 3 | Shared API client | Task 2 owns job methods; Task 3 edits only Career page/tests unless coordinated. |
| 1 / 4 | Service tests vs HTTP contract | Both persisted state and fail-closed HTTP behavior required. |
| 2 / 4 | Legacy 204 and revision advancement | Keep status, expose committed revision; characterize unchanged old test separately. |
| 3 / 4 | UI optimism vs acknowledged persistence | Acknowledgement only after successful response; draft retention independently tested. |
| 1 | Existing internal test setup callers | Update callers explicitly; no production blind-save overload. |
| 2 | All tracked writers vs partial endpoints | Token prevents overlapping server writes; caller preconditions required for stale snapshot replacements. |
| 3 | Draft revision vs server response ordering | Never attach a later revision to an older draft automatically. |
| 4 | Broad suites vs bounded provider/manual evidence | Preserve 56 PARTIAL/C11 DEFERRED until their actual criteria pass. |
Ruling: use the user-requested dedicated branch in the existing clean checkout. Extra worktree
creation and routine approval pauses are unnecessary; autonomous execution and pushes are
explicitly authorized in REQUEST.md.
@@ -0,0 +1,333 @@
# Jobjakt — RD21 Critical Data-Loss Remediation
## Phase 9 Blocking Defect · Root Cause, Repair and Requalification
### Mission
Resolve the Phase 9 blocking defect RD21: acknowledged Career Profile and job edits can subsequently be lost.
This is a high-priority data-integrity defect.
The goal is to eliminate the root cause, prove that acknowledged changes are durably persisted, prevent stale writes from silently replacing newer data, and obtain sufficient regression evidence to reassess Phase 9 acceptance.
Work autonomously through safe investigation, implementation, testing, documentation, commits and remote pushes.
Do not start Phase 10, deploy production or implement unrelated features.
## 1. Confirm the baseline
The final Phase 9 state is:
- Branch: `feature/phase9-full-qa`
- Commit: `8e7e3f6387544e24082ebe2c107c0b868897959e`
- Phase 9: NO-GO.
- Phase 10 readiness: NO-GO.
- Production: NO-GO.
- Working tree clean and remote verified at handoff.
Reported passing tests:
- Backend: 1,084.
- Frontend: 919.
- Browser: 362.
- Browser contracts: 23.
- AI sidecar: 56.
- Diagnostics: 18 + 3.
Read:
`docs/redesign/phase-9/p9-07/FINAL_ACCEPTANCE.md`
Locate all RD21 reproduction evidence, logs, issue records and related tests.
Inspect the exact reproduction rather than inferring the cause from the issue title.
## 2. Branching
Create a dedicated branch:
`fix/rd21-acknowledged-edit-loss`
Base it on the accepted Phase 9 development HEAD containing the commit above.
Do not branch from older production `main`.
Preserve the original Phase 9 acceptance report and its historical NO-GO decision.
Do not force-push or modify unrelated branches.
## 3. Reproduce and isolate RD21
Before implementing a fix:
1. Locate the exact known RD21 reproduction sequence.
2. Reproduce it in a controlled development environment.
3. Identify which Career Profile and job workflows are affected.
4. Capture request/response ordering and relevant revision information.
5. Determine when the application reports a successful save.
6. Verify what the database actually contains immediately afterward.
7. Verify what remains after refresh, navigation and a new session.
8. Distinguish frontend display inconsistencies from actual persisted data loss.
Determine whether the problem is caused by one mechanism affecting multiple workflows or separate defects.
Do not weaken the reproduction test or change its conditions simply to obtain a pass.
Use synthetic data.
## 4. Investigate the complete save lifecycle
Trace each affected workflow end-to-end:
User edit → Client draft/state → Save request → API validation → Authorization → Database transaction → Durable commit → Server response → Client acknowledgement → Subsequent reads.
Investigate potential causes, including:
- Stale frontend state.
- Overlapping requests.
- Autosave and explicit-save races.
- Debounced requests completing out of order.
- Optimistic UI acknowledgements.
- Lost or ignored request failures.
- Stale API responses overwriting newer client state.
- Concurrent updates.
- Missing revision/concurrency checks.
- Background refresh or cache invalidation.
- Navigation/unmount timing.
- Database transaction behaviour.
- Partial aggregate updates.
- Incorrect mapping between API DTOs and persisted entities.
- Shared abstractions affecting Career Profile and jobs.
These are investigation hypotheses, not assumed findings.
Find the actual root cause and support it with evidence.
## 5. Implement a durable fix
Requirements:
- A successful-save acknowledgement must correspond to confirmed successful persistence.
- Failed saves must not be presented as successful.
- The latest user-approved change must not be silently replaced by stale data.
- Concurrent updates must be handled predictably.
- Revision conflicts must be surfaced appropriately.
- Failed operations must retain recoverable user edits.
- Related fields must not be inadvertently cleared by partial updates.
- Correct user and tenant ownership must be preserved.
- Existing Career Profile canonical-data and CV-specific override rules must remain intact.
- Submitted/frozen documents must not be unintentionally modified.
- Job timelines and associations must remain consistent.
Prefer a focused correction to the existing save infrastructure.
If shared infrastructure is defective, fix the common cause rather than applying unrelated UI workarounds.
Do not add arbitrary delays or unbounded retries to hide a race condition.
Do not change save semantics without testing all affected consumers.
## 6. Regression tests
Create deterministic tests that reproduce the original RD21 failure before accepting the fix.
Include suitable coverage for:
- Career Profile edits.
- Job edits.
- Consecutive rapid edits.
- Overlapping saves.
- Delayed and out-of-order responses.
- Navigation immediately after save acknowledgement.
- Refresh after save.
- Logout/login or independent session.
- Failed network requests.
- Database transaction failures.
- Concurrent editing in two sessions.
- Conflict detection and recovery.
- Partial updates and unchanged fields.
- Cross-tenant isolation.
- Existing CV and document revision behaviour.
Tests must establish persisted database state, not merely successful API status codes or optimistic UI state.
Use controlled concurrency and deterministic synchronisation where possible, avoiding timing-dependent flaky assertions.
## 7. Database-provider verification
Run the relevant tests against the existing supported development provider.
Also verify MariaDB using an isolated disposable test instance if the environment supports it.
Do not claim MariaDB compatibility from SQLite-only evidence.
Do not access or mutate the production database.
If MariaDB cannot be exercised, record it as a specific outstanding qualification requirement.
## 8. Broader regression review
Investigate whether similar save patterns exist in:
- Dashboard preferences.
- Settings.
- CV Builder.
- Cover letters.
- Documents.
- Application timelines.
- Other editable user-owned records.
Do not assume they are affected.
Use code inspection and targeted tests to establish whether the RD21 root cause extends beyond the original affected workflows.
Fix directly related vulnerabilities to user-data integrity within this bounded remediation scope.
Record unrelated findings separately.
## 9. Verification
After fixing RD21:
- Run the original reproduction.
- Repeat it across multiple deterministic test cases.
- Run affected backend and frontend tests.
- Run applicable browser tests.
- Run provider and tenant-isolation checks.
- Run the complete regression suite.
- Run type checking, lint and build.
- Run applicable security checks.
- Reconcile the functionality preservation matrix.
Record exact commands, results, failures and limitations.
Do not conceal failures through skips or weakened assertions.
## 10. Phase 9 requalification
Once the fix is validated, reassess Phase 9 against the original acceptance requirements.
Distinguish:
A. RD21 resolved.
B. Phase 9 functional QA readiness.
C. Remaining manual accessibility/device checks.
D. Provider/database qualification.
E. Phase 10 readiness.
F. Production release readiness.
RD21 resolution must not automatically convert every outstanding PARTIAL or DEFERRED preservation item into PASS.
Treat the existing 56 PARTIAL groups and C11 DEFERRED according to their authoritative acceptance criteria.
Update findings only where new evidence justifies doing so.
Preserve the original Phase 9 NO-GO report and create a dated remediation/requalification addendum.
## 11. Documentation
Create an RD21 remediation directory using repository conventions, for example:
`docs/redesign/phase-9/rd21-remediation/`
Include:
- Original reproduction.
- Root-cause analysis.
- Affected workflows.
- Implemented fix.
- Concurrency/save semantics.
- Before/after regression evidence.
- Database verification.
- Full test results.
- Preservation impact.
- Remaining risks.
- Phase 9 acceptance addendum.
- Phase 10 handoff decision.
Document any recovery limitations for edits that may already have been lost. Do not claim historical data can be recovered without evidence.
## 12. Autonomous execution
Work autonomously through:
1. Baseline and reproduction.
2. Root-cause investigation.
3. Implementation.
4. Targeted regression tests.
5. Broad regression qualification.
6. Acceptance addendum and handoff.
Commit completed logical checkpoints.
Push to:
`fix/rd21-acknowledged-edit-loss`
Verify remote synchronisation and clean working tree.
Do not stop for routine low-risk decisions.
Escalate destructive operations, significant scope changes or mandatory external approvals.
## 13. Release boundaries
The live server remains production NO-GO.
Do not:
- Merge into `main`.
- Deploy production.
- Run production migrations.
- Restart live services.
- Modify live databases.
- Perform filesystem repairs.
- Change live Ollama models.
- Start Phase 10.
- Begin LinkedIn, FINN/NAV or new AI features.
The production filesystem, backup, restore, security and manual acceptance requirements remain independently gated.
## 14. Final acceptance
Report separate decisions:
**RD21:** RESOLVED / UNRESOLVED.
**Phase 9:** GO / CONDITIONAL GO / NO-GO.
**Phase 10 readiness:** GO / CONDITIONAL GO / NO-GO.
**Production:** GO / NO-GO.
Include:
- Exact root cause.
- Reproduction evidence.
- Code changes.
- New regression tests.
- Actual full test results.
- MariaDB qualification status.
- Remaining defects and risks.
- Preservation status.
- Branch name and final commit.
- Remote verification.
- Phase 10 recommendation.
### Final instruction
Begin by reproducing RD21.
Fix the proven underlying cause, not the visible symptom.
Validate that an acknowledged edit remains durably saved after subsequent requests, navigation, refresh and independent-session reads.
Complete all safe remediation and requalification checkpoints autonomously.
Push validated work to the dedicated fix branch.
**Do not merge to `main` or begin Phase 10 automatically.**
@@ -0,0 +1,67 @@
# RD21 investigation (23 October 2026)
Status: repair and qualification in progress; this is not acceptance.
## Original failure
The unchanged Phase 9 characterization was executed on the exact baseline
`8e7e3f6387544e24082ebe2c107c0b868897959e`. Both independently authenticated
editors read the same synthetic record. Editor A changes and successfully saves one field.
Editor B changes a different field in its older complete snapshot. B's request is accepted and
replaces A's field with its original value. Subsequent authenticated reads show the loss.
See [original-reproduction.json](original-reproduction.json). Career stale response: 200;
job stale response: 204. Both `lostFirstEdit` values are true. The characterization's one passing
test establishes that the unsafe behavior was reproduced; it is not a safety pass.
New native SQLite tests additionally query persisted rows through independent DbContexts
immediately after acknowledgement. The original job safety assertion failed because the stale
request returned `NoContentResult` instead of a conflict. Career safety tests failed because the
stale request returned success and because an injected Identity failure left canonical rows
committed despite the failure response. These are separate deterministic regression assertions.
## Mechanisms
1. Generic job PUT reads the current row, maps the client's complete replacement over it and
calls SaveChanges. The request has no source revision; the entity has no concurrency token.
A fresh request context therefore accepts an arbitrarily old snapshot. This is persisted data
replacement, independently of client display state. The optional awaited summary generation
also creates an interval between read and write; checking only at controller entry is unsafe.
2. Career PUT and restore similarly lack an editor revision. The canonical service increments
Version but does not compare the editor's version or use it as an EF concurrency token.
Children and history are transactionally saved, then the transaction commits. The controller
updates ApplicationUser.ProfileCvStructureJson/CvText afterward. Identity failure can leave
canonical and legacy projections inconsistent despite a failed operation response.
3. Career parse and reviewed import acceptance also call the canonical replacement service and
update the projection/application metadata separately. They must participate in the same guard
and transaction; fixing only manual PUT would leave alternate replacement paths open.
## Directly related text writers
SQLite tests reproduce the same persisted replacement through cover-letter save and application
drafts: a first acknowledged `First approved text` becomes `Stale snapshot text` after a later
request from the stale editor. These routes replace fields also writable through the generic job
snapshot. They are included in the bounded repair. Cover-letter history alone does not reject
stale approval or keep the current approved text intact. Legacy tailored-CV text replacement is
also an application-owned snapshot writer and must fail closed without its source revision.
## Client lifecycle
Career has an existing synchronous mutation guard and disables its editor while saving. Its error
paths retain drafts, but it cannot detect another session's change without server revision data.
The shared job dialog does not guard older GET responses, and remains editable/closable during
PUT; a successful submitted snapshot can close a dialog containing later unsent edits. The
frontend correction will preserve the revision belonging to each draft, block conflicting
in-flight actions, reject old responses, and display recoverable conflicts without blind retry.
## Provider limitation
The read-only `rtk proxy docker info --format '{{json .ServerVersion}}'` probe on 23 October
failed: `dockerDesktopLinuxEngine` named pipe does not exist. No daemon installation/restart was
attempted. MariaDB runtime qualification is outstanding; SQLite and provider script generation
cannot substitute for a MariaDB persistence/concurrency run.
## Historical recovery
No evidence establishes recovery of edits already overwritten before this repair. Canonical
history may contain a prior profile snapshot, but availability and user intent must be examined
per record. No production database has been accessed and no historical recovery is claimed.
@@ -0,0 +1,21 @@
{
"id": "RD21",
"career": {
"staleStatus": 200,
"expectedFirstSummary": [
"First editor committed summary"
],
"actualSummary": [
"Initial summary"
],
"lostFirstEdit": true
},
"job": {
"staleStatus": 204,
"expectedFirstTitle": "First editor committed title",
"actualTitle": "Original job title",
"lostFirstEdit": true
},
"safetyQualification": "FAIL: stale writes can overwrite acknowledged edits",
"limitation": "This passing characterization records observed behavior; it does not accept unsafe stale writes or provide a new revision contract. Creation-intent idempotency remains separate."
}