Files
jobtrackingapp/docs/audits/user-journey-audit.md
T

75 lines
14 KiB
Markdown

# JobTracker user-journey audit
Audit date: 2026-08-02
## Evidence classification
- **Tested in a running browser:** repository Playwright test drove real Chromium against isolated running API/UI services.
- **Tested with mocked services:** backend unit tests or frontend Jest/RTL tests replaced external or HTTP boundaries.
- **Inspected only in code:** execution path was read but not operated end to end.
- **Blocked:** the required browser-control client, safe external provider, or a functioning prerequisite endpoint was unavailable.
- **Not applicable:** the product has no such workflow.
No screenshot references exist: interactive browser control failed before navigation because its required client module was absent. The four repository Playwright tests capture screenshots only on failure and all passed.
## Journey results
Each record includes persona, preconditions, steps, expected/actual results, classification, status, console/network evidence, screenshots, and related findings.
| ID | Journey | Persona / preconditions | Steps and expected result | Actual result | Classification | Status | Console errors / failed requests / screenshot | Findings |
|---|---|---|---|---|---|---|---|---|
| UJ-01 | Landing and public navigation | Logged-out visitor | Open landing; follow public navigation without gaining private access | Landing/routes inspected; public CV opened in real Chromium. Landing navigation itself was not interactively exercised. | Inspected only in code / browser for public CV | Partial | No captured console error; screenshot none | JT-018 |
| UJ-02 | Local sign-in and invalid credentials | Returning local user; isolated Playwright DB | Enter credentials; expect Dashboard and visible identity. Invalid credentials should stay unauthenticated with generic response. | Real Chromium login passed. Generic 401/lockout logic and UI error state pass unit/component tests; invalid login was not manually typed. | Running browser / mocked | Pass for valid login; partial for invalid | No Playwright failure; screenshot none | — |
| UJ-03 | Manual registration and email verification | New synthetic user; isolated API with verification required and email disabled | Register; expect a verification-required state and no authenticated access until confirmation | Registration returned 200, stored `EmailConfirmed=0`, but registration-issued session immediately accessed `/auth/me` (200). | Running API; email mocked/disabled | Fail | Failed expectation at auth lifecycle; no email sent; screenshot none | JT-007 |
| UJ-04 | Sign-out and session expiry | Returning synthetic user with copied pre-logout cookie | Log out; expect current server session/token to be unusable | Browser cookie was cleared (subsequent 401), but copied pre-logout cookie remained accepted (200). Natural expiry was inspected, not waited out. | Running API / code-inspected | Fail for revocation; partial for expiry | No console; requests 204, 401, and copied-session 200 | JT-008 |
| UJ-05 | Password reset/change and email change | Local account; verification-required disposable API | Change email/password or reset; require verification/recovery semantics and revoke compromised sessions | Email changed with 204 and stayed confirmed. Password reset/change revoke no session rows by code inspection. Reset email delivery was not invoked against real SMTP. | Running API for email / mocked+code for passwords | Fail/partial | No real email; screenshot none | JT-007, JT-008, JT-002 |
| UJ-06 | Google/Microsoft sign-in, linking, 2FA and recovery | Synthetic principals/mocked validators only | Validate provider identity, explicit linking, 2FA/recovery, failure handling | Extensive unit tests cover success/failure; no real provider or TOTP browser journey. Microsoft path trusts mutable email-like claims and weak issuer shape for auto-linking. | Mocked services / code-inspected | Partial | External providers intentionally not contacted | JT-001 |
| UJ-07 | Protected routes and unauthorized API | Logged-out visitor and User B | Open private API/route without valid owner/session; expect 401/404/403 | Unauthenticated API returned 401; User B got 404 for A's resources and 403 for admin. UI redirect logic inspected. | Running API / code-inspected UI | Pass at API level | No screenshot | — |
| UJ-08 | First-run empty account and onboarding | User B with no data/incomplete profile | Expect actionable empty state, profile/integration guidance, and resumable setup | B received empty job/company lists and 404 own profile. Onboarding/empty-state components pass mocked tests; no browser keyboard/navigation review. | Running API / mocked UI | Partial | CV list instead returned 500 even for empty B | JT-003, JT-014 |
| UJ-09 | Create a job manually | Returning Playwright user | Add Job → manual details → company/title → skip optional steps → create; expect saved job visible | Passed in real Chromium. Synthetic Unicode/Norwegian job also existed in local API data. | Running browser | Pass | No Playwright failure; screenshot none | — |
| UJ-10 | Edit/delete/move/search/filter/sort/return | Returning user with jobs | Modify a job, move stages, soft-delete/restore, search/filter/sort, navigate away/back, prevent accidental destructive actions | Component and controller tests cover these flows; code uses disabled save states and confirm helpers. Not manually exercised in browser. No idempotency key for repeated submissions. | Mocked services / code-inspected | Partial | No browser console/network capture | JT-014 |
| UJ-11 | Job workspace, notes, deadlines, contacts and follow-ups | User A owns job 1 | Open workspace; expect overview/assets/activity/AI and linked subpanels | `/workspace` returned 500 under default SQLite. `/timeline` and `/interview-prep` returned 500 due duplicate routes; checklist/analysis/match returned 200. | Running API | Fail | Failed requests: workspace/timeline/interview-prep 500 | JT-003, JT-004 |
| UJ-12 | Attachments | User A/B and synthetic text file | Upload/list/download/rename/delete with ownership and file/DB consistency | A upload/list/download passed; B copied file ID returned 404. Rename/delete inspected only. Failure paths can orphan files or desynchronise path/row. | Running API / code-inspected failure paths | Partial | No failed owner request; screenshot none | JT-010 |
| UJ-13 | Import job from URL | Authenticated user; no external site contacted | Validate URL, reject private networks, safely bound fetch, populate draft | Parser/SSRF unit tests pass and direct/private IP checks exist. Real site import blocked; response body is buffered before the four-megabyte check. | Mocked services / code-inspected | Partial | External fetch not performed | JT-011, JT-024 |
| UJ-14 | Career Profile and CV import review | User with synthetic CV/profile | Create/edit profile; import CV; review diffs; accept selected, reject others, edit before acceptance; preserve existing data | Profile/diff/pipeline backend and frontend tests are broad and approval is explicit. No manual browser run. CV run/list endpoints contain default-SQLite failures. | Mocked services / code-inspected; blocked browser | Partial/fail | `GET /api/profile-cv/runs` 500 in local runtime | JT-003, JT-014 |
| UJ-15 | CV variants, edit/reorder/hide/preview/export | User with career data | Create variant; edit/reorder/hide; preview; export; return without lost work | Public synthetic CV creation/publish/render/PDF passed in Chromium. Authenticated variant list returned 500; editor/autosave/DOCX-related paths were component/code-only. | Browser for public PDF / mocked and code-only for editor | Partial/fail | CV list 500; screenshot none | JT-003, JT-015 |
| UJ-16 | Deterministic match and AI assistance | User A with job/profile; cloud/local AI not configured | View match; generate suggestion; accept/reject/edit; handle empty/malformed/delay/failure; never auto-apply | Deterministic match returned 200. Mocked tests show AI results are suggestions/history and user approval is required. Usage/history fail on SQLite; no paid provider invoked. | Running API for match / mocked AI / blocked provider | Partial | `/api/ai/usage` and AI history 500 | JT-003, JT-022 |
| UJ-17 | AI trust and private-data boundary | Synthetic untrusted job/email text | Treat content as untrusted, communicate limitations, restrict payload to relevant job/profile/selected attachments | Prompts include guardrails and labelled source text; Markdown renders as React nodes. AI workspace sends the current job plus full master profile for most modules. No per-user global opt-out/provider-recipient explanation exists. Prompt-injection resilience was code-inspected, not adversarially provider-tested. | Code-inspected / mocked | Partial | External AI intentionally blocked | JT-022, JT-025 |
| UJ-18 | Connect/disconnect email provider | Synthetic provider mocks | Connect/cancel/reject/expire/disconnect safely and show state | Gmail/Graph/IMAP controller/provider/component tests cover mocked cases. No real OAuth or mailbox used. | Mocked services / code-inspected | Partial | No external requests | — |
| UJ-19 | Link/view/draft/cancel/send correspondence | Synthetic messages only | Associate correct job/category; view sent/received; draft/discard; require explicit send; archive/pin/read-later/spam/trash; render safely | Local correspondence ownership passed. Provider imports, category state, and explicit send actions covered by tests/code. No real message sent. Some requested mailbox categories are not implemented as a unified workflow. | Running API for local records / mocked provider / code-only | Partial | No real email; no screenshot | JT-014 |
| UJ-20 | Dashboard and analytics | Empty B and small synthetic A | Verify empty/data KPIs, counts, trends, follow-ups, dates/timezones, drill-down | Analytics services/tests cover calculations, but no manual browser KPI comparison against substantial data. One-job local dataset is insufficient for accuracy claims. | Mocked/code-inspected | Partial | Browser blocked; no screenshot | JT-014 |
| UJ-21 | Settings/preferences/AI controls | Authenticated user | Persist real preferences; enable/disable AI; select/understand provider/privacy; show usage | Theme/language/table settings persist client-side. Notification checkboxes only update localStorage and do not govern server reminders. AI usage returns 500 on SQLite; no global per-user AI disable/provider choice. | Code-inspected / running API for usage | Fail/partial | AI usage 500 | JT-003, JT-012, JT-022 |
| UJ-22 | Data export and account deletion | Authenticated user/admin | Export all user data; confirm irreversible deletion; remove DB/files/tokens/backups as documented | No self-service account deletion. Admin delete removes only Identity user. Downloadable backup is app-key-encrypted and omits career/CV/AI/provider/session data and file bytes. | Code-inspected | Fail | Not irreversibly executed | JT-009 |
| UJ-23 | Administrator | Normal User B; admin implementation present | Normal user must be denied; admin can safely manage users/system/audit | B got 403. Admin browser workflows and destructive admin delete were not run. | Running API for denial / code-inspected admin | Partial | 403 as expected | JT-009, JT-014 |
| UJ-24 | Edge cases and failure states | Synthetic users/data | Empty/invalid/long/Unicode, duplicates, double-click, refresh/back, tabs, slow/interrupted network, missing records, concurrent edits, unsupported/oversized uploads | Validation and many failure paths have tests; Unicode job stored. Unsupported/oversized upload guards inspected. Multi-tab, throttling, refresh mid-save, concurrent edit and double-click were not manually exercised; no optimistic concurrency tokens are evident. | Mocked/code-inspected; browser blocked | Partial | No screenshots/console; route failures above | JT-010, JT-011, JT-014 |
| UJ-25 | Responsive and accessibility | Keyboard-only and 375/768/1440px users | No clipping; readable contrast; named controls; visible focus; modal focus; usable tables/boards/previews/themes | Manual checks blocked. Code confirms unnamed icon buttons, keyboard-inaccessible CV cards, and fixed 210mm public-CV iframe. Positive MUI labels/dialog semantics also observed. | Inspected only in code / blocked browser | Fail for confirmed semantics; blocked visually | Screenshot none | JT-015 |
| UJ-26 | Two-user isolation | Disposable User A and B | B must not access A by UI or copied/guessed IDs | No disclosure in meaningful API results; several CV/AI/workspace routes blocked by 500s. UI-level navigation not run. | Running API / code-inspected UI | Pass/partial | Detailed matrix in evidence; screenshot none | JT-003, JT-004 |
## Persona coverage
| Persona | Coverage |
|---|---|
| Logged-out visitor | Browser public CV; unauthorized API; public route code inspection |
| New user | Registration/verification running API; onboarding mocked/code-only |
| Returning user | Real Chromium login and job creation |
| User with no data | User B running API; UI empty states mocked |
| User with substantial synthetic data | Blocked; only small disposable dataset used |
| User with incomplete profile | User B/API plus mocked UI |
| User encountering API failures | Running API failures captured; UI display mostly mocked |
| User with expired session | Natural expiry not waited; copied-session/logout behaviour tested |
| Keyboard-only/mobile-width user | Blocked; code inspection only |
| Administrator | Normal-user denial tested; admin UI blocked |
## Most serious journey failures
1. Default SQLite breaks CV lists/runs, AI history/usage, and the application workspace.
2. Timeline and interview-prep URLs are ambiguous and always fail before user logic.
3. Registration grants an authenticated session to an unverified address; profile email changes remain confirmed.
4. Logout does not revoke the server session represented by a copied cookie.
5. Rules, enrichment, reminders, and scheduled exports cannot see tenant rows in background scopes.
6. Notification preferences and full data-deletion/export expectations are not enforced by server behaviour.
## Browser limitation
The exact browser-plugin blocker and the four genuine Chromium workflows are recorded in `evidence/browser-evidence.md`. No other workflow is claimed as browser-tested.