651 lines
17 KiB
Markdown
651 lines
17 KiB
Markdown
# Jobjakt — Phase 9: Comprehensive Application QA and Regression Qualification
|
||
|
||
## Mission
|
||
|
||
Complete Phase 9 of the Jobjakt redesign.
|
||
|
||
Perform a comprehensive, evidence-based quality-assurance assessment of the entire application following Phases 1–8.
|
||
|
||
Identify and resolve confirmed in-scope regressions, validate existing functionality, test user journeys, assess accessibility and responsiveness, and produce a defensible acceptance decision.
|
||
|
||
This phase is not a new redesign.
|
||
|
||
The priorities are:
|
||
|
||
1. Functional correctness and data integrity.
|
||
2. Ease of use and workflow consistency.
|
||
3. Accessibility and responsive behaviour.
|
||
4. Security and user/tenant isolation.
|
||
5. CV/document/PDF reliability.
|
||
6. English and Norwegian Bokmål quality.
|
||
7. Performance and stability.
|
||
8. Release readiness and transparent risk reporting.
|
||
|
||
Work autonomously through bounded checkpoints, testing, documentation, commits and remote pushes.
|
||
|
||
Do not begin Phase 10 or new integration features.
|
||
|
||
Do not deploy production.
|
||
|
||
## 1. Confirmed Phase 8 baseline
|
||
|
||
Phase 8 has completed its implemented scope.
|
||
|
||
- Final branch: `feature/phase8-dashboard-customisation`
|
||
- Final commit: `1c24e406f194f0f42277d837b3008a7a28dca26a`
|
||
- Phase 8: CONDITIONAL GO.
|
||
- Phase 9 readiness: CONDITIONAL GO.
|
||
- Production: NO-GO.
|
||
- Development remote matched HEAD.
|
||
- Working tree was clean.
|
||
|
||
Reported passing checks:
|
||
|
||
- Backend: 1,084.
|
||
- Frontend: 917.
|
||
- Complete browser suite: 360.
|
||
- Separate browser contracts: 23.
|
||
- AI sidecar: 56.
|
||
- Sizing/performance diagnostics: 3.
|
||
|
||
Accepted runs had zero failures or skips.
|
||
|
||
All 33 routes remain present.
|
||
|
||
All 57 functionality-preservation groups were reconciled, but their statuses were 56 PARTIAL and C11 DEFERRED.
|
||
|
||
Existing risk items include:
|
||
|
||
- 64 incomplete axe accessibility findings requiring triage/manual assessment.
|
||
- Outstanding real-device and manual accessibility checks.
|
||
- Database-provider runtime verification, particularly MariaDB.
|
||
- Unprofiled sustained performance.
|
||
- Pre-existing duplicate reminder read.
|
||
- Outstanding live advisories.
|
||
- Previously deferred P5-13 AI qualification.
|
||
- Existing production release blockers.
|
||
|
||
Read:
|
||
|
||
`docs/redesign/phase-8/p8-05/FINAL_ACCEPTANCE.md`
|
||
|
||
Read the Phase 9 handoff and the preceding redesign's authoritative preservation and acceptance records.
|
||
|
||
Do not treat unresolved requirements as passed.
|
||
|
||
## 2. Git and execution boundaries
|
||
|
||
Verify the latest accepted Phase 8 commit and remote state.
|
||
|
||
Create:
|
||
|
||
`feature/phase9-full-qa`
|
||
|
||
Base this branch on the accepted Phase 8 development HEAD.
|
||
|
||
Do not branch from the older production main if main does not contain the redesign.
|
||
|
||
Preserve existing working-tree changes and branch history.
|
||
|
||
Commit and push validated checkpoints to the Phase 9 feature branch.
|
||
|
||
Verify every remote push.
|
||
|
||
Do not force-push.
|
||
|
||
Do not merge to main.
|
||
|
||
Do not deploy production.
|
||
|
||
## 3. Establish the QA inventory
|
||
|
||
Before testing, construct a complete inventory from the actual application.
|
||
|
||
Include:
|
||
|
||
- All 33 routes.
|
||
- All 57 functionality-preservation groups.
|
||
- Existing user roles.
|
||
- All major APIs.
|
||
- Backend/database workflows.
|
||
- Frontend page components.
|
||
- Shared design-system components.
|
||
- AI sidecar contracts.
|
||
- CV/document rendering.
|
||
- Dashboard preferences.
|
||
- Authentication and authorisation.
|
||
- Admin-only functionality.
|
||
|
||
Identify the existing automated coverage associated with each major workflow.
|
||
|
||
Create a QA matrix connecting:
|
||
|
||
Route → Workflow → Feature → Test evidence → Known risks → Acceptance status.
|
||
|
||
Do not invent coverage where no tests exist.
|
||
|
||
## 4. End-to-end user journey testing
|
||
|
||
Test the application as a real job seeker would use it.
|
||
|
||
Cover:
|
||
|
||
### Authentication and account management
|
||
Registration, verification, login, logout, password recovery, session management, account security and error handling.
|
||
|
||
### Onboarding
|
||
New-user flows, incomplete onboarding, resuming onboarding, localisation and navigation.
|
||
|
||
### Dashboard
|
||
Statistics, widget visibility, ordering, sizing, persistence, conflicts, reset and responsive layouts.
|
||
|
||
### Jobs
|
||
Job creation, editing, statuses, table, Kanban, searching, filtering, sorting, timelines, reminders and deletion where applicable.
|
||
|
||
### Job Detail
|
||
Navigation, associated documents, history, actions and data preservation.
|
||
|
||
### Career Profile
|
||
Canonical data editing, profile sections, validation, saving, recovery and relationships to CVs.
|
||
|
||
### CV Builder
|
||
Templates, editing, section ordering, content overrides, revisions, preview, exports and document fidelity.
|
||
|
||
### Cover letters and documents
|
||
Creation, editing, version handling, association with applications and safe persistence.
|
||
|
||
### Analytics
|
||
Data correctness, calculations, filters, empty states and dashboard relationships.
|
||
|
||
### Notifications
|
||
Read/unread state, duplicate handling, reminders, failure recovery and preferences.
|
||
|
||
### Settings
|
||
Profile, account, security, localisation, themes, preferences and other supported sections.
|
||
|
||
### Admin
|
||
Permissions, statistics, workflows, isolation and user management.
|
||
|
||
Test complete connected workflows, not just individual pages.
|
||
|
||
Preserve submitted/frozen document state and canonical Career Profile boundaries.
|
||
|
||
## 5. Visual and responsive QA
|
||
|
||
Inspect every significant route and user workflow across:
|
||
|
||
- Desktop.
|
||
- Tablet.
|
||
- Mobile.
|
||
- English.
|
||
- Norwegian Bokmål.
|
||
- Light theme.
|
||
- Dark theme.
|
||
|
||
Check:
|
||
|
||
- Consistent design system.
|
||
- Responsive layout.
|
||
- Sidebar and navigation.
|
||
- Dialogs and drawers.
|
||
- Tables and cards.
|
||
- Forms and validation.
|
||
- Chart readability.
|
||
- Touch interactions.
|
||
- Typography and spacing.
|
||
- Overflow and clipping.
|
||
- Loading states.
|
||
- Empty states.
|
||
- Error states.
|
||
- Long text and translations.
|
||
- Realistic dense datasets.
|
||
|
||
Use reproducible browser screenshots and DOM/accessibility evidence where useful.
|
||
|
||
Compare against established redesign decisions.
|
||
|
||
Fix confirmed in-scope regressions without introducing another full redesign.
|
||
|
||
Do not mark a route visually tested unless it was actually inspected.
|
||
|
||
## 6. Accessibility qualification
|
||
|
||
Investigate Phase 8's 64 incomplete axe findings.
|
||
|
||
These are not necessarily violations.
|
||
|
||
Deduplicate findings by underlying cause and affected component.
|
||
|
||
Classify them into:
|
||
|
||
- Confirmed accessibility defect.
|
||
- Requires manual inspection.
|
||
- Tool limitation.
|
||
- Not applicable, with evidence.
|
||
- Resolved.
|
||
|
||
Test:
|
||
|
||
- Keyboard-only navigation.
|
||
- Logical focus order.
|
||
- Visible focus.
|
||
- Dialog focus handling.
|
||
- Accessible names.
|
||
- Semantic structure.
|
||
- Screen-reader announcements.
|
||
- Dynamic status changes.
|
||
- Reordering controls.
|
||
- Drag-and-drop alternatives.
|
||
- Reduced motion.
|
||
- Zoom and text resizing.
|
||
- Colour contrast.
|
||
- Mobile touch targets.
|
||
|
||
Run automated checks across representative routes.
|
||
|
||
Perform genuine manual checks when the available testing environment supports them.
|
||
|
||
Do not claim NVDA, VoiceOver, physical-device or human accessibility testing passed unless those checks actually occurred.
|
||
|
||
Create a precise manual testing checklist for any checks requiring the product owner or another human tester.
|
||
|
||
Carry uncompleted checks into the release gates.
|
||
|
||
## 7. English and Norwegian Bokmål
|
||
|
||
Review every significant workflow.
|
||
|
||
Look for:
|
||
|
||
- Missing translation keys.
|
||
- English strings appearing in Norwegian mode.
|
||
- Inconsistent terminology.
|
||
- Literal or unnatural translations.
|
||
- Incorrect date/number formatting.
|
||
- Layout problems caused by longer text.
|
||
- Inconsistent validation messages.
|
||
- Missing accessibility labels.
|
||
|
||
Prefer natural professional Bokmål.
|
||
|
||
Preserve localisation architecture.
|
||
|
||
Do not silently replace established terminology without a clear reason.
|
||
|
||
## 8. Database and persistence qualification
|
||
|
||
Investigate supported database providers and configurations.
|
||
|
||
Include:
|
||
|
||
- EF Core migrations.
|
||
- SQLite development behaviour.
|
||
- MariaDB deployment compatibility.
|
||
- Database schema expectations.
|
||
- Relationships and constraints.
|
||
- Concurrent updates.
|
||
- Stale writes.
|
||
- Transaction handling.
|
||
- Preference persistence.
|
||
- Career Profile consistency.
|
||
- CV revision safety.
|
||
|
||
Use isolated test databases and synthetic data.
|
||
|
||
Where possible, run actual MariaDB integration testing in an approved non-production environment.
|
||
|
||
Do not claim MariaDB runtime acceptance from SQLite-only tests.
|
||
|
||
Do not modify the live production database.
|
||
|
||
Document any limitations where the provider cannot be tested safely.
|
||
|
||
## 9. Security and privacy
|
||
|
||
Verify:
|
||
|
||
- User and tenant isolation.
|
||
- API authorisation.
|
||
- Admin permissions.
|
||
- Session handling.
|
||
- Upload validation.
|
||
- Document ownership.
|
||
- Sensitive-data exposure.
|
||
- Error message leakage.
|
||
- AI endpoint access controls.
|
||
- URL import protections.
|
||
- Relevant rate limits.
|
||
- Data retention and deletion behaviour.
|
||
- Safe handling of untrusted documents and external content.
|
||
|
||
Run applicable existing security tests and static checks.
|
||
|
||
Do not expand into an uncontrolled security rewrite.
|
||
|
||
Record discovered high-risk issues and address those within the authorised QA scope.
|
||
|
||
Do not hide security failures or weaken assertions.
|
||
|
||
## 10. CV and PDF qualification
|
||
|
||
Test real output fidelity using synthetic CV data.
|
||
|
||
Cover:
|
||
|
||
- One-page CVs.
|
||
- Multi-page CVs.
|
||
- Long employment histories.
|
||
- Long bullet lists.
|
||
- Missing optional sections.
|
||
- Different templates.
|
||
- English and Norwegian.
|
||
- Long words and wrapping.
|
||
- Special characters.
|
||
- Page breaks.
|
||
- Header/footer behaviour.
|
||
- Margins.
|
||
- Fonts.
|
||
- Preview/export agreement.
|
||
- Version preservation.
|
||
|
||
Verify document pagination visually and programmatically where appropriate.
|
||
|
||
Do not infer PDF fidelity from frontend component tests alone.
|
||
|
||
Record renderer and environment differences.
|
||
|
||
## 11. Large-data and edge-case testing
|
||
|
||
Create realistic synthetic scenarios involving:
|
||
|
||
- No applications.
|
||
- Large numbers of applications.
|
||
- Long application timelines.
|
||
- Many CV revisions.
|
||
- Complex Career Profile histories.
|
||
- Missing or partial fields.
|
||
- Long job descriptions.
|
||
- Large filtering and sorting datasets.
|
||
- Many notifications.
|
||
- Slow APIs.
|
||
- Network interruption.
|
||
- Failed saves.
|
||
- Expired sessions.
|
||
- Concurrent edits.
|
||
- Missing or stale external references.
|
||
|
||
Verify that workflows remain usable and that data is not lost.
|
||
|
||
Do not load destructive or excessive test data into production.
|
||
|
||
## 12. Performance and stability
|
||
|
||
Go beyond small sizing diagnostics.
|
||
|
||
Assess representative workloads for:
|
||
|
||
- Page loading.
|
||
- Dashboard rendering.
|
||
- Job Table/Kanban performance.
|
||
- Filtering and search.
|
||
- Large Career Profiles.
|
||
- CV editing and preview.
|
||
- PDF generation.
|
||
- Preference saving.
|
||
- Backend database requests.
|
||
- AI sidecar interactions.
|
||
|
||
Investigate sustained performance under controlled, bounded load.
|
||
|
||
Measure actual resource usage and latency where possible.
|
||
|
||
Use isolated or approved test environments.
|
||
|
||
Do not benchmark destructively against live services.
|
||
|
||
Clearly distinguish measured performance from estimates.
|
||
|
||
Document resource constraints and untested production-scale behaviour.
|
||
|
||
## 13. Known issues and deferred requirements
|
||
|
||
Investigate the duplicate reminder read and other pre-existing advisories.
|
||
|
||
Revisit:
|
||
|
||
- Phase 7 conditional acceptance.
|
||
- Phase 8 conditional acceptance.
|
||
- 56 PARTIAL preservation statuses.
|
||
- C11 DEFERRED.
|
||
- Accessibility incompletes.
|
||
- Database-provider coverage gaps.
|
||
- Sustained performance qualification.
|
||
- P5-13 withholding.
|
||
- Existing production blockers.
|
||
|
||
For every item, determine:
|
||
|
||
- Current status.
|
||
- Reproducibility.
|
||
- User impact.
|
||
- Severity.
|
||
- Owner/next action.
|
||
- Whether it blocks Phase 9 acceptance.
|
||
- Whether it blocks production.
|
||
- Required evidence for closure.
|
||
|
||
Do not automatically reinterpret PARTIAL as PASS or DEFERRED as completed.
|
||
|
||
Record why each item is legitimately accepted, deferred or still blocked.
|
||
|
||
Do not change historic acceptance evidence retroactively.
|
||
|
||
## 14. Remediation policy
|
||
|
||
Phase 9 may fix confirmed regressions and defects discovered during QA.
|
||
|
||
Keep fixes:
|
||
|
||
- Bounded.
|
||
- Tested.
|
||
- Relevant to Phase 9.
|
||
- Compatible with existing architecture.
|
||
- Documented.
|
||
- Independently reviewable.
|
||
|
||
For major architectural changes or new product features, create a separate issue and explain the dependency.
|
||
|
||
Do not begin Phase 10 legacy cleanup.
|
||
|
||
Do not introduce LinkedIn, FINN, NAV or new AI model features.
|
||
|
||
Do not activate P5-13.
|
||
|
||
## 15. Suggested checkpoints
|
||
|
||
Use existing authoritative Phase 9 checkpoint definitions where available; otherwise:
|
||
|
||
### P9-00 — QA baseline and coverage matrix
|
||
|
||
Establish the current test baseline, route/feature coverage, risks and acceptance criteria.
|
||
|
||
### P9-01 — End-to-end functional QA
|
||
|
||
Test full user journeys, account ownership, persistence and application workflows.
|
||
|
||
### P9-02 — Responsive and visual QA
|
||
|
||
Inspect routes, themes, languages, mobile/tablet layouts and complex states.
|
||
|
||
### P9-03 — Accessibility and localisation
|
||
|
||
Triage incomplete findings, resolve confirmed issues and prepare human testing evidence.
|
||
|
||
### P9-04 — Data integrity, database and security
|
||
|
||
Validate providers, concurrency, user isolation, uploads, document ownership and security contracts.
|
||
|
||
### P9-05 — CV/PDF, edge cases and performance
|
||
|
||
Verify document fidelity, large datasets, failure recovery and bounded performance.
|
||
|
||
### P9-06 — Defect remediation and full regression
|
||
|
||
Resolve approved in-scope defects and run the complete relevant suites.
|
||
|
||
### P9-07 — Final acceptance and Phase 10 handoff
|
||
|
||
Consolidate evidence, remaining blockers, updated preservation statuses and recommended cleanup scope.
|
||
|
||
At each validated checkpoint:
|
||
|
||
1. Implement necessary QA tests and in-scope fixes.
|
||
2. Execute verification.
|
||
3. Record exact results.
|
||
4. Commit the checkpoint.
|
||
5. Push to the Phase 9 branch.
|
||
6. Verify remote synchronisation.
|
||
7. Continue autonomously.
|
||
|
||
If an unsafe or unapproved issue blocks a checkpoint, document it and continue independent safe work where possible.
|
||
|
||
## 16. Production restrictions
|
||
|
||
The current production environment remains NO-GO.
|
||
|
||
Previously reported blockers include:
|
||
|
||
- EXT4 filesystem errors on Docker storage.
|
||
- Missing independently verified off-disk backup at the last assessment.
|
||
- Restore and rollback qualification.
|
||
- Production ingress security.
|
||
- Database-provider/runtime validation.
|
||
- Manual accessibility requirements.
|
||
- Outstanding release advisories.
|
||
- AI model integrity and qualification.
|
||
|
||
Do not assume these issues have been resolved.
|
||
|
||
Do not:
|
||
|
||
- Merge into main.
|
||
- Deploy Jobjakt.
|
||
- Restart live Docker services.
|
||
- Modify production secrets.
|
||
- Run production migrations.
|
||
- Repair the production filesystem.
|
||
- Perform live data deletion.
|
||
- Activate Qwen3.5 or replace Qwen2.5.
|
||
- Change production AI inference configuration.
|
||
|
||
Safe local QA and isolated integration testing may continue.
|
||
|
||
Production-readiness investigation can be documented but does not authorise production remediation.
|
||
|
||
## 17. Documentation and evidence
|
||
|
||
Use established repository conventions.
|
||
|
||
Suggested location:
|
||
|
||
`docs/redesign/phase-9/`
|
||
|
||
Deliver:
|
||
|
||
- QA coverage matrix.
|
||
- Route and workflow verification.
|
||
- Visual/responsive evidence.
|
||
- Accessibility findings and manual checklist.
|
||
- Localisation results.
|
||
- Database/provider validation.
|
||
- Security findings.
|
||
- CV/PDF fidelity evidence.
|
||
- Performance results.
|
||
- Defect register.
|
||
- Updated preservation matrix.
|
||
- Full regression results.
|
||
- Release risk register.
|
||
- Phase 10 cleanup eligibility list.
|
||
- Final acceptance report.
|
||
|
||
Suggested final report:
|
||
|
||
`docs/redesign/phase-9/p9-07/FINAL_ACCEPTANCE.md`
|
||
|
||
Maintain original evidence and stable issue IDs.
|
||
|
||
Separate PASS, FAIL, PARTIAL, DEFERRED and NOT TESTED.
|
||
|
||
## 18. Final acceptance
|
||
|
||
Provide independent decisions:
|
||
|
||
**Phase 9 QA:**
|
||
GO / CONDITIONAL GO / NO-GO.
|
||
|
||
**Phase 10 readiness:**
|
||
GO / CONDITIONAL GO / NO-GO.
|
||
|
||
**Production release:**
|
||
GO / NO-GO.
|
||
|
||
Do not claim production readiness solely because QA tests pass.
|
||
|
||
Do not claim a manual check passed without actual human/manual evidence.
|
||
|
||
Do not claim a database provider passed if it was not exercised.
|
||
|
||
Clearly distinguish development acceptance from production release approval.
|
||
|
||
## 19. Required final response
|
||
|
||
Report:
|
||
|
||
1. Completed QA checkpoints.
|
||
2. Routes and workflows tested.
|
||
3. Actual backend/frontend/browser/contract/sidecar results.
|
||
4. New regression tests.
|
||
5. Confirmed defects found and resolved.
|
||
6. Remaining defects and severity.
|
||
7. Accessibility outcomes and outstanding human checks.
|
||
8. Localisation findings.
|
||
9. Database-provider verification.
|
||
10. Security and tenant-isolation outcomes.
|
||
11. CV/PDF rendering results.
|
||
12. Performance results.
|
||
13. Preservation-group status changes.
|
||
14. C11 deferred status.
|
||
15. Phase 10 cleanup eligibility.
|
||
16. Final branch and commit.
|
||
17. Remote push verification.
|
||
18. Phase 9 decision.
|
||
19. Phase 10 readiness decision.
|
||
20. Production decision.
|
||
|
||
## 20. Final execution instruction
|
||
|
||
Begin Phase 9 from the accepted Phase 8 development baseline:
|
||
|
||
`1c24e406f194f0f42277d837b3008a7a28dca26a`
|
||
|
||
Create:
|
||
|
||
`feature/phase9-full-qa`
|
||
|
||
Complete comprehensive application QA autonomously through bounded checkpoints.
|
||
|
||
Fix confirmed, authorised in-scope regressions.
|
||
|
||
Preserve all existing features and data contracts.
|
||
|
||
Maintain transparent unresolved-risk tracking.
|
||
|
||
Commit and push each validated checkpoint.
|
||
|
||
Prepare a complete Phase 10 handoff.
|
||
|
||
**Do not merge into main.**
|
||
|
||
**Do not deploy production.**
|
||
|
||
**Do not begin Phase 10 automatically.** |