diff --git a/docs/audits/evidence/qa-001/README.md b/docs/audits/evidence/qa-001/README.md new file mode 100644 index 0000000..3fb1f49 --- /dev/null +++ b/docs/audits/evidence/qa-001/README.md @@ -0,0 +1,9 @@ +# QA-001 evidence index + +Synthetic/local evidence only. No provider calls, private job descriptions, credentials or personal data were used. + +- Implementation and limits: `docs/verification/qa-001-job-term-quality.md` +- Commands/results: `docs/audits/verification-log.md` V-114–V-116 +- Seven deterministic fixtures: `JobTrackerApi.Tests/JobCvMatchServiceTests.cs` +- UI label regression: `job-tracker-ui/src/match-score-panel.test.tsx` +- Browser screenshots: not captured; required presentation checks remain explicit in the master plan. diff --git a/docs/audits/verification-log.md b/docs/audits/verification-log.md index 27dff52..7d7f9d2 100644 --- a/docs/audits/verification-log.md +++ b/docs/audits/verification-log.md @@ -145,3 +145,6 @@ Output was reduced to filenames and commit counts. The token artifact appears un | V-111 | Complete trace of `themePrefs`, auth user-key transitions, App provider/router lifecycle, Settings selector, Next layout/bootstrap and existing tests | Repository root | Reproduce UX-002 precedence and delayed switch path | PASS — login stored the resolved user key without a theme notification, so anonymous mode remained until refresh; provider key/router dependency remounted app state; no cross-tab listener or pre-paint bootstrap existed | Confirmed source execution path; no backend/identity behavior implicated | Application-related | | V-112 | `npm.cmd test -- --runInBand --forceExit src/theme-state.test.tsx`; full `npm.cmd test -- --runInBand --forceExit`; `npm.cmd run build`; `git diff --check` | `job-tracker-ui` / repository root | Verify precedence, login/logout events, System behavior, pre-paint bootstrap, in-place provider state, regressions and TypeScript | PASS — focused 6/6; full 48/48 suites and 172/172 tests; production build and patch hygiene pass | Initial build found unsupported `noSsr` prop and was corrected; Jest retains existing force-exit/open-handle notice | Application issue corrected before commit | | V-113 | In-app browser Settings: Light → Dashboard → refresh; Dark; System; second-tab Dark → Light sync; 375/768/1440 measurements/screenshots; console inspection | Local frontend | Verify actual theme transitions, persistence, cross-tab behavior and responsive access | PASS/PARTIAL — expected scheme at every transition, first tab updated without reload, no final console warnings/errors, no horizontal overflow after scrollable Settings tabs | Authenticated two-user, host OS preference flip and production not run; transient HMR messages occurred while editing and clean build passes | Environmental/deployment limitation | +| V-114 | Complete trace of import HTML cleanup/language/tagging, deterministic matcher, analysis endpoint/UI, match endpoint/UI, learning-item sync and storage/cache behavior | Repository root | Reproduce QA-001 low-information term path and version implications | PASS — visible terms come from English-only single-token ranking in `JobCvMatchService`; imported HTML is cleaned earlier but manual text may not be; result is on-demand and has no historical analysis cache; AI is not involved | Confirmed source execution path | Application-related | +| V-115 | `dotnet test ... --filter "...JobCvMatchServiceTests|...ApplicationIntelligenceTests|...JobApplicationsEndpointBehaviorTests|...ApplicationChecklist"`; `npm.cmd test ... match-score-panel.test.tsx application-intelligence.test.tsx`; final matcher/UI focused rerun | Repository root / `job-tracker-ui` | Verify seven required fixtures, affected endpoints/learning lifecycle and honest label | PASS — affected backend 54/54; affected UI 13/13; final matcher 15/15 and label 3/3 | Synthetic job/CV text only; no model/provider/browser | N/A | +| V-116 | Full `dotnet test ... --no-restore`; full `npm.cmd test -- --runInBand --forceExit`; `npm.cmd run build`; `git diff --check` | Repository root / `job-tracker-ui` | QA-001 full regression, TypeScript/build and patch hygiene | PASS — backend 601/601; frontend 48/48 suites and 172/172 tests; production build; no whitespace errors | Jest existing force-exit/open-handle notice; browser/production presentation not run | Environmental/deployment limitation | diff --git a/docs/verification/application-action-matrix.md b/docs/verification/application-action-matrix.md index 6cff0dd..225841b 100644 --- a/docs/verification/application-action-matrix.md +++ b/docs/verification/application-action-matrix.md @@ -45,5 +45,6 @@ This is the rolling action-level evidence index. `PASS (automated/runtime)` is n | CV processing | retry provenance, durable refresh state and cancel/retry controls | PASS (backend + component tests) | BLOCKED | NOT RUN | `ai-004-cv-processing-queue.md` | | Authentication UI | unified username/password and provider alternatives; invalid/cancel/return behavior | PASS (components; mocked providers) | PARTIAL — local dark-theme form at 375/768/1440 | NOT RUN | `ux-001-unified-authentication.md` | | Theme state | Light/Dark/System precedence, login/logout scope, refresh/navigation and two-tab synchronization | PASS (state/provider/bootstrap tests) | PASS/PARTIAL — anonymous local browser at 375/768/1440 | NOT RUN | `ux-002-deterministic-theme-state.md` | +| Job match terms | bilingual/noisy/short/technology-heavy/filler extraction and honest labels | PASS (seven fixtures + component tests) | NOT RUN | NOT RUN | `qa-001-job-term-quality.md` | Remaining product actions are `NOT STARTED` in the master plan and will be added as their work packages enter verification. The in-app browser is available for local UI checks; configured/real-provider and production access are not documented/configured. diff --git a/docs/verification/qa-001-job-term-quality.md b/docs/verification/qa-001-job-term-quality.md new file mode 100644 index 0000000..27ea435 --- /dev/null +++ b/docs/verification/qa-001-job-term-quality.md @@ -0,0 +1,52 @@ +# QA-001 job-analysis and important-term quality + +Updated: 2026-08-09 + +Status: `IMPLEMENTED — NOT VERIFIED`. Deterministic fixtures, backend/frontend regressions and production build pass. Browser and production presentation checks remain. + +## Revalidated execution path + +- URL imports already convert structured job-posting HTML to text and run lightweight Norwegian/English language detection plus `SkillTagger`. Manual descriptions and notes can still contain HTML/noise. +- The visible problematic terms originate in `JobCvMatchService`: curated skills are combined with frequency-ranked single tokens from description, translated description and notes. Its stop list was English-only and did not clean raw HTML at this boundary. +- The separate application-analysis endpoint exposes only curated tags in its `Keywords` field, and the current Analysis UI does not render that field. No AI prompt produces the deterministic match terms. +- Match output is recomputed on every request; there is no stored analysis/result row or version to migrate. Missing-term learning items are derived through `SyncLearningRecommendationsAsync`, which preserves user-completed/dismissed decisions and auto-completes only obsolete pending generated items. + +## Implemented contract + +- The shared matcher removes script/style/navigation/header/footer blocks, remaining tags and decoded source chrome before analysis while preserving line boundaries. +- English and Norwegian function words, generic recruitment filler and common consent/navigation text are suppressed by category, not only by the reported examples. +- Stop words split bounded phrase runs. Useful two-to-four-word responsibility/domain phrases are ranked ahead of remaining single terms; tokens already represented by a phrase are not repeated as isolated advice. +- Curated, canonical tags now preserve C++, ASP.NET Core, Entity Framework Core, Go/Golang context, Next.js, Terraform, Azure DevOps, Kafka, Redis and Linux. Existing C#, .NET, Node.js and CI/CD punctuation remains intact. +- The UI says “important terms” rather than implying opaque SEO-style keywords, with equivalent Norwegian copy. +- Deterministic processing remains local and free; no model or external provider was added. + +## Verification + +- Seven required fixtures pass: Norwegian, English, mixed, short, noisy HTML, technology-heavy and repeated recruitment filler. +- Final focused matcher: 15/15. Wider affected backend set: 54/54. +- Full backend: 601/601. +- Focused match/analysis UI: 13/13; final label test 3/3. +- Full frontend: 48/48 suites and 172/172 tests. +- Production frontend build/TypeScript and `git diff --check`: pass. + +## Version and regeneration behavior + +There are no historical analysis blobs to silently rewrite. The next GET recomputes from current job/CV text. When the missing-term set changes, existing sync logic creates new generated learning items, auto-completes obsolete pending generated items, and leaves explicit user decisions intact. No schema or data migration is required. + +## Remaining gates + +- The browser session had already been finalized after UX-002; QA-001 label/chip presentation, empty/error/long Norwegian content and 375/768/1440 Light/Dark browser checks were not run in this package. +- Synthetic production comparison and rollout monitoring remain unavailable without deployment access. +- The curated vocabulary is intentionally bounded. New technologies should be added with representative false-positive tests rather than learned from private job data. + +## Evidence + +- Evidence index: `docs/audits/evidence/qa-001/README.md` +- Commands/results: `docs/audits/verification-log.md` V-114–V-116 +- Backend fixtures: `JobTrackerApi.Tests/JobCvMatchServiceTests.cs` +- Frontend label coverage: `job-tracker-ui/src/match-score-panel.test.tsx` +- Implementation commit: `da1aa8b` + +## Rollback + +Revert `da1aa8b`. No database downgrade, dependency rollback or cache purge is required. Existing generated learning decisions remain in history; the next request will derive the older term set again. diff --git a/docs/work-programmes/decisions.md b/docs/work-programmes/decisions.md index feb7b05..065cc8f 100644 --- a/docs/work-programmes/decisions.md +++ b/docs/work-programmes/decisions.md @@ -329,3 +329,13 @@ - **Consequences:** users without a saved scoped choice inherit the explicit anonymous choice, otherwise the documented default is System. Explicit Light/Dark ignore OS changes. The pre-paint script must remain behaviorally aligned with `themePrefs`; tests cover both. - **User approval required:** No; this directly implements the approved UX-002 contract without schema, dependency or production changes. - **Reversible:** Yes. Revert `11734ee`; existing preference values remain unchanged and readable. + +## DEC-034 — Important-term quality stays deterministic and on demand + +- **Date:** 2026-08-09 +- **Decision:** Harden the existing shared `JobCvMatchService` rather than add a model or stored analysis version. Clean HTML/chrome at the matcher boundary, use bilingual/general filler categories, rank bounded phrase runs before singleton terms, and extend the canonical skill vocabulary for punctuation-sensitive technologies. Rename UI output to “important terms.” +- **Reason/evidence:** the complete path proved the reported Norwegian words came from English-only deterministic token ranking. Imported descriptions may be clean, but manual descriptions/notes reach the same matcher. Results are recomputed and the generated learning sync already preserves user decisions, so a cache/schema migration would solve a nonexistent storage problem. +- **Alternatives considered:** hardcode the five examples; call Ollama for keywords; add a result table/version column; filter only in the UI; silently delete learning items. These are incomplete, less reliable, duplicate state, leave API consumers dirty or discard user history. +- **Consequences:** term changes appear on the next request; obsolete pending generated learning items auto-complete under existing behavior while done/dismissed decisions remain. The curated vocabulary remains intentionally bounded and test-driven. +- **User approval required:** No; this is the deterministic-first implementation explicitly required by both programmes. +- **Reversible:** Yes. Revert `da1aa8b`; no schema, cache or provider state changes. diff --git a/docs/work-programmes/master-progress.md b/docs/work-programmes/master-progress.md index eb24714..6753189 100644 --- a/docs/work-programmes/master-progress.md +++ b/docs/work-programmes/master-progress.md @@ -2,17 +2,17 @@ Updated: 2026-08-09 -- **Overall programme status:** Active. Six packages are locally verified; sixteen packages through UX-002 are implemented with automated/runtime evidence but blocked from applicable provider/production gates; QA-001 is in progress. -- **Current work package:** `QA-001` — job-analysis and keyword quality (`IN PROGRESS`); source requirements and the deterministic analysis/cache pipeline are the next trace. +- **Overall programme status:** Active. Six packages are locally verified; seventeen packages through QA-001 are implemented with automated/runtime evidence but blocked from applicable browser/provider/production gates; CAREER-001 is in progress. +- **Current work package:** `CAREER-001` — Career Workspace action-oriented redesign (`IN PROGRESS`); requirements and current first/returning/incomplete/processing/error hierarchy are the next trace. - **Completed work packages:** None are `DONE`; all repository security/AI packages still have applicable browser, provider and/or production gates. - **Locally verified work:** SEC-001, SEC-002, SEC-003, SEC-005A, CORE-001 and PROD-002 (`VERIFIED LOCALLY`). -- **Implemented, verification incomplete:** SEC-004, SEC-005B, SEC-008, CORE-002, BG-001, OPS-001A/B/C, POL-001/002, AI-001/002/003/004 and UX-001/002 (`IMPLEMENTED — NOT VERIFIED`). UX-002 fixes login/logout preference scope, startup paint, route remount and cross-tab synchronization; automated/build and local Light/Dark/System browser gates pass. +- **Implemented, verification incomplete:** SEC-004, SEC-005B, SEC-008, CORE-002, BG-001, OPS-001A/B/C, POL-001/002, AI-001/002/003/004, UX-001/002 and QA-001 (`IMPLEMENTED — NOT VERIFIED`). QA-001 now cleans HTML/chrome, filters bilingual filler, preserves curated punctuation/phrases and uses honest labels; all automated/build gates pass. - **Production-verified work:** None. - **Blocked work:** SEC-006 requires explicit internet/package-index permission; PROD-001/003/004 and REL-001 require documented production access and unfinished dependencies. Real provider, SMTP/MariaDB and production environments are unavailable; the in-app browser is available for local UI checks. - **Deferred work:** None. Conditional multi-replica coordination, model deletion, realtime operation delivery and unrelated production changes remain outside current packages. -- **Next five work packages:** QA-001 job-analysis/keyword quality; CAREER-001 Career Workspace redesign; CAREER-002 CV Builder redesign; EMAIL-001 consolidated email experience; JOBS-001 job-search/application redesign. SEC-006/007 resume after package-index permission. -- **Status counts:** 6 `VERIFIED LOCALLY`; 16 `IMPLEMENTED — NOT VERIFIED`; 1 `IN PROGRESS`; 11 `NOT STARTED`; 5 `BLOCKED`; 0 `DONE`; 0 `DEFERRED`. -- **Test status:** backend 594/594; AI-004 focused backend 40/40; frontend 48/48 suites and 172/172 tests, UX-002 focused 6/6, production build pass; prior sidecar 22/22. Patch check passes. UX-002 browser passed Light/Dark/System/navigation/refresh/two-tab synchronization at required widths; provider/model/MariaDB/production checks were not run. +- **Next five work packages:** CAREER-001 Career Workspace redesign; CAREER-002 CV Builder redesign; EMAIL-001 consolidated email experience; JOBS-001 job-search/application redesign; HOME-001 homepage/Pro promotion. SEC-006/007 resume after package-index permission. +- **Status counts:** 6 `VERIFIED LOCALLY`; 17 `IMPLEMENTED — NOT VERIFIED`; 1 `IN PROGRESS`; 10 `NOT STARTED`; 5 `BLOCKED`; 0 `DONE`; 0 `DEFERRED`. +- **Test status:** backend 601/601; QA-001 focused matcher 15/15 and affected backend 54/54; frontend 48/48 suites and 172/172 tests, affected UI 13/13; production build and patch check pass. QA-001 browser/production presentation was not run; prior UX-002 browser and sidecar 22/22 remain green. - **Deployment status:** No deployment performed. No production migrations were run. AI operation worker remains disabled by default. - **Production status:** Unchanged and unverified. No provider/model call, model pull, external request or paid API occurred. - **Known regressions:** None found by automated suites. Jest still needs `--forceExit` and reports its existing open-handle notice. Direct clean EF-only SQLite migration still hits the pre-existing historical blank-chain defect before later migrations; normal startup owns reconciliation. Cross-feature monthly AI usage accounting remains a rollout gap. @@ -43,5 +43,6 @@ Updated: 2026-08-09 - `docs/verification/ai-004-cv-processing-queue.md` - `docs/verification/ux-001-unified-authentication.md` - `docs/verification/ux-002-deterministic-theme-state.md` +- `docs/verification/qa-001-job-term-quality.md` - `docs/verification/prod-002-ai-evaluation.md` - `docs/work-programmes/master-work-plan.md` diff --git a/docs/work-programmes/master-work-plan.md b/docs/work-programmes/master-work-plan.md index ef8f6db..f111c6a 100644 --- a/docs/work-programmes/master-work-plan.md +++ b/docs/work-programmes/master-work-plan.md @@ -16,7 +16,7 @@ Allowed statuses are `NOT STARTED`, `IN PROGRESS`, `IMPLEMENTED — NOT VERIFIED `DONE` requires every applicable acceptance criterion, focused and regression tests, browser/accessibility/theme/mobile checks, tenant and entitlement checks, documentation, migration/rollback evidence, and production verification. Repository-only work that still requires production is at most `VERIFIED LOCALLY`. -Exactly one implementation item may be `IN PROGRESS`. As of this revision it is **QA-001**. +Exactly one implementation item may be `IN PROGRESS`. As of this revision it is **CAREER-001**. ## Consolidated dependency order @@ -595,11 +595,11 @@ Ordering differences from the suggested list: - **Required tests:** seven specified Norwegian/English/mixed/short/noisy/tech/filler fixtures. - **Required browser verification:** result presentation/empty/error/long Norwegian text at three widths/themes. - **Required production verification:** synthetic analysis comparison; no silent historical rewrite. -- **Status:** `IN PROGRESS`. -- **Blocker:** none. -- **Evidence:** source requirements and deterministic fixture contract are next for complete pipeline/caching trace; PROD-002 fixtures are available. -- **Commit:** none. -- **Remaining work:** deterministic fix before considering model help. +- **Status:** `IMPLEMENTED — NOT VERIFIED`. +- **Blocker:** browser presentation and production comparison remain; deterministic repository work is complete. +- **Evidence:** `docs/verification/qa-001-job-term-quality.md`; V-114–V-116; seven required fixtures, focused matcher 15/15, affected backend 54/54, full backend 601/601, focused UI 13/13, full frontend 172/172 and build. +- **Commit:** `da1aa8b` (`fix(match): prioritize meaningful job terms`). +- **Remaining work:** browser empty/error/long Norwegian/three-width/theme presentation; synthetic production comparison. No stored analysis migration exists. ### CAREER-001 — Career Workspace action-oriented redesign @@ -613,9 +613,9 @@ Ordering differences from the suggested list: - **Required tests:** first/returning/incomplete/processing/failure state and navigation. - **Required browser verification:** three widths, light/dark, keyboard/focus/loading/empty/error/Norwegian. - **Required production verification:** synthetic account smoke. -- **Status:** `NOT STARTED`. +- **Status:** `IN PROGRESS`. - **Blocker:** browser tooling must be available for completion. -- **Evidence:** current architecture and source requirement. +- **Evidence:** source requirements and current Career Workspace state hierarchy are next for complete trace; CORE-001 and AI-004 contracts are available. - **Commit:** none. - **Remaining work:** no master-profile overwrite. diff --git a/docs/work-programmes/session-handoff.md b/docs/work-programmes/session-handoff.md index ab2c8f3..05a48b3 100644 --- a/docs/work-programmes/session-handoff.md +++ b/docs/work-programmes/session-handoff.md @@ -2,17 +2,17 @@ Updated: 2026-08-09 -- **Exact current task:** QA-001 — re-read the specified Norwegian/English quality fixtures and trace import cleanup, language/token/phrase/skill/scoring/cache/version/UI-label behavior before editing. -- **Last completed step:** UX-002 implementation `11734ee` was committed. Deterministic user/anonymous/System precedence, pre-paint bootstrap, in-place provider switching, auth-user/cross-tab events and mobile Settings tabs pass automated/build/browser verification. Evidence/tracking are ready for their own commit and push. -- **Files currently modified:** UX-002 verification/evidence/tracking documents and three synthetic screenshots under `docs/audits/evidence/ux-002/`. UX-002 application code is committed. -- **Commands already run:** UX-002 complete theme/auth/provider/router/bootstrap trace; focused 6/6 and full 172/172 frontend tests; production build; diff checks; local browser Light/Dark/System/navigation/refresh/two-tab/375/768/1440 checks; implementation commit. See V-111–V-113. -- **Test results:** backend baseline 594/594; frontend 48/48 suites and 172/172 tests; UX-002 focused 6/6; production frontend build and diff check pass. Browser theme checks and final console inspection pass; authenticated multi-user/production remain. +- **Exact current task:** CAREER-001 — re-read Career Workspace requirements and trace first-run, completeness, CV import review, queued processing/failure and recent-document actions before editing. +- **Last completed step:** QA-001 implementation `da1aa8b` was committed. Bilingual/chrome filtering, bounded phrases, expanded canonical technology tags and honest important-term labels pass all automated/build gates. Evidence/tracking are ready for their own commit and push. +- **Files currently modified:** QA-001 verification/evidence/tracking documents only. QA-001 application code is committed; no screenshots were captured. +- **Commands already run:** QA-001 complete import/language/tagger/matcher/API/UI/learning/cache trace; seven required fixtures; affected and full backend/frontend tests; production build; diff checks; implementation commit. See V-114–V-116. +- **Test results:** backend 601/601; QA matcher 15/15; affected backend 54/54; frontend 48/48 suites and 172/172 tests; affected UI 13/13; production build and diff check pass. Browser/production QA presentation remains. - **Services currently running:** none started by this session. The Next development server was stopped and in-app browser tabs finalized. Pre-existing Docker services were not changed. - **Temporary files or processes:** none. No dependency declaration, database or private artifact changed. - **Production changes currently active:** none. No deployment, provider/model call, private CV access, model pull, paid service, production migration or external payload occurred. -- **Rollback status:** UX-002 has no schema/dependency migration; revert `11734ee` while retaining existing preference keys. UX-001/AI-004 rollback records remain current. Production is unchanged. -- **Uncommitted changes:** UX-002 evidence/master tracking documents and screenshots only; commit/push these before QA-001 application code. -- **Known failures:** authenticated two-user/production theme smoke remains; configured/real-provider UX-001 browser checks remain; MariaDB/SMTP/production unavailable; SEC-006 upgrades require explicit internet/package-index permission; parser isolation remains SEC-007; historical clean EF-only SQLite chain needs startup reconciliation; Jest reports its existing force-exit/open-handle notice. -- **Exact next action:** commit and push UX-002 evidence, then read `work.md:174-263`, `ollama.md:177-223`, PROD-002 fixtures and every job-analysis producer/cache/UI consumer; reproduce the seven specified quality cases before editing. -- **Work that can continue independently:** QA-001 and subsequent repository UX packages. SEC-006/007 await package-index permission; PROD-001/003/004 await production access. +- **Rollback status:** QA-001 has no schema/cache/dependency migration; revert `da1aa8b` to restore the prior term set/labels. Earlier rollback records remain current. Production is unchanged. +- **Uncommitted changes:** QA-001 evidence/master tracking documents only; commit/push these before CAREER-001 application code. +- **Known failures:** QA-001 browser/production presentation; authenticated two-user/production theme smoke; configured/real-provider UX-001; MariaDB/SMTP/production unavailable; SEC-006 needs explicit internet/package-index permission; parser isolation remains SEC-007; historical clean EF-only SQLite chain needs startup reconciliation; Jest retains its open-handle notice. +- **Exact next action:** commit and push QA-001 evidence, then read `work.md:265-307`, Career Workspace components/tests, CORE-001 data shapes and AI-004 operation states; reproduce first/returning/incomplete/queued/review/failure paths before editing. +- **Work that can continue independently:** CAREER-001/002 and later repository UX packages. SEC-006/007 await package-index permission; PROD-001/003/004 await production access. - **Decisions still required from the user:** none for synthetic repository work. Internet/package upgrades, private CV access, external/paid providers and production actions retain their explicit approval/safety gates; SEC-009 retention/legal policy remains unresolved.