88 lines
12 KiB
Markdown
88 lines
12 KiB
Markdown
# S01: Smarter Gmail import and matching
|
||
|
||
**Goal:** Make Gmail import job-aware enough that a user reviewing one imported job sees the right candidate messages/threads first, can import a complete thread when that is the safer choice, and then trust the linked correspondence to stay current automatically when new Gmail replies arrive or when the user sends a reply from Gmail.
|
||
**Demo:** From a job opened in the workspace, the user connects Gmail, sees ranked thread/message suggestions built from that job’s company/recruiter context, imports either one message or a full thread, and then sees later inbound or user-sent replies from that linked Gmail thread appear on the same job without needing to manually re-pull or re-import the thread.
|
||
|
||
S01 directly advances active requirements **R001**, **R002**, and **R010**. The work remains backend-first because the highest risk is now continuity, not just initial matching: if ranking, thread identity, and linked-thread refresh stay shallow, the UI can look nicer while the real correspondence loop still feels untrustworthy. The completed tasks below established the initial matching/import contract; the remaining work extends that contract into full-thread continuity so downstream drafting and tracking slices can rely on a living thread instead of a one-time import snapshot.
|
||
|
||
## Must-Haves
|
||
|
||
- A job-scoped Gmail matching API ranks candidate messages/threads for a specific `JobApplication` using company, recruiter, title, and existing correspondence signals rather than a generic inbox search alone.
|
||
- Gmail import persists enough metadata to preserve message/thread identity, expose already-imported state, and reduce duplicate cleanup when the user imports single messages or whole threads into a job.
|
||
- The system can import an entire Gmail thread as a job-linked correspondence history, not just isolated messages, while keeping message ordering and duplicate protection intact.
|
||
- Already-linked Gmail threads can refresh automatically so new inbound messages and user-sent replies from Gmail show up on the job without requiring a manual re-import flow.
|
||
- The job workspace `Correspondence` tab shows ranked Gmail suggestions with match reasons/confidence, keeps a manual search override, and refreshes the visible job-linked correspondence list after import or linked-thread sync.
|
||
|
||
## Proof Level
|
||
|
||
- This slice proves: integration
|
||
- Real runtime required: yes
|
||
- Human/UAT required: yes
|
||
|
||
## Verification
|
||
|
||
- `dotnet test JobTrackerApi.Tests/JobTrackerApi.Tests.csproj --filter GmailControllerTests`
|
||
- `CI=true npm --prefix job-tracker-ui test -- --watch=false --runTestsByPath src/correspondence-gmail-import.test.tsx`
|
||
- Failure-path inspection: hit `api/gmail/status` while disconnected; exercise the job-scoped Gmail matching endpoint with an invalid or inaccessible `jobApplicationId`; and exercise the linked-thread refresh path with an unknown thread id or stale sync cursor. Confirm the API exposes connection state plus clear validation/not-found errors without leaking Gmail content.
|
||
- Manual UAT: run the app, connect a real Gmail account, open one job in the workspace, confirm ranked Gmail suggestions appear before manual search, import one message and one full thread, send or receive another reply in Gmail on that linked thread, and verify the new reply appears on that same job without a manual re-import step and without any auto-send behavior.
|
||
|
||
## Observability / Diagnostics
|
||
|
||
- Runtime signals: ranked match reasons/confidence returned per candidate, import result counts for imported/skipped messages, linked-thread sync result counts, newest synced Gmail history marker/cursor per linked thread, and existing request logs with trace ids around Gmail endpoints.
|
||
- Inspection surfaces: `api/gmail/status`, the job-scoped Gmail matching and linked-thread refresh responses in `JobTrackerApi/Controllers/GmailController.cs`, any Gmail sync/background service introduced for linked threads, the correspondence list in `job-tracker-ui/src/components/Correspondence.tsx`, and backend/frontend tests covering duplicate and sync failure paths.
|
||
- Failure visibility: Gmail connection state, empty/noisy candidate lists, duplicate-skip counts, stale thread-sync state, and per-candidate imported/already-linked flags should remain visible enough to tell whether failure came from query construction, Gmail retrieval, dedupe, sync cursor handling, or UI wiring.
|
||
- Redaction constraints: never expose OAuth tokens or full message bodies in logs; diagnostics should stay at message metadata, ranking reasons, sync counts, and import results.
|
||
|
||
## Integration Closure
|
||
|
||
- Upstream surfaces consumed: `JobTrackerApi/Services/GmailOAuthService.cs`, `Data/JobTrackerContext.cs`, `Models/JobApplication.cs`, `Models/Company.cs`, `Models/Correspondence.cs`, `job-tracker-ui/src/components/JobDetailsDialog.tsx`, and `job-tracker-ui/src/components/Correspondence.tsx`.
|
||
- New wiring introduced in this slice: a job-aware Gmail candidate contract in the API, correspondence persistence that retains Gmail thread/message identity, and job-context-driven rendering in the workspace Gmail import UI.
|
||
- Additional wiring now required by the override: a linked-thread refresh mechanism that can detect new Gmail replies for already-imported threads and merge them into the job’s correspondence timeline without forcing the user to re-run import manually.
|
||
- What remains before the milestone is truly usable end-to-end: later slices still need to consume the imported correspondence for stronger drafting and daily workflow surfaces, but Gmail import itself must now be trustworthy both at first import and during ongoing thread updates.
|
||
|
||
## Tasks
|
||
|
||
- [x] **T01: Add a job-aware Gmail matching contract and backend ranking tests** `est:3h`
|
||
- Why: The slice risk sits in matching quality, so the backend needs to own candidate discovery, ranking, and duplicate awareness before the UI can present trustworthy suggestions.
|
||
- Files: `JobTrackerApi/Controllers/GmailController.cs`, `JobTrackerApi/Services/GmailOAuthService.cs`, `JobTrackerApi.Tests/GmailControllerTests.cs`
|
||
- Do: Add a job-scoped Gmail candidate endpoint that loads the owned job plus company context, builds multiple Gmail queries from recruiter/company/title/correspondence signals, dedupes messages and groups them by thread, and returns ranked suggestions with match reasons, confidence inputs, and already-imported flags; cover the contract with focused controller/service tests instead of leaving ranking in `Correspondence.tsx`.
|
||
- Verify: `dotnet test JobTrackerApi.Tests/JobTrackerApi.Tests.csproj --filter GmailControllerTests`
|
||
- Done when: the API can return ranked Gmail candidates for one job with explicit reasons and duplicate state, and backend tests prove owned-job lookup plus ranking/dedupe behavior.
|
||
- [x] **T02: Persist Gmail thread metadata and harden import continuity** `est:3h`
|
||
- Why: Smarter matching is not enough if imported correspondence still loses thread identity or forces downstream slices to re-derive sender/thread information later.
|
||
- Files: `Models/Correspondence.cs`, `JobTrackerApi/Controllers/GmailController.cs`, `JobTrackerApi/Controllers/CorrespondenceController.cs`, `JobTrackerApi/Program.cs`, `JobTrackerApi.Tests/GmailControllerTests.cs`
|
||
- Do: Extend correspondence persistence and import logic so Gmail imports store thread/message identity plus sender/recipient metadata, update import responses and duplicate handling to reflect imported vs skipped outcomes clearly, and update startup schema guards so older SQLite/MySQL dev databases remain bootable when the new fields land.
|
||
- Verify: `dotnet test JobTrackerApi.Tests/JobTrackerApi.Tests.csproj --filter GmailControllerTests`
|
||
- Done when: imported correspondence keeps enough Gmail metadata for thread continuity and dedupe, compatibility guards are planned with the schema change, and tests prove single-message/thread imports behave correctly on repeat imports.
|
||
- [x] **T03: Wire ranked Gmail suggestions into the job workspace UI** `est:3h`
|
||
- Why: The slice is only complete when the user can act on the smarter backend contract inside the actual job workspace rather than through a generic inbox search UI.
|
||
- Files: `job-tracker-ui/src/components/JobDetailsDialog.tsx`, `job-tracker-ui/src/components/Correspondence.tsx`, `job-tracker-ui/src/types.ts`, `job-tracker-ui/src/correspondence-gmail-import.test.tsx`
|
||
- Do: Pass job context from `JobDetailsDialog` into `Correspondence`, replace client-side primary ranking with the server-provided candidate contract, show match reasons/confidence/import state and thread actions, keep manual query/search as a fallback override, and add a React test that proves ranked suggestions and import refresh behavior in the dialog.
|
||
- Verify: `CI=true npm --prefix job-tracker-ui test -- --watch=false --runTestsByPath src/correspondence-gmail-import.test.tsx`
|
||
- Done when: the Correspondence Gmail tab opens on ranked job-aware suggestions, still supports manual search overrides, refreshes the correspondence list after import, and the new UI test passes.
|
||
- [ ] **T04: Add linked-thread refresh so imported Gmail threads stay current** `est:4h`
|
||
- Why: The user’s trust bar is no longer just “can I import the right thread once?” but “does that thread continue to reflect reality after I reply or receive another response in Gmail?”
|
||
- Files: `JobTrackerApi/Controllers/GmailController.cs`, `JobTrackerApi/Services/GmailOAuthService.cs`, `JobTrackerApi/Program.cs`, `Models/Correspondence.cs`, `JobTrackerApi.Tests/GmailControllerTests.cs`
|
||
- Do: Add a linked-thread refresh path that can pull new Gmail messages for already-imported job threads using stored Gmail thread/message identity, dedupe against existing correspondence, import newly discovered inbound and sent replies into the same job, and expose enough sync status for the UI to know whether the thread is current.
|
||
- Verify: `dotnet test JobTrackerApi.Tests/JobTrackerApi.Tests.csproj --filter GmailControllerTests`
|
||
- Done when: an already-linked Gmail thread can be refreshed without manual re-import, new sent/inbound replies land as correspondence on the same job, and focused tests prove dedupe plus stale-thread handling.
|
||
- [ ] **T05: Surface automatic Gmail thread continuity in the workspace UI** `est:3h`
|
||
- Why: Even with backend sync, the slice will still feel broken if the user cannot tell that a linked thread is auto-updating or whether a newly sent Gmail reply has already been incorporated.
|
||
- Files: `job-tracker-ui/src/components/Correspondence.tsx`, `job-tracker-ui/src/components/JobDetailsDialog.tsx`, `job-tracker-ui/src/types.ts`, `job-tracker-ui/src/correspondence-gmail-import.test.tsx`
|
||
- Do: Update the Correspondence tab to distinguish one-time suggestions from already-linked live threads, show last-sync or updating state, trigger automatic refresh on load and after manual job actions where appropriate, and extend the React test so a newly synced Gmail reply appears in the job correspondence list without a fresh import flow.
|
||
- Verify: `CI=true npm --prefix job-tracker-ui test -- --watch=false --runTestsByPath src/correspondence-gmail-import.test.tsx`
|
||
- Done when: the user can see that a thread is linked/live, newly synced replies appear in the workspace automatically or on lightweight refresh, and the UI test covers the no-manual-reimport continuity path.
|
||
|
||
## Files Likely Touched
|
||
|
||
- `JobTrackerApi/Controllers/GmailController.cs`
|
||
- `JobTrackerApi/Services/GmailOAuthService.cs`
|
||
- `Models/Correspondence.cs`
|
||
- `JobTrackerApi/Controllers/CorrespondenceController.cs`
|
||
- `JobTrackerApi/Program.cs`
|
||
- `JobTrackerApi.Tests/GmailControllerTests.cs`
|
||
- `job-tracker-ui/src/components/JobDetailsDialog.tsx`
|
||
- `job-tracker-ui/src/components/Correspondence.tsx`
|
||
- `job-tracker-ui/src/types.ts`
|
||
- `job-tracker-ui/src/correspondence-gmail-import.test.tsx`
|