diff --git a/docs/specs/README.md b/docs/specs/README.md new file mode 100644 index 0000000..676fcf7 --- /dev/null +++ b/docs/specs/README.md @@ -0,0 +1,69 @@ +# InboxIntel — Feature Build-Out & UI Overhaul Specs + +This folder is the design contract for the Clean.Email-parity feature build-out plus +a full UI overhaul. Specs are written to be executable: each names real types, +files, and the patterns already in the codebase. + +## Locked decisions (2026-06-30) + +| Area | Decision | +|------|----------| +| **Automation safety** | **Hybrid.** Safe, reversible actions (label, archive/skip-inbox, mark-read, star, move-to-label) apply automatically. Destructive actions (trash, delete, keep-newest culling, trash-by-age) are **proposed** and require one-click user approval before touching Gmail. | +| **Scope** | **Tier 1 + 2 only.** Gmail-only. **No sending** (no Compose/Reply/Forward). No multi-provider. | +| **Gmail vs in-app** | **Touch real Gmail.** Screener/Block/Pause/Read-Later/Deliver-To use managed `InboxIntel/…` labels + skip-inbox so the inbox is clean everywhere (phone, web). All reversible. | +| **Privacy Monitor** | Spec around Have I Been Pwned; ship behind a config flag, **off until a key is added**. No cost now. | +| **UI aesthetic** | **Stripe / Notion** — light, airy, generous whitespace, soft shadows. | +| **Color modes** | **Light + dark toggle**, light-first, driven by CSS-variable design tokens. | +| **UI stack** | **Tailwind CSS + shadcn-style** headless primitives (Radix + cva + tailwind-merge), hand-built component set. | +| **UI rollout** | **Incremental** — stand up the design system, then convert page-by-page. App stays working throughout. | +| **Brand** | Keep existing logo. **Propose a new accent + neutral palette** (see `ui-overhaul.md`) for approval. | +| **Mobile** | **Desktop-first, responsive-ok** — usable on phones, but desktop is the primary target. | + +## The unifying idea + +Clean.Email's whole "keeps your inbox clean automatically" story reduces to **one +engine** plus a few specializations: + +``` + ┌─────────────────────────────┐ + │ Automation Engine │ + │ (conditions → action) │ + └──────────────┬──────────────┘ + ┌──────────────────────────┼──────────────────────────┐ + ▼ ▼ ▼ + AutomationRule SenderPolicy per-email / per-thread + (custom rules, (Block, Whitelist, Pinned (Email.IsPinned) + Trash-by-Age) Screener, Pause, Mute (MailThread.IsMuted) + Read-Later, Keep-Newest, + Deliver-To) +``` + +Everything writes through the existing `CleanupService` / `IGmailService`, logs to a +single **`AutomationAction`** table that doubles as the **approval queue** (Proposed) +and the **Activity Log** (Applied / Rejected / Undone). + +## Spec index + +| File | Covers | +|------|--------| +| [`feature-rules-engine.md`](feature-rules-engine.md) | Auto Clean Rules, the execution engine, hybrid approval queue, Pinned, Mute, Trash-by-Age, Keep-Newest, the `AutomationWorker` | +| [`feature-sender-policy.md`](feature-sender-policy.md) | Block, Whitelist, Screener, Pause, Read-Later, Deliver-To (per-sender) | +| [`feature-activity-log.md`](feature-activity-log.md) | Unified `AutomationAction` log, undo, Activity Summaries | +| [`feature-privacy-monitor.md`](feature-privacy-monitor.md) | HIBP breach checking behind a flag | +| [`ui-overhaul.md`](ui-overhaul.md) | Design tokens, palette proposal, Tailwind+shadcn setup, page-by-page migration | +| [`build-plan.md`](build-plan.md) | Increment sequencing, commit plan, what depends on what | + +## Cross-cutting invariants (apply to every feature) + +1. **User-scoping.** Every query, mutation, and background pass filters by `UserId`. No + cross-user data path may exist. (Standing security instruction.) +2. **AI never destroys.** The existing `AiService` doc invariant holds: AI only reads + and suggests. Automation destructive actions come from deterministic rules + user + approval, never directly from an LLM. +3. **Reversibility & managed labels.** Anything that hides mail uses Gmail labels under + the `InboxIntel/` namespace and `skip-inbox` (remove `INBOX`), never hard-delete. + Hard delete is never an automatic or proposed action — it stays manual-only. +4. **Graceful degradation.** A failing Gmail/AI/HIBP call logs and continues; it never + crashes the worker or a sync. +5. **Pinned & Whitelisted are sacrosanct.** No rule, policy, or sweep may act on a + pinned email or a whitelisted sender. diff --git a/docs/specs/build-plan.md b/docs/specs/build-plan.md new file mode 100644 index 0000000..56476c5 --- /dev/null +++ b/docs/specs/build-plan.md @@ -0,0 +1,64 @@ +# Build Plan — sequencing & commits + +Two tracks run in parallel and rarely touch the same files: **Backend (automation)** and +**Frontend (UI overhaul)**. Backend can progress regardless of palette sign-off; the new +feature *pages* wait for the design system (Increment F1). + +Each step ends with a build (`dotnet build` and/or `npm run build`) and a commit, per the +established pattern. + +## Backend track (B) + +| Step | Deliverable | Key files | Migration | +|------|-------------|-----------|-----------| +| **B1** | Automation core domain + engine skeleton | enums, `AutomationRule`, `AutomationAction`, `SenderPolicy`, `Email.IsPinned`, `MailThread.IsMuted`, `User.Screener*` | `AddAutomationCore` | +| **B2** | `EnsureLabelAsync` on Gmail service + label cache | `IGmailService`, `GmailApiService` | — | +| **B3** | `AutomationEngine.RunAsync` (matching, safe-apply, propose) + `Matches()` unit tests | `AutomationEngine`, `RuleMatcher` | — | +| **B4** | `IRuleService` + `AutomationController` (rules CRUD + preview) | service, controller, `AutomationDtos` | — | +| **B5** | Approval queue + activity + undo (`IAutomationActionService`) | service, controller endpoints | — | +| **B6** | `ISenderPolicyService` + Screener + policy controller | service, controller | — | +| **B7** | `AutomationWorker` + hook engine into incremental sync; `AutomationOptions`; DI wiring | worker, `SyncService`, `DependencyInjection`, `Options`, `appsettings` | — | +| **B8** | Activity Summaries + Cleanup Reminders into `DigestService` | `DigestService` | — | +| **B9** | Privacy Monitor (provider, service, controller, options, Null provider) | `Privacy/*`, DI, `appsettings` | `AddPrivacyFields` | + +> Commit boundaries roughly per step (B1–B2 may bundle; B3 stands alone with tests). + +## Frontend track (F) + +| Step | Deliverable | Notes | +|------|-------------|-------| +| **F0** | Tailwind + PostCSS + Radix + cva/lucide installed; `index.css` tokens; `darkMode:class`; `ThemeToggle`; build green | no page swaps yet | +| **F1** | `components/ui/*` primitive set | button/card/input/dialog/sheet/dropdown/tabs/tooltip/toast/table/badge/switch/skeleton | +| **F2** | App shell (`Layout`) on new system + theme toggle in topbar | biggest visual win | +| **F3** | Dashboard | theme chart.js colors via tokens | +| **F4** | Senders (+ policy dropdown, wired to B6) | | +| **F5** | Unsubscribe (confidence meter) | | +| **F6** | Search / Folders / Cleanup | | +| **F7** | New pages: Rules editor, Review queue, Screener, Activity, Read-Later, Privacy | depends on B4–B9 | +| **F8** | Landing polish; delete `styles.css` | | + +## Suggested interleave + +``` +B1+B2 ─▶ B3 ─▶ B4 ─▶ B5 ─▶ B6 ─▶ B7 ─▶ B8 ─▶ B9 +F0 ─▶ F1 ─▶ F2 ─▶ F3 ─▶ F4 ─▶ F5 ─▶ F6 ─▶ F7(needs B4–B9) ─▶ F8 +``` + +Practical order to actually build in: **F0 → F1 → F2** (get the app looking modern fast +and de-risk the stack), then **B1→B3** (the engine core), then alternate +feature-by-feature (B4+F7-rules, B5+F7-review, B6+F7-screener, …), finishing with B8/B9 ++ their pages, then F3–F6 restyles and F8 cleanup. + +## Definition of done (per feature) + +- Backend builds (0 errors), unit/integration tests for engine logic pass. +- Frontend builds; page works in light **and** dark. +- Every new endpoint scoped to `UserId`; destructive paths go through approval. +- Spec checklist items ticked. +- Committed (push remains blocked by the known `git.cesnimda.uk` credential issue — local only). + +## Open items to confirm before/while building + +1. **Accent color** sign-off (indigo proposed; alternatives listed in `ui-overhaul.md`). +2. Whether Activity Summaries need a **separate toggle** from the analytics digest (default: fold in). +3. First-match-wins vs all-matching-rules for rule evaluation (default: **first-match-wins** by priority). diff --git a/docs/specs/feature-activity-log.md b/docs/specs/feature-activity-log.md new file mode 100644 index 0000000..2681c61 --- /dev/null +++ b/docs/specs/feature-activity-log.md @@ -0,0 +1,75 @@ +# Spec: Activity Log, Undo & Activity Summaries + +A unified, trustworthy record of everything automation did — and a way to take it back. +Built entirely on the `AutomationAction` table from `feature-rules-engine.md`; no new +storage. + +## 1. Activity Log + +- Backed by `AutomationAction` rows with `Status in (Applied, Rejected, Undone, Failed)`. +- `IAutomationActionService.GetActivityAsync(userId, take)` returns + `ActivityLogEntryDto`, newest first, grouped where it reads naturally + (e.g. "Archived 38 emails from LinkedIn — Rule: Social noise"). +- Each entry exposes `CanUndo`: + - Safe actions (Archive/SkipInbox/ApplyLabel/MarkRead/Star) — always undoable while we + still hold `UndoStateJson` and the message exists. + - Trash — undoable (Gmail untrash) within Gmail's 30-day window. + - Hard delete — N/A (never automated). + +### UI (`/app/activity`) +- Reverse-chronological feed with source chips (Rule / Policy / Screener / Age sweep), + action icon, affected count, timestamp, and an **Undo** button where `CanUndo`. +- Filter by source and action type. Date range. Search by sender. + +## 2. Undo + +`UndoAsync(userId, actionIds)`: +1. Load the `Applied` actions (verify `UserId`). +2. For each, parse `UndoStateJson` (captured pre-apply: which labels were present, + whether `INBOX` was set, whether it was in Trash). +3. Issue the inverse `BatchModifyAsync` / `BatchUntrash` to restore prior state. +4. Set `Status = Undone`, stamp `AppliedUtc = now` on the undo. +5. Log is append-only in spirit: the original row flips to `Undone` rather than being deleted. + +`UndoStateJson` shape (kept tiny): +```json +{ "hadInbox": true, "labels": ["Label_12","Label_88"], "wasTrashed": false } +``` +Captured by the engine/approval step immediately before mutating. + +## 3. Activity Summaries (extends existing digest) + +Clean.Email's "Activity Summaries" = periodic notification of what automation did. We +already have the SMTP digest infra (`IDigestService`, `DigestWorker`, +`User.DigestEnabled`). Extend rather than add: + +- `DigestService.BuildHtml` gains an **"Automation activity since last digest"** section: + counts of archived/labeled/screened, pending-approval count (with a nudge to review), + top rules by volume, new screener senders. +- Pull from `AutomationAction` where `AppliedUtc > user.LastDigestSentUtc`. +- No new toggle — folds into the existing digest opt-in. (Optional later: a separate + `User.ActivitySummaryEnabled` if users want activity summaries without the analytics digest.) + +## 4. Cleanup Reminders + +A lightweight nudge when the inbox needs attention, reusing `DigestWorker`'s tick: + +- If a user has **pending destructive approvals** older than `ReminderAfterDays` (default 3) + and digests are on, the next digest leads with "You have N actions awaiting approval." +- If automation is **off** but the dashboard health score is poor / unsubscribe backlog + is large, include a "Time to clean up" prompt with a deep link. + +No new infrastructure — just content rules inside `DigestService`. + +## 5. API + +Covered by `feature-rules-engine.md` §7: +`GET /automation/activity`, `POST /automation/activity/undo`. Add query params for +filtering: `?source=&action=&from=&to=&q=`. + +## 6. Security & safety + +- [ ] Activity + undo scoped to `UserId`; action ownership re-checked on undo. +- [ ] Undo is best-effort and idempotent — undoing an already-undone/missing message + logs and no-ops rather than erroring. +- [ ] Log never exposes another user's senders/emails. diff --git a/docs/specs/feature-privacy-monitor.md b/docs/specs/feature-privacy-monitor.md new file mode 100644 index 0000000..ef970a4 --- /dev/null +++ b/docs/specs/feature-privacy-monitor.md @@ -0,0 +1,91 @@ +# Spec: Privacy Monitor (data-breach checking) + +Checks the user's email address (and optionally addresses they've corresponded with) +against known data breaches. Spec'd around **Have I Been Pwned (HIBP)**, shipped behind +a config flag that stays **off until an API key is provided** — zero cost until then. + +## 1. Configuration + +`PrivacyOptions` (new, `Configuration/Options.cs`): + +```csharp +public class PrivacyOptions +{ + public const string SectionName = "Privacy"; + public bool Enabled { get; set; } = false; // master flag + public string Provider { get; set; } = "Hibp"; // "Hibp" | "None" + public string? HibpApiKey { get; set; } // required for Hibp + public int CacheHours { get; set; } = 24; // don't hammer the API +} +``` + +`appsettings.json` gains a `Privacy` section with `Enabled: false`, empty key. +`IsEnabled => Enabled && Provider == "Hibp" && !string.IsNullOrWhiteSpace(HibpApiKey)`. + +## 2. Provider abstraction + +```csharp +public record BreachDto(string Name, string Title, string Domain, DateOnly BreachDate, + IReadOnlyList DataClasses, bool IsVerified, string? Description); + +public interface IBreachProvider +{ + bool IsEnabled { get; } + /// Breaches for an address; empty list if clean; throws only on hard errors. + Task> CheckAsync(string emailAddress, CancellationToken ct = default); +} +``` + +- `HibpBreachProvider` — `GET https://haveibeenpwned.com/api/v3/breachedaccount/{account}?truncateResponse=false`, + header `hibp-api-key`, a descriptive `user-agent`. Handle: `404` = no breaches (return + empty), `401` = misconfig (log, treat as disabled), `429` = rate-limited (respect + `Retry-After`, return cached/empty). Uses a named `HttpClient` "hibp" with a sane timeout. +- `NullBreachProvider` — `IsEnabled => false`, returns empty. Registered when the flag is off + (mirrors the `NullAiProvider` pattern in `DependencyInjection`). + +## 3. Service + +```csharp +public interface IPrivacyService +{ + bool IsEnabled { get; } + /// Check the signed-in user's own address. Cached per PrivacyOptions.CacheHours. + Task CheckSelfAsync(Guid userId, CancellationToken ct = default); +} + +public record PrivacyReportDto(string Address, bool Checked, int BreachCount, + IReadOnlyList Breaches, DateTimeOffset? LastCheckedUtc); +``` + +- Resolve the user's address from `User`/identity. +- Cache the last result + timestamp (new `User.LastBreachCheckUtc` + a small + `BreachCheck`/JSON column, or a dedicated table if we later check multiple addresses). + For v1, store on `User`: `LastBreachCheckUtc`, `BreachCountCached`. +- Respect `CacheHours`: return cached unless stale or `force` requested. + +## 4. API (`PrivacyController : ApiControllerBase`) + +| Method | Route | Purpose | +|--------|-------|---------| +| GET | `/privacy/status` | `{ enabled }` so the UI can hide the feature when off | +| GET | `/privacy/self` | cached report for the signed-in user | +| POST | `/privacy/self/refresh` | force a fresh check (rate-limit aware) | + +All scoped to `UserId`. We **only** check the authenticated user's own address in v1 — +never arbitrary addresses (avoids turning the app into a breach-lookup tool for others). + +## 5. Frontend + +- **Privacy page** (`/app/privacy`), hidden from nav when `GET /privacy/status` is disabled. +- Shows: address checked, breach count, and a card per breach (title, date, what leaked, + verified badge), plus a "Re-check" button and "what this means / next steps" guidance. +- Dashboard widget (optional): a small "Privacy" tile with breach count + link, also + hidden when disabled. + +## 6. Security & safety + +- [ ] Only the authenticated user's own address is ever checked (no lookup of others). +- [ ] API key read from config/secrets, never logged, never sent to the client. +- [ ] Feature fully inert (endpoints return `{ enabled:false }`, nav hidden) until a key is set. +- [ ] Rate-limit/backoff respected; failures degrade to cached/empty, never crash. +- [ ] HIBP responses cached to minimize external calls and avoid leaking usage patterns. diff --git a/docs/specs/feature-rules-engine.md b/docs/specs/feature-rules-engine.md new file mode 100644 index 0000000..f8bfd29 --- /dev/null +++ b/docs/specs/feature-rules-engine.md @@ -0,0 +1,330 @@ +# Spec: Automation Engine, Auto Clean Rules, Pinned, Mute, Age-based cleanup + +The keystone feature. Everything else in the build-out plugs into this engine. + +## 1. Goals + +- Persistent, user-defined **Auto Clean Rules**: *match conditions → action*, run + automatically against new and existing mail. +- **Hybrid safety**: safe actions auto-apply; destructive actions queue for approval. +- Reusable execution path for the per-sender policies (`feature-sender-policy.md`) + and the activity log (`feature-activity-log.md`). +- **Pinned** emails and **Muted** threads as first-class automation exemptions. + +## 2. Domain model + +### 2.1 New enums (`InboxIntel.Domain/Enums/Enums.cs`) + +```csharp +/// What an automation rule/policy does to a matched email. +/// Safe = applied automatically. Destructive = proposed, needs approval. +public enum AutomationActionType +{ + // ── Safe (auto-applied) ── + Archive = 0, // remove INBOX (skip inbox), keep the mail + MarkRead = 1, + Star = 2, + ApplyLabel = 3, // add a Gmail label (Deliver-To, Read-Later, Paused, Screener) + SkipInbox = 4, // remove INBOX only (used by Pause/Read-Later/Screener) + // ── Destructive (proposed, needs approval) ── + Trash = 50, // move to Trash (reversible in Gmail for 30 days) + KeepNewestCull= 51, // trash all-but-newest-N from a sender +} + +public static class AutomationActionTypeExtensions +{ + public static bool IsDestructive(this AutomationActionType t) => (int)t >= 50; +} + +/// Lifecycle of a single proposed/applied automation action. +public enum AutomationActionStatus +{ + Proposed = 0, // destructive, awaiting user approval + Applied = 1, // executed against Gmail + Rejected = 2, // user declined the proposal + Undone = 3, // user reverted an applied action + Failed = 4, // execution errored +} + +/// Where an automation action originated. +public enum AutomationSource +{ + Rule = 0, // an AutomationRule + SenderPolicy = 1, // Block / Pause / Read-Later / Keep-Newest / Deliver-To + Screener = 2, + AgeSweep = 3, // Trash-by-Age +} +``` + +### 2.2 `AutomationRule` (new entity) + +The user-facing "Auto Clean Rules". Structured match columns (no free-form JSON — keeps +EF querying and the UI simple). All match fields are nullable = "don't care"; a rule +matches an email when **every non-null condition** is satisfied (AND semantics). + +```csharp +public class AutomationRule : AuditableEntity +{ + public Guid Id { get; set; } = Guid.NewGuid(); + public Guid UserId { get; set; } + + public string Name { get; set; } = string.Empty; + public bool Enabled { get; set; } = true; + /// Lower number = evaluated first. Ties broken by CreatedUtc. + public int Priority { get; set; } + + // ── Match conditions (all nullable = ignore) ── + public string? SenderAddress { get; set; } // exact, lower-cased + public string? SenderDomain { get; set; } // e.g. "github.com" + public EmailCategory? Category { get; set; } + public string? SubjectContains { get; set; } // case-insensitive substring + public bool? IsUnread { get; set; } + public bool? HasAttachment { get; set; } + public bool? HasListUnsubscribe { get; set; } + public long? MinSizeBytes { get; set; } + public int? OlderThanDays { get; set; } // SentAtUtc older than N days + + // ── Action ── + public AutomationActionType Action { get; set; } + /// Label name for ApplyLabel actions (created under InboxIntel/ if not user-chosen). + public string? ActionLabelName { get; set; } + /// N for KeepNewestCull; null otherwise. + public int? ActionParam { get; set; } + + /// If true, also remove INBOX when applying a label (move vs. just tag). + public bool AlsoSkipInbox { get; set; } + + public int TimesApplied { get; set; } + public DateTimeOffset? LastRunUtc { get; set; } +} +``` + +> **Trash-by-Age** is just an `AutomationRule` with `OlderThanDays` set and +> `Action = Trash`. **Keep-Newest** is `Action = KeepNewestCull, ActionParam = N` +> (and is owned per-sender via `SenderPolicy`, which materializes one of these rules — +> see `feature-sender-policy.md`). No special-case code paths. + +### 2.3 `AutomationAction` (new entity) — queue **and** log + +One row per (action, email) the engine decides to take. This is the approval queue +when `Proposed`, and the activity log once `Applied`/`Rejected`/`Undone`. + +```csharp +public class AutomationAction : AuditableEntity +{ + public Guid Id { get; set; } = Guid.NewGuid(); + public Guid UserId { get; set; } + + public AutomationSource Source { get; set; } + public Guid? RuleId { get; set; } // AutomationRule, if Source=Rule + public Guid? SenderPolicyId { get; set; } // if Source=SenderPolicy/Screener + + public Guid EmailId { get; set; } + public Email? Email { get; set; } + public string GmailMessageId { get; set; } = string.Empty; // captured for undo + + public AutomationActionType Action { get; set; } + public AutomationActionStatus Status { get; set; } + + /// Label/state captured before applying, so Undo can restore it. + /// e.g. "had INBOX; had no InboxIntel/Paused". Serialized small JSON. + public string? UndoStateJson { get; set; } + + public string? ErrorMessage { get; set; } + public DateTimeOffset? AppliedUtc { get; set; } +} +``` + +> **Batching note:** destructive proposals are grouped in the UI by `(RuleId/Source, +> Action, SenderId)` so the user approves "Trash 412 emails from Groupon" as one click, +> not 412 rows. The grouping is a query concern, not a schema one. + +### 2.4 Exemption flags on existing entities + +- `Email.IsPinned` (bool, default false) — **Pinned Messages**. Engine skips pinned emails entirely. +- `MailThread.IsMuted` (bool, default false) — **Mute**. New messages in a muted thread are auto `SkipInbox` + `MarkRead` and never surface in other automation. + +Both require a migration (`AddAutomationCore`). + +## 3. Application layer + +### 3.1 `IGmailService` addition + +Screener/Read-Later/Pause/Deliver-To need to create labels on demand: + +```csharp +/// Returns the labelId for a label name, creating it (and any +/// "Parent/Child" nesting) if it does not exist. Idempotent. +Task EnsureLabelAsync(Guid userId, string name, CancellationToken ct = default); +``` + +Implement in `GmailApiService` using `Users.Labels.List` (already wired via +`ListLabelsAsync`) then `Users.Labels.Create` when missing. Cache name→id per request scope. + +### 3.2 New service interfaces (`IServices.cs`) + +```csharp +public interface IRuleService +{ + Task> ListRulesAsync(Guid userId, CancellationToken ct = default); + Task CreateRuleAsync(Guid userId, AutomationRuleInputDto input, CancellationToken ct = default); + Task UpdateRuleAsync(Guid userId, Guid ruleId, AutomationRuleInputDto input, CancellationToken ct = default); + Task DeleteRuleAsync(Guid userId, Guid ruleId, CancellationToken ct = default); + /// Dry-run: how many existing emails would this rule match right now? + Task PreviewRuleAsync(Guid userId, AutomationRuleInputDto input, CancellationToken ct = default); + Task SetEnabledAsync(Guid userId, Guid ruleId, bool enabled, CancellationToken ct = default); +} + +/// The execution engine. Evaluates rules + policies and applies/queues actions. +public interface IAutomationEngine +{ + /// Evaluate all enabled rules + policies for a user against + /// candidate emails (newly synced, or all if full=true). Safe actions apply + /// immediately; destructive ones are written as Proposed. + Task RunAsync(Guid userId, bool full = false, CancellationToken ct = default); +} + +/// The hybrid approval queue + activity log. +public interface IAutomationActionService +{ + Task> GetPendingAsync(Guid userId, CancellationToken ct = default); + Task> GetActivityAsync(Guid userId, int take = 100, CancellationToken ct = default); + Task ApproveAsync(Guid userId, IReadOnlyList actionIds, CancellationToken ct = default); + Task RejectAsync(Guid userId, IReadOnlyList actionIds, CancellationToken ct = default); + Task UndoAsync(Guid userId, IReadOnlyList actionIds, CancellationToken ct = default); +} +``` + +### 3.3 DTOs (`AutomationDtos.cs`, new file) + +```csharp +public record AutomationRuleInputDto( + string Name, bool Enabled, int Priority, + string? SenderAddress, string? SenderDomain, EmailCategory? Category, + string? SubjectContains, bool? IsUnread, bool? HasAttachment, + bool? HasListUnsubscribe, long? MinSizeBytes, int? OlderThanDays, + AutomationActionType Action, string? ActionLabelName, int? ActionParam, bool AlsoSkipInbox); + +public record AutomationRuleDto( /* input fields + */ Guid Id, int TimesApplied, DateTimeOffset? LastRunUtc); + +public record RuleMatchPreviewDto(int MatchCount, IReadOnlyList Sample); + +public record PendingActionGroupDto( + string GroupKey, AutomationActionType Action, string Description, + int Count, IReadOnlyList ActionIds, IReadOnlyList Sample); + +public record ActivityLogEntryDto( + Guid Id, AutomationSource Source, string Description, AutomationActionType Action, + AutomationActionStatus Status, int Count, DateTimeOffset When, bool CanUndo); +``` + +## 4. Engine semantics (`AutomationEngine`) + +Pseudo-flow of `RunAsync(userId, full)`: + +``` +1. Load enabled rules (ordered by Priority, CreatedUtc) and sender policies. +2. Determine candidate emails: + full == true → all non-trashed emails for the user + full == false → emails added/updated since LastAutomationRunUtc (SyncState) +3. Pre-load the whitelist (SenderPolicy where Kind=Allow) and pinned email ids. +4. For each candidate email: + skip if email.IsPinned + skip if sender is whitelisted + skip if thread.IsMuted (handled by its own SkipInbox+MarkRead pass) + for each rule in priority order: + if Matches(rule, email): + plan = (rule.Action, labelName, param) + if plan.Action.IsDestructive(): + upsert AutomationAction(Proposed) // no Gmail call + else: + apply via Gmail BatchModify (batched per action+label) + write AutomationAction(Applied) + break // first matching rule wins (priority); configurable later +5. Run age-based + keep-newest evaluation (see §5). +6. Flush batched safe actions to Gmail in BatchModify groups (≤1000 ids/call). +7. Update SyncState.LastAutomationRunUtc. +``` + +**Matching** is a pure function `bool Matches(AutomationRule, Email, Sender)` — +unit-testable, no I/O. Each non-null condition must hold. + +**Batching:** collect `(addLabelIds, removeLabelIds)` per email, group identical +label-sets, and issue one `BatchModifyAsync` per group. Trash proposals never call +Gmail in the engine — only on approval. + +## 5. Age-based & Keep-Newest + +- **Trash-by-Age** (`OlderThanDays` + `Trash`): matched in the normal candidate loop, + but because age changes over time independent of new mail, it must also run in a + **periodic full sweep**. The `AutomationWorker` (next section) calls `RunAsync(full:true)` + on a daily cadence so age rules catch up. +- **Keep-Newest** (`KeepNewestCull`, param N): evaluated per sender — order that + sender's non-pinned mail by `SentAtUtc desc`, skip the newest N, propose `Trash` for + the rest. Runs in the same daily full sweep. + +Both produce `Proposed` actions (destructive) → approval queue. + +## 6. Background worker (`AutomationWorker`) + +New `BackgroundService` mirroring `GmailSyncWorker`/`DigestWorker`: + +- Hourly tick. +- **After each incremental sync** the engine should also run on just-synced mail. + Cleanest hook: have `SyncService.RunIncrementalSyncAsync` (and the manual sync path) + call `IAutomationEngine.RunAsync(userId, full:false)` at the end, inside the same + scope. This gives near-real-time automation without a separate schedule. +- **Daily full sweep** at a configured hour (`AutomationOptions.SweepHourUtc`, default 3) + → `RunAsync(userId, full:true)` for age/keep-newest catch-up. +- Per-user try/catch, logs and continues. Early-out if the user has no enabled rules + or policies. + +`AutomationOptions` (new, `Configuration/Options.cs`): `Enabled` (default true), +`SweepHourUtc` (3), `MaxAutoActionsPerRun` (safety cap, default 5000). + +## 7. API (`AutomationController : ApiControllerBase`) + +All actions scoped to `UserId`. + +| Method | Route | Purpose | +|--------|-------|---------| +| GET | `/automation/rules` | list rules | +| POST | `/automation/rules` | create | +| PUT | `/automation/rules/{id}` | update | +| DELETE | `/automation/rules/{id}` | delete | +| POST | `/automation/rules/preview` | dry-run match count + sample | +| PUT | `/automation/rules/{id}/enabled` | toggle | +| GET | `/automation/pending` | grouped approval queue | +| POST | `/automation/pending/approve` | `{ actionIds[] }` → execute + log | +| POST | `/automation/pending/reject` | `{ actionIds[] }` | +| GET | `/automation/activity?take=100` | activity log | +| POST | `/automation/activity/undo` | `{ actionIds[] }` → revert | +| POST | `/email/{id}/pin` · `/unpin` | Pinned Messages | +| POST | `/thread/{id}/mute` · `/unmute` | Mute | + +## 8. Frontend (built on the new design system — see `ui-overhaul.md`) + +- **Rules page** (`/app/rules`): table of rules with enable toggles; a rule editor + drawer (Sheet) with condition builder + action picker + live "matches N emails" + preview; priority drag-reorder. +- **Review queue** (`/app/review` or a badge in the topbar): grouped pending + destructive actions, Approve/Reject per group, "Approve all". +- **Pin** affordance on email rows/detail (📌). **Mute** affordance on thread views. +- Pending count surfaces as a badge in the sidebar/topbar. + +## 9. Security & safety checklist + +- [ ] Every endpoint filters by `UserId`; rule/action ownership verified before mutate. +- [ ] Destructive actions can ONLY be executed via `ApproveAsync`, never by the engine. +- [ ] Hard delete is never an `AutomationActionType` — not automatable. +- [ ] Pinned/whitelisted exemptions enforced in `Matches`/candidate selection, with tests. +- [ ] `MaxAutoActionsPerRun` cap prevents a misconfigured rule from mass-acting; overflow logged. +- [ ] Undo restores prior label state from `UndoStateJson`. + +## 10. Test plan + +- Unit: `Matches()` truth table across every condition + AND combinations. +- Unit: destructive vs safe routing (proposed vs applied). +- Unit: Keep-Newest ordering & pinned exemption. +- Integration: engine run with a fake `IGmailService` asserts BatchModify groups + queue rows. +- Integration: approve → Gmail trash called + status Applied; undo → labels restored. diff --git a/docs/specs/feature-sender-policy.md b/docs/specs/feature-sender-policy.md new file mode 100644 index 0000000..0c1c70a --- /dev/null +++ b/docs/specs/feature-sender-policy.md @@ -0,0 +1,148 @@ +# Spec: Sender Policy — Block, Whitelist, Screener, Pause, Read-Later, Keep-Newest, Deliver-To + +Per-sender ongoing behaviors. These are toggles the user sets from the Senders UI (or +the Screener queue), distinct from the condition-based `AutomationRule`s. They share the +same execution engine and `AutomationAction` queue/log (`feature-rules-engine.md`). + +## 1. Domain model + +### 1.1 `SenderPolicy` (new entity) + +One row per sender that has any non-default policy. Created lazily. + +```csharp +public class SenderPolicy : AuditableEntity +{ + public Guid Id { get; set; } = Guid.NewGuid(); + public Guid UserId { get; set; } + public Guid SenderId { get; set; } + public Sender? Sender { get; set; } + + public SenderDisposition Disposition { get; set; } = SenderDisposition.None; + + // Independent toggles (a sender can be Read-Later AND Keep-Newest, etc.) + public bool ReadLater { get; set; } // route new mail to InboxIntel/Read Later, skip inbox + public bool Paused { get; set; } // hold new mail in InboxIntel/Paused, skip inbox + public int? KeepNewestCount { get; set; } // trash all-but-newest-N (destructive → proposed) + public int? TrashOlderThanDays { get; set; } // per-sender trash-by-age (destructive → proposed) + public string? DeliverToLabel { get; set; } // auto-apply this label to new mail + public bool DeliverToSkipInbox { get; set; } // move vs. tag for Deliver-To + + public DateTimeOffset? UpdatedUtc { get; set; } +} + +public enum SenderDisposition +{ + None = 0, + Allow = 1, // Whitelist — exempt from ALL automation, always reaches inbox + Block = 2, // auto-trash new mail (destructive → proposed) + Screening= 3, // unknown sender quarantined pending approve/block +} +``` + +> `Allow` (Whitelist) is checked first in the engine and short-circuits every other +> rule/policy for that sender. `Block` proposes Trash on each new message. `Screening` +> applies the `InboxIntel/Screener` label + skip-inbox and surfaces in the Screener queue. + +### 1.2 Managed labels (created via `EnsureLabelAsync`) + +| Feature | Label | On apply | +|---------|-------|----------| +| Screener | `InboxIntel/Screener` | + skip inbox | +| Pause | `InboxIntel/Paused` | + skip inbox | +| Read-Later | `InboxIntel/Read Later` | + skip inbox | +| Deliver-To | user-chosen (any Gmail label) | optional skip inbox | + +Whitelist/Block need no label (exempt / trash). + +## 2. Screener semantics + +The Screener catches mail from **first-seen senders**. + +- A sender is "known" if the user has ever received mail from them **before** the + screener was enabled, or has explicitly Allowed/Blocked them. Establish a baseline + `ScreenerEnabledUtc` on the `User` when the user turns Screener on, so existing + contacts aren't all quarantined retroactively. +- During automation, a candidate email from a sender with **no prior history before + `ScreenerEnabledUtc`** and **no disposition** → create `SenderPolicy { Disposition = + Screening }`, apply `InboxIntel/Screener` + skip inbox, log `AutomationAction(Source=Screener, + Applied)` (this is a *safe* action — just labeling/skip-inbox, nothing destroyed). +- **Screener queue UI**: lists screening senders with a sample subject + count. + - **Approve** → `Disposition = Allow`; remove `InboxIntel/Screener`, restore to inbox + for held mail; future mail flows normally. + - **Block** → `Disposition = Block`; propose Trash for held mail; future mail auto-proposed for trash. +- Screener is **opt-in** (a `User.ScreenerEnabled` flag, default false) because it + actively reroutes mail. + +`User` additions (migration): `ScreenerEnabled` (bool), `ScreenerEnabledUtc` (DateTimeOffset?). + +## 3. Pause vs Read-Later vs Deliver-To + +All three are **safe** (label + optional skip-inbox), so they auto-apply: + +- **Pause**: temporarily stop a sender cluttering the inbox without unsubscribing. New + mail → `InboxIntel/Paused` + skip inbox. **Resume** removes the policy and (optionally) + re-inboxes held mail. +- **Read-Later**: newsletters you want to read on your own time → `InboxIntel/Read Later` + + skip inbox. A "Read Later" view in-app lists them. +- **Deliver-To**: auto-file a sender's mail under a chosen label (e.g. "Receipts"), + optionally skipping the inbox. + +## 4. Block & Keep-Newest & per-sender Trash-by-Age + +Destructive → **proposed**, surfaced in the Review queue: + +- **Block**: each new message from a blocked sender → propose Trash. (We never + hard-delete; Gmail Trash auto-purges after 30 days.) +- **Keep-Newest** / **per-sender Trash-by-Age**: evaluated in the daily full sweep + (`feature-rules-engine.md` §5), proposing Trash for the cull set. + +## 5. Application layer + +```csharp +public interface ISenderPolicyService +{ + Task GetAsync(Guid userId, Guid senderId, CancellationToken ct = default); + Task SetAsync(Guid userId, Guid senderId, SenderPolicyInputDto input, CancellationToken ct = default); + Task ClearAsync(Guid userId, Guid senderId, CancellationToken ct = default); + + // Screener + Task> GetScreenerQueueAsync(Guid userId, CancellationToken ct = default); + Task ApproveSenderAsync(Guid userId, Guid senderId, CancellationToken ct = default); // → Allow + Task BlockSenderAsync(Guid userId, Guid senderId, CancellationToken ct = default); // → Block + Task SetScreenerEnabledAsync(Guid userId, bool enabled, CancellationToken ct = default); +} +``` + +The engine reads `SenderPolicy` rows alongside `AutomationRule`s in `RunAsync`. Policy +evaluation order: **Allow (exempt) → Block → Pause → Read-Later → Deliver-To → +Keep-Newest/Trash-by-Age**. A whitelisted sender exits immediately. + +## 6. API (`SenderPolicyController` or extend an existing senders controller) + +| Method | Route | Purpose | +|--------|-------|---------| +| GET | `/senders/{id}/policy` | current policy | +| PUT | `/senders/{id}/policy` | set disposition/toggles | +| DELETE | `/senders/{id}/policy` | clear | +| GET | `/screener` | screener queue | +| POST | `/screener/{senderId}/approve` | whitelist | +| POST | `/screener/{senderId}/block` | block + propose trash | +| PUT | `/screener/enabled` | enable/disable screener | + +## 7. Frontend + +- **Senders page**: each sender row gets a policy menu (DropdownMenu): Whitelist, Block, + Pause, Read-Later, Deliver-To→(label picker), Keep-Newest→(N), Trash-by-Age→(days). + Active policies shown as small badges on the row. +- **Screener page** (`/app/screener`): queue of screening senders, Approve/Block per row, + bulk approve/block, and a master enable toggle with an explainer. +- **Read Later view** (`/app/read-later`): mail tagged `InboxIntel/Read Later`. + +## 8. Security & safety + +- [ ] Policy rows verified to belong to `UserId` before any read/write. +- [ ] Block/Keep-Newest/Trash-by-Age only ever **propose** (hybrid rule) — never auto-trash. +- [ ] Whitelist exemption enforced before any other policy/rule, with a test. +- [ ] Screener baseline (`ScreenerEnabledUtc`) prevents retroactive mass-quarantine. +- [ ] Resume/Approve restores inbox state from `AutomationAction.UndoStateJson`. diff --git a/docs/specs/ui-overhaul.md b/docs/specs/ui-overhaul.md new file mode 100644 index 0000000..66ff40c --- /dev/null +++ b/docs/specs/ui-overhaul.md @@ -0,0 +1,145 @@ +# Spec: UI Overhaul — Stripe/Notion aesthetic, Tailwind + shadcn-style, light+dark + +A full visual rebuild: clean, airy, modern, light-first with a polished dark mode. +Rolled out **incrementally** — design system first, then page-by-page — so the app +keeps working throughout. + +## 1. Stack + +Add to `frontend`: + +- **tailwindcss** (+ `postcss`, `autoprefixer`) — utility styling. +- **Radix UI primitives** (`@radix-ui/react-*`: dialog, dropdown-menu, tabs, tooltip, + switch, popover, toast, separator, scroll-area) — accessible behavior. +- **class-variance-authority** + **tailwind-merge** + **clsx** — the shadcn component pattern. +- **lucide-react** — icon set (clean, consistent; replaces ad-hoc emoji where it helps). + +Config: +- `tailwind.config.js` — content globs over `index.html` + `src/**/*.{js,jsx}`; theme + extends map to CSS variables (below); `darkMode: 'class'`. +- `postcss.config.js`. A `src/index.css` with `@tailwind base/components/utilities` + + the token `:root` / `.dark` blocks. Keep the old `styles.css` importing until a page is + migrated, then drop per-page. + +## 2. Design tokens (CSS variables, HSL) + +Defined once in `src/index.css`; Tailwind theme references them so `bg-background`, +`text-foreground`, `bg-primary`, etc. just work and flip with `.dark`. + +```css +:root { + /* Neutrals — warm-tinted slate (Notion-ish paper) */ + --background: 0 0% 100%; + --foreground: 222 22% 12%; + --card: 0 0% 100%; + --muted: 220 16% 96%; + --muted-foreground: 220 9% 46%; + --border: 220 16% 90%; + --input: 220 16% 90%; + --ring: 245 75% 60%; + + /* Brand accent — indigo/iris (modern SaaS, Stripe-blurple cousin) */ + --primary: 245 75% 59%; /* #5b5bf0-ish */ + --primary-foreground: 0 0% 100%; + + /* Semantic */ + --success: 152 56% 40%; + --warning: 38 92% 50%; + --danger: 0 72% 51%; + --danger-foreground: 0 0% 100%; + + --radius: 0.625rem; /* soft, modern corners */ +} + +.dark { + --background: 224 32% 9%; /* deep slate, not pure black */ + --foreground: 220 18% 92%; + --card: 224 28% 12%; + --muted: 223 22% 17%; + --muted-foreground: 220 12% 64%; + --border: 223 20% 20%; + --input: 223 20% 22%; + --ring: 245 80% 66%; + --primary: 245 80% 67%; + --primary-foreground: 224 32% 9%; + --success: 152 50% 50%; + --warning: 38 92% 58%; + --danger: 0 70% 60%; +} +``` + +### Proposed palette (for sign-off) + +| Token | Light | Dark | Use | +|-------|-------|------|-----| +| Primary (accent) | **Indigo `#5b5bf0`** | `#7c7cf5` | buttons, links, active nav, focus ring | +| Background | `#ffffff` | `#11151f` | app canvas | +| Card/surface | `#ffffff` | `#161b27` | panels, cards | +| Muted surface | `#f3f5f9` | `#1f2533` | subtle fills, hover | +| Border | `#e3e8ef` | `#2b3242` | hairlines | +| Text | `#191e2b` | `#e7eaf2` | body | +| Muted text | `#6b7280` | `#9aa3b2` | secondary | +| Success | `#2f9e6b` | `#3dbd86` | healthy, succeeded | +| Warning | `#f5a623` | `#f7b84b` | caution, pending | +| Danger | `#e23b3b` | `#ef5a5a` | destructive, failed | + +> **Alternatives if indigo isn't your taste** (pick one and I'll swap the single token): +> Emerald `#10b981` (calm, "clean"), Violet `#7c3aed` (premium), Teal `#0d9488` (fresh), +> Blue `#2563eb` (classic/trustworthy). Logo stays as-is; accent just needs to sit well beside it. + +## 3. Component library (`src/components/ui/`) + +Hand-built shadcn-style primitives, each a thin `cva` wrapper over Tailwind + (where +interactive) a Radix primitive: + +`button`, `card`, `input`, `textarea`, `select`, `checkbox`, `switch`, `badge`, +`dialog`, `sheet` (side drawer), `dropdown-menu`, `tabs`, `tooltip`, `toast` (+ a +`useToast` hook to replace ad-hoc toast state), `table`, `skeleton`, `separator`, +`avatar`, `empty-state`. + +Plus app-level shells: `PageHeader`, `Sidebar`, `Topbar`, `StatCard`, +`ThemeToggle` (writes `.dark` on ``, persists to `localStorage`). + +## 4. Layout language + +- **Sidebar**: 248px, `bg-card`, hairline border, grouped nav with section labels + (Overview · Cleanup · Automation · Account). Lucide icons. Active item = soft primary + tint pill. Collapsible to icon-rail on narrow widths. +- **Topbar**: page title + breadcrumbs left; sync status, theme toggle, digest toggle, + pending-review badge, "Sync now", avatar right. Sticky, subtle bottom border. +- **Content**: max-width container, generous padding (`p-6`/`p-8`), cards with + `rounded-[--radius]`, `border`, soft shadow (`shadow-sm`), 16–24px gaps. +- **Density**: comfortable default; tables get a compact variant for big lists. +- **Motion**: 150–200ms ease transitions on hover/expand; Radix-driven enter/exit on + dialogs/sheets/toasts. Respect `prefers-reduced-motion`. + +## 5. Page-by-page migration map + +| Order | Page | Notes | +|-------|------|-------| +| 0 | **Tooling + tokens + primitives** | no visual swap yet; build the system | +| 1 | **App shell** (`Layout.jsx`) | sidebar + topbar + theme toggle — biggest immediate lift | +| 2 | **Dashboard** | StatCards, chart cards restyled (keep chart.js, theme its colors via tokens) | +| 3 | **Senders** | list/detail split, new email rows, policy dropdown (ties to sender-policy spec) | +| 4 | **Unsubscribe** | table → new `table` primitive, confidence as colored `badge`/meter | +| 5 | **Search / Folders / Cleanup** | shared list components, filter bar, bulk toolbar restyle | +| 6 | **New feature pages** | Rules, Review queue, Screener, Activity, Privacy, Read-Later — built native | +| 7 | **Landing** | polish to match the new system | +| 8 | **Retire `styles.css`** | delete once nothing imports it | + +Each increment: convert the page, verify `npm run build`, screenshot/sanity-check, commit. + +## 6. Accessibility & quality bar + +- Radix primitives give focus management, ESC/overlay behavior, ARIA for free — don't + hand-roll dialogs/menus. +- Visible focus ring (`--ring`) on all interactive elements. +- Color is never the only signal (icons/labels alongside semantic colors). +- Contrast ≥ WCAG AA in both themes for text and primary buttons. +- Keyboard shortcuts (already present) preserved and surfaced in a `?` cheat-sheet dialog. + +## 7. Out of scope + +- No logo redesign (keeping current `Logo.jsx`). +- No mobile-dedicated layouts beyond responsive degradation (desktop-first decision). +- No new charting library (theme the existing chart.js).