Inboxintel/docs/specs/feature-rules-engine.md

# Spec: Automation Engine, Auto Clean Rules, Pinned, Mute, Age-based cleanup

The keystone feature. Everything else in the build-out plugs into this engine.

## 1. Goals

- Persistent, user-defined **Auto Clean Rules**: *match conditions → action*, run
  automatically against new and existing mail.
- **Hybrid safety**: safe actions auto-apply; destructive actions queue for approval.
- Reusable execution path for the per-sender policies (`feature-sender-policy.md`)
  and the activity log (`feature-activity-log.md`).
- **Pinned** emails and **Muted** threads as first-class automation exemptions.

## 2. Domain model

### 2.1 New enums (`InboxIntel.Domain/Enums/Enums.cs`)

```csharp
/// <summary>What an automation rule/policy does to a matched email.
/// Safe = applied automatically. Destructive = proposed, needs approval.</summary>
public enum AutomationActionType
{
    // ── Safe (auto-applied) ──
    Archive       = 0,  // remove INBOX (skip inbox), keep the mail
    MarkRead      = 1,
    Star          = 2,
    ApplyLabel    = 3,  // add a Gmail label (Deliver-To, Read-Later, Paused, Screener)
    SkipInbox     = 4,  // remove INBOX only (used by Pause/Read-Later/Screener)
    // ── Destructive (proposed, needs approval) ──
    Trash         = 50, // move to Trash (reversible in Gmail for 30 days)
    KeepNewestCull= 51, // trash all-but-newest-N from a sender
}

public static class AutomationActionTypeExtensions
{
    public static bool IsDestructive(this AutomationActionType t) => (int)t >= 50;
}

/// <summary>Lifecycle of a single proposed/applied automation action.</summary>
public enum AutomationActionStatus
{
    Proposed = 0,  // destructive, awaiting user approval
    Applied  = 1,  // executed against Gmail
    Rejected = 2,  // user declined the proposal
    Undone   = 3,  // user reverted an applied action
    Failed   = 4,  // execution errored
}

/// <summary>Where an automation action originated.</summary>
public enum AutomationSource
{
    Rule         = 0,  // an AutomationRule
    SenderPolicy = 1,  // Block / Pause / Read-Later / Keep-Newest / Deliver-To
    Screener     = 2,
    AgeSweep     = 3,  // Trash-by-Age
}
```

### 2.2 `AutomationRule` (new entity)

The user-facing "Auto Clean Rules". Structured match columns (no free-form JSON — keeps
EF querying and the UI simple). All match fields are nullable = "don't care"; a rule
matches an email when **every non-null condition** is satisfied (AND semantics).

```csharp
public class AutomationRule : AuditableEntity
{
    public Guid Id { get; set; } = Guid.NewGuid();
    public Guid UserId { get; set; }

    public string Name { get; set; } = string.Empty;
    public bool Enabled { get; set; } = true;
    /// <summary>Lower number = evaluated first. Ties broken by CreatedUtc.</summary>
    public int Priority { get; set; }

    // ── Match conditions (all nullable = ignore) ──
    public string? SenderAddress { get; set; }     // exact, lower-cased
    public string? SenderDomain { get; set; }      // e.g. "github.com"
    public EmailCategory? Category { get; set; }
    public string? SubjectContains { get; set; }   // case-insensitive substring
    public bool? IsUnread { get; set; }
    public bool? HasAttachment { get; set; }
    public bool? HasListUnsubscribe { get; set; }
    public long? MinSizeBytes { get; set; }
    public int? OlderThanDays { get; set; }         // SentAtUtc older than N days

    // ── Action ──
    public AutomationActionType Action { get; set; }
    /// <summary>Label name for ApplyLabel actions (created under InboxIntel/ if not user-chosen).</summary>
    public string? ActionLabelName { get; set; }
    /// <summary>N for KeepNewestCull; null otherwise.</summary>
    public int? ActionParam { get; set; }

    /// <summary>If true, also remove INBOX when applying a label (move vs. just tag).</summary>
    public bool AlsoSkipInbox { get; set; }

    public int TimesApplied { get; set; }
    public DateTimeOffset? LastRunUtc { get; set; }
}
```

> **Trash-by-Age** is just an `AutomationRule` with `OlderThanDays` set and
> `Action = Trash`. **Keep-Newest** is `Action = KeepNewestCull, ActionParam = N`
> (and is owned per-sender via `SenderPolicy`, which materializes one of these rules —
> see `feature-sender-policy.md`). No special-case code paths.

### 2.3 `AutomationAction` (new entity) — queue **and** log

One row per (action, email) the engine decides to take. This is the approval queue
when `Proposed`, and the activity log once `Applied`/`Rejected`/`Undone`.

```csharp
public class AutomationAction : AuditableEntity
{
    public Guid Id { get; set; } = Guid.NewGuid();
    public Guid UserId { get; set; }

    public AutomationSource Source { get; set; }
    public Guid? RuleId { get; set; }              // AutomationRule, if Source=Rule
    public Guid? SenderPolicyId { get; set; }      // if Source=SenderPolicy/Screener

    public Guid EmailId { get; set; }
    public Email? Email { get; set; }
    public string GmailMessageId { get; set; } = string.Empty; // captured for undo

    public AutomationActionType Action { get; set; }
    public AutomationActionStatus Status { get; set; }

    /// <summary>Label/state captured before applying, so Undo can restore it.
    /// e.g. "had INBOX; had no InboxIntel/Paused". Serialized small JSON.</summary>
    public string? UndoStateJson { get; set; }

    public string? ErrorMessage { get; set; }
    public DateTimeOffset? AppliedUtc { get; set; }
}
```

> **Batching note:** destructive proposals are grouped in the UI by `(RuleId/Source,
> Action, SenderId)` so the user approves "Trash 412 emails from Groupon" as one click,
> not 412 rows. The grouping is a query concern, not a schema one.

### 2.4 Exemption flags on existing entities

- `Email.IsPinned` (bool, default false) — **Pinned Messages**. Engine skips pinned emails entirely.
- `MailThread.IsMuted` (bool, default false) — **Mute**. New messages in a muted thread are auto `SkipInbox` + `MarkRead` and never surface in other automation.

Both require a migration (`AddAutomationCore`).

## 3. Application layer

### 3.1 `IGmailService` addition

Screener/Read-Later/Pause/Deliver-To need to create labels on demand:

```csharp
/// <summary>Returns the labelId for a label name, creating it (and any
/// "Parent/Child" nesting) if it does not exist. Idempotent.</summary>
Task<string> EnsureLabelAsync(Guid userId, string name, CancellationToken ct = default);
```

Implement in `GmailApiService` using `Users.Labels.List` (already wired via
`ListLabelsAsync`) then `Users.Labels.Create` when missing. Cache name→id per request scope.

### 3.2 New service interfaces (`IServices.cs`)

```csharp
public interface IRuleService
{
    Task<IReadOnlyList<AutomationRuleDto>> ListRulesAsync(Guid userId, CancellationToken ct = default);
    Task<AutomationRuleDto> CreateRuleAsync(Guid userId, AutomationRuleInputDto input, CancellationToken ct = default);
    Task<AutomationRuleDto> UpdateRuleAsync(Guid userId, Guid ruleId, AutomationRuleInputDto input, CancellationToken ct = default);
    Task DeleteRuleAsync(Guid userId, Guid ruleId, CancellationToken ct = default);
    /// <summary>Dry-run: how many existing emails would this rule match right now?</summary>
    Task<RuleMatchPreviewDto> PreviewRuleAsync(Guid userId, AutomationRuleInputDto input, CancellationToken ct = default);
    Task SetEnabledAsync(Guid userId, Guid ruleId, bool enabled, CancellationToken ct = default);
}

/// <summary>The execution engine. Evaluates rules + policies and applies/queues actions.</summary>
public interface IAutomationEngine
{
    /// <summary>Evaluate all enabled rules + policies for a user against
    /// candidate emails (newly synced, or all if full=true). Safe actions apply
    /// immediately; destructive ones are written as Proposed.</summary>
    Task RunAsync(Guid userId, bool full = false, CancellationToken ct = default);
}

/// <summary>The hybrid approval queue + activity log.</summary>
public interface IAutomationActionService
{
    Task<IReadOnlyList<PendingActionGroupDto>> GetPendingAsync(Guid userId, CancellationToken ct = default);
    Task<IReadOnlyList<ActivityLogEntryDto>> GetActivityAsync(Guid userId, int take = 100, CancellationToken ct = default);
    Task ApproveAsync(Guid userId, IReadOnlyList<Guid> actionIds, CancellationToken ct = default);
    Task RejectAsync(Guid userId, IReadOnlyList<Guid> actionIds, CancellationToken ct = default);
    Task UndoAsync(Guid userId, IReadOnlyList<Guid> actionIds, CancellationToken ct = default);
}
```

### 3.3 DTOs (`AutomationDtos.cs`, new file)

```csharp
public record AutomationRuleInputDto(
    string Name, bool Enabled, int Priority,
    string? SenderAddress, string? SenderDomain, EmailCategory? Category,
    string? SubjectContains, bool? IsUnread, bool? HasAttachment,
    bool? HasListUnsubscribe, long? MinSizeBytes, int? OlderThanDays,
    AutomationActionType Action, string? ActionLabelName, int? ActionParam, bool AlsoSkipInbox);

public record AutomationRuleDto( /* input fields + */ Guid Id, int TimesApplied, DateTimeOffset? LastRunUtc);

public record RuleMatchPreviewDto(int MatchCount, IReadOnlyList<EmailSummaryDto> Sample);

public record PendingActionGroupDto(
    string GroupKey, AutomationActionType Action, string Description,
    int Count, IReadOnlyList<Guid> ActionIds, IReadOnlyList<EmailSummaryDto> Sample);

public record ActivityLogEntryDto(
    Guid Id, AutomationSource Source, string Description, AutomationActionType Action,
    AutomationActionStatus Status, int Count, DateTimeOffset When, bool CanUndo);
```

## 4. Engine semantics (`AutomationEngine`)

Pseudo-flow of `RunAsync(userId, full)`:

```
1. Load enabled rules (ordered by Priority, CreatedUtc) and sender policies.
2. Determine candidate emails:
     full == true  → all non-trashed emails for the user
     full == false → emails added/updated since LastAutomationRunUtc (SyncState)
3. Pre-load the whitelist (SenderPolicy where Kind=Allow) and pinned email ids.
4. For each candidate email:
     skip if email.IsPinned
     skip if sender is whitelisted
     skip if thread.IsMuted (handled by its own SkipInbox+MarkRead pass)
     for each rule in priority order:
        if Matches(rule, email):
           plan = (rule.Action, labelName, param)
           if plan.Action.IsDestructive():
               upsert AutomationAction(Proposed)   // no Gmail call
           else:
               apply via Gmail BatchModify (batched per action+label)
               write AutomationAction(Applied)
           break   // first matching rule wins (priority); configurable later
5. Run age-based + keep-newest evaluation (see §5).
6. Flush batched safe actions to Gmail in BatchModify groups (≤1000 ids/call).
7. Update SyncState.LastAutomationRunUtc.
```

**Matching** is a pure function `bool Matches(AutomationRule, Email, Sender)` —
unit-testable, no I/O. Each non-null condition must hold.

**Batching:** collect `(addLabelIds, removeLabelIds)` per email, group identical
label-sets, and issue one `BatchModifyAsync` per group. Trash proposals never call
Gmail in the engine — only on approval.

## 5. Age-based & Keep-Newest

- **Trash-by-Age** (`OlderThanDays` + `Trash`): matched in the normal candidate loop,
  but because age changes over time independent of new mail, it must also run in a
  **periodic full sweep**. The `AutomationWorker` (next section) calls `RunAsync(full:true)`
  on a daily cadence so age rules catch up.
- **Keep-Newest** (`KeepNewestCull`, param N): evaluated per sender — order that
  sender's non-pinned mail by `SentAtUtc desc`, skip the newest N, propose `Trash` for
  the rest. Runs in the same daily full sweep.

Both produce `Proposed` actions (destructive) → approval queue.

## 6. Background worker (`AutomationWorker`)

New `BackgroundService` mirroring `GmailSyncWorker`/`DigestWorker`:

- Hourly tick.
- **After each incremental sync** the engine should also run on just-synced mail.
  Cleanest hook: have `SyncService.RunIncrementalSyncAsync` (and the manual sync path)
  call `IAutomationEngine.RunAsync(userId, full:false)` at the end, inside the same
  scope. This gives near-real-time automation without a separate schedule.
- **Daily full sweep** at a configured hour (`AutomationOptions.SweepHourUtc`, default 3)
  → `RunAsync(userId, full:true)` for age/keep-newest catch-up.
- Per-user try/catch, logs and continues. Early-out if the user has no enabled rules
  or policies.

`AutomationOptions` (new, `Configuration/Options.cs`): `Enabled` (default true),
`SweepHourUtc` (3), `MaxAutoActionsPerRun` (safety cap, default 5000).

## 7. API (`AutomationController : ApiControllerBase`)

All actions scoped to `UserId`.

| Method | Route | Purpose |
|--------|-------|---------|
| GET | `/automation/rules` | list rules |
| POST | `/automation/rules` | create |
| PUT | `/automation/rules/{id}` | update |
| DELETE | `/automation/rules/{id}` | delete |
| POST | `/automation/rules/preview` | dry-run match count + sample |
| PUT | `/automation/rules/{id}/enabled` | toggle |
| GET | `/automation/pending` | grouped approval queue |
| POST | `/automation/pending/approve` | `{ actionIds[] }` → execute + log |
| POST | `/automation/pending/reject` | `{ actionIds[] }` |
| GET | `/automation/activity?take=100` | activity log |
| POST | `/automation/activity/undo` | `{ actionIds[] }` → revert |
| POST | `/email/{id}/pin` · `/unpin` | Pinned Messages |
| POST | `/thread/{id}/mute` · `/unmute` | Mute |

## 8. Frontend (built on the new design system — see `ui-overhaul.md`)

- **Rules page** (`/app/rules`): table of rules with enable toggles; a rule editor
  drawer (Sheet) with condition builder + action picker + live "matches N emails"
  preview; priority drag-reorder.
- **Review queue** (`/app/review` or a badge in the topbar): grouped pending
  destructive actions, Approve/Reject per group, "Approve all".
- **Pin** affordance on email rows/detail (📌). **Mute** affordance on thread views.
- Pending count surfaces as a badge in the sidebar/topbar.

## 9. Security & safety checklist

- [ ] Every endpoint filters by `UserId`; rule/action ownership verified before mutate.
- [ ] Destructive actions can ONLY be executed via `ApproveAsync`, never by the engine.
- [ ] Hard delete is never an `AutomationActionType` — not automatable.
- [ ] Pinned/whitelisted exemptions enforced in `Matches`/candidate selection, with tests.
- [ ] `MaxAutoActionsPerRun` cap prevents a misconfigured rule from mass-acting; overflow logged.
- [ ] Undo restores prior label state from `UndoStateJson`.

## 10. Test plan

- Unit: `Matches()` truth table across every condition + AND combinations.
- Unit: destructive vs safe routing (proposed vs applied).
- Unit: Keep-Newest ordering & pinned exemption.
- Integration: engine run with a fake `IGmailService` asserts BatchModify groups + queue rows.
- Integration: approve → Gmail trash called + status Applied; undo → labels restored.