Inboxintel/docs/specs/feature-rules-engine.md

Spec: Automation Engine, Auto Clean Rules, Pinned, Mute, Age-based cleanup

The keystone feature. Everything else in the build-out plugs into this engine.

1. Goals

• Persistent, user-defined Auto Clean Rules: match conditions → action, run automatically against new and existing mail.
• Hybrid safety: safe actions auto-apply; destructive actions queue for approval.
• Reusable execution path for the per-sender policies (feature-sender-policy.md) and the activity log (feature-activity-log.md).
• Pinned emails and Muted threads as first-class automation exemptions.

2. Domain model

2.1 New enums (InboxIntel.Domain/Enums/Enums.cs)

/// <summary>What an automation rule/policy does to a matched email.
/// Safe = applied automatically. Destructive = proposed, needs approval.</summary>
public enum AutomationActionType
{
    // ── Safe (auto-applied) ──
    Archive       = 0,  // remove INBOX (skip inbox), keep the mail
    MarkRead      = 1,
    Star          = 2,
    ApplyLabel    = 3,  // add a Gmail label (Deliver-To, Read-Later, Paused, Screener)
    SkipInbox     = 4,  // remove INBOX only (used by Pause/Read-Later/Screener)
    // ── Destructive (proposed, needs approval) ──
    Trash         = 50, // move to Trash (reversible in Gmail for 30 days)
    KeepNewestCull= 51, // trash all-but-newest-N from a sender
}

public static class AutomationActionTypeExtensions
{
    public static bool IsDestructive(this AutomationActionType t) => (int)t >= 50;
}

/// <summary>Lifecycle of a single proposed/applied automation action.</summary>
public enum AutomationActionStatus
{
    Proposed = 0,  // destructive, awaiting user approval
    Applied  = 1,  // executed against Gmail
    Rejected = 2,  // user declined the proposal
    Undone   = 3,  // user reverted an applied action
    Failed   = 4,  // execution errored
}

/// <summary>Where an automation action originated.</summary>
public enum AutomationSource
{
    Rule         = 0,  // an AutomationRule
    SenderPolicy = 1,  // Block / Pause / Read-Later / Keep-Newest / Deliver-To
    Screener     = 2,
    AgeSweep     = 3,  // Trash-by-Age
}

2.2 AutomationRule (new entity)

The user-facing "Auto Clean Rules". Structured match columns (no free-form JSON — keeps EF querying and the UI simple). All match fields are nullable = "don't care"; a rule matches an email when every non-null condition is satisfied (AND semantics).

public class AutomationRule : AuditableEntity
{
    public Guid Id { get; set; } = Guid.NewGuid();
    public Guid UserId { get; set; }

    public string Name { get; set; } = string.Empty;
    public bool Enabled { get; set; } = true;
    /// <summary>Lower number = evaluated first. Ties broken by CreatedUtc.</summary>
    public int Priority { get; set; }

    // ── Match conditions (all nullable = ignore) ──
    public string? SenderAddress { get; set; }     // exact, lower-cased
    public string? SenderDomain { get; set; }      // e.g. "github.com"
    public EmailCategory? Category { get; set; }
    public string? SubjectContains { get; set; }   // case-insensitive substring
    public bool? IsUnread { get; set; }
    public bool? HasAttachment { get; set; }
    public bool? HasListUnsubscribe { get; set; }
    public long? MinSizeBytes { get; set; }
    public int? OlderThanDays { get; set; }         // SentAtUtc older than N days

    // ── Action ──
    public AutomationActionType Action { get; set; }
    /// <summary>Label name for ApplyLabel actions (created under InboxIntel/ if not user-chosen).</summary>
    public string? ActionLabelName { get; set; }
    /// <summary>N for KeepNewestCull; null otherwise.</summary>
    public int? ActionParam { get; set; }

    /// <summary>If true, also remove INBOX when applying a label (move vs. just tag).</summary>
    public bool AlsoSkipInbox { get; set; }

    public int TimesApplied { get; set; }
    public DateTimeOffset? LastRunUtc { get; set; }
}

Trash-by-Age is just an AutomationRule with OlderThanDays set and Action = Trash. Keep-Newest is Action = KeepNewestCull, ActionParam = N (and is owned per-sender via SenderPolicy, which materializes one of these rules — see feature-sender-policy.md). No special-case code paths.

2.3 AutomationAction (new entity) — queue and log

One row per (action, email) the engine decides to take. This is the approval queue when Proposed, and the activity log once Applied/Rejected/Undone.

public class AutomationAction : AuditableEntity
{
    public Guid Id { get; set; } = Guid.NewGuid();
    public Guid UserId { get; set; }

    public AutomationSource Source { get; set; }
    public Guid? RuleId { get; set; }              // AutomationRule, if Source=Rule
    public Guid? SenderPolicyId { get; set; }      // if Source=SenderPolicy/Screener

    public Guid EmailId { get; set; }
    public Email? Email { get; set; }
    public string GmailMessageId { get; set; } = string.Empty; // captured for undo

    public AutomationActionType Action { get; set; }
    public AutomationActionStatus Status { get; set; }

    /// <summary>Label/state captured before applying, so Undo can restore it.
    /// e.g. "had INBOX; had no InboxIntel/Paused". Serialized small JSON.</summary>
    public string? UndoStateJson { get; set; }

    public string? ErrorMessage { get; set; }
    public DateTimeOffset? AppliedUtc { get; set; }
}

Batching note: destructive proposals are grouped in the UI by (RuleId/Source, Action, SenderId) so the user approves "Trash 412 emails from Groupon" as one click, not 412 rows. The grouping is a query concern, not a schema one.

2.4 Exemption flags on existing entities

• Email.IsPinned (bool, default false) — Pinned Messages. Engine skips pinned emails entirely.
• MailThread.IsMuted (bool, default false) — Mute. New messages in a muted thread are auto SkipInbox + MarkRead and never surface in other automation.

Both require a migration (AddAutomationCore).

3. Application layer

3.1 IGmailService addition

Screener/Read-Later/Pause/Deliver-To need to create labels on demand:

/// <summary>Returns the labelId for a label name, creating it (and any
/// "Parent/Child" nesting) if it does not exist. Idempotent.</summary>
Task<string> EnsureLabelAsync(Guid userId, string name, CancellationToken ct = default);

Implement in GmailApiService using Users.Labels.List (already wired via ListLabelsAsync) then Users.Labels.Create when missing. Cache name→id per request scope.

3.2 New service interfaces (IServices.cs)

public interface IRuleService
{
    Task<IReadOnlyList<AutomationRuleDto>> ListRulesAsync(Guid userId, CancellationToken ct = default);
    Task<AutomationRuleDto> CreateRuleAsync(Guid userId, AutomationRuleInputDto input, CancellationToken ct = default);
    Task<AutomationRuleDto> UpdateRuleAsync(Guid userId, Guid ruleId, AutomationRuleInputDto input, CancellationToken ct = default);
    Task DeleteRuleAsync(Guid userId, Guid ruleId, CancellationToken ct = default);
    /// <summary>Dry-run: how many existing emails would this rule match right now?</summary>
    Task<RuleMatchPreviewDto> PreviewRuleAsync(Guid userId, AutomationRuleInputDto input, CancellationToken ct = default);
    Task SetEnabledAsync(Guid userId, Guid ruleId, bool enabled, CancellationToken ct = default);
}

/// <summary>The execution engine. Evaluates rules + policies and applies/queues actions.</summary>
public interface IAutomationEngine
{
    /// <summary>Evaluate all enabled rules + policies for a user against
    /// candidate emails (newly synced, or all if full=true). Safe actions apply
    /// immediately; destructive ones are written as Proposed.</summary>
    Task RunAsync(Guid userId, bool full = false, CancellationToken ct = default);
}

/// <summary>The hybrid approval queue + activity log.</summary>
public interface IAutomationActionService
{
    Task<IReadOnlyList<PendingActionGroupDto>> GetPendingAsync(Guid userId, CancellationToken ct = default);
    Task<IReadOnlyList<ActivityLogEntryDto>> GetActivityAsync(Guid userId, int take = 100, CancellationToken ct = default);
    Task ApproveAsync(Guid userId, IReadOnlyList<Guid> actionIds, CancellationToken ct = default);
    Task RejectAsync(Guid userId, IReadOnlyList<Guid> actionIds, CancellationToken ct = default);
    Task UndoAsync(Guid userId, IReadOnlyList<Guid> actionIds, CancellationToken ct = default);
}

3.3 DTOs (AutomationDtos.cs, new file)

public record AutomationRuleInputDto(
    string Name, bool Enabled, int Priority,
    string? SenderAddress, string? SenderDomain, EmailCategory? Category,
    string? SubjectContains, bool? IsUnread, bool? HasAttachment,
    bool? HasListUnsubscribe, long? MinSizeBytes, int? OlderThanDays,
    AutomationActionType Action, string? ActionLabelName, int? ActionParam, bool AlsoSkipInbox);

public record AutomationRuleDto( /* input fields + */ Guid Id, int TimesApplied, DateTimeOffset? LastRunUtc);

public record RuleMatchPreviewDto(int MatchCount, IReadOnlyList<EmailSummaryDto> Sample);

public record PendingActionGroupDto(
    string GroupKey, AutomationActionType Action, string Description,
    int Count, IReadOnlyList<Guid> ActionIds, IReadOnlyList<EmailSummaryDto> Sample);

public record ActivityLogEntryDto(
    Guid Id, AutomationSource Source, string Description, AutomationActionType Action,
    AutomationActionStatus Status, int Count, DateTimeOffset When, bool CanUndo);

4. Engine semantics (AutomationEngine)

Pseudo-flow of RunAsync(userId, full):

1. Load enabled rules (ordered by Priority, CreatedUtc) and sender policies.
2. Determine candidate emails:
     full == true  → all non-trashed emails for the user
     full == false → emails added/updated since LastAutomationRunUtc (SyncState)
3. Pre-load the whitelist (SenderPolicy where Kind=Allow) and pinned email ids.
4. For each candidate email:
     skip if email.IsPinned
     skip if sender is whitelisted
     skip if thread.IsMuted (handled by its own SkipInbox+MarkRead pass)
     for each rule in priority order:
        if Matches(rule, email):
           plan = (rule.Action, labelName, param)
           if plan.Action.IsDestructive():
               upsert AutomationAction(Proposed)   // no Gmail call
           else:
               apply via Gmail BatchModify (batched per action+label)
               write AutomationAction(Applied)
           break   // first matching rule wins (priority); configurable later
5. Run age-based + keep-newest evaluation (see §5).
6. Flush batched safe actions to Gmail in BatchModify groups (≤1000 ids/call).
7. Update SyncState.LastAutomationRunUtc.

Matching is a pure function bool Matches(AutomationRule, Email, Sender) — unit-testable, no I/O. Each non-null condition must hold.

Batching: collect (addLabelIds, removeLabelIds) per email, group identical label-sets, and issue one BatchModifyAsync per group. Trash proposals never call Gmail in the engine — only on approval.

5. Age-based & Keep-Newest

• Trash-by-Age (OlderThanDays + Trash): matched in the normal candidate loop, but because age changes over time independent of new mail, it must also run in a periodic full sweep. The AutomationWorker (next section) calls RunAsync(full:true) on a daily cadence so age rules catch up.
• Keep-Newest (KeepNewestCull, param N): evaluated per sender — order that sender's non-pinned mail by SentAtUtc desc, skip the newest N, propose Trash for the rest. Runs in the same daily full sweep.

Both produce Proposed actions (destructive) → approval queue.

6. Background worker (AutomationWorker)

New BackgroundService mirroring GmailSyncWorker/DigestWorker:

• Hourly tick.
• After each incremental sync the engine should also run on just-synced mail. Cleanest hook: have SyncService.RunIncrementalSyncAsync (and the manual sync path) call IAutomationEngine.RunAsync(userId, full:false) at the end, inside the same scope. This gives near-real-time automation without a separate schedule.
• Daily full sweep at a configured hour (AutomationOptions.SweepHourUtc, default 3) → RunAsync(userId, full:true) for age/keep-newest catch-up.
• Per-user try/catch, logs and continues. Early-out if the user has no enabled rules or policies.

AutomationOptions (new, Configuration/Options.cs): Enabled (default true), SweepHourUtc (3), MaxAutoActionsPerRun (safety cap, default 5000).

7. API (AutomationController : ApiControllerBase)

All actions scoped to UserId.

Method	Route	Purpose
GET	/automation/rules	list rules
POST	/automation/rules	create
PUT	/automation/rules/{id}	update
DELETE	/automation/rules/{id}	delete
POST	/automation/rules/preview	dry-run match count + sample
PUT	/automation/rules/{id}/enabled	toggle
GET	/automation/pending	grouped approval queue
POST	/automation/pending/approve	{ actionIds[] } → execute + log
POST	/automation/pending/reject	{ actionIds[] }
GET	/automation/activity?take=100	activity log
POST	/automation/activity/undo	{ actionIds[] } → revert
POST	/email/{id}/pin · /unpin	Pinned Messages
POST	/thread/{id}/mute · /unmute	Mute

8. Frontend (built on the new design system — see ui-overhaul.md)

• Rules page (/app/rules): table of rules with enable toggles; a rule editor drawer (Sheet) with condition builder + action picker + live "matches N emails" preview; priority drag-reorder.
• Review queue (/app/review or a badge in the topbar): grouped pending destructive actions, Approve/Reject per group, "Approve all".
• Pin affordance on email rows/detail (📌). Mute affordance on thread views.
• Pending count surfaces as a badge in the sidebar/topbar.

9. Security & safety checklist

• Every endpoint filters by UserId; rule/action ownership verified before mutate.
• Destructive actions can ONLY be executed via ApproveAsync, never by the engine.
• Hard delete is never an AutomationActionType — not automatable.
• Pinned/whitelisted exemptions enforced in Matches/candidate selection, with tests.
• MaxAutoActionsPerRun cap prevents a misconfigured rule from mass-acting; overflow logged.
• Undo restores prior label state from UndoStateJson.

10. Test plan

• Unit: Matches() truth table across every condition + AND combinations.
• Unit: destructive vs safe routing (proposed vs applied).
• Unit: Keep-Newest ordering & pinned exemption.
• Integration: engine run with a fake IGmailService asserts BatchModify groups + queue rows.
• Integration: approve → Gmail trash called + status Applied; undo → labels restored.