docs: discovery blueprint + multi-provider design (#8)
CI / backend (push) Successful in 52s
CI / frontend (push) Successful in 14s
Deploy Staging / deploy (push) Successful in 18s
CI / backend (pull_request) Successful in 52s
CI / frontend (pull_request) Successful in 15s
Security / secrets (push) Successful in 4s
Security / dependencies (push) Successful in 55s
Security / secrets (pull_request) Successful in 4s
Security / dependencies (pull_request) Successful in 54s
CI / backend (push) Successful in 52s
CI / frontend (push) Successful in 14s
Deploy Staging / deploy (push) Successful in 18s
CI / backend (pull_request) Successful in 52s
CI / frontend (pull_request) Successful in 15s
Security / secrets (push) Successful in 4s
Security / dependencies (push) Successful in 55s
Security / secrets (pull_request) Successful in 4s
Security / dependencies (pull_request) Successful in 54s
This commit was merged in pull request #8.
This commit is contained in:
@@ -0,0 +1,80 @@
|
||||
# 01 — Provider Abstraction (Part 1)
|
||||
|
||||
A unified layer so Gmail, Outlook/Graph, and future IMAP look identical to the rest of the
|
||||
app. **No provider-specific logic in Domain**; specifics live in Infrastructure adapters.
|
||||
|
||||
## Layering
|
||||
```
|
||||
Domain : Account, EmailMessage, EmailThread, Label (provider-agnostic)
|
||||
Application : IEmailProvider (contract) · ISyncOrchestrator · DTOs
|
||||
Infrastructure : GmailProvider · OutlookProvider · ImapProvider (adapters)
|
||||
ProviderFactory (ProviderType → adapter) · ITokenStore
|
||||
```
|
||||
|
||||
## The contract
|
||||
```csharp
|
||||
public enum ProviderType { Google, Microsoft, Imap }
|
||||
|
||||
public interface IEmailProvider {
|
||||
ProviderType Type { get; }
|
||||
ProviderCapabilities Capabilities { get; } // read, modifyFlags, folders, delta, send?
|
||||
|
||||
// Auth (details in 02-auth-and-signin.md)
|
||||
Task<OAuthResult> ExchangeCodeAsync(string code, CancellationToken ct);
|
||||
Task<TokenSet> RefreshAsync(TokenSet current, CancellationToken ct);
|
||||
Task<ProviderIdentity> GetIdentityAsync(TokenSet tokens, CancellationToken ct); // sub + email
|
||||
|
||||
// Sync (pull-based, incremental)
|
||||
Task<SyncPage> SyncAsync(SyncCursor cursor, TokenSet tokens, CancellationToken ct);
|
||||
Task<RawMessage> FetchMessageAsync(string providerMessageId, TokenSet tokens, CancellationToken ct);
|
||||
|
||||
// Mutations (only if Capabilities allow; mirrors current gmail.modify scope)
|
||||
Task ApplyFlagAsync(string providerMessageId, MailFlagChange change, TokenSet tokens, CancellationToken ct);
|
||||
}
|
||||
```
|
||||
- `SyncPage` = `{ IReadOnlyList<RawMessage> upserts, IReadOnlyList<string> deletes, SyncCursor next, bool hasMore }`.
|
||||
- `RawMessage` is the **provider-shaped** payload; a **normaliser** maps it to the domain
|
||||
`EmailMessage`. The rest of the app never sees `RawMessage`.
|
||||
- Capabilities let the UI/engine **degrade gracefully** (e.g., an IMAP server without
|
||||
CONDSTORE falls back to full-scan sync; no `send` today for any provider).
|
||||
|
||||
## Provider implementations
|
||||
| Provider | API | Incremental cursor | Threading | Folders/Labels | Notes |
|
||||
|----------|-----|--------------------|-----------|----------------|-------|
|
||||
| **Gmail** | Gmail REST | `historyId` (History API) | `threadId` | labels | Reuses existing client; read + modify (no send), as today |
|
||||
| **Outlook/365** | Microsoft Graph | **delta query** `@odata.deltaLink` | `conversationId` | mailFolders | OAuth via Microsoft identity platform |
|
||||
| **IMAP** | IMAP4rev1 | `UIDVALIDITY`+`UIDNEXT`, `HIGHESTMODSEQ` (CONDSTORE/QRESYNC) | heuristic (References/In-Reply-To) | folders | Fallback = periodic UID scan if no CONDSTORE; MailKit already a dependency |
|
||||
|
||||
## Normalisation (the unified model)
|
||||
Each adapter maps provider fields → domain via a `IMessageNormaliser`:
|
||||
| Domain field | Gmail | Graph | IMAP |
|
||||
|--------------|-------|-------|------|
|
||||
| `ProviderMessageId` | message id | message id | `UIDVALIDITY:UID` |
|
||||
| `ProviderThreadId` | threadId | conversationId | derived (References) |
|
||||
| flags (unread/star/important/trashed/inbox) | labelIds | isRead/flag/folder | `\Seen \Flagged`, folder |
|
||||
| labels/folders | labels | mailFolders | folders |
|
||||
| sent/received, from, subject, snippet, body, attachments, size | headers/parts | message resource | RFC822 parse (MailKit) |
|
||||
- **Threads are per-account** (each provider defines its own). *Cross-provider* thread
|
||||
linking is a later semantic/AI feature (see blueprint [07](../07-ai-feature-catalogue.md)),
|
||||
not part of core normalisation.
|
||||
|
||||
## Sync engine
|
||||
- **`ISyncOrchestrator`** replaces the Gmail-specific worker: for each active `Account`, it
|
||||
loads the `SyncCursor`, calls `provider.SyncAsync`, **upserts** normalised messages
|
||||
(idempotent on `(AccountId, ProviderMessageId)`), applies deletes, and **persists the next
|
||||
cursor** atomically.
|
||||
- Runs as the existing hosted-worker pattern (`AccountSyncWorker`), one logical job per
|
||||
account, bounded concurrency, Polly backoff, resumable.
|
||||
- **Token refresh:** `SyncAsync`/mutations get a valid `TokenSet` from `ITokenStore`, which
|
||||
refreshes on expiry/401 and **re-encrypts** at rest; a failed refresh flips the account to
|
||||
`reauth_needed` (surfaced in UI, see [07](07-ux-flows.md)) — never crashes sync.
|
||||
- **New mail** triggers AI enrichment + embedding jobs (blueprint [08](../08-technical-architecture.md)).
|
||||
|
||||
## Search across providers
|
||||
Because all providers normalise into **one `email_messages` store scoped by `UserId`**,
|
||||
search (structured + FTS + semantic) **already spans every account a user has connected** —
|
||||
no per-provider search code. An optional `AccountId` facet lets users scope to one mailbox.
|
||||
|
||||
## Extensibility
|
||||
Adding a provider = one `IEmailProvider` adapter + one normaliser + register in
|
||||
`ProviderFactory` + a feature flag to enable it. **Zero changes to Domain, search, or AI.**
|
||||
@@ -0,0 +1,70 @@
|
||||
# 02 — Auth & Sign-in (Part 2)
|
||||
|
||||
**OAuth is the login.** No passwords. A user's identity is the set of provider accounts
|
||||
linked to them; any one can authenticate the session. Identity key is **`(provider, sub)`**
|
||||
— never email (emails change; `sub` is stable, and the same email can exist on Google *and*
|
||||
Microsoft as distinct accounts).
|
||||
|
||||
## Provider-selection sign-in (first-time)
|
||||
```
|
||||
[ Choose how to sign in ]
|
||||
▸ Continue with Google ▸ Continue with Microsoft ( ▸ IMAP — future )
|
||||
```
|
||||
1. User picks a provider → redirect to provider OAuth (**PKCE**, `state`, `nonce`).
|
||||
2. Callback → exchange code → `GetIdentityAsync` returns `(provider, sub, email, name)`.
|
||||
3. **Resolve:** look up `accounts (provider, sub)`.
|
||||
- **No match →** first-time. Create `user` (**first user ever = Admin**, otherwise `Member`
|
||||
if `system_settings.registration_open`, else reject) + `account` (`is_login_identity=true`)
|
||||
+ encrypted `provider_tokens`. Start session.
|
||||
- **Match →** existing user. Refresh tokens, start session.
|
||||
4. Kick off the account's initial sync.
|
||||
|
||||
## Adding another account later (linking) — the security-critical flow
|
||||
The user is **already authenticated**. "Add account" → provider OAuth in **link mode**:
|
||||
- Callback identity `(provider, sub)`:
|
||||
- **Unlinked →** attach a new `account` (mailbox) to the **current** user. ✅
|
||||
- **Already linked to *this* user →** no-op / "already connected."
|
||||
- **Already linked to *another* user →** **blocked** by the unique `(provider, provider_account_id)`
|
||||
constraint + explicit check → error "This mailbox is connected to a different InboxIntel
|
||||
user." **This is the anti-hijack guarantee** — you can only link an identity you can
|
||||
authenticate *and* that no one else owns.
|
||||
- A single user can hold **N accounts** across Google/Microsoft/IMAP; each can also serve as a
|
||||
login identity (any of them signs you into the same user).
|
||||
|
||||
## Switching
|
||||
- **Switch mailbox (same user):** an **account switcher** changes the active mailbox context
|
||||
(or "All accounts" unified view). No re-auth — it's all one user. Search can scope to one
|
||||
account or span all.
|
||||
- **Switch user (different person):** full sign-out → sign-in. Optional "fast switch" could
|
||||
hold multiple sessions, but for a small team, explicit re-auth is simplest and safest.
|
||||
|
||||
## Sessions
|
||||
- **Opaque server-side session** (`sessions` table) referenced by an **HttpOnly · Secure ·
|
||||
SameSite=Lax** cookie. **Provider tokens are never exposed to the browser.**
|
||||
- Rotate session id on login (anti-fixation); **idle (e.g., 7d) + absolute (e.g., 30d)**
|
||||
expiry; revoke on logout; **"sign out everywhere"** and **admin revoke** delete session rows.
|
||||
- CSRF: SameSite + anti-CSRF token on state-changing requests.
|
||||
|
||||
## Token lifecycle
|
||||
- Stored **encrypted at rest** (Data Protection); decrypted only in-memory for API calls.
|
||||
- **Refresh** on expiry/401 via `ITokenStore` → re-encrypt + persist; failure flips account to
|
||||
**`ReauthNeeded`** (banner + "Reconnect" CTA, see [07](07-ux-flows.md)) — sync/AI for that
|
||||
account pause, the rest of the app is unaffected.
|
||||
|
||||
## Provider OAuth specifics
|
||||
| | Google | Microsoft (Graph) |
|
||||
|--|--------|-------------------|
|
||||
| Endpoint | accounts.google.com | login.microsoftonline.com (`common`) |
|
||||
| Scopes | `openid email profile gmail.readonly gmail.modify` | `openid email profile offline_access Mail.Read Mail.ReadWrite` |
|
||||
| Identity | `sub` (+ verified email) | `oid`/`sub` (+ email) |
|
||||
| Refresh | refresh_token (offline) | refresh_token (`offline_access`) |
|
||||
| Redirect | `/signin/google` | `/signin/microsoft` |
|
||||
- **Least privilege:** request read/modify only (no send today — matches current posture).
|
||||
Extra scopes are added per-feature with consent, never up-front.
|
||||
|
||||
## Edge cases
|
||||
- **Same email, two providers** → two distinct accounts (identity is `sub`), unless the user
|
||||
links both to one InboxIntel user.
|
||||
- **Provider disabled by admin flag** (`provider.microsoft=false`) → hide it on the picker;
|
||||
existing accounts of that provider pause sync and show a notice.
|
||||
- **Reused browser / stale cookie** → session validated server-side each request; revoked/expired → re-auth.
|
||||
@@ -0,0 +1,79 @@
|
||||
# 03 — Database Design (Part 6)
|
||||
|
||||
Schema for a **small-team, self-hosted, one-org** platform: multi-provider accounts per
|
||||
user, a normalised email store scaling to **millions of messages**, settings, feature
|
||||
flags, and audit. PostgreSQL + EF Core (+ pgvector for the blueprint's semantic tier).
|
||||
|
||||
## Entity map
|
||||
```
|
||||
users ─┬─< accounts ─┬─< provider_tokens (1:1, encrypted)
|
||||
│ ├─< email_threads ─< email_messages ─┬─< attachments
|
||||
│ │ └─< message_labels >─ labels
|
||||
│ └─ sync_state (cursor)
|
||||
├─< user_settings (1:1)
|
||||
└─< audit_logs (actor)
|
||||
system_settings (singleton) feature_flags
|
||||
```
|
||||
|
||||
## Tables
|
||||
|
||||
### Identity & access
|
||||
- **`users`** — the app identity (from OAuth).
|
||||
`id (uuid pk) · primary_email (citext, unique) · display_name · avatar_url · role (enum: Admin|Member) · status (enum: Active|Suspended) · created_at · last_login_at`
|
||||
*First-ever user is bootstrapped as **Admin** (see [05](05-admin-system.md)).*
|
||||
- **`accounts`** — a connected mailbox **and** a login identity (OAuth-is-login).
|
||||
`id (uuid pk) · user_id (fk) · provider (enum: Google|Microsoft|Imap) · provider_account_id (text, the OAuth 'sub' — immutable) · email (citext) · display_name · is_login_identity (bool) · status (enum: Active|ReauthNeeded|Disabled) · scopes (text[]) · added_at · last_sync_at`
|
||||
**Unique:** `(provider, provider_account_id)` → resolves an OAuth login to exactly one account→user; prevents the same mailbox linking twice.
|
||||
- **`provider_tokens`** — 1:1 with `accounts`, **encrypted at rest** (Data Protection API).
|
||||
`account_id (pk/fk) · access_token_enc (bytea) · refresh_token_enc (bytea) · expires_at_utc · token_type · rotated_at`
|
||||
*Never logged; see [06](06-security-model.md).*
|
||||
- **`sessions`** — server-side app sessions (opaque cookie).
|
||||
`id · user_id · created_at · expires_at · ip · user_agent · revoked_at` (supports "sign out everywhere" + admin revoke).
|
||||
|
||||
### Email (normalised, provider-agnostic)
|
||||
- **`email_threads`** — per account.
|
||||
`id (uuid pk) · account_id (fk) · user_id (denorm) · provider_thread_id · subject · participants (jsonb) · message_count · last_message_at`
|
||||
**Unique:** `(account_id, provider_thread_id)`.
|
||||
- **`email_messages`** — the big table.
|
||||
`id (uuid pk) · account_id (fk) · user_id (denorm) · thread_id (fk) · provider_message_id · sender_id (fk) · subject · snippet · body_text · sent_at_utc · received_at_utc · size_bytes · flags (unread/starred/important/in_inbox/trashed as bits or bools) · has_attachments · category (enum) · has_list_unsubscribe · supports_one_click_unsub · search_vector (tsvector, generated, weighted) · embedding (vector(768), nullable) · created_at`
|
||||
**Unique:** `(account_id, provider_message_id)` (idempotent upsert).
|
||||
- **`labels`** (`id · account_id · provider_label_id · name · type`) + **`message_labels`** (`message_id · label_id`, pk both).
|
||||
- **`attachments`** (`id · message_id · filename · mime · size · provider_attachment_id · content_text nullable` for future OCR/search).
|
||||
- **`senders`** / **`domains`** (existing) — kept, scoped per user (or global with per-user stats materialised in `sender_importance`).
|
||||
|
||||
### Settings, flags, audit
|
||||
- **`user_settings`** — 1:1 with `users`.
|
||||
`user_id (pk) · theme (enum: system|light|dark) · inbox_layout (jsonb) · notifications (jsonb) · ai_prefs (jsonb) · provider_prefs (jsonb) · updated_at`
|
||||
- **`system_settings`** — singleton (org-wide, admin-managed).
|
||||
`id (const) · maintenance_mode (bool) · default_theme · registration_open (bool) · updated_by · updated_at` (+ arbitrary `values jsonb` for growth).
|
||||
- **`feature_flags`** — the flag system (drives AI gating).
|
||||
`key (pk text) · enabled (bool) · scope (enum: SystemOnly|UserOverridable) · description · rollout (jsonb, e.g. per-role) · updated_by · updated_at`
|
||||
Seeded flags: `ai.enabled`, `ai.semantic_search`, `ai.ask_inbox`, `provider.google`, `provider.microsoft`, `provider.imap`, `maintenance.readonly`.
|
||||
- **`audit_logs`** — admin + security events.
|
||||
`id (bigserial) · actor_user_id (fk, nullable for system) · action (text) · target_type · target_id · metadata (jsonb) · ip · created_at`
|
||||
Append-only; indexed on `(created_at)`, `(actor_user_id)`, `(action)`.
|
||||
|
||||
## Indexing & performance
|
||||
| Index | Column(s) | For |
|
||||
|-------|-----------|-----|
|
||||
| GIN | `email_messages.search_vector` | FTS |
|
||||
| GIN `pg_trgm` | sender/subject | fuzzy (fixes non-sargable `.Contains`) |
|
||||
| HNSW | `email_messages.embedding` | semantic k-NN |
|
||||
| btree | `(user_id, sent_at_utc desc)`, `(account_id, sent_at_utc desc)`, `(thread_id)` | keyset pagination, scoping, threading |
|
||||
| unique | `(account_id, provider_message_id)`, `(provider, provider_account_id)` | idempotency, login resolution |
|
||||
|
||||
## Scalability (millions of emails, multi-account, incremental)
|
||||
- **Multi-account** = first-class via `accounts`; every email carries `account_id` + denormalised `user_id` (so cross-account per-user search is a single indexed scan).
|
||||
- **Millions of rows:** keyset (cursor) pagination, top-N-by-score ranking, `pg_trgm`/GIN/HNSW indexes. If a single user exceeds ~1–2M messages, **list-partition `email_messages` by `account_id`** (or hash by `user_id`).
|
||||
- **Incremental sync:** per-account `sync_state` cursor (`historyId` / `deltaLink` / `uidvalidity+modseq`) → only deltas fetched; idempotent upserts on the unique key.
|
||||
- **Idempotency & resumability:** all sync writes keyed on `(account_id, provider_message_id)`; cursor advanced atomically with the batch.
|
||||
|
||||
## RBAC in the schema
|
||||
Small-team model = a `role` column on `users` (`Admin|Member`) — no separate roles/permissions tables yet. **Extensible** to `roles`/`permissions`/`org_id` if this ever grows to multi-tenant, without reshaping the email tables.
|
||||
|
||||
## Migration from today (see full [Migration Guide](12-migration-guide.md))
|
||||
1. Create `users` from existing OAuth identity; set first user = **Admin**.
|
||||
2. Create one `Google` **`account`** per existing user; move current Gmail tokens → `provider_tokens`.
|
||||
3. Backfill `email_messages.account_id`/`user_id`, rename/extend from the current `Email` table; widen `search_vector`; add nullable `embedding`.
|
||||
4. Add `user_settings`, `system_settings`, `feature_flags` (seed `ai.enabled` from the current `Ai:Mode`), `audit_logs`.
|
||||
5. All additive + backfill; no destructive step — safe to run behind a maintenance window.
|
||||
@@ -0,0 +1,68 @@
|
||||
# 04 — Settings & Feature Flags (Part 3)
|
||||
|
||||
Two layers of configuration — **per-user preferences** and **org-wide system settings /
|
||||
feature flags** — with a clear precedence. Critically: **AI is governed by a feature flag
|
||||
(admin, global), not merely a user preference.**
|
||||
|
||||
## User settings (per user)
|
||||
Stored in `user_settings`; editable by the user.
|
||||
| Setting | Values |
|
||||
|---------|--------|
|
||||
| `theme` | system · light · dark (dark-first default) |
|
||||
| `inbox_layout` | density, pane layout, default view/lane, per-account or unified |
|
||||
| `notifications` | channels, quiet hours, priority-only |
|
||||
| `ai_prefs` | per-feature opt-in (summaries, replies, semantic, ask-inbox…) — **only effective if the flag allows** |
|
||||
| `provider_prefs` | default account, sync frequency, signature per account |
|
||||
|
||||
## System settings (admin, org-wide)
|
||||
Stored in `system_settings` (singleton); Admin-only ([05](05-admin-system.md)).
|
||||
- `maintenance_mode` (off · read-only · locked-except-admin) · `default_theme` ·
|
||||
`registration_open` · org display name · retention defaults.
|
||||
|
||||
## Feature flag system
|
||||
`feature_flags` rows: `key · enabled · scope · rollout · description · updated_by/at`.
|
||||
- **`scope = SystemOnly`** — a hard org-wide switch; users cannot override (e.g., `provider.microsoft`, `maintenance.readonly`).
|
||||
- **`scope = UserOverridable`** — a default that a user preference can turn *off* (never *on* beyond what the flag permits) — e.g., `ai.summaries`.
|
||||
- **`rollout`** (jsonb) — optional per-role/percentage gating (e.g., enable a beta for Admins first).
|
||||
|
||||
### Evaluation service
|
||||
```csharp
|
||||
public interface IFeatureFlags {
|
||||
bool IsEnabled(string key, UserContext user); // system flag ∧ scope ∧ role rollout
|
||||
}
|
||||
public interface IAiGate { // the AI-specific resolver
|
||||
bool IsAiEnabled(UserContext user); // ai.enabled (system) ∧ user.ai_prefs.master
|
||||
bool IsAiFeatureEnabled(string feature, UserContext user); // ∧ ai.<feature> ∧ user opt-in
|
||||
}
|
||||
```
|
||||
- Flags are **cached** with change notification (hot-reload on admin edit); every read is cheap.
|
||||
- All flag reads are **fail-closed**: unknown/errored flag ⇒ treated as **off**.
|
||||
|
||||
## AI gating precedence (the key requirement)
|
||||
Effective AI availability is an **AND** down a chain — the system flag is the master gate:
|
||||
```
|
||||
AI feature X is available for user U ⇔
|
||||
feature_flags["ai.enabled"].enabled (admin master switch — SYSTEM)
|
||||
∧ feature_flags["ai." + X].enabled (per-feature flag — SYSTEM)
|
||||
∧ providerCapability(X) (Ollama/provider actually available)
|
||||
∧ user.ai_prefs.master_opt_in (user hasn't disabled AI for themselves)
|
||||
∧ user.ai_prefs[X] (user opted into this feature)
|
||||
```
|
||||
- **Admin turns `ai.enabled` off ⇒ AI vanishes for everyone**, regardless of any user
|
||||
preference. This is the behaviour the brief mandates.
|
||||
- With AI on at the system level, users still choose per-feature. The **Null AI provider +
|
||||
capability flags** (blueprint [06](../06-ai-strategy.md)) mean a disabled path **falls back
|
||||
or hides** — never errors, never blocks core email.
|
||||
|
||||
## Settings precedence (general)
|
||||
```
|
||||
system default → feature flag (may hard-disable) → user override (only where UserOverridable)
|
||||
```
|
||||
|
||||
## Maintenance mode
|
||||
- `read-only`: mutations (send/cleanup/label) blocked with a banner; browsing/search stay up.
|
||||
- `locked-except-admin`: only Admins can use the app (for migrations/upgrades).
|
||||
- Enforced at the API via a middleware policy + surfaced as a global banner in the UI.
|
||||
|
||||
## Auditing
|
||||
Flag and system-setting changes are **admin actions → audit-logged** (who/what/old→new/when).
|
||||
@@ -0,0 +1,45 @@
|
||||
# 05 — Admin System (Part 4)
|
||||
|
||||
An Admin-only panel to run the instance. **Admins manage the platform, not people's
|
||||
inboxes** — no admin route can read another user's mail (see [06](06-security-model.md)).
|
||||
|
||||
## Sections
|
||||
| Section | Admin can | Notes |
|
||||
|---------|-----------|-------|
|
||||
| **Users** | List users; view role/status/last-login; **promote/demote** (Admin↔Member); **suspend/reactivate**; **revoke sessions**; remove user (with data-deletion policy) | **Never** view a user's email contents |
|
||||
| **Feature flags** | List all flags; toggle `enabled`; set scope/rollout; per-role rollout | Includes AI + provider flags |
|
||||
| **AI (global)** | Master `ai.enabled` toggle + per-feature (`ai.summaries`, `ai.semantic_search`, `ai.ask_inbox`…); see Ollama/model status | Off ⇒ AI hidden for everyone ([04](04-settings-and-flags.md)) |
|
||||
| **Providers** | Enable/disable `provider.google` / `provider.microsoft` / `provider.imap` | Disabled ⇒ hidden on login picker; existing accounts pause |
|
||||
| **System config** | Maintenance mode (off/read-only/locked); `registration_open`; org name; default theme; retention | Sensitive → step-up + audit |
|
||||
| **Monitoring** | Basic health overview (below) | Read-only |
|
||||
| **Audit log** | Search/filter admin + security events | Append-only |
|
||||
|
||||
## Monitoring overview (basic)
|
||||
- **Sync health:** per-account last-sync time, `ReauthNeeded` count, error rate; job-queue depth.
|
||||
- **AI/Ollama:** reachable? loaded models, VRAM headroom, recent latency, failure rate.
|
||||
- **Sessions:** active session count; recent logins.
|
||||
- **System:** DB size / message count; background-job backlog; recent errors (from Serilog).
|
||||
- Deliberately **overview-only** — deep observability is a future opportunity, not v1.
|
||||
|
||||
## Access control & bootstrap
|
||||
- Every admin route requires the **Admin policy**; sensitive mutations require **confirmation/
|
||||
step-up** + are **rate-limited** and **audited**.
|
||||
- **Bootstrap:** the first user to sign in becomes **Admin** (one-time). Afterwards, admin is
|
||||
granted only by an existing Admin (audited, forces target session refresh so new/removed
|
||||
privileges take effect immediately).
|
||||
- Guardrails: an Admin cannot demote/suspend the **last remaining Admin** (lock-out prevention).
|
||||
|
||||
## Audit logging (what's recorded)
|
||||
Actor · action · target (user/flag/setting/provider) · old→new · ip · timestamp — for **all**
|
||||
admin mutations and security events (role change, flag toggle, provider disable, maintenance
|
||||
on/off, session revoke, user suspend). Append-only `audit_logs`; visible in the Audit section;
|
||||
exportable.
|
||||
|
||||
## API surface (Admin-scoped, all audited)
|
||||
```
|
||||
GET/PATCH /admin/users /admin/users/{id}/role /admin/users/{id}/status
|
||||
GET/PATCH /admin/flags /admin/flags/{key}
|
||||
GET/PATCH /admin/system-settings
|
||||
GET /admin/monitoring /admin/audit
|
||||
```
|
||||
All behind the Admin policy + maintenance-aware middleware.
|
||||
@@ -0,0 +1,79 @@
|
||||
# 06 — Security Model (Part 5)
|
||||
|
||||
Builds on the existing hardening (read-only scope, encrypted tokens, IDOR global query
|
||||
filters, SSRF egress guard, non-root containers, confirmed destructive actions) and adds
|
||||
what multi-user + admin + multi-provider require.
|
||||
|
||||
## RBAC
|
||||
- Roles: **Admin** · **Member** (small-team, one org). First user bootstraps as Admin.
|
||||
- Enforced by **policy-based authorization** at the API (ASP.NET Core policies), not in the UI.
|
||||
|
||||
| Capability | Member | Admin |
|
||||
|------------|:------:|:-----:|
|
||||
| Read/manage **own** mail & accounts | ✅ | ✅ |
|
||||
| Own user settings | ✅ | ✅ |
|
||||
| Link/unlink **own** provider accounts | ✅ | ✅ |
|
||||
| View/manage **other users** | ❌ | ✅ |
|
||||
| Toggle **feature flags** (incl. AI global) | ❌ | ✅ |
|
||||
| Enable/disable **providers** | ❌ | ✅ |
|
||||
| **Maintenance mode**, system settings | ❌ | ✅ |
|
||||
| View **audit log** & monitoring | ❌ | ✅ |
|
||||
- **No cross-user data access, ever** — Admin manages *accounts/flags/system*, **not** other
|
||||
users' email contents (privacy). Admin power is over the *platform*, not people's inboxes.
|
||||
|
||||
## OAuth token storage
|
||||
- Refresh/access tokens **encrypted at rest** with the Data Protection API (AES); keys persist
|
||||
to the mounted `/keys` volume (existing). Decrypted **only in-memory** for the moment of an
|
||||
API call. **Never logged, never sent to the browser.**
|
||||
- 1:1 `provider_tokens` per account; rotation timestamped; a compromised/rotated token is
|
||||
replaced atomically. Token columns are `bytea` ciphertext, not readable in DB dumps.
|
||||
|
||||
## Session handling
|
||||
- Opaque **server-side sessions** (DB-backed) + HttpOnly/Secure/SameSite cookie; **id rotated
|
||||
on login** (anti-fixation); idle + absolute expiry; server-side **revocation** (logout,
|
||||
sign-out-everywhere, admin revoke, role change). CSRF via SameSite + token.
|
||||
|
||||
## Admin access protection
|
||||
- Admin routes require the **Admin policy**; sensitive mutations (toggle AI global, disable a
|
||||
provider, suspend a user, enter maintenance) require a **confirmation / step-up** and are
|
||||
**rate-limited**.
|
||||
- **Every admin action is audit-logged** (`audit_logs`: actor, action, target, metadata, ip,
|
||||
time) — append-only.
|
||||
- First-admin bootstrap is one-time; afterwards admin is grant-only by an existing Admin
|
||||
(logged). Guard against privilege escalation: role changes are Admin-only + audited + force
|
||||
session refresh.
|
||||
|
||||
## API security boundaries
|
||||
- **Per-user isolation** via EF **global query filters** (extended to `account_id`/`user_id`)
|
||||
so a query can *never* return another user's rows — the IDOR safeguard, now multi-account.
|
||||
- **Input validation** (FluentValidation) on all DTOs; **mass-assignment safe** (explicit DTOs,
|
||||
no entity binding).
|
||||
- **Rate limiting** on auth, admin, search, and AI endpoints.
|
||||
- **SSRF egress guard** (existing) constrains all outbound calls — provider APIs, IMAP hosts,
|
||||
Ollama, and any opt-in cloud AI — to an allowlist; user-supplied IMAP hosts are validated.
|
||||
- **Security headers** (CSP, HSTS, X-Frame-Options, etc.) via the reverse proxy/API; strict CORS.
|
||||
|
||||
## Multi-provider & AI specifics
|
||||
- **Least-privilege scopes** per provider; extra scopes added per-feature with consent.
|
||||
- **Provider isolation:** disabling a provider flag revokes its use cleanly; per-account tokens
|
||||
are independent (one reauth doesn't affect others).
|
||||
- **Prompt injection:** email content is untrusted → LLM output is **advisory only, never
|
||||
triggers actions**; a human/rule confirms. AI runs **local by default**; cloud AI is explicit
|
||||
opt-in with per-feature consent + egress logging.
|
||||
- **Attachments/vision:** sandboxed parsing, size/type limits, never executed.
|
||||
|
||||
## Threat model (summary)
|
||||
| Threat | Mitigation |
|
||||
|--------|------------|
|
||||
| Account hijack via linking | Must authenticate as target user; unique `(provider, sub)`; linking an owned identity blocked |
|
||||
| Token theft / DB exposure | Encryption at rest; tokens never in logs/browser; rotation |
|
||||
| Privilege escalation | Admin-only role changes, audited, session refresh; policy checks server-side |
|
||||
| IDOR / cross-user leakage | Global query filters on user_id/account_id |
|
||||
| CSRF / session fixation | SameSite + token; session id rotation; server-side revoke |
|
||||
| SSRF (providers/IMAP/AI) | Egress allowlist guard; validate user-supplied hosts |
|
||||
| Prompt injection | AI advisory-only; never acts; local-first |
|
||||
| Mass admin abuse | Rate limit + step-up + full audit trail |
|
||||
|
||||
## Non-negotiables
|
||||
Admins manage the platform, **not** users' inboxes · tokens encrypted & browser-invisible ·
|
||||
every privileged action audited · AI never required and never acts autonomously.
|
||||
@@ -0,0 +1,75 @@
|
||||
# 07 — UX Flows (Part 7)
|
||||
|
||||
Applies the [Design Brief](../00-design-brief.md) (Notion/Arc-professional, dark-first,
|
||||
green accent, pointer-first, responsive). **Keep it simple** — these are utility surfaces,
|
||||
not the daily driver.
|
||||
|
||||
## 1. Provider selection on login
|
||||
A calm, centred card — the *only* thing on screen.
|
||||
```
|
||||
InboxIntel
|
||||
───────────────────────────
|
||||
Sign in to continue
|
||||
[ ▸ Continue with Google ]
|
||||
[ ▸ Continue with Microsoft ]
|
||||
( IMAP — coming soon, disabled )
|
||||
───────────────────────────
|
||||
Your email stays on your machine.
|
||||
```
|
||||
- Only **enabled** providers show (driven by `provider.*` flags).
|
||||
- One click → provider OAuth → back into the app. First-timer lands on an empty, friendly
|
||||
inbox with a "syncing your mail…" state.
|
||||
|
||||
## 2. Connected accounts page
|
||||
Reached from the account switcher or Settings → Accounts.
|
||||
```
|
||||
Accounts [ + Add account ]
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ 🟢 Google me@gmail.com Synced 2m ago ⋯ │
|
||||
│ 🟢 Microsoft me@outlook.com Synced 5m ago ⋯ │
|
||||
│ 🟠 Google old@gmail.com Reconnect needed → [Reconnect]│
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
```
|
||||
- Status badges: 🟢 Active · 🟠 ReauthNeeded (with **Reconnect**) · ⚪ Disabled.
|
||||
- Per-account row menu (⋯): set as default · sync now · rename · **remove account** (confirm +
|
||||
explains local data deletion).
|
||||
- **+ Add account** → provider picker → OAuth in link-mode → new row appears.
|
||||
- **Account switcher** (top bar): "All accounts" (unified) or pick one to scope the inbox/search.
|
||||
|
||||
## 3. Settings page (user)
|
||||
Left sub-nav, one panel at a time — no overwhelm.
|
||||
```
|
||||
Settings
|
||||
Appearance ▸ Theme (System/Light/Dark) · density · layout
|
||||
Accounts ▸ (the page above)
|
||||
Notifications ▸ channels · quiet hours · priority-only
|
||||
AI ▸ (visible only if ai.enabled; see below)
|
||||
Privacy ▸ data, export, clear search history
|
||||
```
|
||||
|
||||
## 4. AI toggle visibility
|
||||
- If **`ai.enabled` (system) is OFF** → the **AI section is hidden entirely** (or shown as a
|
||||
single disabled note: "AI features are turned off by your administrator"). No dead toggles.
|
||||
- If **ON** → a master **"Use AI features"** switch (user opt-in) + per-feature toggles
|
||||
(Summaries · Reply suggestions · Semantic search · Ask your inbox), each reflecting its
|
||||
`ai.<feature>` flag. Toggling off a feature instantly falls back to the non-AI path.
|
||||
- A small **status chip** ("Local · Ollama · ready") reassures it's on-device.
|
||||
|
||||
## 5. Admin dashboard
|
||||
Only visible to Admins (nav item appears for the Admin role).
|
||||
```
|
||||
Admin
|
||||
Overview ▸ sync health · Ollama/VRAM · sessions · errors (cards)
|
||||
Users ▸ table: name · role · status · last login · [actions]
|
||||
Flags ▸ toggles grouped: AI · Providers · Maintenance
|
||||
System ▸ maintenance mode · registration · defaults
|
||||
Audit ▸ searchable event log
|
||||
```
|
||||
- Clean tables + toggles; destructive/sensitive actions show a **confirm dialog** (step-up).
|
||||
- Maintenance mode shows a **global banner** to all users while active.
|
||||
|
||||
## Cross-cutting
|
||||
- **Reduced-motion & keyboard** reachability on all of the above (baseline a11y).
|
||||
- **Empty/loading/error states** per the design system (skeletons, friendly empties, calm errors).
|
||||
- Fully **responsive**: settings/admin sub-nav collapses to a top tab bar on mobile; the login
|
||||
card is centred on all sizes.
|
||||
@@ -0,0 +1,68 @@
|
||||
# 08 — AI Feature-Flag Integration (Part 8)
|
||||
|
||||
How the AI layer plugs into the flag system while staying **completely separable** from core
|
||||
email logic. Extends the blueprint AI strategy ([../06](../06-ai-strategy.md)); the gate
|
||||
math lives in [04](04-settings-and-flags.md).
|
||||
|
||||
## Principle: AI is a guest, never a host
|
||||
Core email (sync, search-lexical, cleanup, settings, admin) **never references an AI type**.
|
||||
It calls domain services; those *optionally* consult AI through a single gate + facade. Remove
|
||||
AI entirely and nothing in the core path breaks.
|
||||
|
||||
```
|
||||
Core feature code
|
||||
│ (never touches Ollama/IAiProvider directly)
|
||||
▼
|
||||
IAiGate.IsAiFeatureEnabled("summaries", user) ──► false ─► non-AI path / hide
|
||||
│ true
|
||||
▼
|
||||
IInboxAi facade (Application) ──► model router ──► IAiProvider / IEmbeddingProvider
|
||||
(Null | Ollama | future)
|
||||
```
|
||||
|
||||
## Toggle behaviour (flag-driven)
|
||||
- Before *any* AI call, code asks `IAiGate` (which folds in `ai.enabled` + `ai.<feature>` +
|
||||
provider capability + user opt-in — the AND-chain from [04](04-settings-and-flags.md)).
|
||||
- **Admin `ai.enabled` = off** ⇒ gate returns false everywhere ⇒ AI UI hidden, AI code paths
|
||||
skipped. Flip on ⇒ features reappear (hot-reloaded flag cache) with **no redeploy**.
|
||||
- Per-feature flags allow shipping AI features **dark** and enabling gradually (rollout).
|
||||
|
||||
## Fallback when AI is disabled (per feature)
|
||||
| AI feature | Fallback with AI off |
|
||||
|------------|----------------------|
|
||||
| Semantic / NL search | Lexical + structured + fuzzy search (still excellent) |
|
||||
| Thread summary | Hidden; show first snippet + metadata |
|
||||
| Reply suggestions | Hidden; normal compose |
|
||||
| Follow-up detection | Heuristic-only (sent + question + no reply in N days) |
|
||||
| Categorisation | `HeuristicClassifier` rules only |
|
||||
| Ask-your-inbox | Feature hidden |
|
||||
| Dedup | Exact-hash only (no near-dup) |
|
||||
Every fallback is **first-class**, not a broken/greyed feature — this satisfies "AI must never
|
||||
be required for core functionality."
|
||||
|
||||
## Ollama integration layer (local models)
|
||||
- `OllamaProvider` (`IAiProvider`) + `OllamaEmbeddingProvider` (`IEmbeddingProvider`) talk to a
|
||||
local Ollama (own container, optional Compose `ai` profile).
|
||||
- **Model router** maps logical task → model via config (`Ai:Models:{Chat,Embed,Vision}`);
|
||||
swapping a model is a config change, not code.
|
||||
- **VRAM guard** (RTX 3080 / 10 GB): embeddings hot, 7B warm, vision on-demand
|
||||
([../06](../06-ai-strategy.md)).
|
||||
- **Health surfaced to admin** ([05](05-admin-system.md)): reachable? models loaded? VRAM?
|
||||
latency? If Ollama is down, capability = false ⇒ gate falls back gracefully (no user errors).
|
||||
|
||||
## Safe abstraction (`IAIProvider`) — separation guarantees
|
||||
1. **Interface boundary:** only Infrastructure implements providers; Application depends on
|
||||
`IInboxAi`/`IAiGate` abstractions.
|
||||
2. **Null objects:** `NullAiProvider`/`NullEmbeddingProvider` return "unavailable" so the DI
|
||||
graph is always valid, AI on or off.
|
||||
3. **Analyzer pipeline:** AI enrichers (`IEmailAnalyzer`) declare required capabilities and are
|
||||
**skipped** when unavailable — adding/removing AI features never touches core sync/search.
|
||||
4. **Bounded:** every AI call has timeout + `CancellationToken` + Polly fallback to the Null
|
||||
path; AI can never hang or crash the core.
|
||||
5. **Provider-swap:** adding a future AI provider = one class + config + (optionally) a flag —
|
||||
no feature-code changes.
|
||||
|
||||
## Precedence recap (single source of truth)
|
||||
The effective availability chain and admin master-switch semantics are defined once in
|
||||
[04 — Settings & Feature Flags](04-settings-and-flags.md#ai-gating-precedence-the-key-requirement);
|
||||
this document is the *architecture* of how features consume that decision.
|
||||
@@ -0,0 +1,57 @@
|
||||
# 09 — Implementation Plan (Part 9)
|
||||
|
||||
Six phases (from the brief), each **shippable behind feature flags** so `main`/staging never
|
||||
break and providers activate only when ready. Follows the established
|
||||
[../../WORKFLOW.md](../../WORKFLOW.md) pipeline (PR → checks → staging → tag→prod).
|
||||
|
||||
## Phase 1 — Provider abstraction + Google login refactor
|
||||
- **Goal:** introduce the seam and move the *existing* Gmail behaviour behind it, with
|
||||
OAuth-as-login and the multi-user identity foundation.
|
||||
- **Deliverables:** `IEmailProvider` + `ProviderFactory`; **`GmailProvider`** adapter wrapping
|
||||
today's Gmail code; `users` / `accounts` / `provider_tokens` / `sessions` tables;
|
||||
OAuth-as-login for Google; first-user→Admin bootstrap.
|
||||
- **Flags:** `provider.google` (on). **Exit:** existing Gmail users function unchanged through
|
||||
the new abstraction; sign-in creates a `user`+`account`; all tests green.
|
||||
|
||||
## Phase 2 — Microsoft Outlook integration
|
||||
- **Goal:** prove the abstraction with a second provider.
|
||||
- **Deliverables:** **`OutlookProvider`** (Microsoft Graph, delta query); Microsoft OAuth login
|
||||
+ link-mode; scope config; normaliser mappings.
|
||||
- **Flags:** `provider.microsoft` (**off** until verified, then rollout). **Exit:** a user can
|
||||
link an Outlook mailbox; it syncs and searches alongside Gmail; no core changes needed.
|
||||
|
||||
## Phase 3 — Unified email model + sync engine
|
||||
- **Goal:** formalise the normalised store and provider-agnostic sync.
|
||||
- **Deliverables:** normalised `email_messages`/`email_threads` (widened `search_vector`,
|
||||
nullable `embedding`); **`ISyncOrchestrator`** + `AccountSyncWorker` (per-account cursors,
|
||||
idempotent upserts, incremental); **data migration** of existing Gmail rows → default account.
|
||||
- **Flags:** none user-facing; migration behind a maintenance window. **Exit:** all providers
|
||||
sync through one orchestrator into one store; cross-account search works.
|
||||
|
||||
## Phase 4 — Settings system
|
||||
- **Goal:** user + system settings + the flag engine.
|
||||
- **Deliverables:** `user_settings`, `system_settings`, **`feature_flags`** + `IFeatureFlags`/
|
||||
`IAiGate` (cached, fail-closed); settings UI; maintenance-mode middleware.
|
||||
- **Flags:** self-hosting (the engine that hosts the rest). **Exit:** users edit prefs; admins
|
||||
can flip flags; AI gate resolves the AND-chain.
|
||||
|
||||
## Phase 5 — Admin panel + feature flags
|
||||
- **Goal:** the Admin surface + RBAC + audit.
|
||||
- **Deliverables:** Admin API (policy-gated) + UI (Users/Flags/AI/Providers/System/Monitoring/
|
||||
Audit); `audit_logs`; role management; basic monitoring.
|
||||
- **Flags:** admin nav shown by role. **Exit:** an Admin can manage users/flags/providers,
|
||||
every action audited; step-up + rate-limit enforced.
|
||||
|
||||
## Phase 6 — AI integration layer
|
||||
- **Goal:** wire optional AI behind the gate.
|
||||
- **Deliverables:** extend `IAiProvider` (+`CompleteStructuredAsync`, `IEmbeddingProvider`);
|
||||
`IInboxAi` facade + model router + VRAM guard; analyzer pipeline; first AI features
|
||||
(summaries, reply, follow-up-confirm) each **flag-gated + fallback**.
|
||||
- **Flags:** `ai.enabled` + `ai.<feature>` (rollout). **Exit:** AI features work when enabled,
|
||||
**vanish/fallback** when off; core unaffected; Ollama health in admin.
|
||||
|
||||
## Sequencing notes
|
||||
- Phases 1–3 are the platform spine; 4–5 the control plane; 6 the optional intelligence.
|
||||
- **Nothing activates on merge** — flags gate everything, so partial phases are safe on `develop`/`main`.
|
||||
- Aligns with the blueprint roadmap ([../09](../09-roadmap.md)): this multi-provider work is a
|
||||
**v1.x platform epic** that the search/AI features then build on.
|
||||
@@ -0,0 +1,50 @@
|
||||
# 10 — Git Workflow (Part 11)
|
||||
|
||||
Extends [../../WORKFLOW.md](../../WORKFLOW.md) for multi-phase, multi-provider work. The
|
||||
core idea: **feature flags decouple *merging* from *activating*, which makes every
|
||||
integration safe to land and trivial to roll back.**
|
||||
|
||||
## Branching per phase
|
||||
| Phase | Milestone | Branches |
|
||||
|-------|-----------|----------|
|
||||
| 1 Provider abstraction + Google | `v1.x` | `epic/provider-platform` → `feature/email-provider-interface` · `feature/gmail-adapter` · `feature/oauth-login-users` |
|
||||
| 2 Microsoft | `v1.x` | `feature/outlook-provider` · `feature/microsoft-oauth` |
|
||||
| 3 Unified model + sync | `v1.x` | `feature/normalised-email-model` · `feature/sync-orchestrator` · `feature/data-migration` |
|
||||
| 4 Settings | `v1.x` | `feature/settings-store` · `feature/feature-flags-engine` |
|
||||
| 5 Admin | `v1.x` | `feature/admin-api` · `feature/admin-ui` · `feature/audit-log` |
|
||||
| 6 AI layer | `v1.x` | `feature/ai-abstraction-ext` · `feature/ai-analyzers` · `feature/ai-flag-gating` |
|
||||
- `feature/* → develop` (squash), `develop → main` (merge commit) — as established. Epics are
|
||||
tracked by milestone/label; features integrate continuously (no long epic branch).
|
||||
|
||||
## PR structure per provider integration
|
||||
Each provider is a self-contained PR set that lands **dark**:
|
||||
1. **Adapter PR** — `IEmailProvider` impl + normaliser + unit tests (mocked provider).
|
||||
2. **Auth PR** — OAuth login/link for that provider.
|
||||
3. **Enablement PR** — register in `ProviderFactory` + seed `provider.<x>` flag **OFF**.
|
||||
4. **Activation** — flip the flag on in staging → verify end-to-end → roll out in prod.
|
||||
- **PR checklist adds:** provider behind a flag (off by default) · normaliser tests · token
|
||||
encryption verified · no Domain leakage · docs updated.
|
||||
|
||||
## Feature flags prevent breaking changes
|
||||
- Merge = code present but **inert** until its flag is on. So half-finished providers/AI can
|
||||
live on `main` safely; CI stays green; no long-lived divergence.
|
||||
- AI ships behind `ai.*`; providers behind `provider.*`; risky changes behind their own flag.
|
||||
|
||||
## Rollback strategy (per provider / per feature)
|
||||
| Level | Action | Speed |
|
||||
|-------|--------|-------|
|
||||
| **Flag** (first resort) | Admin flips `provider.<x>` / `ai.<x>` **off** | **Instant, no deploy** — feature disappears, existing data untouched |
|
||||
| **Deploy** | Redeploy the previous **tag** (`vX.Y.Z-1`) | Minutes (pipeline) |
|
||||
| **Revert** | `git revert` the PR → PR → merge → deploy | Minutes–hours |
|
||||
| **Data** | Provider accounts are isolated; disabling a provider **pauses** its sync — no destructive change to migrate back | Safe by design |
|
||||
- Because providers are isolated and flag-gated, a bad integration **never blocks the others**
|
||||
and never requires a risky data rollback.
|
||||
|
||||
## Release milestones
|
||||
- Cut a tag when a phase reaches its exit criteria (`deploy-prod.yml` fires on `v*`).
|
||||
- Suggested: `v1.1` provider platform + Google · `v1.2` +Outlook · `v1.3` unified sync ·
|
||||
`v1.4` settings+admin · `v1.5` AI layer — folded into the blueprint roadmap ([../09](../09-roadmap.md)).
|
||||
|
||||
## Docs alongside code
|
||||
Every feature PR updates the relevant `multi-provider/*` doc + `CHANGELOG.md`; on approval the
|
||||
design docs graduate into living `docs/` references (provider system, settings, admin, security).
|
||||
@@ -0,0 +1,31 @@
|
||||
# 11 — Risk Analysis
|
||||
|
||||
Risks specific to the multi-provider + multi-user + admin evolution (the blueprint-wide risks
|
||||
are in [../11](../11-risks-and-future.md)). L/I = Likelihood/Impact (H/M/L).
|
||||
|
||||
| # | Risk | L | I | Mitigation |
|
||||
|---|------|---|---|------------|
|
||||
| M1 | **Account-linking hijack** (attach someone's mailbox to your user) | L | H | Must authenticate as the target user; unique `(provider, sub)`; explicit "already owned" block; audited ([02](02-auth-and-signin.md)) |
|
||||
| M2 | **Cross-user data leakage** (multi-user IDOR) | M | H | EF global query filters on `user_id`/`account_id`; policy-based authz; no admin route reads user mail; tests for isolation |
|
||||
| M3 | **OAuth token theft / exposure** | L | H | Encrypted at rest (Data Protection); never logged or sent to browser; rotation; `bytea` ciphertext |
|
||||
| M4 | **Provider quirks break sync** (Graph delta resets, IMAP `UIDVALIDITY` change, Gmail history gaps) | M | M | Per-provider cursor handling with **full-resync fallback**; idempotent upserts; capability flags; account flips to needs-attention, never crashes |
|
||||
| M5 | **Migration corrupts existing Gmail data** | L | H | Additive-only schema; backfill is idempotent; maintenance window; tested on a staging copy; reversible ([12](12-migration-guide.md)) |
|
||||
| M6 | **RBAC bug grants Member admin powers** | L | H | Server-side policies (not UI); admin-only role changes audited + force session refresh; can't demote last Admin; authz tests |
|
||||
| M7 | **Feature-flag misconfiguration** (AI/provider on when not ready) | M | M | Flags default **off/fail-closed**; land dark; enable in staging first; audited toggles; rollout by role |
|
||||
| M8 | **AI gate bypass** (feature runs when disabled) | L | M | Single `IAiGate` chokepoint before any AI call; Null providers; capability checks; no direct provider refs in core |
|
||||
| M9 | **Session/CSRF weaknesses** across new surfaces | M | M | Server-side sessions, id rotation, SameSite + CSRF token, revoke-all, short idle expiry |
|
||||
| M10 | **Admin abuse / mistake** (mass suspend, wrong flag) | M | M | Step-up confirm + rate-limit + full audit trail + reversible flags |
|
||||
| M11 | **Scaling: millions of msgs × multiple accounts** | M | M | Denormalised `user_id`, keyset pagination, GIN/trgm/HNSW indexes, optional `account_id` partitioning, per-account sync throttling |
|
||||
| M12 | **Provider OAuth app setup burden** (separate Google + Microsoft app registrations, redirect URIs, verification) | M | L | Documented setup per provider; providers flag-gated so an unconfigured one is simply hidden |
|
||||
| M13 | **Scope/verification friction** (Google restricted scopes, MS admin consent) | M | M | Least-privilege scopes; document the consent/verification path; self-host uses the operator's own OAuth apps |
|
||||
|
||||
## Top watch-items
|
||||
- **M2 (isolation)** and **M6 (RBAC)** — the two ways multi-user can go wrong; both mitigated
|
||||
by server-side authz + query filters + isolation tests as a release gate.
|
||||
- **M4 (provider sync quirks)** — the most likely *operational* pain; the full-resync fallback
|
||||
and per-account isolation contain it.
|
||||
|
||||
## Overall
|
||||
The flag-gated, additive, isolated design makes this evolution **low-blast-radius**: each
|
||||
provider and the AI layer land dark and roll back by flag, migrations are additive, and the
|
||||
security model closes the new multi-user gaps. Proceed **phase by phase behind flags**.
|
||||
@@ -0,0 +1,52 @@
|
||||
# 12 — Migration Guide (Part 10)
|
||||
|
||||
Moving the current **single-account Gmail** app to the **multi-provider, multi-user** model
|
||||
— **additive and reversible**, no destructive step. Executed as ordered EF Core migrations +
|
||||
idempotent backfills, behind a short maintenance window.
|
||||
|
||||
## Principles
|
||||
- **Additive first:** create new tables/columns before moving data; keep old columns until parity is verified.
|
||||
- **Idempotent backfills:** safe to re-run; keyed on stable ids.
|
||||
- **Flag-gated cutover:** the new sign-in/model activates behind flags; the old path stays until removed.
|
||||
- **Reversible:** each step has a documented rollback; no data is deleted during migration.
|
||||
|
||||
## Step-by-step
|
||||
1. **Schema (additive migration)**
|
||||
- Create `users`, `accounts`, `provider_tokens`, `sessions`, `user_settings`,
|
||||
`system_settings`, `feature_flags`, `audit_logs`.
|
||||
- Add `account_id`, `user_id` (nullable) to the current email/thread tables; widen
|
||||
`search_vector`; add nullable `embedding vector(768)` + `pgvector` extension.
|
||||
2. **Identity backfill**
|
||||
- For the existing operator/user, create a `users` row; mark the **first user = Admin**.
|
||||
- Create one **`Google` `account`** per existing identity (`is_login_identity=true`); move
|
||||
current encrypted Gmail tokens → `provider_tokens`.
|
||||
3. **Email backfill**
|
||||
- Set `account_id`/`user_id` on all existing `Email`/thread rows to the default Google account.
|
||||
- Regenerate the widened `search_vector`; leave `embedding` null (backfilled later by the AI phase).
|
||||
- Enforce the new unique keys `(account_id, provider_message_id)` / `(account_id, provider_thread_id)`.
|
||||
4. **Config seed**
|
||||
- Seed `feature_flags`: `provider.google=on`, `provider.microsoft=off`, `provider.imap=off`,
|
||||
`ai.enabled` = derived from the current `Ai:Mode` (Disabled→off), `ai.*`=off, `maintenance.*`=off.
|
||||
- Create `system_settings` singleton; create `user_settings` from any existing per-user prefs (else defaults).
|
||||
5. **Cutover**
|
||||
- Enable the new OAuth-as-login + unified sync behind their flags; verify on **staging** first
|
||||
(the pipeline we built), then production via a tagged release.
|
||||
6. **Cleanup (later, separate migration)**
|
||||
- Once parity is confirmed in production, drop obsolete columns/paths. Not part of the cutover.
|
||||
|
||||
## Verification checklist
|
||||
- Existing user signs in via Google → lands on their mail unchanged.
|
||||
- Email counts match pre/post; search returns identical results for sample queries.
|
||||
- Tokens decrypt and refresh; sync resumes from the correct cursor.
|
||||
- No cross-user rows visible (isolation test).
|
||||
|
||||
## Rollback
|
||||
- **Pre-cutover:** additive changes are inert → simply don't flip the flags; drop new tables if aborting.
|
||||
- **Post-cutover issue:** flip flags off / redeploy previous **tag**; old columns still present →
|
||||
the legacy path still works. No data was deleted, so no data rollback is needed.
|
||||
|
||||
## Provider-app prerequisites (operator setup)
|
||||
- **Google:** OAuth client (existing) + redirect `/(…)/signin/google`; scopes `gmail.readonly gmail.modify`.
|
||||
- **Microsoft:** register an Entra app; redirect `/(…)/signin/microsoft`; scopes `Mail.Read Mail.ReadWrite offline_access`; admin consent if required.
|
||||
- **IMAP (future):** per-account host/credentials; validated against the SSRF allowlist.
|
||||
- Because providers are flag-gated, an unconfigured provider is simply hidden — configure, then enable.
|
||||
@@ -0,0 +1,48 @@
|
||||
# Multi-Provider Email Platform + Admin/Settings — Design
|
||||
|
||||
**Design + architecture phase. No implementation until approved.**
|
||||
|
||||
Evolves InboxIntel from a single-account Gmail tool into a **multi-provider platform**
|
||||
(Gmail · Outlook/Graph · future IMAP) for a **small self-hosted team**, with settings,
|
||||
feature flags, and an admin panel. Extends — does not discard — the
|
||||
[discovery blueprint](../README.md).
|
||||
|
||||
## Locked decisions (from interview)
|
||||
1. **Tenancy:** **small team, self-hosted, one org.** Roles = **Admin / Member**. Shared
|
||||
system settings + feature flags; each member's mail is private to them. No multi-tenant
|
||||
org table (one implicit org); the model stays extensible to multi-org later.
|
||||
2. **App identity = provider OAuth.** The first Google/Microsoft sign-in **creates/authenticates
|
||||
the InboxIntel user**; additional mailboxes **link** to that same user. **No passwords stored.**
|
||||
|
||||
## How this reshapes the blueprint (the "review" deltas)
|
||||
| Blueprint assumption | New reality |
|
||||
|----------------------|-------------|
|
||||
| Single-user, local-first | **Multi-user (small team)** with Admin/Member RBAC + admin panel |
|
||||
| Gmail-centric `Email`/`Sender` | **Account-scoped, provider-normalised** model (`IEmailProvider`) |
|
||||
| AI gated by `Ai:Mode` + user pref | **AI gated by system feature flag → user pref → capability** (flag wins) |
|
||||
| One implicit mailbox | **N provider accounts per user** (`accounts` table + per-account sync cursors) |
|
||||
| Sync = `GmailSyncWorker` | **Provider-agnostic sync orchestrator** dispatching to provider adapters |
|
||||
|
||||
These deltas will be back-ported into main-blueprint docs [08](../08-technical-architecture.md)
|
||||
and [09](../09-roadmap.md) when this design is approved.
|
||||
|
||||
## Documents
|
||||
| # | Doc | Covers (brief part) | Status |
|
||||
|---|-----|---------------------|--------|
|
||||
| 01 | [Provider Abstraction](01-provider-abstraction.md) | Part 1 | ✅ draft |
|
||||
| 02 | [Auth & Sign-in](02-auth-and-signin.md) | Part 2 | ✅ draft |
|
||||
| 03 | [Database Design](03-database-design.md) | Part 6 | ✅ draft |
|
||||
| 04 | [Settings & Feature Flags](04-settings-and-flags.md) | Part 3 | ✅ draft |
|
||||
| 05 | [Admin System](05-admin-system.md) | Part 4 | ✅ draft |
|
||||
| 06 | [Security Model](06-security-model.md) | Part 5 | ✅ draft |
|
||||
| 07 | [UX Flows](07-ux-flows.md) | Part 7 | ✅ draft |
|
||||
| 08 | [AI Feature-Flag Integration](08-ai-feature-flags.md) | Part 8 | ✅ draft |
|
||||
| 09 | [Implementation Plan](09-implementation-plan.md) | Part 9 | ✅ draft |
|
||||
| 10 | [Git Workflow](10-git-workflow.md) | Part 11 | ✅ draft |
|
||||
| 11 | [Risk Analysis](11-risk-analysis.md) | output | ✅ draft |
|
||||
| 12 | [Migration Guide](12-migration-guide.md) | Part 10 | ✅ draft |
|
||||
|
||||
## Non-negotiables carried forward
|
||||
Provider logic **never leaks into Domain** · search works **across all a user's accounts**
|
||||
· emails stored in **one unified format** · **AI never required** for core function ·
|
||||
tokens **encrypted at rest** · admin actions **audit-logged**.
|
||||
Reference in New Issue
Block a user