docs: discovery blueprint + multi-provider design (#8)

docs/discovery/06-ai-strategy.md

@@ -0,0 +1,113 @@
+# 06 — AI Strategy & Model Recommendations (Phase 5)
+
+AI is **optional, modular, local-first, and explainable**. The app must be fully usable
+with AI disabled. Primary hardware: **NVIDIA RTX 3080 (10 GB VRAM)**. All AI sits behind
+an abstraction so models/providers swap without touching application logic.
+
+---
+
+## Part A — The AI abstraction (extends the existing seam)
+Today there is one interface, `IAiProvider.CompleteAsync(system, user)`, with
+`NullAiProvider` / `OllamaProvider` / `OpenAiProvider`. We extend it into a small set of
+capability interfaces plus a task-oriented facade.
+
+### Provider-level interfaces (Infrastructure)
+```csharp
+public interface IAiProvider {                    // chat / generation (exists)
+    AiProviderMode Mode { get; }
+    Task<string> CompleteAsync(string system, string user, CancellationToken ct = default);
+    // NEW: schema-constrained JSON (Ollama `format`, OpenAI `response_format`)
+    Task<T?> CompleteStructuredAsync<T>(string system, string user, CancellationToken ct = default);
+}
+
+public interface IEmbeddingProvider {             // NEW: vectors for semantic search/dedup
+    Task<float[]> EmbedAsync(string text, CancellationToken ct = default);
+    Task<IReadOnlyList<float[]>> EmbedBatchAsync(IReadOnlyList<string> texts, CancellationToken ct = default);
+}
+
+public interface IAiCapabilities {                // NEW: lets the UI hide what isn't available
+    bool Chat { get; } bool Embeddings { get; } bool Vision { get; } bool StructuredJson { get; }
+}
+```
+`NullAiProvider`/`NullEmbeddingProvider` return empty/"unavailable" so **every caller has a
+graceful non-AI path** (fall back to lexical search, heuristics, or hide the feature).
+
+### Task facade (Application) — the only thing feature code calls
+```csharp
+public interface IInboxAi {
+    Task<AiResult<string>>        SummarizeThreadAsync(Guid threadId, CancellationToken ct);
+    Task<AiResult<Extraction>>    ExtractAsync(string body, ExtractKinds kinds, CancellationToken ct);
+    Task<AiResult<Classification>>ClassifyAsync(string subject, string body, CancellationToken ct);
+    Task<AiResult<Answer>>        AskAsync(string question, SearchScope scope, CancellationToken ct); // RAG
+    Task<float[]?>                EmbedAsync(string text, CancellationToken ct);
+    Task<AiResult<RiskVerdict>>   AssessPhishingAsync(EmailContext ctx, CancellationToken ct);
+}
+```
+- `AiResult<T>` carries `{ value, available, model, latencyMs, explanation }` → powers the
+  **"explainability"** and graceful-degradation requirements everywhere.
+- A **model router** maps *logical task → model name* from config, so swapping a model is a
+  config change: `Ai:Models:{Chat,Summarize,Embed,Classify,Extract,Vision}`.
+- Prompt templates are **versioned files**, separate from code.
+- Cross-cutting: async, `CancellationToken`, per-call timeout + **Polly** fallback to the
+  Null path, and a token/VRAM-aware concurrency limiter.
+
+### Config shape
+```
+Ai:Mode = Disabled | LocalOllama | Cloud...
+Ai:Ollama:BaseUrl, KeepAliveSeconds, VramBudgetMb
+Ai:Models:Chat = qwen2.5:7b-instruct
+Ai:Models:Embed = nomic-embed-text
+Ai:Models:Vision = qwen2.5vl:7b        (load-on-demand)
+```
+
+---
+
+## Part B — Model recommendations for the RTX 3080 (10 GB)
+
+**Budget reality:** a 7–8B instruct model at Q4/Q5 (~5 GB + ~1 GB KV cache at 8k context)
+runs **alongside** a small embedding model (~0.5 GB) inside 10 GB with headroom. 14B is
+possible (~9 GB) but leaves no room for concurrency and is slower — **7–8B is the sweet spot.**
+
+| Task | Recommended (primary) | Alt | ~VRAM (Q4) | Latency | License | Notes |
+|------|----------------------|-----|-----------|---------|---------|-------|
+| **Chat / reasoning / RAG answers** | **Qwen2.5-7B-Instruct** | Llama-3.1-8B, Gemma-2-9B | ~5.5 GB | ~30–60 tok/s | Apache-2.0 | Strong reasoning, multilingual, great JSON |
+| **Summarisation** | reuse **Qwen2.5-7B** | Llama-3.1-8B | (shared) | fast | Apache-2.0 | No separate model needed |
+| **Extraction (tasks/dates/entities)** | reuse **Qwen2.5-7B** + `format:json` | Phi-4 | (shared) | fast | Apache-2.0 | Schema-constrained JSON output |
+| **Classification / tagging** | **Qwen2.5-3B** (fast) *or* embeddings-zero-shot | rules-first (exists) | ~2.2 GB | very fast | Apache-2.0/Qwen | High-volume → small model or embeddings, not 7B |
+| **Embeddings (semantic search, dedup, zero-shot class.)** | **nomic-embed-text** (768-d, 8k ctx) | mxbai-embed-large, **bge-m3** (multilingual) | ~0.5 GB | very fast | Apache-2.0 | **Keep permanently loaded** |
+| **OCR / attachment vision** | **Qwen2.5-VL-7B** *or* **MiniCPM-V** | llama3.2-vision-11B, moondream (tiny) | ~7 GB | slow | Apache/varied | **Load-on-demand**; evicts the text LLM |
+| **Language detection** | **library, not an LLM** (fastText-lid / CLD3) | — | ~0 | instant | MIT/Apache | Don't waste VRAM on this |
+| **Translation** | reuse **Qwen2.5-7B** (multilingual) | NLLB-200 (dedicated MT) | (shared) | med | Apache-2.0 | Good for common pairs; on-demand |
+| **Spam detection** | **rules + small classifier/embeddings** | 3B LLM for edge cases | ~0–2 GB | fast | — | Traditional-first; LLM only for ambiguity |
+| **Phishing detection** | **rules/URL analysis + embeddings**, LLM **reasoning** for suspicious | Qwen2.5-7B for explanation | shared | on-demand | — | LLM explains *why*; runs async on flagged mail |
+| **Duplicate detection** | **hashing (exact) + embedding cosine (near)** | — | (uses embed) | fast | — | No chat model required |
+
+**Selection rationale:** accuracy + **Apache-2.0 licensing** (commercial-safe) + fits 10 GB
++ strong **structured-JSON** and multilingual (mailboxes aren't English-only). Qwen2.5-7B is
+a single versatile workhorse for chat/summarise/extract/translate; nomic-embed-text is a
+tiny always-on retrieval engine; a vision model is a heavy, rare, on-demand guest.
+
+### Load policy (VRAM management)
+| State | Models | Rationale |
+|-------|--------|-----------|
+| **Hot (permanent)** | `nomic-embed-text` (~0.5 GB) | Used constantly (search, dedup, classify, backfill) — must be instant |
+| **Warm (keep-alive)** | `qwen2.5:7b-instruct` (~5.5 GB) | Loaded on first chat/summarise, kept warm ~5–10 min idle via Ollama `keep_alive`, then unloaded |
+| **Cold (on-demand)** | `qwen2.5vl:7b` vision, `nllb` translation | Loaded only for the specific job; unloaded after (would otherwise exceed budget alongside the 7B) |
+- Ollama handles load/unload; we set `keep_alive` per task and a **VRAM budget guard** that
+  serialises a vision job behind unloading the text LLM.
+- Embedding **backfill** runs as a hosted background worker, batched, low priority, so it
+  never starves interactive requests.
+
+---
+
+## Part C — Principles
+1. **Traditional-first.** If rules/heuristics/library solve it well (language detection,
+   exact dedup, basic spam, most categorisation), use them — AI only where it *clearly* wins.
+2. **Always optional.** Null providers + capability flags → the app is whole without AI.
+3. **Local & private by default.** Ollama on-device; data never leaves unless a cloud
+   provider is explicitly chosen.
+4. **Explainable.** Every AI output carries a short rationale + the model used.
+5. **Swappable.** Logical-task→model routing via config; prompts versioned; no feature code
+   references a model name.
+6. **Bounded.** Timeouts, cancellation, Polly fallback, VRAM-aware concurrency — AI can never
+   hang or crash the core experience.