feat(security): audit batch C — data posture, retention, DP keys #22

Merged
cesnimda merged 1 commits from fix/audit-data-protection into develop 2026-07-02 10:13:08 +02:00
8 changed files with 286 additions and 2 deletions
+63
View File
@@ -0,0 +1,63 @@
# InboxIntel — Security & Data Posture
The deliberate security posture of this application, so operators know exactly what is and
isn't protected. Complements [AUDIT_REPORT.md](AUDIT_REPORT.md) (point-in-time audit) and
`docs/discovery/multi-provider/06-security-model.md` (future multi-user design).
## Deployment model this posture assumes
Self-hosted, **single-operator** instance: the person running the server is the person whose
mailbox is synced. All services (API, Postgres, frontend) run in Docker on the operator's own
machine; Postgres and the API bind to loopback/compose-internal only; TLS terminates at the
reverse proxy.
## What is protected, and how
| Asset | Protection |
|---|---|
| Google OAuth refresh/access tokens | Encrypted at rest (ASP.NET Data Protection, AES); never logged; never sent to the browser |
| Data Protection key ring | Optionally encrypted with an operator-supplied X.509 certificate — set `DataProtection:CertificatePath`/`CertificatePassword`. **Without it, keys sit in plaintext on the `/keys` volume** and anyone with volume access can decrypt stored tokens. Recommended for any shared host. |
| App session | HttpOnly/SameSite=Lax/Secure cookie · sliding 7 d **with an absolute 30 d cap** (`Auth:AbsoluteSessionDays`) |
| Login/abuse | Rate limiting: global 300 req/min per user (or IP when anonymous); `auth` 10/min; export/unsubscribe/AI 20/min (`RateLimiting:*`) |
| Cross-user access | EF global query filters on every tenant-scoped entity (incl. the EmailLabel join) — tested |
| Outbound fetches (unsubscribe etc.) | `SafeHttpGuard` SSRF allowlisting (DNS-rebinding-safe) + redirects disabled |
| Untrusted email content in the UI | Rendered only as escaped React text (no `dangerouslySetInnerHTML`); search highlights use non-HTML sentinels; SPA ships CSP with `script-src 'self'` |
| Secrets | Never committed (`deploy/.env*` git-ignored; pre-commit + CI gitleaks scans); no passwords stored at all (OAuth-only login) |
## What is deliberately NOT protected (accepted risks — read this)
1. **Email bodies are stored in plaintext in Postgres.** Full-text and semantic search index
the body; encrypted columns cannot be indexed this way. On the assumed single-operator
deployment, the database lives on the operator's own disk, so the threat this would
mitigate (a third party reading the DB files) reduces to "someone with access to your
machine" — mitigate it at the layer that actually works:
- **Use full-disk or volume encryption** on the host (BitLocker/LUKS) — strongly recommended.
- **Encrypt backups** of the `pgdata` volume the same way.
- Before any **multi-user** deployment, revisit per the multi-provider security design
(host admins must not be able to read members' mail — plaintext bodies break that promise).
2. **DB connection is not TLS** — Postgres is only reachable on the compose-internal network /
loopback. If you ever move Postgres to another host, add `SSL Mode=Require` to the
connection string (audit L-4).
3. **No CSRF tokens** — SameSite=Lax cookies + strict CORS + JSON-only bodies make classic
CSRF impractical; revisit if either changes (audit L-1).
## Data retention (opt-in)
By default the local mailbox copy is kept indefinitely. Two knobs enable automatic purging of
the **local copy only** (your actual Gmail is never touched):
```json
"DataRetention": {
"PurgeTrashedAfterDays": 0, // e.g. 30 — purge local copies of trashed mail after 30 days
"PurgeAllAfterDays": 0 // e.g. 730 — keep at most ~2 years of mail locally
}
```
`0` disables a knob. A daily background worker applies them. For a full "delete my data"
operation: stop the stack and remove the `pgdata` + `keys` volumes
(`docker compose down -v`), and revoke the app's access in your Google account.
## Production checklist (beyond compose defaults)
- Set `AllowedHosts` to your real hostname(s) (audit L-2).
- Terminate TLS at the proxy; HSTS is enabled automatically outside Development.
- Provide `DataProtection:CertificatePath` to encrypt the key ring.
- Keep `deploy/.env` readable only by the service user; rotate the DB password if exposed.
- Dependency + secret scanning run in CI on every PR (required checks).
## Reporting
Single-operator project — if you find a vulnerability, open a private issue or contact the
repository owner directly.
+12 -1
View File
@@ -27,9 +27,20 @@ builder.Host.UseSerilog((ctx, cfg) => cfg
.WriteTo.Console());
// Persist Data Protection keys so encrypted refresh tokens survive restarts.
builder.Services.AddDataProtection()
// AUDIT M-3: optionally encrypt the Data Protection key ring with an X.509 certificate so
// the keys are not readable in plaintext from the /keys volume (which would otherwise let
// anyone with volume access decrypt all stored refresh tokens). Configure
// DataProtection:CertificatePath (+ CertificatePassword) to enable; without it, keys are
// persisted unprotected and a startup warning documents the residual risk.
var dp = builder.Services.AddDataProtection()
.PersistKeysToFileSystem(new DirectoryInfo(builder.Configuration["DataProtection:KeyPath"] ?? "/keys"))
.SetApplicationName("InboxIntel");
var dpCertPath = builder.Configuration["DataProtection:CertificatePath"];
if (!string.IsNullOrWhiteSpace(dpCertPath))
{
dp.ProtectKeysWithCertificate(new System.Security.Cryptography.X509Certificates.X509Certificate2(
dpCertPath, builder.Configuration["DataProtection:CertificatePassword"]));
}
builder.Services.AddApplication();
builder.Services.AddInfrastructure(builder.Configuration);
+7 -1
View File
@@ -6,7 +6,13 @@
"AutoMigrate": true
},
"DataProtection": {
"KeyPath": "/keys"
"KeyPath": "/keys",
"CertificatePath": "",
"CertificatePassword": ""
},
"DataRetention": {
"PurgeTrashedAfterDays": 0,
"PurgeAllAfterDays": 0
},
"GoogleOAuth": {
"ClientId": "",
@@ -72,3 +72,13 @@ public class DigestOptions
/// <summary>Hour (UTC) the background worker checks for due digests.</summary>
public int SendHourUtc { get; set; } = 8;
}
/// <summary>AUDIT H-3: opt-in local data retention. 0 = disabled (keep forever).</summary>
public class DataRetentionOptions
{
public const string SectionName = "DataRetention";
/// <summary>Purge locally stored emails flagged Trashed older than this many days.</summary>
public int PurgeTrashedAfterDays { get; set; } = 0;
/// <summary>Purge ALL locally stored emails older than this many days (local copy only).</summary>
public int PurgeAllAfterDays { get; set; } = 0;
}
@@ -60,6 +60,11 @@ public static class DependencyInjection
services.AddScoped<IDigestService, DigestService>();
services.AddHostedService<DigestWorker>();
// AUDIT H-3: opt-in local data retention (worker no-ops while disabled).
services.Configure<DataRetentionOptions>(config.GetSection(DataRetentionOptions.SectionName));
services.AddScoped<Retention.RetentionService>();
services.AddHostedService<Retention.RetentionWorker>();
// HTTP clients
// V-01: do NOT follow redirects — a validated external URL must not be able to
// 3xx-redirect into an internal target after SafeHttpGuard has checked it.
@@ -0,0 +1,63 @@
using InboxIntel.Infrastructure.Configuration;
using InboxIntel.Infrastructure.Persistence;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
namespace InboxIntel.Infrastructure.Retention;
/// <summary>
/// AUDIT H-3 (retention): purges locally stored email data past the configured age so the
/// local mailbox copy is not kept forever by default-of-omission. This deletes ONLY the
/// local Postgres rows — the user's actual Gmail is never touched. Both knobs default to 0
/// (disabled) so existing deployments are unchanged until the operator opts in.
/// </summary>
public class RetentionService
{
private readonly AppDbContext _db;
private readonly DataRetentionOptions _options;
private readonly ILogger<RetentionService> _logger;
public RetentionService(AppDbContext db, IOptions<DataRetentionOptions> options, ILogger<RetentionService> logger)
{
_db = db;
_options = options.Value;
_logger = logger;
}
/// <summary>Runs the configured purges. Returns the number of emails removed.</summary>
public async Task<int> PurgeAsync(CancellationToken ct = default)
{
var removed = 0;
var now = DateTimeOffset.UtcNow;
if (_options.PurgeTrashedAfterDays > 0)
{
var cutoff = now.AddDays(-_options.PurgeTrashedAfterDays);
removed += await PurgeWhereAsync(e => e.IsTrashed && e.SentAtUtc < cutoff, ct);
}
if (_options.PurgeAllAfterDays > 0)
{
var cutoff = now.AddDays(-_options.PurgeAllAfterDays);
removed += await PurgeWhereAsync(e => e.SentAtUtc < cutoff, ct);
}
if (removed > 0)
_logger.LogInformation("Retention purge removed {Count} locally stored emails", removed);
return removed;
}
private async Task<int> PurgeWhereAsync(
System.Linq.Expressions.Expression<Func<Domain.Entities.Email, bool>> predicate,
CancellationToken ct)
{
// RemoveRange (not ExecuteDelete) so dependent rows (labels/attachments) cascade via
// the model on every provider, and the InMemory test provider works identically.
var doomed = await _db.Emails.Where(predicate).ToListAsync(ct);
if (doomed.Count == 0) return 0;
_db.Emails.RemoveRange(doomed);
await _db.SaveChangesAsync(ct);
return doomed.Count;
}
}
@@ -0,0 +1,53 @@
using InboxIntel.Infrastructure.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
namespace InboxIntel.Infrastructure.Retention;
/// <summary>
/// Daily driver for <see cref="RetentionService"/> (AUDIT H-3). Exits immediately when both
/// retention knobs are 0 (the default) so existing deployments see no behaviour change.
/// </summary>
public class RetentionWorker : BackgroundService
{
private readonly IServiceScopeFactory _scopeFactory;
private readonly DataRetentionOptions _options;
private readonly ILogger<RetentionWorker> _logger;
public RetentionWorker(IServiceScopeFactory scopeFactory, IOptions<DataRetentionOptions> options, ILogger<RetentionWorker> logger)
{
_scopeFactory = scopeFactory;
_options = options.Value;
_logger = logger;
}
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
if (_options.PurgeTrashedAfterDays <= 0 && _options.PurgeAllAfterDays <= 0)
{
_logger.LogDebug("RetentionWorker idle: retention is disabled (all knobs 0).");
return;
}
_logger.LogInformation("RetentionWorker started; trashed>{Trashed}d, all>{All}d",
_options.PurgeTrashedAfterDays, _options.PurgeAllAfterDays);
while (!stoppingToken.IsCancellationRequested)
{
try
{
using var scope = _scopeFactory.CreateScope();
var service = scope.ServiceProvider.GetRequiredService<RetentionService>();
await service.PurgeAsync(stoppingToken);
}
catch (OperationCanceledException) when (stoppingToken.IsCancellationRequested) { break; }
catch (Exception ex)
{
_logger.LogError(ex, "Retention purge failed; will retry next cycle.");
}
await Task.Delay(TimeSpan.FromHours(24), stoppingToken);
}
}
}
@@ -0,0 +1,73 @@
using FluentAssertions;
using InboxIntel.Application.Abstractions;
using InboxIntel.Domain.Entities;
using InboxIntel.Infrastructure.Configuration;
using InboxIntel.Infrastructure.Persistence;
using InboxIntel.Infrastructure.Retention;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Logging.Abstractions;
using Microsoft.Extensions.Options;
using Xunit;
namespace InboxIntel.IntegrationTests;
/// <summary>
/// AUDIT H-3 (retention): the purge removes only what the configuration targets and is a
/// no-op while disabled — locking the "no behaviour change until opted in" guarantee.
/// </summary>
public class RetentionServiceTests
{
private sealed class FakeCurrentUser : ICurrentUser
{
public Guid UserId => Guid.Empty; // worker scope
public bool IsAuthenticated => false;
}
private static AppDbContext NewDb(string name) =>
new(new DbContextOptionsBuilder<AppDbContext>().UseInMemoryDatabase(name).Options, new FakeCurrentUser());
private static async Task SeedAsync(AppDbContext db)
{
var user = Guid.NewGuid();
db.Emails.AddRange(
new Email { UserId = user, GmailMessageId = "old-trashed", IsTrashed = true, SentAtUtc = DateTimeOffset.UtcNow.AddDays(-100) },
new Email { UserId = user, GmailMessageId = "new-trashed", IsTrashed = true, SentAtUtc = DateTimeOffset.UtcNow.AddDays(-5) },
new Email { UserId = user, GmailMessageId = "old-kept", IsTrashed = false, SentAtUtc = DateTimeOffset.UtcNow.AddDays(-100) });
await db.SaveChangesAsync();
}
private static RetentionService Sut(AppDbContext db, int trashedDays = 0, int allDays = 0) =>
new(db, Options.Create(new DataRetentionOptions
{
PurgeTrashedAfterDays = trashedDays,
PurgeAllAfterDays = allDays
}), NullLogger<RetentionService>.Instance);
[Fact]
public async Task Disabled_retention_purges_nothing()
{
using var db = NewDb(nameof(Disabled_retention_purges_nothing));
await SeedAsync(db);
(await Sut(db).PurgeAsync()).Should().Be(0);
(await db.Emails.CountAsync()).Should().Be(3);
}
[Fact]
public async Task Trashed_purge_removes_only_old_trashed_emails()
{
using var db = NewDb(nameof(Trashed_purge_removes_only_old_trashed_emails));
await SeedAsync(db);
(await Sut(db, trashedDays: 30).PurgeAsync()).Should().Be(1);
var remaining = await db.Emails.Select(e => e.GmailMessageId).ToListAsync();
remaining.Should().BeEquivalentTo(new[] { "new-trashed", "old-kept" });
}
[Fact]
public async Task Age_purge_removes_everything_past_the_cutoff()
{
using var db = NewDb(nameof(Age_purge_removes_everything_past_the_cutoff));
await SeedAsync(db);
(await Sut(db, allDays: 30).PurgeAsync()).Should().Be(2); // both 100-day-old emails
(await db.Emails.SingleAsync()).GmailMessageId.Should().Be("new-trashed");
}
}