# Database ownership and startup order

> 2026-07-19. Which component creates which table, in what order, and why a clean MariaDB install used
> to fail. Read this before adding a table or touching `StartupInitializationExtensions`.

## The problem this document exists to prevent

EF Core bakes **provider-specific type names into a migration at scaffold time**. Every migration in
this repo was scaffolded against SQLite, so run against MariaDB it emits:

- `TEXT` for `DateTimeOffset` and every unbounded string
- `INTEGER` for `bool` and `int`
- a `PRIMARY KEY` with **no** `AUTO_INCREMENT`

A composite index over one of those `TEXT`/`longtext` columns then exceeds MySQL's 3072-byte key
limit and startup dies with `Specified key was too long`. This is not theoretical — it crashed
production once (Phase 4 `CvVariants`) and made every clean MariaDB install fail until 2026-07-19.

**Rule: a table whose migration was scaffolded against SQLite must not be created by that migration
on MariaDB.** Empty the migration and give the table to the reconciler, which carries correct DDL per
provider.

## Startup order

`InitializeJobTrackerAsync` runs exactly this sequence:

```
1. Connect
2. ReconcileSchema()      ← pass 1: repair existing schema, create reconciler-owned tables
3. Database.Migrate()     ← create every migration-owned table
4. ReconcileSchema()      ← pass 2: everything pass 1 had to skip
5. Seed admin, start services
```

### Why the reconciler runs twice

Neither position alone works:

- **Pass 1 must come first.** A legacy database has hand-added columns and Identity tables that
  predate the migrations; without repairing them (and stamping the legacy migration id into
  `__EFMigrationsHistory`) `Migrate()` collides with them. `AddCareerProfileRelationalChildren` also
  adds children that reference `CareerProfiles`, a **reconciler-owned** table — so it must exist
  before migrations run.
- **Pass 2 must come after.** On a brand-new database the migration-owned tables do not exist during
  pass 1, so every reconciler table that references one (FK into `JobApplications`) is skipped, as
  are the index and `AUTO_INCREMENT` repairs.

Every statement in `ReconcileSchema` is existence-guarded, so the second pass is a no-op scan on an
already-correct database. Two consequences worth knowing:

- The `DbConnection` is **not** wrapped in `using` — it belongs to the `DbContext`, and disposing it
  in pass 1 made pass 2 throw `ObjectDisposedException`.
- `conn.Open()` is guarded on `ConnectionState`, because pass 2 may inherit an open connection.

## Ownership

### Migration-owned

Created by EF migrations, never by the reconciler:

`Companies`, `JobApplications`, `Jobs`, `Correspondences`, `Attachments`, `JobEvents`,
`RuleSettings`, and the ASP.NET Identity tables.

The reconciler may **repair** these (add a missing column, add an index, fix a non-`AUTO_INCREMENT`
primary key) and may seed the default `RuleSettings` row — but it must never `CREATE TABLE` them.
It used to create `RuleSettings`, which is precisely why a clean install failed with
`Table 'RuleSettings' already exists` once `Migrate()` reached the initial migration.

### Reconciler-owned

Created by `StartupInitializationExtensions`, with a **no-op migration** holding the model snapshot:

`UserRuleSettings`, `SystemEmailSettings`, `CvUploadArtifacts`, `CvExtractionRuns`,
`GmailConnections`, `MicrosoftGraphConnections`, `ImapConnections`, `TailoredCvDrafts`,
`CareerProfiles`, `CareerProfileVersions`, the six CareerProfile children (`CareerExperiences`,
`CareerEducations`, `CareerSkills`, `CareerProjects`, `CareerCertifications`, `CareerLanguages`),
`InterviewPrepNotes`, `AiWorkspaceNotes`, `CvVariants`, `CvVariantVersions`, `AiInteractions`,
`ApplicationChecklistItems`, `TwoFactorRecoveryCodes`, `TrustedDevices`, `UserSessions`.

No-op migrations, each with a comment explaining why:

| Migration | Tables |
|---|---|
| `20260717222917_AddCareerProfileRelationalChildren` | the six CareerProfile children |
| `20260718074509_AddCvVariants` | `CvVariants`, `CvVariantVersions` |
| `20260718131138_AddAiInteractions` | `AiInteractions` |
| `20260719085904_AddApplicationChecklistItems` | `ApplicationChecklistItems` |
| `20260719094728_SyncCareerChildKeyLengths` | snapshot sync only |

### Dependency guards

A reconciler table that references another table is guarded on **its parent existing**, so pass 1
skips it on a fresh database and pass 2 creates it:

| Table | Waits for |
|---|---|
| `TailoredCvDrafts`, `InterviewPrepNotes`, `AiWorkspaceNotes`, `CvVariants`, `AiInteractions`, `ApplicationChecklistItems` | `JobApplications` (migration-owned) |
| `CvVariantVersions` | `CvVariants` |
| `CareerProfileVersions`, the six CareerProfile children | `CareerProfiles` |
| `CvExtractionRuns` | `CvUploadArtifacts` |

Index creation goes through one helper, `EnsureMySqlIndex`, which is guarded on **table** existence
as well as index existence — repairing an absent table is not pass 1's job.

## Adding a new table

1. Add the entity and its `DbSet`, and **bound every indexed string** with `HasMaxLength` — an
   unbounded string becomes `longtext`, which MariaDB cannot index without a prefix length. This is
   what broke the CareerProfile children.
2. `dotnet ef migrations add …`, then **empty the `Up`/`Down`** and say why in a comment.
3. Add SQLite DDL (`CREATE TABLE IF NOT EXISTS`) and MySQL DDL (`int AUTO_INCREMENT`, `varchar(n)`,
   `datetime(6)`, `tinyint(1)`) to the reconciler. Guard the MySQL create on any parent table.
4. Create indexes via `EnsureMySqlIndex` / `CREATE INDEX IF NOT EXISTS`.
5. Verify on a real MariaDB container — see below. EF InMemory will not catch any of this.

## Fresh install

No manual database preparation. Point the app at an **empty** database and start it:

```bash
# MariaDB
Database__Provider=mysql \
ConnectionStrings__JobTracker="Server=…;Database=jobtracker;User=…;Password=…;" \
dotnet run --project JobTrackerApi/JobTrackerApi.csproj

# SQLite (default) — creates Data__Root/jobtracker.db
dotnet run --project JobTrackerApi/JobTrackerApi.csproj
```

Create the empty schema/database itself (`CREATE DATABASE jobtracker;`); the application builds
everything inside it.

## Production upgrade

Deploy and restart. The reconciler is idempotent and additive:

- it never drops a table that holds rows (`DropMalformedMySqlTable` checks the row count first)
- it only adds missing columns, tables and indexes
- migrations already recorded in `__EFMigrationsHistory` are not re-run, so emptying a migration's
  `Up` changes nothing for an existing database

No downtime step, no manual SQL, no data migration.

## Verified

All four scenarios, 2026-07-19, against MariaDB 11 and SQLite:

| Scenario | Result |
|---|---|
| Empty MariaDB | 40 tables created, app starts |
| Populated MariaDB, restart | idempotent — still 40 tables, rows preserved |
| Empty MariaDB via the Docker image | 40 tables created, app starts |
| Fresh SQLite | 42 tables created, app starts |
| Existing partially-migrated SQLite dev DB (34 tables) | upgraded to 44 tables, 13 applications and 8 companies preserved |

Column types on MariaDB spot-checked: `int AUTO_INCREMENT` primary keys, `varchar(255)` owner keys,
`datetime(6)` timestamps, `tinyint(1)` booleans, and every composite index inside the key limit.