# Database ownership and startup order

> Updated 2026-08-30. Which component creates which table, in what order, and why a clean MariaDB install used
> to fail. Read this before adding a table or touching `StartupInitializationExtensions`.

## The problem this document exists to prevent

EF Core bakes **provider-specific type names into a migration at scaffold time**. Every migration in
this repo was scaffolded against SQLite, so run against MariaDB it emits:

- `TEXT` for `DateTimeOffset` and every unbounded string
- `INTEGER` for `bool` and `int`
- a `PRIMARY KEY` with **no** `AUTO_INCREMENT`

A composite index over one of those `TEXT`/`longtext` columns then exceeds MySQL's 3072-byte key
limit and startup dies with `Specified key was too long`. This is not theoretical — it crashed
production once (Phase 4 `CvVariants`) and made every clean MariaDB install fail until 2026-07-19.

That historical workaround created two schema owners and is now being retired under JT-019. New
tables belong to migrations. When the generated operation is not provider-safe, hand-author a
provider-aware migration (using `ActiveProvider`) instead of adding more startup DDL. The reconciler
is compatibility code for tables and columns that already shipped under its ownership.

## Startup order

`InitializeJobTrackerAsync` runs this provider-aware sequence:

```
1. Connect
2. ReconcileSchema()      ← repair legacy schema/create prerequisites
3a. SQLite: apply one migration, reconcile, repeat
3b. MariaDB: apply the complete migration chain
4. ReconcileSchema()      ← create/repair everything skipped before migrations
5. Seed admin, start services
```

### Why migration sequencing differs by provider

Neither a single reconciliation position nor one shared provider sequence works:

- **Pass 1 must come first.** A legacy database has hand-added columns and Identity tables that
  predate the migrations; without repairing them (and stamping the legacy migration id into
  `__EFMigrationsHistory`) `Migrate()` collides with them.
- **The final pass must come after.** On a brand-new database the migration-owned tables do not exist during
  pass 1, so every reconciler table that references one (FK into `JobApplications`) is skipped, as
  are the index and `AUTO_INCREMENT` repairs.
- **SQLite reconciles between migrations.** Historical SQLite table rebuilds read the current model
  shape, including columns that were originally supplied by reconciliation. The per-migration pass
  establishes that shape before a later rebuild reads it.
- **MariaDB does not reconcile between migrations.** Its ALTER operations do not use SQLite table
  rebuilds, and an intermediate pass could create a later migration's column early and cause a
  duplicate-column failure. It applies the chain first and uses the shared final repair pass.

Every statement in `ReconcileSchema` is existence-guarded, so the second pass is a no-op scan on an
already-correct database. Two consequences worth knowing:

- The `DbConnection` is **not** wrapped in `using` — it belongs to the `DbContext`, and disposing it
  in pass 1 made pass 2 throw `ObjectDisposedException`.
- `conn.Open()` is guarded on `ConnectionState`, because pass 2 may inherit an open connection.

## Ownership

### Migration-owned

Created by EF migrations, never by the reconciler:

`AccountDeletionFiles`, `AccountDeletionRequests`, `AiInteractions`, `AiUsageRecords`, `AiWorkspaceNotes`,
`ApplicationChecklistItems`, `AspNetRoleClaims`, `AspNetRoles`, `AspNetUserClaims`, `AspNetUserLogins`,
`AspNetUserRoles`, `AspNetUsers`, `AspNetUserTokens`, `Attachments`, `CareerCertifications`, `CareerEducations`,
`CareerExperiences`, `CareerLanguages`, `CareerProfiles`, `CareerProfileVersions`, `CareerProjects`,
`CareerSkills`, `Companies`, `CoverLetterVersions`,
`Correspondences`, `CvExtractionRuns`, `CvUploadArtifacts`, `CvVariants`, `CvVariantVersions`,
`EmailDrafts`, `EmailSendAttempts`, `GmailConnections`,
`GmailReviewDecisions`, `ImapConnections`, `InterviewPrepItems`, `InterviewPrepNotes`, `JobApplications`, `JobEvents`, `Jobs`,
`MicrosoftGraphConnections`, `RuleSettings`, `SystemEmailSettings`, `TailoredCvDrafts`,
`TrustedDevices`, `TwoFactorRecoveryCodes`, `UserNotifications`, `UserOperations`, `UserRuleSettings`,
and `UserSessions`.

`SystemEmailSettings` is the first completed ownership transfer: migration
`20260830120000_AdoptSystemEmailSettingsSchema` creates it for both supported providers and preserves
an existing reconciler-created MariaDB table. Startup no longer creates it.

`UserRuleSettings` followed in `20260830121000_AdoptUserRuleSettingsSchema`; its owner-keyed rows are
preserved across adoption, downgrade and retry, and startup no longer creates it.

`GmailReviewDecisions` moved in `20260830122000_AdoptGmailReviewDecisionsSchema`. This also supplies
the table on MariaDB, where the old reconciler had no creation path.

The authentication support group (`TwoFactorRecoveryCodes`, `TrustedDevices`, and `UserSessions`)
moved in `20260830123000_AdoptAuthenticationSupportSchema`. These tables deliberately have no
database foreign key to `AspNetUsers` because they are queried during authentication before a
current-user scope exists.

The email-provider connection group (`GmailConnections`, `MicrosoftGraphConnections`, and
`ImapConnections`) moved in `20260830124000_AdoptEmailConnectionSchema`. Startup retains only
additive Gmail sync-column repair plus guarded MariaDB identity/index repair; it does not create
these tables.

The CV import persistence group (`CvUploadArtifacts` and `CvExtractionRuns`) moved in
`20260830125000_AdoptCvExtractionSchema`. The extraction run's optional artifact foreign key remains
`ON DELETE SET NULL`, so retention cleanup cannot erase the extraction/review record.

`TailoredCvDrafts` moved in `20260830126000_AdoptTailoredCvDraftSchema`; its generated and edited
content is retained during adoption while the one-to-one job-application relationship remains
`ON DELETE CASCADE`.

The persisted job-workspace note group (`InterviewPrepNotes` and `AiWorkspaceNotes`) moved in
`20260830127000_AdoptJobWorkspaceNotesSchema`. Both caches retain their unique owner/job keys and
remain application-owned through `ON DELETE CASCADE`.

The CV builder aggregate (`CvVariants` and `CvVariantVersions`) moved in
`20260830128000_AdoptCvVariantSchema`. Public slugs remain unique, deleting a linked application
sets the CV link to null, and deleting a CV cascades through its append-only revision history.

`AiInteractions` moved in `20260830129000_AdoptAiInteractionSchema`. The older cross-feature usage
migration retains its compatibility bootstrap for historical chain traversal, but startup no longer
creates the table; only additive counter and guarded MariaDB shape/index repairs remain.

`ApplicationChecklistItems` moved in `20260830130000_AdoptApplicationChecklistSchema`. Its stable
system-key uniqueness and owner/job/sort index preserve idempotent seeding alongside freely ordered
manual tasks; application deletion remains cascading.

`CoverLetterVersions` moved in `20260830131000_AdoptCoverLetterVersionSchema`. Manual and AI-approved
revisions retain their source/action metadata and owner/job/version ordering; application deletion
remains cascading.

`InterviewPrepItems` moved in `20260830132000_AdoptInterviewPrepItemSchema`. User-authored and
AI-generated questions, answers, preparation state, sources, and ordering are retained; application
deletion remains cascading.

The Career Profile aggregate (`CareerProfiles`, `CareerProfileVersions`, and all six relational
child tables) moved in `20260830133000_AdoptCareerProfileSchema`. Canonical/long-tail JSON,
append-only history, stable child keys and ordering are retained; bounded MariaDB owner/item keys
keep all aggregate indexes provider-safe.

The ASP.NET Identity aggregate moved in `20260830134000_AdoptIdentitySchema`. Accounts, password
hashes, security/2FA state, UI preferences, roles, claims, external logins, tokens, indexes, and
cascades are retained. The earlier guarded `AspNetUsers` bootstrap remains only so historical
standalone migration traversal can reach later additive user-column migrations.

The reconciler may **repair** these (add a missing column, add an index, fix a non-`AUTO_INCREMENT`
primary key) and may seed the default `RuleSettings` row — but it must never `CREATE TABLE` them.
It used to create `RuleSettings`, which is precisely why a clean install failed with
`Table 'RuleSettings' already exists` once `Migrate()` reached the initial migration.

### Reconciler-owned

None. The reconciler contains guarded historical repair logic, but it no longer creates EF model
tables. Compatibility bootstraps inside old migrations are chain prerequisites, not current owners.

`StartupSchemaOwnership` is the executable inventory. Its tests require every EF model table to
have exactly one creation owner and keep compatibility bootstraps out of the migration-owned set.

No-op migrations, each with a comment explaining why:

| Migration | Tables |
|---|---|
| `20260717222917_AddCareerProfileRelationalChildren` | historical no-op; ownership transferred by `20260830133000_AdoptCareerProfileSchema` |
| `20260718074509_AddCvVariants` | historical no-op; ownership transferred by `20260830128000_AdoptCvVariantSchema` |
| `20260718131138_AddAiInteractions` | historical no-op; ownership transferred by `20260830129000_AdoptAiInteractionSchema` |
| `20260719085904_AddApplicationChecklistItems` | historical no-op; ownership transferred by `20260830130000_AdoptApplicationChecklistSchema` |
| `20260719094728_SyncCareerChildKeyLengths` | historical snapshot sync; ownership transferred by `20260830133000_AdoptCareerProfileSchema` |
| `20260719120954_AddCoverLetterVersions` | historical no-op; ownership transferred by `20260830131000_AdoptCoverLetterVersionSchema` |
| `20260719145044_AddInterviewPrepItems` | historical no-op; ownership transferred by `20260830132000_AdoptInterviewPrepItemSchema` |

### Dependency guards

A reconciler table that references another table is guarded on **its parent existing**, so pass 1
skips it on a fresh database and pass 2 creates it:

| Table | Waits for |
|---|---|
| `TailoredCvDrafts`, `InterviewPrepNotes`, `AiWorkspaceNotes`, `CvVariants`, `AiInteractions`, `ApplicationChecklistItems`, `CoverLetterVersions`, `InterviewPrepItems` | `JobApplications` (migration-owned) |
| `CvVariantVersions` | `CvVariants` |
| `CareerProfileVersions`, the six CareerProfile children | `CareerProfiles` (migration-owned) |
| `CvExtractionRuns` | `CvUploadArtifacts` |

Index creation goes through one helper, `EnsureMySqlIndex`, which is guarded on **table** existence
as well as index existence — repairing an absent table is not pass 1's job.

## Adding a new table

1. Add the entity and its `DbSet`, and **bound every indexed string** with `HasMaxLength` — an
   unbounded string becomes `longtext`, which MariaDB cannot index without a prefix length. This is
   what broke the CareerProfile children.
2. Add an EF migration. Review its generated SQL for both SQLite and MariaDB; do not assume a
   SQLite-scaffolded type is valid on MariaDB.
3. If needed, replace the generated operation with provider-aware migration SQL (`int
   AUTO_INCREMENT`, bounded `varchar(n)`, `datetime(6)`, `tinyint(1)` on MariaDB). Do not add the
   table to `StartupInitializationExtensions`.
4. Classify the table in `StartupSchemaOwnership.MigrationOwnedTables`; the ownership test will fail
   if it is omitted or duplicated.
5. Verify blank, populated-upgrade and restart paths on SQLite and a real MariaDB container. EF
   InMemory will not catch provider DDL defects.

## Retiring a reconciler-owned table

Move one leaf/dependency group at a time:

1. Add an idempotent provider-aware migration that creates the legacy shape when absent.
2. Make downgrade preserve a table that may predate migration ownership; never drop ambiguous data.
3. Remove only that table's startup `CREATE TABLE` block and move it between the executable ownership sets.
4. Prove a blank database, a pre-migration database with a representative row, migration retry, and
   provider SQL/runtime behaviour.
5. Leave column/index repairs in place until historical upgrade fixtures prove they are redundant;
   table creation and legacy repair are separate ownership decisions.

## Fresh install

No manual database preparation. Point the app at an **empty** database and start it:

```bash
# MariaDB
Database__Provider=mysql \
ConnectionStrings__JobTracker="Server=…;Database=jobtracker;User=…;Password=…;" \
dotnet run --project JobTrackerApi/JobTrackerApi.csproj

# SQLite (default) — creates Data__Root/jobtracker.db
dotnet run --project JobTrackerApi/JobTrackerApi.csproj
```

Create the empty schema/database itself (`CREATE DATABASE jobtracker;`); the application builds
everything inside it.

Standalone EF tooling is also supported for a blank SQLite database. The historical initial
migration now supplies the stable JobApplication columns required by later SQLite rebuilds, and
guarded compatibility bootstraps provide source tables used by later additive migrations.
Application startup may subsequently apply guarded historical column/index repairs without losing rows.

## Production upgrade

Deploy and restart. The reconciler is idempotent and additive:

- it never drops a table that holds rows (`DropMalformedMySqlTable` checks the row count first)
- it only adds missing columns, tables and indexes
- migrations already recorded in `__EFMigrationsHistory` are not re-run, so emptying a migration's
  `Up` changes nothing for an existing database

No downtime step, no manual SQL, no data migration.

## Verified

All four scenarios, 2026-07-19, against MariaDB 11 and SQLite:

| Scenario | Result |
|---|---|
| Empty MariaDB | 42 tables created, app starts |
| Populated MariaDB, restart | idempotent — still 42 tables, rows preserved |
| Partially-migrated MariaDB (Phase 4/5 tables dropped) | healed 35 → 42, surviving rows preserved |
| Empty MariaDB via the Docker image | 42 tables created, app starts |
| Fresh SQLite | 42 tables created, app starts |
| Existing partially-migrated SQLite dev DB (34 tables) | upgraded to 44 tables, 13 applications and 8 companies preserved |

Column types on MariaDB spot-checked: `int AUTO_INCREMENT` primary keys, `varchar(255)` owner keys,
`datetime(6)` timestamps, `tinyint(1)` booleans, and every composite index inside the key limit.

On 2026-08-15 the current 29-migration chain was additionally verified against a blank standalone
SQLite database, an older populated SQLite checkpoint, and a disposable MariaDB 11.8 database.
Standalone SQLite migration and retry both reached the latest migration; populated title/date and
reconciler-owned owner/summary data survived. Starting the application over that EF-only database
served `/health` successfully. Fresh MariaDB startup and restart both served `/health` with 29
migrations and 49 tables; provider-sensitive ID, owner, decimal and timestamp column types were
spot-checked. No production database was changed.
