jobtrackingapp/docs/infrastructure/database-ownership.md

Database ownership and startup order

2026-07-19. Which component creates which table, in what order, and why a clean MariaDB install used to fail. Read this before adding a table or touching StartupInitializationExtensions.

The problem this document exists to prevent

EF Core bakes provider-specific type names into a migration at scaffold time. Every migration in this repo was scaffolded against SQLite, so run against MariaDB it emits:

• TEXT for DateTimeOffset and every unbounded string
• INTEGER for bool and int
• a PRIMARY KEY with no AUTO_INCREMENT

A composite index over one of those TEXT/longtext columns then exceeds MySQL's 3072-byte key limit and startup dies with Specified key was too long. This is not theoretical — it crashed production once (Phase 4 CvVariants) and made every clean MariaDB install fail until 2026-07-19.

Rule: a table whose migration was scaffolded against SQLite must not be created by that migration on MariaDB. Empty the migration and give the table to the reconciler, which carries correct DDL per provider.

Startup order

InitializeJobTrackerAsync runs exactly this sequence:

1. Connect
2. ReconcileSchema()      ← pass 1: repair existing schema, create reconciler-owned tables
3. Database.Migrate()     ← create every migration-owned table
4. ReconcileSchema()      ← pass 2: everything pass 1 had to skip
5. Seed admin, start services

Why the reconciler runs twice

Neither position alone works:

• Pass 1 must come first. A legacy database has hand-added columns and Identity tables that predate the migrations; without repairing them (and stamping the legacy migration id into __EFMigrationsHistory) Migrate() collides with them. AddCareerProfileRelationalChildren also adds children that reference CareerProfiles, a reconciler-owned table — so it must exist before migrations run.
• Pass 2 must come after. On a brand-new database the migration-owned tables do not exist during pass 1, so every reconciler table that references one (FK into JobApplications) is skipped, as are the index and AUTO_INCREMENT repairs.

Every statement in ReconcileSchema is existence-guarded, so the second pass is a no-op scan on an already-correct database. Two consequences worth knowing:

• The DbConnection is not wrapped in using — it belongs to the DbContext, and disposing it in pass 1 made pass 2 throw ObjectDisposedException.
• conn.Open() is guarded on ConnectionState, because pass 2 may inherit an open connection.

Ownership

Migration-owned

Created by EF migrations, never by the reconciler:

Companies, JobApplications, Jobs, Correspondences, Attachments, JobEvents, RuleSettings, and the ASP.NET Identity tables.

The reconciler may repair these (add a missing column, add an index, fix a non-AUTO_INCREMENT primary key) and may seed the default RuleSettings row — but it must never CREATE TABLE them. It used to create RuleSettings, which is precisely why a clean install failed with Table 'RuleSettings' already exists once Migrate() reached the initial migration.

Reconciler-owned

Created by StartupInitializationExtensions, with a no-op migration holding the model snapshot:

UserRuleSettings, SystemEmailSettings, CvUploadArtifacts, CvExtractionRuns, GmailConnections, MicrosoftGraphConnections, ImapConnections, TailoredCvDrafts, CareerProfiles, CareerProfileVersions, the six CareerProfile children (CareerExperiences, CareerEducations, CareerSkills, CareerProjects, CareerCertifications, CareerLanguages), InterviewPrepNotes, AiWorkspaceNotes, CvVariants, CvVariantVersions, AiInteractions, ApplicationChecklistItems, CoverLetterVersions, TwoFactorRecoveryCodes, TrustedDevices, UserSessions.

No-op migrations, each with a comment explaining why:

Migration	Tables
20260717222917_AddCareerProfileRelationalChildren	the six CareerProfile children
20260718074509_AddCvVariants	CvVariants, CvVariantVersions
20260718131138_AddAiInteractions	AiInteractions
20260719085904_AddApplicationChecklistItems	ApplicationChecklistItems
20260719094728_SyncCareerChildKeyLengths	snapshot sync only
20260719120954_AddCoverLetterVersions	CoverLetterVersions

Dependency guards

A reconciler table that references another table is guarded on its parent existing, so pass 1 skips it on a fresh database and pass 2 creates it:

Table	Waits for
TailoredCvDrafts, InterviewPrepNotes, AiWorkspaceNotes, CvVariants, AiInteractions, ApplicationChecklistItems, CoverLetterVersions	JobApplications (migration-owned)
CvVariantVersions	CvVariants
CareerProfileVersions, the six CareerProfile children	CareerProfiles
CvExtractionRuns	CvUploadArtifacts

Index creation goes through one helper, EnsureMySqlIndex, which is guarded on table existence as well as index existence — repairing an absent table is not pass 1's job.

Adding a new table

1. Add the entity and its DbSet, and bound every indexed string with HasMaxLength — an unbounded string becomes longtext, which MariaDB cannot index without a prefix length. This is what broke the CareerProfile children.
2. dotnet ef migrations add …, then empty the Up/Down and say why in a comment.
3. Add SQLite DDL (CREATE TABLE IF NOT EXISTS) and MySQL DDL (int AUTO_INCREMENT, varchar(n), datetime(6), tinyint(1)) to the reconciler. Guard the MySQL create on any parent table.
4. Create indexes via EnsureMySqlIndex / CREATE INDEX IF NOT EXISTS.
5. Verify on a real MariaDB container — see below. EF InMemory will not catch any of this.

Fresh install

No manual database preparation. Point the app at an empty database and start it:

# MariaDB
Database__Provider=mysql \
ConnectionStrings__JobTracker="Server=…;Database=jobtracker;User=…;Password=…;" \
dotnet run --project JobTrackerApi/JobTrackerApi.csproj

# SQLite (default) — creates Data__Root/jobtracker.db
dotnet run --project JobTrackerApi/JobTrackerApi.csproj

Create the empty schema/database itself (CREATE DATABASE jobtracker;); the application builds everything inside it.

Production upgrade

Deploy and restart. The reconciler is idempotent and additive:

• it never drops a table that holds rows (DropMalformedMySqlTable checks the row count first)
• it only adds missing columns, tables and indexes
• migrations already recorded in __EFMigrationsHistory are not re-run, so emptying a migration's Up changes nothing for an existing database

No downtime step, no manual SQL, no data migration.

Verified

All four scenarios, 2026-07-19, against MariaDB 11 and SQLite:

Scenario	Result
Empty MariaDB	40 tables created, app starts
Populated MariaDB, restart	idempotent — still 40 tables, rows preserved
Empty MariaDB via the Docker image	40 tables created, app starts
Fresh SQLite	42 tables created, app starts
Existing partially-migrated SQLite dev DB (34 tables)	upgraded to 44 tables, 13 applications and 8 companies preserved

Column types on MariaDB spot-checked: int AUTO_INCREMENT primary keys, varchar(255) owner keys, datetime(6) timestamps, tinyint(1) booleans, and every composite index inside the key limit.