cesnimda
3d74baef78
Phase 5.5. Completes the lifecycle after submission: prepare, communicate, chase. Interview preparation gets a durable, user-owned store. There were already two per-application AI stores, InterviewPrepNote and AiWorkspaceNote, but both are caches that regenerate when their context signature changes — anything a user typed into them would eventually be overwritten. InterviewPrepItem is the side nothing regenerates, covering company research, technical notes, behavioural answers, STAR examples and the user's own questions in one table, because those categories differ only by label and adding one must not need a migration. Each item records whether the user wrote it or accepted a suggestion, and an IsPrepared flag makes the section double as the preparation checklist. Generation stays in the existing AiWorkspaceService "interview" module, appended to AiInteraction as before. A suggestion is history until the user adds it as a prep item; opening the section generates nothing. Follow-up reuses what exists rather than adding a tracker. The date is JobApplication.FollowUpAt, the same field RulesEngine and the reminder hosted service already act on, so reminders keep working with no new wiring. The task stays an ApplicationChecklistItem in the follow-up category — the section counts open tasks without owning them. The record is a FollowUpSet JobEvent, the same type the rest of the app emits. Communication is untouched: Correspondence already owns recruiter contacts, history and notes, and the workspace already mounted it. The timeline interpreter learned five more types — InterviewScheduled, InterviewCompleted and OfferReceived as milestones, FollowUpCreated and FollowUpCompleted as routine, deliberately outside the milestone spine so it stays a summary of what actually happened. JobEvent remains the history source. InterviewPrepItems is reconciler-owned with a no-op migration, guarded on JobApplications, and verified on a fresh MariaDB 11: int AUTO_INCREMENT primary key, varchar owner and title, tinyint flag, datetime(6), composite index inside the key limit. 371 backend tests, 128 frontend tests, Release build and the production build all pass locally. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
161 lines
7.4 KiB
Markdown
161 lines
7.4 KiB
Markdown
# Database ownership and startup order
|
|
|
|
> 2026-07-19. Which component creates which table, in what order, and why a clean MariaDB install used
|
|
> to fail. Read this before adding a table or touching `StartupInitializationExtensions`.
|
|
|
|
## The problem this document exists to prevent
|
|
|
|
EF Core bakes **provider-specific type names into a migration at scaffold time**. Every migration in
|
|
this repo was scaffolded against SQLite, so run against MariaDB it emits:
|
|
|
|
- `TEXT` for `DateTimeOffset` and every unbounded string
|
|
- `INTEGER` for `bool` and `int`
|
|
- a `PRIMARY KEY` with **no** `AUTO_INCREMENT`
|
|
|
|
A composite index over one of those `TEXT`/`longtext` columns then exceeds MySQL's 3072-byte key
|
|
limit and startup dies with `Specified key was too long`. This is not theoretical — it crashed
|
|
production once (Phase 4 `CvVariants`) and made every clean MariaDB install fail until 2026-07-19.
|
|
|
|
**Rule: a table whose migration was scaffolded against SQLite must not be created by that migration
|
|
on MariaDB.** Empty the migration and give the table to the reconciler, which carries correct DDL per
|
|
provider.
|
|
|
|
## Startup order
|
|
|
|
`InitializeJobTrackerAsync` runs exactly this sequence:
|
|
|
|
```
|
|
1. Connect
|
|
2. ReconcileSchema() ← pass 1: repair existing schema, create reconciler-owned tables
|
|
3. Database.Migrate() ← create every migration-owned table
|
|
4. ReconcileSchema() ← pass 2: everything pass 1 had to skip
|
|
5. Seed admin, start services
|
|
```
|
|
|
|
### Why the reconciler runs twice
|
|
|
|
Neither position alone works:
|
|
|
|
- **Pass 1 must come first.** A legacy database has hand-added columns and Identity tables that
|
|
predate the migrations; without repairing them (and stamping the legacy migration id into
|
|
`__EFMigrationsHistory`) `Migrate()` collides with them. `AddCareerProfileRelationalChildren` also
|
|
adds children that reference `CareerProfiles`, a **reconciler-owned** table — so it must exist
|
|
before migrations run.
|
|
- **Pass 2 must come after.** On a brand-new database the migration-owned tables do not exist during
|
|
pass 1, so every reconciler table that references one (FK into `JobApplications`) is skipped, as
|
|
are the index and `AUTO_INCREMENT` repairs.
|
|
|
|
Every statement in `ReconcileSchema` is existence-guarded, so the second pass is a no-op scan on an
|
|
already-correct database. Two consequences worth knowing:
|
|
|
|
- The `DbConnection` is **not** wrapped in `using` — it belongs to the `DbContext`, and disposing it
|
|
in pass 1 made pass 2 throw `ObjectDisposedException`.
|
|
- `conn.Open()` is guarded on `ConnectionState`, because pass 2 may inherit an open connection.
|
|
|
|
## Ownership
|
|
|
|
### Migration-owned
|
|
|
|
Created by EF migrations, never by the reconciler:
|
|
|
|
`Companies`, `JobApplications`, `Jobs`, `Correspondences`, `Attachments`, `JobEvents`,
|
|
`RuleSettings`, and the ASP.NET Identity tables.
|
|
|
|
The reconciler may **repair** these (add a missing column, add an index, fix a non-`AUTO_INCREMENT`
|
|
primary key) and may seed the default `RuleSettings` row — but it must never `CREATE TABLE` them.
|
|
It used to create `RuleSettings`, which is precisely why a clean install failed with
|
|
`Table 'RuleSettings' already exists` once `Migrate()` reached the initial migration.
|
|
|
|
### Reconciler-owned
|
|
|
|
Created by `StartupInitializationExtensions`, with a **no-op migration** holding the model snapshot:
|
|
|
|
`UserRuleSettings`, `SystemEmailSettings`, `CvUploadArtifacts`, `CvExtractionRuns`,
|
|
`GmailConnections`, `MicrosoftGraphConnections`, `ImapConnections`, `TailoredCvDrafts`,
|
|
`CareerProfiles`, `CareerProfileVersions`, the six CareerProfile children (`CareerExperiences`,
|
|
`CareerEducations`, `CareerSkills`, `CareerProjects`, `CareerCertifications`, `CareerLanguages`),
|
|
`InterviewPrepNotes`, `AiWorkspaceNotes`, `CvVariants`, `CvVariantVersions`, `AiInteractions`,
|
|
`ApplicationChecklistItems`, `CoverLetterVersions`, `InterviewPrepItems`, `TwoFactorRecoveryCodes`,
|
|
`TrustedDevices`, `UserSessions`.
|
|
|
|
No-op migrations, each with a comment explaining why:
|
|
|
|
| Migration | Tables |
|
|
|---|---|
|
|
| `20260717222917_AddCareerProfileRelationalChildren` | the six CareerProfile children |
|
|
| `20260718074509_AddCvVariants` | `CvVariants`, `CvVariantVersions` |
|
|
| `20260718131138_AddAiInteractions` | `AiInteractions` |
|
|
| `20260719085904_AddApplicationChecklistItems` | `ApplicationChecklistItems` |
|
|
| `20260719094728_SyncCareerChildKeyLengths` | snapshot sync only |
|
|
| `20260719120954_AddCoverLetterVersions` | `CoverLetterVersions` |
|
|
| `20260719145044_AddInterviewPrepItems` | `InterviewPrepItems` |
|
|
|
|
### Dependency guards
|
|
|
|
A reconciler table that references another table is guarded on **its parent existing**, so pass 1
|
|
skips it on a fresh database and pass 2 creates it:
|
|
|
|
| Table | Waits for |
|
|
|---|---|
|
|
| `TailoredCvDrafts`, `InterviewPrepNotes`, `AiWorkspaceNotes`, `CvVariants`, `AiInteractions`, `ApplicationChecklistItems`, `CoverLetterVersions`, `InterviewPrepItems` | `JobApplications` (migration-owned) |
|
|
| `CvVariantVersions` | `CvVariants` |
|
|
| `CareerProfileVersions`, the six CareerProfile children | `CareerProfiles` |
|
|
| `CvExtractionRuns` | `CvUploadArtifacts` |
|
|
|
|
Index creation goes through one helper, `EnsureMySqlIndex`, which is guarded on **table** existence
|
|
as well as index existence — repairing an absent table is not pass 1's job.
|
|
|
|
## Adding a new table
|
|
|
|
1. Add the entity and its `DbSet`, and **bound every indexed string** with `HasMaxLength` — an
|
|
unbounded string becomes `longtext`, which MariaDB cannot index without a prefix length. This is
|
|
what broke the CareerProfile children.
|
|
2. `dotnet ef migrations add …`, then **empty the `Up`/`Down`** and say why in a comment.
|
|
3. Add SQLite DDL (`CREATE TABLE IF NOT EXISTS`) and MySQL DDL (`int AUTO_INCREMENT`, `varchar(n)`,
|
|
`datetime(6)`, `tinyint(1)`) to the reconciler. Guard the MySQL create on any parent table.
|
|
4. Create indexes via `EnsureMySqlIndex` / `CREATE INDEX IF NOT EXISTS`.
|
|
5. Verify on a real MariaDB container — see below. EF InMemory will not catch any of this.
|
|
|
|
## Fresh install
|
|
|
|
No manual database preparation. Point the app at an **empty** database and start it:
|
|
|
|
```bash
|
|
# MariaDB
|
|
Database__Provider=mysql \
|
|
ConnectionStrings__JobTracker="Server=…;Database=jobtracker;User=…;Password=…;" \
|
|
dotnet run --project JobTrackerApi/JobTrackerApi.csproj
|
|
|
|
# SQLite (default) — creates Data__Root/jobtracker.db
|
|
dotnet run --project JobTrackerApi/JobTrackerApi.csproj
|
|
```
|
|
|
|
Create the empty schema/database itself (`CREATE DATABASE jobtracker;`); the application builds
|
|
everything inside it.
|
|
|
|
## Production upgrade
|
|
|
|
Deploy and restart. The reconciler is idempotent and additive:
|
|
|
|
- it never drops a table that holds rows (`DropMalformedMySqlTable` checks the row count first)
|
|
- it only adds missing columns, tables and indexes
|
|
- migrations already recorded in `__EFMigrationsHistory` are not re-run, so emptying a migration's
|
|
`Up` changes nothing for an existing database
|
|
|
|
No downtime step, no manual SQL, no data migration.
|
|
|
|
## Verified
|
|
|
|
All four scenarios, 2026-07-19, against MariaDB 11 and SQLite:
|
|
|
|
| Scenario | Result |
|
|
|---|---|
|
|
| Empty MariaDB | 40 tables created, app starts |
|
|
| Populated MariaDB, restart | idempotent — still 40 tables, rows preserved |
|
|
| Empty MariaDB via the Docker image | 40 tables created, app starts |
|
|
| Fresh SQLite | 42 tables created, app starts |
|
|
| Existing partially-migrated SQLite dev DB (34 tables) | upgraded to 44 tables, 13 applications and 8 companies preserved |
|
|
|
|
Column types on MariaDB spot-checked: `int AUTO_INCREMENT` primary keys, `varchar(255)` owner keys,
|
|
`datetime(6)` timestamps, `tinyint(1)` booleans, and every composite index inside the key limit.
|