feat(ops): automate backups and private CV benchmarks

2026-09-02 00:49:06 +02:00
parent bd7fa05b8c
commit de85492373
10 changed files with 301 additions and 46 deletions
@@ -0,0 +1,48 @@
+# Encrypted Google Drive backups
+
+Yes—JobTracker recovery sets can be copied to Google Drive. Use an `rclone crypt` remote so database,
+CV, attachment and key archives are encrypted before they leave the production host.
+
+## One-time production setup
+
+1. Install `rclone` from the operating system's trusted package source.
+2. Run `rclone config` as the same operating-system user used by deployment.
+3. Create a Google Drive remote, for example `gdrive`.
+4. Complete Google OAuth in your browser. Do not paste its refresh token into Git, chat or CI.
+5. Create a second remote of type `crypt`, for example `gdrive-crypt`, targeting a dedicated folder
+   such as `gdrive:JobTrackerBackups`.
+6. Enable filename and directory-name encryption. Store the crypt password and salt in the protected
+   production credential store/rclone configuration, not in the repository.
+7. Verify the encrypted remote without uploading private application data:
+
+   ```bash
+   printf 'backup connectivity test\n' >/tmp/jobtracker-backup-test.txt
+   rclone copy /tmp/jobtracker-backup-test.txt gdrive-crypt:jobtracker-test
+   rclone ls gdrive-crypt:jobtracker-test
+   rm -f /tmp/jobtracker-backup-test.txt
+   rclone purge gdrive-crypt:jobtracker-test
+   ```
+
+8. Add this to `/opt/job-tracker/shared/.env`:
+
+   ```text
+   BACKUP_RCLONE_REMOTE=gdrive-crypt:jobtracker
+   ```
+
+9. Run the **Production backup** workflow manually. It must create the local verified recovery set
+   first, copy that set to Drive, and fail visibly if the off-host copy fails.
+
+## Schedule and retention
+
+- Scheduled workflow: 00:00 and 01:00 UTC triggers with a Europe/Oslo guard; one backup runs at local
+  02:00 even when daylight-saving time changes.
+- Recovery-point objective: 24 hours.
+- Local retention: 30 days plus 12 monthly recovery sets.
+- Google Drive retention: the upload is immutable. Apply a matching Drive/rclone lifecycle only after
+  a scratch restore has proven the off-host copy.
+
+## Recovery rule
+
+Download into a scratch directory and restore into a separate MariaDB/database-volume environment.
+Never rehearse by restoring over live production. Preserve the newest deletion-tombstone ledger when
+restoring older application data.