feat(ops): protect complete pre-deploy recovery set
CI and Deploy / test (push) Successful in 6m56s
CI and Deploy / deploy (push) Successful in 1m10s

This commit is contained in:
cesnimda
2026-09-01 20:06:25 +02:00
parent bf3dc2d06e
commit bd7fa05b8c
6 changed files with 223 additions and 11 deletions
+9 -7
View File
@@ -86,9 +86,10 @@ MYSQL_PWD='<pw>' mariadb --host=<host> --user=<user> --default-character-set=utf
## Production access
A sanitized read-only host inventory was completed on 2026-08-15. It verified the integrity of the
existing compressed database dumps without reading private rows or restoring data. The newest
observed dump was dated 2026-08-02, and the set is database-only: it does not prove recovery of owned
files, data-protection keys, configuration, or account-deletion tombstones.
then-existing compressed database dumps without reading private rows or restoring data. The newest
observed dump was dated 2026-08-02, and that set was database-only. Current deployments now create a
verified database backup plus application-data and deletion-tombstone volume archives before any
build or replacement. A scratch restore is still required before recovery is considered proven.
**Authorized production steps still required:**
@@ -98,10 +99,11 @@ files, data-protection keys, configuration, or account-deletion tombstones.
- table count (~42) and row counts for `AspNetUsers`, `JobApplications`, `Companies`,
`CareerProfiles` match production;
- a real record containing `æ`/`ø`/`å` reads back correctly (the check above).
3. Build and restore a complete recovery bundle covering the `jobtracker_data` volume and deployment
data-protection keys as well as MariaDB. Preserve the separately mounted
`jobtracker_deletion_tombstones` volume across application-data restores; never overwrite it with
an older backup capable of resurrecting a deleted identity.
3. Restore the matching `jobtracker_data` archive alongside the database dump. It contains owner
files and deployment Data Protection keys.
4. Verify the separately archived `jobtracker_deletion_tombstones` ledger, but preserve the newest
ledger across application-data restores; never overwrite it with an older backup capable of
resurrecting a deleted identity.
Until that is done, backup/restore is proven **on the mechanism and on synthetic Norwegian data**, not
on the production dataset.
+3
View File
@@ -1,5 +1,8 @@
# Stripe activation
For a dashboard-by-dashboard operator walkthrough, use
[`stripe-billing-setup-walkthrough.md`](stripe-billing-setup-walkthrough.md).
The application code is complete. Activate it in Stripe test mode first; test and live mode have
different API keys, price ids, webhook endpoints/signing secrets, and customer data.
@@ -0,0 +1,133 @@
# Stripe billing setup walkthrough
This closes production blocker 2. Complete the test-mode checklist first, prove the full lifecycle,
then repeat it in live mode. Stripe test and live objects are separate: keys, prices, webhook secrets,
customers and subscriptions cannot be mixed.
The current production misconfiguration is specific: `STRIPE_PRICE_PREMIUM` contains a Product ID
(`prod_...`). JobTracker needs the recurring Price ID attached to that product (`price_...`). The API
deliberately treats the Product ID as disabled instead of sending an invalid Checkout request.
## What you need
- Access to the Stripe account that will receive JobTracker subscriptions.
- Access to `/opt/job-tracker/shared/.env` on the production host.
- One disposable JobTracker user for the test purchase.
- A decision on the monthly Pro price and currency. Start with one monthly price; annual billing can
be added after the monthly lifecycle is proven.
Never paste Stripe keys or webhook signing secrets into Git, chat, screenshots, tickets or CI logs.
## Part 1 — create the test subscription price
1. Sign in to the Stripe Dashboard and enable **Test mode**.
2. Open **Product catalog** and either open the existing JobTracker Pro product or create one named
`JobTracker Pro`.
3. Add a price with:
- pricing model: standard/flat rate;
- billing: recurring;
- interval: monthly;
- currency and amount: your chosen commercial terms.
4. Save the price.
5. Open the new price and copy its identifier. It must start with `price_`, not `prod_`.
The product describes what is sold. The price describes how it is billed. Checkout needs the latter.
## Part 2 — configure the customer portal
1. In Stripe, open the customer-portal settings.
2. Enable the portal in test mode.
3. Allow customers to update payment details and cancel subscriptions.
4. Decide whether cancellation is immediate or at the end of the billing period. The application
follows Stripe's resulting subscription status; Stripe remains authoritative.
5. Save the portal configuration.
## Part 3 — create the test webhook
1. Open Stripe's webhook/event-destination area in test mode.
2. Add this public HTTPS endpoint:
```text
https://jobs.cesnimda.uk/api/billing/webhook
```
3. Subscribe it to exactly these events:
```text
customer.subscription.created
customer.subscription.updated
customer.subscription.deleted
```
4. Save the endpoint.
5. Reveal and copy that endpoint's signing secret. It starts with `whsec_`.
The endpoint signing secret belongs to this specific webhook endpoint and mode. It is not the API
secret key shown on Stripe's API keys page.
## Part 4 — install the three test values
On the production host, edit `/opt/job-tracker/shared/.env` and set:
```text
STRIPE_SECRET_KEY=<test secret key beginning sk_test_>
STRIPE_PRICE_PREMIUM=<test recurring price id beginning price_>
STRIPE_WEBHOOK_SECRET=<test endpoint secret beginning whsec_>
```
Before saving, verify all three values came from test mode. Do not mix a live key with a test price or
test webhook secret. Redeploy after saving so the backend receives the new environment.
## Part 5 — prove the test lifecycle
Use a disposable authenticated JobTracker account:
1. Open account/settings and confirm billing is available.
2. Select **Upgrade**. Stripe-hosted Checkout must open for the expected monthly amount and currency.
3. Complete Checkout using a Stripe test card.
4. Confirm Stripe redirects to `/settings?billing=success`.
5. In Stripe's webhook delivery log, confirm the subscription event received a successful `2xx`
response.
6. Refresh JobTracker and confirm the user has Pro/Premium entitlements.
7. Open **Manage billing** and confirm the Stripe customer portal opens.
8. Cancel the test subscription using the portal.
9. Confirm the updated/deleted subscription webhook succeeds and JobTracker removes Premium after
the subscription is no longer active or trialling.
10. Replay one already-successful event from Stripe. JobTracker's stored state and role must remain
correct; replay must not duplicate or corrupt billing state.
Do not continue to live mode if any webhook delivery is red, Checkout shows the wrong amount, or the
role does not follow Stripe's subscription state.
## Part 6 — live-mode cutover
1. Disable Test mode in Stripe.
2. Create or open the live JobTracker Pro product and create a live recurring monthly price.
3. Configure the live customer portal.
4. Create the live webhook endpoint with the same URL and three subscription events.
5. Replace all three production environment values together:
- live `sk_live_...` secret key;
- live `price_...` recurring Price ID;
- live endpoint-specific `whsec_...` signing secret.
6. Redeploy.
7. Complete one low-risk real subscription, portal visit and cancellation.
8. Confirm every live webhook delivery is successful and the JobTracker role follows the live
subscription state.
## Troubleshooting
| Symptom | Check |
| --- | --- |
| Billing appears unavailable | `STRIPE_PRICE_PREMIUM` must begin `price_`; all three settings must be present. |
| Checkout says the price does not exist | The API key and Price ID probably come from different Stripe modes or accounts. |
| Webhook signature fails | Use the endpoint's `whsec_...` secret, not an API key or a Stripe CLI temporary secret. |
| Checkout succeeds but Premium is absent | Inspect the webhook delivery response; JobTracker provisions from signed subscription events. |
| Portal does not open | Enable and save the customer portal in the same Stripe mode as the customer. |
| Cancellation does not remove Premium | Check the resulting Stripe status and replay the authoritative subscription event. |
## Safe rollback
Removing all three Stripe environment values and redeploying disables new Checkout and portal
sessions without deleting billing history. Do not manually remove Premium while Stripe still reports
an active subscription. Repair webhook delivery and replay the authoritative event instead.