test(release): exercise rollback handler offline
This commit is contained in:
@@ -34,3 +34,21 @@ Proposed future remediation: parse with the same supported connection-string sem
|
||||
6. Measure recovery duration and data-loss window against owner-approved RTO/RPO. No target or measurement exists in this checkpoint. Preserve cleanup and evidence identities; no production execution or release approval implied.
|
||||
|
||||
Fresh checks: Git Bash `bash -n deploy/deploy.sh` PASSED (syntax only); `python scripts/test-mariadb-workflow.py` PASSED 4/4 (source quarantine guards only). These are separate from the earlier 5/5 offline runner tests. Restore, failure injection, hosted isolation, live encryption and rollback health remain NOT TESTED.
|
||||
|
||||
## RR-12 offline failure injection
|
||||
|
||||
Tested the unchanged `finish_deploy` function and EXIT trap from baseline `47c6834cee192a5f1f054ddcd500b4d45c48c985`, `deploy/deploy.sh` blob `8341110de5a827698efc6eeb6951f296770562da`. The test extracts only the handler and trap and supplies synthetic image refs and mock Compose and cleanup functions. The full deploy script was never sourced or executed.
|
||||
|
||||
| Case | Observed behavior |
|
||||
| --- | --- |
|
||||
| Replacement started, both previous core image references exist | Recreates backend and frontend with those refs only. The failed deployment status is preserved. No health probe follows the successful Compose call. |
|
||||
| Compose rejects rollback | Attempts status and backend/frontend log collection. The original failure status is preserved, even when the rollback command returns a different status. |
|
||||
| Either or both previous references are missing | Skips automatic recreation and reports that no complete prior core release is available. |
|
||||
| Deployment fails before replacement, or succeeds | Skips automatic recreation. |
|
||||
|
||||
Both auth-config temporary-file cleanup calls occur before the rollback decision. The test stubs deletion and checks the exact call sequence.
|
||||
|
||||
- `python scripts/test-deploy-rollback.py -v` **PASSED 5/5**, including three missing-reference combinations.
|
||||
- `python scripts/test-deploy-backup.py -v` **PASSED 5/5** and `python scripts/test-mariadb-workflow.py -v` **PASSED 4/4**. Git Bash `bash -n deploy/deploy.sh` passed syntax validation.
|
||||
|
||||
RR-12 remains open for an authorised isolated rehearsal. After the prior images are recreated, verify exact image identity, `/health`, `/ready`, and authenticated reads and writes against representative migrated and post-write data. A successful Compose call does not establish service health or recover schema, data, or the AI image. RR-13 still needs authorised off-host retrieval/decryption and recovery-set consistency verification. No Docker, network service, database, remote storage, deployment or restore was invoked by these checks.
|
||||
|
||||
@@ -4,7 +4,7 @@ Discovery only, 2026-11-02. These are qualification gaps unless a concrete defec
|
||||
|
||||
| ID | Historical item / current finding | Status / closure evidence needed |
|
||||
| --- | --- | --- |
|
||||
| RR-01 | Phase 9/10 and production NO-GO | Phase 10 assessment now independently published, implementation still awaits exact approval. Later roadmap acceptance absent. BLOCKED for final RC |
|
||||
| RR-01 | Phase 9/10 and production NO-GO | CLEAN-20 implementation and verification are published. Independent Phase 10 acceptance and later roadmap acceptance remain open. BLOCKED for final RC |
|
||||
| RR-02 | Old Linux 11/24 and later S01 failure narratives | Latest handoff records approved S01 36/36 per platform and owner visual acceptance for unchanged development output. Do not reopen these metrics without affected change; physical print/device remain NOT TESTED |
|
||||
| RR-03 | Python advisory/native parser/image qualification | Latest scoped conditional disposition retained; current advisories and production-equivalent native isolation NOT TESTED. No new scan or risk waiver |
|
||||
| RR-04 | Hosted MariaDB/isolation | Source explicitly fails before execution until isolated runner proof. BLOCKED; fresh local 5 guard tests do not close it |
|
||||
@@ -14,8 +14,8 @@ Discovery only, 2026-11-02. These are qualification gaps unless a concrete defec
|
||||
| RR-08 | MySQL TLS / dependency / production identity | Latest handoff retains MYSQL-TLS-01 and OPS-01. No live configuration read; current advisory disposition and deployed equivalence unverified |
|
||||
| RR-09 | Backup/restore and rollback | Existing procedures and historical seeded rehearsal are not a current restore. Need isolated recovery-set completeness, decryption, tombstone behavior, post-write rollback compatibility and agreed RPO/RTO |
|
||||
| RR-10 | AI quality/activation | Deterministic Docling fixtures are distinct from real-model semantic accuracy. Optional unqualified adapter remains disabled; no AI invocation authorised here |
|
||||
| RR-11 | Backup host parsing rejects supported `Host=` alias; uppercase spelling also unrecognised | REPRODUCED with synthetic input and shell-native positive control. Deployment aborts before backup; proposed P2 compatibility defect. Fix scope awaits authorised release remediation |
|
||||
| RR-12 | Rollback restores core images without post-rollback health qualification; no schema/data/AI reversal | VERIFIED source limitation, runtime impact NOT TESTED. Requires isolated post-migration/post-write recovery rehearsal; no rollback guarantee |
|
||||
| RR-11 | Backup host parsing rejected supported `Host=` alias and uppercase spelling | FIXED AND OFFLINE-TESTED. See [bounded remediation](RR-11_IMPLEMENTATION.md). Real dump/restore qualification remains open |
|
||||
| RR-12 | Rollback recreates core images without post-rollback health qualification; no schema/data/AI reversal | OFFLINE HANDLER CHECKS PASSED 5/5. See [failure-injection evidence](11_BACKUP_RESTORE_ROLLBACK.md#rr-12-offline-failure-injection). Actual image recovery and health/readiness after migration/writes still require isolated rehearsal |
|
||||
| RR-13 | Off-host encryption and recovery-set consistency not established by script success messages | NOT TESTED operational qualification. Verify remote encryption, retrieval/decryption, database/files/key/tombstone consistency in approved isolation |
|
||||
|
||||
Next remediation is evidence work: complete the source/endpoint/environment census and resolve historical records against their exact latest evidence. RR-11 now has a bounded future fix proposal in 11_BACKUP_RESTORE_ROLLBACK.md; no runtime implementation is authorised by this discovery checkpoint. Every fix needs minimal scope, targeted verification and the applicable stage approval.
|
||||
Continue evidence work against the [source endpoint and environment census](03_ENDPOINT_ENVIRONMENT_CENSUS.md). Actual qualification requires an accepted candidate and isolated environment. Every fix needs minimal scope, targeted verification and the applicable stage approval.
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# Release discovery handoff
|
||||
|
||||
RR-12 offline failure injection passed 5/5 against the unchanged deploy exit handler. The evidence covers synthetic prior-image recreation, failed rollback diagnostics, original exit status and conditions that skip rollback. [Recovery report](11_BACKUP_RESTORE_ROLLBACK.md#rr-12-offline-failure-injection) distinguishes this from a real post-migration/post-write rehearsal. RR-13 off-host retrieval/decryption and consistent restore remain open. No production qualification.
|
||||
|
||||
10 November: [endpoint and environment census](03_ENDPOINT_ENVIRONMENT_CENSUS.md) maps 41 source route families, anonymous exceptions and failure cases to qualification owners on the backup-parser branch. No runtime qualification. Carry the map into Stage 2 only after the release candidate and environment are accepted.
|
||||
|
||||
Branch `planning/release-readiness-discovery`, baseline `5cb903b989742b7a96cb22ddf4c0e24993b18980`; documentation only. Stage 1 IN_PROGRESS, not complete. Full authoritative release prompt read. No final GO/NO-GO completion report or deployable release claimed.
|
||||
|
||||
@@ -0,0 +1,82 @@
|
||||
"""Exercise the deployment exit handler with synthetic image references and no Docker."""
|
||||
import os
|
||||
from pathlib import Path
|
||||
import shutil
|
||||
import subprocess
|
||||
import unittest
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
BASH = 'C:/Program Files/Git/bin/bash.exe' if os.name == 'nt' else shutil.which('bash')
|
||||
|
||||
|
||||
class DeployRollbackTests(unittest.TestCase):
|
||||
def exit_deploy(self, status, *, replaced=True, backend=True, frontend=True, compose_status=0):
|
||||
source = (ROOT / 'deploy/deploy.sh').read_text(encoding='utf-8')
|
||||
start = source.index('finish_deploy() {')
|
||||
handler = source[start:source.index('\ntrap finish_deploy EXIT\n', start) + len('\ntrap finish_deploy EXIT\n')]
|
||||
script = '\n'.join([
|
||||
'set -euo pipefail',
|
||||
f'replacement_started={str(replaced).lower()}',
|
||||
'previous_backend_ref=' + ('synthetic/backend:previous' if backend else ''),
|
||||
'previous_frontend_ref=' + ('synthetic/frontend:previous' if frontend else ''),
|
||||
'auth_config_body_file=synthetic-body',
|
||||
'auth_config_headers_file=synthetic-headers',
|
||||
'rm() { printf "CALL rm %s\\n" "$*"; }',
|
||||
'docker() { echo "CALL unexpected docker"; return 99; }',
|
||||
'curl() { echo "CALL unexpected curl"; return 99; }',
|
||||
'compose() {',
|
||||
' printf "CALL compose %s [%s] [%s]\\n" "$*" "${BACKEND_IMAGE:-}" "${FRONTEND_IMAGE:-}";',
|
||||
f' if [ "$1" = up ]; then return {compose_status}; fi;',
|
||||
' return 0;',
|
||||
'}',
|
||||
handler,
|
||||
f'exit {status}',
|
||||
])
|
||||
# Exclude inherited deployment settings from the fixture.
|
||||
environment = {key: value for key, value in os.environ.items()
|
||||
if key.upper() in {'PATH', 'SYSTEMROOT', 'WINDIR', 'TEMP', 'TMP'}}
|
||||
result = subprocess.run([BASH, '--noprofile', '--norc'], input=script,
|
||||
text=True, encoding='utf-8', capture_output=True,
|
||||
cwd=ROOT, env=environment, timeout=10)
|
||||
self.assertEqual(status, result.returncode, result.stdout + result.stderr)
|
||||
self.assertEqual('', result.stderr)
|
||||
calls = [line for line in result.stdout.splitlines() if line.startswith('CALL ')]
|
||||
self.assertEqual(['CALL rm -f synthetic-body', 'CALL rm -f synthetic-headers'], calls[:2])
|
||||
return result.stdout, calls[2:]
|
||||
|
||||
def test_failed_replacement_recreates_only_prior_core_images(self):
|
||||
stdout, calls = self.exit_deploy(41)
|
||||
self.assertEqual([
|
||||
'CALL compose up -d --force-recreate --remove-orphans backend frontend '
|
||||
'[synthetic/backend:previous] [synthetic/frontend:previous]',
|
||||
], calls)
|
||||
self.assertIn('Deployment remains failed', stdout)
|
||||
|
||||
def test_rollback_failure_retains_original_status_and_collects_diagnostics(self):
|
||||
stdout, calls = self.exit_deploy(41, compose_status=19)
|
||||
self.assertEqual([
|
||||
'CALL compose up -d --force-recreate --remove-orphans backend frontend '
|
||||
'[synthetic/backend:previous] [synthetic/frontend:previous]',
|
||||
'CALL compose ps [] []',
|
||||
'CALL compose logs --tail=200 backend frontend [] []',
|
||||
], calls)
|
||||
self.assertIn('Manual recovery is required', stdout)
|
||||
|
||||
def test_incomplete_previous_release_does_not_recreate_any_service(self):
|
||||
for backend, frontend in [(False, True), (True, False), (False, False)]:
|
||||
with self.subTest(backend=backend, frontend=frontend):
|
||||
stdout, calls = self.exit_deploy(41, backend=backend, frontend=frontend)
|
||||
self.assertEqual([], calls)
|
||||
self.assertIn('no complete previous core release', stdout)
|
||||
|
||||
def test_failure_before_replacement_does_not_recreate_any_service(self):
|
||||
_, calls = self.exit_deploy(41, replaced=False)
|
||||
self.assertEqual([], calls)
|
||||
|
||||
def test_success_does_not_recreate_any_service(self):
|
||||
_, calls = self.exit_deploy(0)
|
||||
self.assertEqual([], calls)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
Reference in New Issue
Block a user