45725acc7c
The backend test step fails on the self-hosted runner after 8s while passing on Windows, in a clean Linux container, and in CI's exact build-then-test order. The job log is not readable via the Gitea API (401), so step boundaries are the only available telemetry: splitting restore / build / test makes the failing phase identifiable from step timings alone. Restore retries once, mirroring the npm ci and dotnet SDK retries already in this workflow for the same runner's known flakiness. The suite itself is unchanged — still the whole suite, nothing filtered or skipped. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
154 lines
6.6 KiB
YAML
154 lines
6.6 KiB
YAML
name: CI and Deploy
|
|
|
|
on:
|
|
push:
|
|
branches:
|
|
- main
|
|
pull_request:
|
|
|
|
jobs:
|
|
test:
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- name: Checkout
|
|
uses: actions/checkout@v4
|
|
|
|
- name: Setup .NET (resilient)
|
|
shell: bash
|
|
# actions/setup-dotnet on this single self-hosted runner intermittently
|
|
# leaves a partial extraction in the shared tool-cache ("tar: Cannot open:
|
|
# File exists") or corrupts the SDK download. Install into a clean private
|
|
# dir via dotnet-install.sh and retry once on failure, mirroring the
|
|
# npm ci / NuGet retries elsewhere in this workflow.
|
|
run: |
|
|
install() {
|
|
curl -fsSL https://dot.net/v1/dotnet-install.sh -o /tmp/dotnet-install.sh
|
|
rm -rf "$HOME/.dotnet"
|
|
bash /tmp/dotnet-install.sh --channel 9.0 --install-dir "$HOME/.dotnet"
|
|
}
|
|
install || ( echo "dotnet install failed ($?) — retrying once..." && install )
|
|
echo "$HOME/.dotnet" >> "$GITHUB_PATH"
|
|
"$HOME/.dotnet/dotnet" --info
|
|
|
|
- name: Setup Node
|
|
uses: actions/setup-node@v4
|
|
with:
|
|
node-version: '20'
|
|
cache: 'npm'
|
|
cache-dependency-path: job-tracker-ui/package-lock.json
|
|
|
|
- name: Build backend
|
|
run: dotnet build JobTrackerApi/JobTrackerApi.csproj --configuration Release
|
|
|
|
# The "Build backend" step above only builds JobTrackerApi, so the test project needs its own
|
|
# restore + build. These are separate steps on purpose: this runner is flaky (see the SDK and
|
|
# npm retries in this file) and the job log is not readable from the API, so distinct steps make
|
|
# the failing phase identifiable from step boundaries alone. Restore retries once, mirroring the
|
|
# npm ci pattern below. The suite itself is never weakened or filtered.
|
|
- name: Restore backend tests
|
|
run: |
|
|
restore() { dotnet restore JobTrackerApi.Tests/JobTrackerApi.Tests.csproj; }
|
|
restore || ( echo "NuGet restore failed ($?) — retrying once..." && restore )
|
|
|
|
- name: Build backend tests
|
|
run: dotnet build JobTrackerApi.Tests/JobTrackerApi.Tests.csproj --configuration Release --no-restore
|
|
|
|
- name: Test backend
|
|
run: dotnet test JobTrackerApi.Tests/JobTrackerApi.Tests.csproj --configuration Release --no-build
|
|
|
|
- name: Install frontend deps
|
|
working-directory: job-tracker-ui
|
|
env:
|
|
npm_config_audit: 'false'
|
|
npm_config_fund: 'false'
|
|
run: |
|
|
node -v
|
|
npm -v
|
|
# npm ci occasionally segfaults on the runner (SIGSEGV/139, a memory/native
|
|
# flake). Retry once with a clean node_modules before failing the job.
|
|
npm ci --no-audit --no-fund \
|
|
|| ( echo "npm ci failed ($?) — cleaning node_modules and retrying once..." \
|
|
&& rm -rf node_modules \
|
|
&& npm ci --no-audit --no-fund )
|
|
|
|
- name: Test frontend
|
|
working-directory: job-tracker-ui
|
|
# Run the WHOLE suite. Never whitelist test files here again: the previous
|
|
# whitelist silently skipped new suites and let two regressions reach main.
|
|
run: npm test -- --watchAll=false --runInBand
|
|
|
|
- name: Build frontend
|
|
working-directory: job-tracker-ui
|
|
env:
|
|
CI: 'false'
|
|
GENERATE_SOURCEMAP: 'false'
|
|
NODE_OPTIONS: --max-old-space-size=4096
|
|
# CRA's build (Terser minify + fork-ts-checker workers) has repeatedly died silently on
|
|
# this runner with no error output (OOM/SIGSEGV signature — same resource-starved-runner
|
|
# class as the npm ci and dotnet-install flakes elsewhere in this workflow). Retry once.
|
|
run: |
|
|
npm run build \
|
|
|| ( echo "Frontend build failed ($?) — retrying once..." && npm run build )
|
|
|
|
deploy:
|
|
needs: test
|
|
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- name: Run remote deploy
|
|
uses: appleboy/ssh-action@v1.0.3
|
|
with:
|
|
host: ${{ secrets.PROD_HOST }}
|
|
username: ${{ secrets.PROD_USER }}
|
|
key: ${{ secrets.PROD_SSH_KEY }}
|
|
command_timeout: 40m
|
|
script: |
|
|
set -euo pipefail
|
|
if [ ! -d /opt/job-tracker/app/.git ]; then
|
|
echo "Expected git checkout at /opt/job-tracker/app but .git was not found."
|
|
exit 1
|
|
fi
|
|
cd /opt/job-tracker/app
|
|
if ! git fetch --all --prune; then
|
|
echo "git fetch failed on server. Check remote auth/URL for /opt/job-tracker/app."
|
|
exit 1
|
|
fi
|
|
if ! git rev-parse --verify --quiet ${{ github.sha }} >/dev/null; then
|
|
echo "Commit ${{ github.sha }} is not available in the server checkout after fetch."
|
|
exit 1
|
|
fi
|
|
git reset --hard ${{ github.sha }}
|
|
git clean -fd
|
|
chmod +x deploy/deploy.sh
|
|
APP_VERSION=${{ github.run_number }} \
|
|
APP_COMMIT_SHA=${{ github.sha }} \
|
|
APP_BUILD_STAMP="$(date -u +'%Y-%m-%d %H:%M UTC')" \
|
|
./deploy/deploy.sh
|
|
docker compose ps
|
|
AI_CONTAINER_ID="$(docker compose ps -q ai-service)"
|
|
if [ -z "$AI_CONTAINER_ID" ]; then
|
|
echo "AI service container id could not be resolved after deploy. Continuing because AI is not a deploy gate for the core app."
|
|
else
|
|
ATTEMPTS=90
|
|
SLEEP_SECS=2
|
|
i=1
|
|
while [ "$i" -le "$ATTEMPTS" ]; do
|
|
HEALTH_STATUS="$(docker inspect -f '{{if .State.Health}}{{.State.Health.Status}}{{else}}none{{end}}' "$AI_CONTAINER_ID" 2>/dev/null || echo unknown)"
|
|
if [ "$HEALTH_STATUS" = "healthy" ]; then
|
|
break
|
|
fi
|
|
if [ "$HEALTH_STATUS" = "unhealthy" ]; then
|
|
echo "AI service became unhealthy during deploy readiness wait. Continuing because AI is not a deploy gate for the core app."
|
|
docker compose logs --tail=200 ai-service || true
|
|
break
|
|
fi
|
|
sleep "$SLEEP_SECS"
|
|
i=$((i + 1))
|
|
done
|
|
if [ "${HEALTH_STATUS:-unknown}" != "healthy" ]; then
|
|
echo "AI service did not become healthy within $((ATTEMPTS * SLEEP_SECS)) seconds. Final status: ${HEALTH_STATUS:-unknown}. Continuing because AI is not a deploy gate for the core app."
|
|
docker compose ps
|
|
docker compose logs --tail=200 ai-service || true
|
|
fi
|
|
fi
|