cesnimda
60 lines
8.5 KiB
Markdown
60 lines
8.5 KiB
Markdown
# Local workers and models
|
|
|
|
This overlay targets the separately configured `jobjakt-dev` stack in the primary checkout. It does not replace that checkout's standalone compose file. Production services, databases and model caches are excluded.
|
|
|
|
The local runtime uses Ollama 0.40.11 and `qwen2.5:7b` (Q4_K_M, Apache-2.0), matching the model name in the repository's production configuration. Exact deployed digest parity has not been established. The frozen official registry manifest is in `model-manifests/qwen2.5-7b.json`; its SHA256 is `845dbda0ea48ed749caafd9e6037047aa19acfcfd82e704d7ca97d631a0b697e`. The image digest and resource limits are pinned in `ollama.compose.yml`. CPU inference is limited to four CPUs and 8GB; cloud processing is disabled. Model files use a dedicated development volume; the API binds only `127.0.0.1:11434`.
|
|
|
|
Initial local setup, explicitly authorised by the owner on 27 October 2026:
|
|
|
|
```powershell
|
|
rtk proxy docker compose -f dev/ollama.compose.yml up -d --wait
|
|
rtk proxy docker compose -f dev/ollama.compose.yml exec ollama ollama pull qwen2.5:7b
|
|
rtk proxy python dev/verify_ollama.py --manifest dev/model-manifests/qwen2.5-7b.json --output .artifacts/pre-phase10-ux/local-ai/verification.json
|
|
```
|
|
|
|
Verification checks the actual image, dedicated cache and every model/config/template/license blob against the frozen manifest before inference. A future changed upstream tag must fail verification; review the artifact before updating the pin. The download network remains available for explicit model maintenance; no cloud provider is enabled.
|
|
|
|
The existing approved DistilBART CPU checkpoint is reused through a read-only mount after all files are checked against `docs/redesign/phase-9/p9-sec-02/full-model/ARTIFACTS.json`. Transformers is offline, so this operation cannot fetch replacement weights. The parser-capable sidecar is also checked against its previously qualified local image ID.
|
|
|
|
Preflight and apply (run from this worktree; replace paths only with the verified development checkout and approved artifact):
|
|
|
|
```powershell
|
|
rtk proxy python dev/enable_local_workers.py --checkout "E:/Storage/Projects/Active/Job tracker" --bart-artifact "E:/Storage/Projects/Active/Job tracker/.artifacts/p9-sec-02/full-model/artifact" --output .artifacts/pre-phase10-ux/local-ai/worker-preflight.json
|
|
rtk proxy python dev/enable_local_workers.py --checkout "E:/Storage/Projects/Active/Job tracker" --bart-artifact "E:/Storage/Projects/Active/Job tracker/.artifacts/p9-sec-02/full-model/artifact" --output .artifacts/pre-phase10-ux/local-ai/workers-applied.json --apply
|
|
rtk proxy python dev/check_local_ai.py --output .artifacts/pre-phase10-ux/local-ai/sidecar-smoke.json
|
|
```
|
|
|
|
The apply helper validates project, Development environment, loopback backend ports, development database, current backend image, offline model mount and Mailpit settings. It recreates only the development backend/sidecar without a build or pull, preserving the database and volumes. It enables rules, reminders, saved-search alerts, daily exports, job enrichment and queued AI processing. These features still respect account entitlements, AI consent, configured schedules and available input. Email goes only to Mailpit at `http://localhost:8026`; it is not delivered externally.
|
|
|
|
This configuration helper does not build or publish source changes. Its initial backend image used a30-second limit, which could interrupt CPU generation. The separately qualified source activation below supplies the new240-second development timeout.
|
|
|
|
OAuth/mail synchronisation, billing and external data providers need their own approved credentials and configuration. Enabling workers cannot provide them. CV document import uses deterministic Docling; generative CV-import assistance stays disabled pending its separate source-grounding/accuracy qualification.
|
|
|
|
## Evidence, 27 October 2026
|
|
|
|
The local setup and worker apply passed. Independent model-blob verification passed. A real constrained JSON CPU response passed in 20.546 seconds including model loading. Authenticated sidecar checks passed: trained local job summarisation in 2.518 seconds and an Ollama-backed CV rewrite in 40.997 seconds. These checks assert actual nonempty content, not semantic correctness. No production digest parity or CV extraction accuracy is claimed.
|
|
|
|
Official sources: [Ollama model](https://ollama.com/library/qwen2.5:7b), [frozen registry source](https://registry.ollama.ai/v2/library/qwen2.5/manifests/7b), [Ollama Docker guidance](https://docs.ollama.com/docker), [local/cloud configuration](https://docs.ollama.com/faq).
|
|
|
|
## Local engine recovery, 28 October 2026
|
|
|
|
The resumed session found Docker Desktop4.86.0 stopped. Startup failed with Windows error1920 on orphaned AF_UNIX entries in `Docker/run/dockerInference` and `docker-secrets-engine/engine.sock`. Exact socket rename/delete attempts failed. Directory inspection confirmed only zero-byte reparse-point runtime entries; the Docker WSL distribution was stopped.
|
|
|
|
After stopping only the failed local Desktop/backend processes, the runtime directories were moved aside and recreated. Backups remain in `C:/Users/Connor/AppData/Local/Docker/run.pre-p10-stale-20261028`, `run.pre-p10-stale-20261028-02` and `C:/Users/Connor/AppData/Local/docker-secrets-engine.pre-p10-stale-20261028`. They contain only the inspected stale IPC entries. The Linux engine then responded successfully as `docker-desktop`. Model files, database volumes and Docker settings were not reset. This was local development recovery; production was not accessed.
|
|
|
|
The same failure pattern is reported in [Docker's Windows issue tracker](https://github.com/docker/for-win/issues/15064). The recovery above is supported by current local inspection and a successful engine response; it is not a general instruction to move Docker data directories or reset an installation.
|
|
|
|
## Qualified local source activation, 28 October 2026
|
|
|
|
29 October recovery addendum: the same stale IPC startup failure recurred after Desktop had stopped. Exact inspection again found only zero-byte runtime sockets. Failed local Desktop processes were stopped and the bounded socket directories were moved to `Docker/run.stale-20261029-0622`, `Docker/run.stale-20261029-0624` and `docker-secrets-engine.stale-20261029-0624` under the same local AppData parent. No Docker data/settings/volume/model directories were moved. The engine and existing local development services recovered; a fresh isolated MariaDB lane then passed 90 tests with cleanup exit zero. This does not change the source activation identity below.
|
|
|
|
The existing development stack now runs source checkpoint `3f4db8ca8183704945915ae4d2053e641bafa5b2`. Local URL: **http://jobbjakt.local**. The frontend deliberately rejects an unmatched `localhost` Host header; the canonical host was preserved. Mailpit remains at `http://localhost:8026`.
|
|
|
|
Backend image: `sha256:462b1d14fe871d64b16a2db28fda30a433657dd4d8ed5f61587a6b80685fb0cf`. Frontend image: `sha256:a59b8313233a20f6a2f0a8cc23785d224b58678e5758c3985fa55f238402886f`. All qualified source-file hashes were rechecked before activation. The backend is the same immutable image used by the passing Linux qualification. The frontend was rebuilt from that source with `/api` routing and the existing local nginx configuration. The qualified static test export was archived separately before this build.
|
|
|
|
Raw activation/rollback identities, source-image compose override, build logs and actual HTTP checks are in `.artifacts/pre-phase10-ux/local-ai/source-3f4db8c/`. The one-off activation script is retained alongside that directory as execution evidence. Backend/frontend were recreated with no dependency recreation, builds, pulls or volume removal during the compose apply. Previous image IDs remain recorded. Existing database, sidecar, model and email services were retained.
|
|
|
|
Confirmed: both images healthy; served index matches the built bytes; backend health200; anonymous proxied account request401; actual backend environment has all seven authorised workers enabled and the240-second AI timeout. Ollama blobs were reverified after engine recovery. These checks establish the running source/configuration; model-quality and broader feature qualifications remain separately stated in B_ACCEPTANCE.md.
|
|
|
|
For subsequent compose operations, retain the generated `source-images.override.json` **after** `dev/local-workers.override.yml`, the primary standalone development compose file and its `.env.dev`. Supply the same read-only `JOBTRACKER_DEV_BART_ARTIFACT` path. Running only the primary compose file can revert these local image/configuration selections. Do not substitute production configuration.
|