feat(dev): enable isolated local workers and verified Ollama

This commit is contained in:
cesnimda
2026-10-27 11:55:31 +01:00
parent 663023a088
commit 73ea888375
9 changed files with 356 additions and 1 deletions
+37
View File
@@ -0,0 +1,37 @@
# Local workers and models
This overlay targets the separately configured `jobjakt-dev` stack in the primary checkout. It does not replace that checkout's standalone compose file. Production services, databases and model caches are excluded.
The local runtime uses Ollama 0.40.11 and `qwen2.5:7b` (Q4_K_M, Apache-2.0), matching the model name in the repository's production configuration. Exact deployed digest parity has not been established. The frozen official registry manifest is in `model-manifests/qwen2.5-7b.json`; its SHA256 is `845dbda0ea48ed749caafd9e6037047aa19acfcfd82e704d7ca97d631a0b697e`. The image digest and resource limits are pinned in `ollama.compose.yml`. CPU inference is limited to four CPUs and 8GB; cloud processing is disabled. Model files use a dedicated development volume; the API binds only `127.0.0.1:11434`.
Initial local setup, explicitly authorised by the owner on 27 October 2026:
```powershell
rtk proxy docker compose -f dev/ollama.compose.yml up -d --wait
rtk proxy docker compose -f dev/ollama.compose.yml exec ollama ollama pull qwen2.5:7b
rtk proxy python dev/verify_ollama.py --manifest dev/model-manifests/qwen2.5-7b.json --output .artifacts/pre-phase10-ux/local-ai/verification.json
```
Verification checks the actual image, dedicated cache and every model/config/template/license blob against the frozen manifest before inference. A future changed upstream tag must fail verification; review the artifact before updating the pin. The download network remains available for explicit model maintenance; no cloud provider is enabled.
The existing approved DistilBART CPU checkpoint is reused through a read-only mount after all files are checked against `docs/redesign/phase-9/p9-sec-02/full-model/ARTIFACTS.json`. Transformers is offline, so this operation cannot fetch replacement weights. The parser-capable sidecar is also checked against its previously qualified local image ID.
Preflight and apply (run from this worktree; replace paths only with the verified development checkout and approved artifact):
```powershell
rtk proxy python dev/enable_local_workers.py --checkout "E:/Storage/Projects/Active/Job tracker" --bart-artifact "E:/Storage/Projects/Active/Job tracker/.artifacts/p9-sec-02/full-model/artifact" --output .artifacts/pre-phase10-ux/local-ai/worker-preflight.json
rtk proxy python dev/enable_local_workers.py --checkout "E:/Storage/Projects/Active/Job tracker" --bart-artifact "E:/Storage/Projects/Active/Job tracker/.artifacts/p9-sec-02/full-model/artifact" --output .artifacts/pre-phase10-ux/local-ai/workers-applied.json --apply
rtk proxy python dev/check_local_ai.py --output .artifacts/pre-phase10-ux/local-ai/sidecar-smoke.json
```
The apply helper validates project, Development environment, loopback backend ports, development database, current backend image, offline model mount and Mailpit settings. It recreates only the development backend/sidecar without a build or pull, preserving the database and volumes. It enables rules, reminders, saved-search alerts, daily exports, job enrichment and queued AI processing. These features still respect account entitlements, AI consent, configured schedules and available input. Email goes only to Mailpit at `http://localhost:8026`; it is not delivered externally.
This configuration operation does not publish source changes. In particular, the running older backend image cannot use the new `Ai:RequestTimeoutSeconds=240` setting until built from the validated feature source. Its old 30-second limit can interrupt CPU generation. Source builds and application checks have separate evidence.
OAuth/mail synchronisation, billing and external data providers need their own approved credentials and configuration. Enabling workers cannot provide them. CV document import uses deterministic Docling; generative CV-import assistance stays disabled pending its separate source-grounding/accuracy qualification.
## Evidence, 27 October 2026
The local setup and worker apply passed. Independent model-blob verification passed. A real constrained JSON CPU response passed in 20.546 seconds including model loading. Authenticated sidecar checks passed: trained local job summarisation in 2.518 seconds and an Ollama-backed CV rewrite in 40.997 seconds. These checks assert actual nonempty content, not semantic correctness. No production digest parity or CV extraction accuracy is claimed.
Official sources: [Ollama model](https://ollama.com/library/qwen2.5:7b), [frozen registry source](https://registry.ollama.ai/v2/library/qwen2.5/manifests/7b), [Ollama Docker guidance](https://docs.ollama.com/docker), [local/cloud configuration](https://docs.ollama.com/faq).
+55
View File
@@ -0,0 +1,55 @@
"""Exercise the isolated development sidecar with fictional data only."""
import argparse
import json
from pathlib import Path
import subprocess
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--output", type=Path, required=True)
args = parser.parse_args()
# Authentication stays inside the container; neither secrets nor model text are logged.
program = r'''
import json
import os
import time
import urllib.request
records = []
def request(path, payload, field):
started = time.monotonic()
req = urllib.request.Request("http://127.0.0.1:8001" + path,
data=json.dumps(payload).encode(),
headers={"Content-Type": "application/json", "X-AI-Service-Token": os.environ["AI_SERVICE_TOKEN"]})
with urllib.request.urlopen(req, timeout=240) as response:
result = json.load(response)
value = result.get(field)
assert isinstance(value, str) and value.strip(), "Missing generated content: " + field
records.append({"path": path, "status": "PASS", "seconds": round(time.monotonic() - started, 3), "characters": len(value)})
assert os.environ["OLLAMA_BASE_URL"] == "http://jobjakt-dev-ollama:11434"
assert os.environ["AI_ROUTING_MODE"] == "local_only"
assert os.environ["EXTERNAL_AI_ENABLED"].lower() == "false"
request("/summarize", {"text": "Fictional Systems seeks a software engineer. Responsibilities include developing C# APIs, writing SQL queries and maintaining Docker services. Candidates should communicate clearly and work with a small team. This fictional advertisement is solely a development check.", "max_length": 60, "min_length": 10}, "summary")
request("/cv/rewrite", {"instruction": "Correct grammar only. Return exactly one sentence. Preserve the facts and add no headings.", "text": "A fictional engineer build reliable software.", "max_length": 48, "min_length": 8}, "rewritten_text")
print(json.dumps({"status": "PASS", "model": os.environ["OLLAMA_MODEL"], "routing": "local_only", "checks": records,
"limits": "Synthetic sidecar connectivity and nonempty response only; no factual extraction accuracy or production parity claim."}))
'''
identity = json.loads(subprocess.check_output(["rtk", "proxy", "docker", "inspect", "jobjakt-dev-ai-service-1"], text=True))[0]
assert identity["Config"]["Labels"]["com.docker.compose.project"] == "jobjakt-dev"
assert identity["Image"] == "sha256:8a29cb90fcc2817b0d784826d55a172c943f5ad7e1b6105e690a2d79ee15fe87"
result = subprocess.run(["rtk", "proxy", "docker", "exec", "-i", "jobjakt-dev-ai-service-1", "python", "-"],
input=program, text=True, capture_output=True, timeout=500)
if result.returncode:
# Avoid forwarding exception bodies which might contain request data.
raise SystemExit("Local synthetic sidecar check failed; exit " + str(result.returncode))
record = json.loads(result.stdout)
assert record["status"] == "PASS" and len(record["checks"]) == 2
args.output.parent.mkdir(parents=True, exist_ok=True)
args.output.write_text(json.dumps(record, indent=2) + "\n", encoding="utf-8")
print(json.dumps(record))
if __name__ == "__main__":
main()
+74
View File
@@ -0,0 +1,74 @@
"""Guard and apply worker settings to the existing isolated local development stack."""
import argparse
import hashlib
import json
import os
from pathlib import Path
import subprocess
ROOT = Path(__file__).resolve().parent.parent
SIDECAR = "sha256:8a29cb90fcc2817b0d784826d55a172c943f5ad7e1b6105e690a2d79ee15fe87"
def capture(arguments, environment):
return subprocess.check_output(["rtk", "proxy", *arguments], env=environment, text=True)
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--checkout", type=Path, required=True)
parser.add_argument("--bart-artifact", type=Path, required=True)
parser.add_argument("--output", type=Path, required=True)
parser.add_argument("--apply", action="store_true")
args = parser.parse_args()
checkout = args.checkout.resolve()
artifact = args.bart_artifact.resolve()
environment = {**os.environ, "JOBTRACKER_DEV_BART_ARTIFACT": str(artifact)}
manifest = json.loads((ROOT / "docs/redesign/phase-9/p9-sec-02/full-model/ARTIFACTS.json").read_text())["weights-only"]
for row in manifest["files"]:
path = artifact / row["file"]
assert path.is_file() and path.stat().st_size == row["bytes"], "Local model file missing or wrong size"
with path.open("rb") as stream:
assert hashlib.file_digest(stream, "sha256").hexdigest() == row["sha256"], "Local model integrity mismatch"
assert json.loads((ROOT / ".artifacts/pre-phase10-ux/local-ai/verification.json").read_text())["status"] == "PASS"
compose = ["docker", "compose", "--project-directory", str(checkout), "--env-file", str(checkout / ".env.dev"),
"-f", str(checkout / "docker-compose.dev.yml"), "-f", str(ROOT / "dev/local-workers.override.yml")]
config = json.loads(capture([*compose, "config", "--format", "json"], environment))
assert config["name"] == "jobjakt-dev", "Only the local development project is permitted"
backend = config["services"]["backend"]
sidecar = config["services"]["ai-service"]
settings = backend["environment"]
assert settings["ASPNETCORE_ENVIRONMENT"] == "Development"
connection = settings["ConnectionStrings__JobTracker"].lower()
assert "server=mariadb;" in connection and "database=jobtracker_dev;" in connection
assert settings["Ai__ExternalProcessingEnabled"].lower() == "false"
assert settings["Email__SmtpHost"] == "mailpit" and settings["Email__SmtpPort"] == "1025"
assert not settings["Email__SmtpUser"] and not settings["Email__SmtpPassword"]
assert all(port.get("host_ip") == "127.0.0.1" for port in backend.get("ports", []))
assert sidecar["environment"]["HF_HUB_OFFLINE"] == "1"
assert sidecar["environment"]["TRANSFORMERS_OFFLINE"] == "1"
assert sidecar["environment"]["EXTERNAL_AI_ENABLED"].lower() == "false"
assert sidecar["environment"]["OLLAMA_BASE_URL"] == "http://jobjakt-dev-ollama:11434"
assert any(volume.get("source") == str(artifact) and volume.get("read_only") for volume in sidecar["volumes"])
assert json.loads(capture(["docker", "image", "inspect", sidecar["image"]], environment))[0]["Id"] == SIDECAR
current = json.loads(capture(["docker", "inspect", "jobjakt-dev-backend-1"], environment))[0]
assert current["Config"]["Labels"]["com.docker.compose.project"] == "jobjakt-dev"
assert json.loads(capture(["docker", "image", "inspect", backend["image"]], environment))[0]["Id"] == current["Image"], "This operation changes worker configuration, not backend code"
record = {"project": config["name"], "verifiedBartRevision": manifest["sourceRevision"], "sidecarImage": SIDECAR,
"enabledWorkers": {key: value for key, value in settings.items() if key.startswith("Workers__")},
"emailSink": "Mailpit", "externalAI": False, "applied": False}
args.output.parent.mkdir(parents=True, exist_ok=True)
if args.apply:
with args.output.with_suffix(".log").open("w", encoding="utf-8") as log:
result = subprocess.run(["rtk", "proxy", *compose, "up", "-d", "--no-deps", "--no-build", "--pull", "never",
"--wait", "--wait-timeout", "120", "backend", "ai-service"], env=environment, stdout=log, stderr=subprocess.STDOUT)
record["exitCode"] = result.returncode
record["applied"] = result.returncode == 0
args.output.write_text(json.dumps(record, indent=2) + "\n", encoding="utf-8")
print(json.dumps(record))
if args.apply:
raise SystemExit(record["exitCode"])
if __name__ == "__main__":
main()
+47
View File
@@ -0,0 +1,47 @@
# Apply only to the separately verified standalone jobjakt-dev stack.
services:
backend:
environment:
Workers__RulesEnabled: "true"
Workers__FollowUpRemindersEnabled: "true"
Workers__SavedJobSearchAlertsEnabled: "true"
Workers__DailyExportEnabled: "true"
Workers__JobEnrichmentEnabled: "true"
Workers__AiOperationsEnabled: "true"
Workers__CvDocumentImportsEnabled: "true"
Exports__DailyEnabled: "true"
Email__FollowUpReminders__Enabled: "true"
Email__Enabled: "true"
Email__SmtpHost: mailpit
Email__SmtpPort: "1025"
Email__SmtpEnableSsl: "false"
Email__SmtpUser: ""
Email__SmtpPassword: ""
Ai__ExternalProcessingEnabled: "false"
Ai__RequestTimeoutSeconds: "240"
ai-service:
# Previously qualified local image contains both the legacy and isolated Docling runtimes.
image: jobtracker-cv-import-sidecar:dev-04
environment:
OLLAMA_BASE_URL: http://jobjakt-dev-ollama:11434
OLLAMA_MODEL: qwen2.5:7b
AI_PROVIDER: ollama
AI_ROUTING_MODE: local_only
EXTERNAL_AI_ENABLED: "false"
CV_IMPORT_AI_ENABLED: "false"
AI_SERVICE_SKIP_MODEL_LOAD: "0"
SKIP_MODEL_LOAD: "0"
HF_HOME: /models
HF_HUB_OFFLINE: "1"
TRANSFORMERS_OFFLINE: "1"
HF_HUB_DISABLE_TELEMETRY: "1"
OMP_NUM_THREADS: "2"
MKL_NUM_THREADS: "2"
volumes:
- type: bind
source: ${JOBTRACKER_DEV_BART_ARTIFACT:?Verify the isolated trained checkpoint first}
target: /models/hub/models--sshleifer--distilbart-cnn-12-6/snapshots/a4f8f3ea906ed274767e9906dbaede7531d660ff
read_only: true
mem_limit: 4g
cpus: 2
pids_limit: 256
+1
View File
@@ -0,0 +1 @@
{"schemaVersion":2,"mediaType":"application/vnd.docker.distribution.manifest.v2+json","config":{"mediaType":"application/vnd.docker.container.image.v1+json","digest":"sha256:2f15b3218f0552c60647ce60ada83632d2c09755b16259b13e3e4458e9ae419d","size":487},"layers":[{"mediaType":"application/vnd.ollama.image.model","digest":"sha256:2bada8a7450677000f678be90653b85d364de7db25eb5ea54136ada5f3933730","size":4683073952},{"mediaType":"application/vnd.ollama.image.system","digest":"sha256:66b9ea09bd5b7099cbb4fc820f31b575c0366fa439b08245566692c6784e281e","size":68},{"mediaType":"application/vnd.ollama.image.template","digest":"sha256:eb4402837c7829a690fa845de4d7f3fd842c2adee476d5341da8a46ea9255175","size":1482},{"mediaType":"application/vnd.ollama.image.license","digest":"sha256:832dd9e00a68dd83b3c3fb9f5588dad7dcf337a0db50f7d9483f310cd292e92e","size":11343}]}
+47
View File
@@ -0,0 +1,47 @@
# Local development only. Uses the existing isolated jobjakt-dev AI network.
name: jobjakt-dev-ollama
services:
ollama:
image: ollama/ollama:0.40.11@sha256:459985308d740b99071719baec2740a60f5b2b2d9b835c171991ae25690203da
environment:
OLLAMA_HOST: 0.0.0.0:11434
OLLAMA_NO_CLOUD: "1"
OLLAMA_NUM_PARALLEL: "1"
OLLAMA_MAX_LOADED_MODELS: "1"
OLLAMA_CONTEXT_LENGTH: "4096"
OLLAMA_KEEP_ALIVE: 5m
ports:
- "127.0.0.1:11434:11434"
volumes:
- models:/root/.ollama
networks:
ai_internal:
aliases: [jobjakt-dev-ollama]
model_download: {}
mem_limit: 8g
cpus: 4
pids_limit: 256
read_only: true
tmpfs:
- /tmp:size=256m,mode=1777
cap_drop: [ALL]
security_opt: [no-new-privileges:true]
restart: unless-stopped
healthcheck:
test: [CMD, ollama, list]
interval: 15s
timeout: 5s
retries: 5
start_period: 15s
logging:
options:
max-size: 10m
max-file: "3"
networks:
ai_internal:
external: true
name: jobjakt-dev_ai_internal
model_download: {}
volumes:
models:
name: jobjakt-dev-ollama-models
+56
View File
@@ -0,0 +1,56 @@
"""Verify the dedicated development model cache before local inference."""
import argparse
import hashlib
import json
import re
import subprocess
import urllib.request
from pathlib import Path
MODEL = "qwen2.5:7b"
MANIFEST = "845dbda0ea48ed749caafd9e6037047aa19acfcfd82e704d7ca97d631a0b697e"
IMAGE = "ollama/ollama:0.40.11@sha256:459985308d740b99071719baec2740a60f5b2b2d9b835c171991ae25690203da"
CONTAINER = "jobjakt-dev-ollama-ollama-1"
def run(*arguments):
return subprocess.check_output(["rtk", "proxy", *arguments], text=True).strip()
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--manifest", required=True, type=Path)
parser.add_argument("--output", required=True, type=Path)
args = parser.parse_args()
source = args.manifest.read_bytes()
assert hashlib.sha256(source).hexdigest() == MANIFEST, "Registry manifest changed"
manifest = json.loads(source)
container = json.loads(run("docker", "inspect", CONTAINER))[0]
assert container["Config"]["Labels"]["com.docker.compose.project"] == "jobjakt-dev-ollama"
assert container["Image"] == json.loads(run("docker", "image", "inspect", IMAGE))[0]["Id"]
assert "OLLAMA_NO_CLOUD=1" in container["Config"]["Env"]
assert container["HostConfig"]["ReadonlyRootfs"]
assert container["HostConfig"]["Memory"] == 8 * 1024**3
assert all(mount.get("Name") == "jobjakt-dev-ollama-models" for mount in container["Mounts"] if mount["Type"] == "volume")
descriptors = [manifest["config"], *manifest["layers"], {"digest": "sha256:" + MANIFEST}]
checked = []
for descriptor in descriptors:
digest = descriptor["digest"]
assert re.fullmatch(r"sha256:[a-f0-9]{64}", digest)
path = "/root/.ollama/models/blobs/" + digest.replace(":", "-")
actual = run("docker", "exec", CONTAINER, "sha256sum", path).split()[0]
assert actual == digest[7:], "Downloaded model blob changed"
checked.append(digest)
with urllib.request.urlopen("http://127.0.0.1:11434/api/tags", timeout=5) as response:
models = json.load(response)["models"]
actual = next(model for model in models if model["name"] == MODEL)
assert actual["digest"] == MANIFEST
args.output.parent.mkdir(parents=True, exist_ok=True)
record = {"status": "PASS", "model": MODEL, "manifestSha256": MANIFEST, "checkedBlobs": checked,
"image": IMAGE, "details": actual["details"], "limits": "Local artifact verification only; no inference, production digest parity or extraction accuracy claim."}
args.output.write_text(json.dumps(record, indent=2) + "\n", encoding="utf-8")
print(json.dumps(record))
if __name__ == "__main__":
main()
+38
View File
@@ -0,0 +1,38 @@
# Owner additions, 27 October 2026
The owner extended the active workstream with these requirements; they supplement the original PRE-P10-UX plan.
- More space below the main title/header in Classic (`ats-classic`), Modern and Code.
- Full-height sidebars on every rendered page.
- Actual server-rendered CV template previews in the selection gallery.
- Matching cover-letter styles for the CV catalogue.
- Repair uploads accepted into the queue without extraction.
- Make existing local-development options/workers operational, including AI operations and job enrichment.
- Create a separate local Ollama instance matching the production model configuration; local downloads are now explicitly authorised. This does not authorise production downloads, cache changes, model replacement or production qualification claims.
## Initial findings (before local remediation)
The running local stack is `jobjakt-dev`, from the primary checkout's independently modified standalone development compose file. Backend and frontend bind loopback8081/80, with an isolated MariaDB and Mailpit. Its six automation flags are all false. This file is unrelated existing local work and must not be replaced by the older compose override tracked in this worktree.
The local sidecar points at `http://host.docker.internal:11434` on an internal Docker network. A read-only tags request failed with network unreachable; no native local Ollama executable, process or model manifest cache was found. Thus enabling a boolean alone cannot establish working inference.
Repository production `.env` names `qwen2.5:7b` and a Gemini provider; these are configuration observations, not verification of the deployed server. Local inference will remain self-hosted. No external keys or production personal data are copied. Exact deployed model-digest parity remains unverified until read-only current metadata establishes it.
The official `qwen2.5:7b` manifest was fetched from the Ollama registry. Manifest SHA256: `845dbda0ea48ed749caafd9e6037047aa19acfcfd82e704d7ca97d631a0b697e`; model layer SHA256: `2bada8a7450677000f678be90653b85d364de7db25eb5ea54136ada5f3933730`; layers total4,683,086,845bytes. Downloaded bytes must be independently verified before inference. This is local configuration parity with the recorded model name, not a permanent model selection or an AI-accuracy acceptance.
Local runtime target: official Ollama0.40.11 linux/amd64 image digest `sha256:459985308d740b99071719baec2740a60f5b2b2d9b835c171991ae25690203da`. Docker has approximately16GB RAM and16CPUs; host approximately32GB. Use bounded CPU execution/concurrency and independent dev storage. Do not alter production services or host drivers.
## Checkpoint additions
- [ ] B-owner: qualify title spacing/full-page sidebar backgrounds and real template thumbnails; match letter designs through versioned settings and existing revision owners.
- [ ] Import-recovery: document-only worker can consume prior document imports while model jobs remain disabled; disabled workers return an actionable503; real Docling journey verifies completion.
- [x] Local-options configuration: guarded standalone dev stack identity/loopback/network/storage boundaries; authorised workers enabled; email forced to Mailpit. Account entitlements, schedules and missing provider credentials remain applicable. This is configuration activation, not full feature acceptance.
- [x] Local-Ollama: independent pinned container/cache; all official manifest blobs verified; actual constrained JSON CPU inference and authenticated sidecar summary/rewrite checks passed. Deployed digest parity and real CV extraction accuracy remain unqualified.
Original dashboard/admin/final integration checkpoints remain required. No Phase10 cleanup, main merge or production deployment is authorised.
## Local runtime result
`dev/README.md` records reproducible setup, guardrails, dependency pins and measured synthetic checks. Raw records are `.artifacts/pre-phase10-ux/local-ai/{verification,inference-smoke,worker-preflight,workers-applied,sidecar-smoke}.json`. Actual summarisation2.518s and CV rewrite40.997s returned nonempty content. This required the qualified parser-capable sidecar and the previously approved trained DistilBART artifact mounted read-only, with Transformers offline. No production service or cache was accessed.
The running backend/frontend retain their previous image identities at this configuration checkpoint. A validated source build is still needed for the new configurable backend AI timeout and UI changes. CV-import generative interpretation remains off; real inference here does not qualify source-grounded factual extraction.
+1 -1
View File
@@ -12,7 +12,7 @@
## Global constraints
No main merge, production service/database/storage/model/cache/deployment operation, destructive migration, live inference, FINN/NAV/LinkedIn initiative or Phase10 cleanup. All feature branches cumulatively contain4a8c907. Use safe synthetic fixtures, isolated resources and exact independent counts. Commit/push/verify every validated checkpoint. No false human/device/model acceptance.
No main merge, production service/database/storage/model/cache/deployment operation, destructive migration, FINN/NAV/LinkedIn initiative or Phase10 cleanup. The owner's 27 October local-model authorisation supersedes the original blanket live-inference restriction for a verified isolated local runtime and synthetic checks only; see OWNER_ADDENDUM.md. All feature branches cumulatively contain4a8c907. Use safe synthetic fixtures, isolated resources and exact independent counts. Commit/push/verify every validated checkpoint. No false human/device/model acceptance.
## Review focus