GJ-04 — Country Provider Connectors, Rights Enforcement and Lifecycle

Suggested branch: feature/gj04-provider-connectors

Objective

Implement a scalable source architecture that can serve Norway, Sweden and UK while accurately representing differing API permissions and content completeness. Provider blockers must not block already-authorised sources or manual user entry.

Mandatory source strategy

Norway NAV: Verify current pam-stilling-feed docs/terms. This is an authenticated change feed, not a free-text search endpoint; permitted local indexing/search requires consumer access and incremental updates. Verify JWT consumer registration (the public experimentation token is not a production solution), updates, tombstones/inactive masking, source application deep links, attribution and PII requirements. FINN listings are not included in NAV's feed.

Norway FINN: Official FINN API requires authorised advertiser/business relationship and ownership of accessible data; general third-party search is NOT available by default. Implement user-pasted URL recognition, deduplication, legitimate manual record entry, and original-site link. Do not silently fetch FINN pages, circumvent protection or call this 'full FINN integration'. Only implement full details if explicit suitable rights are established.

Sweden: Verify official JobTech JobSearch and JobAd Links capabilities and licence. JobAd Links provides truncated details/reference links, not guaranteed full vacancy content. Respect original-site link requirements, daily freshness and occupational-code mappings. If other JobTech APIs permit full descriptions, expose source-specific completeness and follow those actual terms.

United Kingdom: Evaluate Adzuna documented job search; results contain a description snippet, and ongoing commercial SaaS use requires licensing/written approval. Evaluate appropriate public-sector/other licensed sources and externally linked searches; don't assume Reed/Indeed/Totaljobs/CV-Library/Find a Job/NHS APIs allow third-party bulk search. For blocked providers, create operator instructions and a permitted fallback, not an invented API.

Architecture

Provider registry with immutable ID, geo/occupation search capabilities, feed vs query mode, allowed detail level, supported filters, source link policy, commercial rights, credential status, effective dates, retention, updates/deletes, quota and telemetry. Make unavailable/blocked sources visible to admins with actionable reason but do not expose keys.
Typed interfaces: Search, ReadDetails, IngestChanges, MapLocation, MapOccupation, Normalize, GetLifecycle, BuildApplyLink, conditional on capabilities. Do not force every provider to implement methods it does not support.
Canonical SourceAdvertisement plus ProviderSourceRecord with provider ID, source ID, canonical link, employer/title, structured location, pay amount/range/currency/period, shifts, occupation, qualification, languages, contract, description format/completeness, created/modified/expiry, original apply URL, source-restriction flags and provenance. Preserve unknown values; avoid inventing salaries/hours or dropping original titles.
Safe ingestion/index queue with cursor/checkpoint, bounded pagination, backoff/rate limit, idempotency, incremental updates/deletes, stale-data policy, audit events, resumability, transactional MariaDB and provider-specific compliance. No infinite poll loops or API-key logs.
Deterministic, conservative cross-provider duplicate detection (source IDs, canonical links and supporting content/time/location) with false-positive review. Do not merge distinct teaching vacancies at same school or identical restaurant positions automatically. Display multiple sources only when identity is sufficiently proven.
Separate external ad cache from user-owned application tracking. Remove/mask inactive content per source terms; preserve user-owned notes/status legally and transparently, not secretly republish deleted ads.
Provider fixture/contracts and authorised bounded live smoke tests. Do not run live provider tests without actual access.

Checkpoints

GJ04-A rights & provider registry; GJ04-B canonical/ad lifecycle; GJ04-C NAV connector if authorised; GJ04-D Swedish connectors; GJ04-E UK approved connector or external-link-only fallback; GJ04-F manual FINN URL workflow; GJ04-G dedupe, security, operational tests. Block unavailable subcheckpoint honestly; continue independent ones. Docs: docs/global-jobs/04/, source by source eligibility, test evidence, permissions, rate limits, update/delete compliance and owner actions. Commit/push each completed checkpoint. Stop.

Shared rules for every INT-GLOBAL-JOBS-01 stage

Product: Jobjakt, a general-purpose, commercial job-search and career-management SaaS. It is not an IT-only product. Current architecture includes React/TypeScript, .NET backend, MariaDB, an AI sidecar, existing Career Profile/CV/document/application features, and an established EN/NO localisation approach. Verify actual repository reality before making assumptions.
Entry gates: Do not implement new global-job-discovery features until the authoritative Phase 10 and product-review/development-entry decisions permit this initiative. Planning-only reconnaissance may precede them. If an entry gate remains NO-GO, produce safe analysis and stop before code changes.
Baseline: Find the latest accepted, integration-complete development commit by reading Git history and the latest authoritative acceptance reports. Do not assume main is current. main deploys production: NEVER merge to main or trigger production. Never change live MariaDB, Docker, Ollama, storage, secrets or other production services.
Branches: Use the specified independent stage branch from the latest accepted development integration history including the preceding accepted stage. Verify ancestry, remote, clean tree and existing branch collisions. Use checkpoint commits; push each validated checkpoint; verify remote HEAD equals local HEAD. Do not force-push. Do not advance to another stage without its actual entry criteria.
Autonomy: Investigate, design, implement, add tests, document, commit and push safe work without pausing for routine decisions. Stop only for external API agreements, new paid resources, security-risk acceptance, destructive changes, privileged infrastructure actions and production operations. Continue independent work when one provider or model capability is blocked.
Sources: Integrate only documented, authorised sources. Verify current licensing, commercial eligibility, credentials, permitted display, storage, cache lifetime, update/delete/tombstone, attribution, user-click-through and rate-limit rules. No covert scraping, private APIs, stolen cookies, bypassing anti-bot controls or fetching protected listing content without authorisation. An external-link-only source is not a direct searchable provider.
Data trust: All provider data, job descriptions and user-provided URLs are untrusted. Guard SSRF, XSS/unsafe HTML, prompt injection, HTML injection, unsafe redirects, credential leakage, denial of service and cross-user/tenant access. Do not present missing facts, inferred salaries, regulated-profession eligibility, or AI-generated job details as verified employer facts.
Availability: Preserve offline/disabled provider states. Use realistic fixtures in automated tests; any live contract test needs authorised test credentials and bounded rates. Never claim unavailable external API functionality passed.
User-facing quality: Desktop/tablet/phone, keyboard and screen-reader access, light/dark, EN/Bokmål plus future qualified locales, loading/empty/error states, responsive design, sensible defaults, performance, source transparency, internationalisation and accessible non-drag alternatives.
Data integrity: Source advertisements and user-owned saved jobs/applications are separate records and lifecycles. Preserve existing Career Profile, historical CVs, cover letters, saved applications, revisions, RD21 protections and MariaDB compatibility. Follow source rules for removing or masking withdrawn advertisements, while handling user-owned records under a separately justified retention policy.
AI: Existing live-model inference and semantic quality must remain gated unless separately qualified. Rule-based/ESCO-backed title suggestions and auditable matching should function without a generative model. Do not invent user qualifications or automate application submissions.
Evidence: At every checkpoint provide exact tests (pass/fail/skip/not run), security/coverage evidence, dated docs, commit hashes, branch and remote verification. Report individual stage GO / CONDITIONAL GO / NO-GO; separate production NO-GO from development readiness.

8.7 KiB Raw Blame History