Compare commits

..

1 Commits

Author SHA1 Message Date
cesnimda f1c42a5245 docs(i18n): inventory document mail and operational message boundaries 2026-11-18 13:18:07 +01:00
6 changed files with 14163 additions and 0 deletions
File diff suppressed because it is too large Load Diff
+65
View File
@@ -0,0 +1,65 @@
# B03: document, mail, discovery and operational message boundaries
18 November 2026. I18N-00 research only, branch `codex/i18n-message-census-b03`, research base `ea18422a3f94b6bf970cedace4f87d3f8f51eb0a`. Application evidence is pinned to **`1db5f488f6138e4f0f33749452b7d9f47658f4b0`**, read with `git show`, never substituted with the older research checkout or changing A3 files. The full frozen I18N master and I18N-00 prompt, B01/B02 inventories/reports, Q01 and handoff were read. I18N-00 permits inventory/documentation and schema research; it does not authorise runtime stages 01–08.
The [inventory](B03_CENSUS.json) adds **346 source loci in 28 named backend owners** and ten source-level producer/consumer associations. **38 raw Git blobs** bind the producer owners and additional transport/client evidence. Each locus retains the full expression, line, UTF-8 line hash, file hash, namespace, owner/declaration context, boundary-specific translator context, raw lexical segments, provisional classification, parameter expressions and sensitivity limits. Actual user/provider/account/document values were never loaded or exported.
## Scope and exact accounting
| Namespace | Source loci | Bounded owners |
| --- | ---: | --- |
| Documents | 183 | CV variant/preview, profile/import/pipeline, application assets, document index, public CV, Career Profile, attachments, letter renderer |
| Communications | 85 | Send/drafts/correspondence, Gmail/Graph/IMAP controllers and provider facades |
| Discovery | 5 | Existing discovery validation/provider-unavailable responses and import-result boundary |
| Operations | 16 | Processing metadata boundary, system/maintenance errors, audit Undo results |
| Account/billing | 57 | Account lifecycle, billing, two-factor and admin user/mail operations |
| **Total** | **346** | The JSON owner coverage table enumerates all 28 files, including two provider facades with zero selected prose/error loci |
The record unit is a selected source line, not a unique sentence or translation key. Sixteen owned-payload/empty-status loci and two private-content boundaries have no literal message. Duplicated validation sites remain separate source loci; trace lines are not extra messages. The source-level associations have **80 evidence-line occurrences**, bound independently to exact expressions/hashes. Repeated occurrences are not claimed as distinct files or unique lines.
Raw lexical captures partition exactly: **402 occurrences = 258 provisional prose/label segments + 61 invariant segments + 83 unresolved segments**. Diagnostic templates, synthetic preview facts and model-instruction frames remain unresolved rather than being exported as UI translations. The document filename is also unresolved. Candidate classification is deliberately provisional: C# lexical captures do not resolve nested interpolation, raw strings, external Identity/provider message domains, or actual display reachability. Protected code/property/proper-name tokens are retained separately inside candidate prose. Placeholder expressions are recorded, but their declared types still require focused declaration/caller review.
All selected loci under the versioned boundary predicate are accounted for. **708 other quote-bearing lines remain explicitly listed per owner** in `otherQuotedLinesNotClassified`. That frontier includes route/schema/format/CSS/prompt literals and additional context; it is not 708 known missing messages. The predicate plus reviewed explicit continuation lines is a reproducible scope definition, not an AST, a whole-file every-string inventory, or proof that all critical surfaces are complete. This batch must not close full I18N-00.
## Source findings and European expansion implications
| ID | Bound evidence | Implication for the gated implementation plan |
| --- | --- | --- |
| B03-F01 | `draft-validation-to-presenter`: draft validation L220/222/224 enters create/update L92/128; the inbox presenter uses `getApiErrorMessage` | Controlled recipient/subject/body validation can bypass translated fallback copy. Preserve numeric limits and field identities, then assign one deliberate server/client display owner; do not bulk-translate arbitrary exception text. |
| B03-F02 | `cv-revision-code-client-display`: variant response L140/202/212; editor code checks L227/315/548 | Stable `cv_revision_required`/`cv_revision_conflict` codes already drive client state. Translate display without changing revision/conflict meaning or relying on English text comparison. Other error paths use server-message presentation. |
| B03-F03 | `discovery-masks-provider-detail`: backend L46/76/125; frontend catch paths L121/140/156 | Existing Norway discovery deliberately renders local fallback keys rather than provider/validation details. Translating backend strings alone does not establish visible coverage. Source/job-ad content and the future GJ country/provider programme remain separate and gated. |
| B03-F04 | `processing-outcome-mail-and-history`: persisted exception/failure assignments L383/428/537/564/571, failure mail L655/659 and profile consumers L506/890/892 | Controlled outcomes and upstream exceptions share persisted string boundaries. Mail embeds run trigger/status/id, error and ISO completion time. Define locale-at-creation/live-display and private-parameter policy before expanding languages; do not translate original CV text or exception payloads as catalogue entries. |
| B03-F05 | `admin-security-mail-transport`: user-controller L237/238/263/265/273 and SMTP contract L8/22 | Security and test-mail frames lack a recipient-locale argument in the inspected transport contract. Token-bearing reset URL, supplied subject/body and runtime timestamp remain opaque/private parameters. Native security wording and recipient-language ownership require separate review. |
| B03-F06 | `imap-send-failure-boundary`: provider L61 and send controller L209/210 | The IMAP unsupported-send prose is an exception producer, while the controller returns its category in `SendResult`. Do not assume every exception string is displayed. Provider transport errors and private provider data need their own complete domain inventory. |
| B03-F07 | `admin-error-versus-diagnostic-display` and `processing-stable-code-and-private-labels` | Admin HTTP errors use server-message presentation; AI `lastError` is displayed separately. Processing task/state/failure labels are mapped through local keys while job/document names remain original data. Keep diagnostics, invariant codes and private labels distinct from translator-owned text. This is classification, not security acceptance. |
| B03-F08 | `letter-output-language-and-aria`: renderer L36/37/42/56/59, asset render calls L300/309 | Document settings own HTML language and physical-document ARIA labels. The current explicit Bokmål branch otherwise falls back to English. UI locale must not become the output owner; the fixed `cover-letter.pdf` filename needs a separate future policy. No PDF/font/native rendering was executed. |
| B03-F09 | `persisted-notification-client-boundary`: popover L204/206; earlier B02 producer contracts | The client displays persisted title/message strings. Changing interface locale does not itself translate existing rows. A persisted-message strategy is still needed; this batch does not replay notifications or retroactively refresh B02 semantic findings. |
| B03-F10 | Preview samples and profile rewrite frames remain unresolved in the segment partition | Sample engineering career facts and AI instruction frames are neither interface chrome nor real career facts to rewrite on a locale switch. European cross-occupation fixtures and independently owned output-language instructions need deliberate later qualification. |
The newer-source comparison is explicit in the inventory: **B01 7/7 source blobs are byte-identical** to its recorded `07fbd5f7` baseline; **B02 14/15 are identical**. `SavedJobSearchAlertHostedService.cs` has changed. Its older B02 evidence remains historical and is not promoted to current-source acceptance. Source equality alone also does not requalify changed external consumers, native language or runtime behaviour. B01/B02 records, decisions and counts were not modified or added to B03 totals.
## Locale scope, QA and reproduction
The target remains **all 24 EU official language groups and the unchanged 60 planned candidate tags**, including wider national/script/regional variants. They are candidate tags, not 60 distinct languages or shipped translations. B03 checks the exact catalogue/schema hashes retained by Q01, candidate count, EU language groups and planned surface statuses. The existing Q01 Ajv validator was also invoked directly: 60 candidates/24 groups passed, all surfaces planned; `cnr-ME` canonicalises to `sr-ME` observationally, with no identity or alias decision. No catalogue/schema was changed.
From repository root:
```powershell
rtk proxy python docs/i18n/00/B03_inventory.py
rtk proxy python docs/i18n/00/B03_tests.py -v
# Explicit research regeneration only; writes B03_CENSUS.json, not app/catalogue files:
rtk proxy python docs/i18n/00/B03_inventory.py --build
```
The validator requires the recorded Git objects locally and fails rather than fetching missing objects or reading mutable application files. No package installation is needed. It independently re-reads immutable Git blobs and recomputes the declared selector, contexts, raw segments, partitions, placeholders, counts, trace expressions and prior-batch source comparison. This checks faithful deterministic evidence reproduction, not semantic correctness of the selector/classifier. Review the source expression and complete containing declaration before creating translation keys.
- [Validation receipt](B03_VALIDATION.json): **384 file/record bindings + 80 trace-line occurrence bindings**, 346 segment partitions and owner/frontier/count reconciliation passed. These are integrity checks, not product tests.
- [Native Python runner receipt](B03_TEST_RESULTS.txt): **4/4 research-integrity methods passed**, including **15 rejected mutations inside one method**. Mutations cover omitted loci/owners/traces, duplicate loci, source expression/file/line hashes, line number, segments/placeholders, forged native/released status, prior-source equivalence and counts. Do not call this 19 separate tests or add overlapping runs.
- Positive evidence checks distinguish server fallback masking, stable revision categories, document-language/ARIA ownership and category-only IMAP result handling. They assert recorded source contracts, not executed application behaviour.
- CLI, Q01 direct catalogue validation and `git diff --check` passed. Source/tool/test/input hashes are retained in the receipt. Graph-first navigation used the existing A3 root graph; actual claims were checked against pinned Git lines. No application source changed, and no graph rebuild or heavy application lane ran for this documentation package.
## True remaining blockers
Full critical-surface/every-string inventory, translator context and declared parameter typing remain incomplete. The 708-line frontier, unselected controller/service/renderer/middleware surfaces, external Identity/provider domains, frontend JSX/ARIA/non-catalogue text, full prompt/formatter domains and broader Python-sidecar strings need further source review. A regex inventory must never be marketed as completed localisation. Country/legal relationships, reviewer/market priorities, native/device/AT/PDF/font and all per-locale acceptance also remain open.
No application/unit/browser/backend/database/model/provider/mail/queue/hosted/production or native-language test ran. No translation, locale release, catalogue status change or approval was made. Phase 10 D02, Product Review, accepted I18N-00 and explicit entry/publication decisions remain mandatory for formal stages 01–08. **I18N-00 remains partial; production NO-GO.** Root owns integration/publication and central records.
+9
View File
@@ -0,0 +1,9 @@
test_current_source_envelope_and_planned_european_scope (__main__.ResearchIntegrity.test_current_source_envelope_and_planned_european_scope) ... ok
test_diagnostic_example_and_prompt_fragments_are_not_translation_ready (__main__.ResearchIntegrity.test_diagnostic_example_and_prompt_fragments_are_not_translation_ready) ... ok
test_mutations_of_source_scope_semantics_and_approval_are_rejected (__main__.ResearchIntegrity.test_mutations_of_source_scope_semantics_and_approval_are_rejected) ... ok
test_producer_and_consumer_evidence_retain_original_language_owners (__main__.ResearchIntegrity.test_producer_and_consumer_evidence_retain_original_language_owners) ... ok
----------------------------------------------------------------------
Ran 4 tests in 3.977s
OK
+58
View File
@@ -0,0 +1,58 @@
{
"status": "inventory-integrity-only",
"sourceCommit": "1db5f488f6138e4f0f33749452b7d9f47658f4b0",
"counts": {
"sources": 38,
"records": 346,
"byNamespace": {
"documents": 183,
"communications": 85,
"discovery": 5,
"operations": 16,
"account-billing": 57
},
"byBoundary": {
"http-error-boundary": 214,
"message-field": 16,
"synthetic-preview-example": 7,
"exception-boundary": 37,
"generation-prompt-frame": 8,
"diagnostic-template": 26,
"upstream-error-forwarding": 1,
"processing-outcome-message": 2,
"persisted-error-forwarding": 2,
"processing-email-template": 3,
"owned-payload-or-empty-status": 16,
"private-content-boundary": 2,
"validation-producer": 3,
"mail-subject-fallback": 1,
"security-email-template": 2,
"test-email-template": 3,
"document-label-or-filename": 3
},
"traces": 10,
"traceLineOccurrences": 80
},
"segmentAccounting": {
"sourceLiteralSegments": 402,
"translationCandidateSegments": 258,
"nonTranslatableSegments": 61,
"unresolvedSegments": 83
},
"candidateTags": 60,
"euLanguageGroups": 24,
"fileRecordBindings": 384,
"traceLineBindings": 80,
"otherQuotedLineFrontier": 708,
"inventorySha256": "b4e91fb2c4821cfadcf58445dbc358f20abb26c505c219681757c0ae22f5b8e3",
"validatorSha256": "3c4f7cce984306766b50a09059e2fda7c5a86ca86309c7ed40c0b2aa003a26e4",
"testsSha256": "f8d4019b6ec934cdd34a2f20e1db58dfd87b25b09a251a34d62ff92c10e57c3a",
"priorBatchChangedFiles": {
"B01": [],
"B02": [
"JobTrackerApi/Services/SavedJobSearchAlertHostedService.cs"
]
},
"productNativeTests": "NOT RUN",
"localeStageAcceptance": "NOT GRANTED"
}
+251
View File
@@ -0,0 +1,251 @@
"""Reproduce bounded I18N-00 lexical evidence; never translate or approve locales."""
from collections import Counter
from functools import lru_cache
import hashlib
import json
from pathlib import Path
import re
import subprocess
HERE = Path(__file__).resolve().parent
ROOT = HERE.parents[2]
SOURCE = '1db5f488f6138e4f0f33749452b7d9f47658f4b0'
OWNERS = {
'documents': ['CvVariantController', 'ProfileCvController', 'ProfileCvController.Import',
'ProfileCvController.Pipeline', 'ApplicationAssetsController', 'DocumentsController',
'PublicCvController', 'CareerProfileController', 'AttachmentsController'],
'communications': ['EmailController', 'EmailDraftsController', 'CorrespondenceController',
'GmailController', 'MicrosoftGraphController', 'ImapController'],
'discovery': ['JobDiscoveryController', 'JobImportController'],
'operations': ['AdminProcessingController', 'AdminSystemController', 'AdminAuditController'],
'account-billing': ['AccountLifecycleController', 'BillingController', 'TwoFactorController', 'UsersController'],
}
SCOPE = {f'JobTrackerApi/Controllers/{owner}.cs': namespace
for namespace, owners in OWNERS.items() for owner in owners}
SCOPE.update({f'JobTrackerApi/Services/EmailProviders/{owner}.cs': 'communications'
for owner in ['GmailProvider', 'MicrosoftGraphProvider', 'ImapProvider']})
SCOPE['JobTrackerApi/Services/CoverLetterDocumentRenderer.cs'] = 'documents'
LITERAL = re.compile(r'"((?:\\.|[^"\\])*)"')
DECLARATION = re.compile(r'^\s*(?:public|private|internal|protected)\s+.*?\b(\w+)\s*\(')
BOUNDARY = re.compile(r'\b(?:BadRequest|Conflict|NotFound|Unauthorized|Problem|ValidationProblem|StatusCode)\s*\(')
EXTRA_LINES = {
'JobTrackerApi/Controllers/UsersController.cs': {237: 'security-email-template', 238: 'security-email-template',
265: 'test-email-template', 273: 'test-email-template'},
'JobTrackerApi/Controllers/ProfileCvController.cs': {422: 'generation-prompt-frame', 424: 'generation-prompt-frame',
425: 'generation-prompt-frame', 426: 'generation-prompt-frame', 427: 'generation-prompt-frame',
428: 'generation-prompt-frame', 431: 'generation-prompt-frame', 442: 'http-error-boundary',
443: 'http-error-boundary', 446: 'http-error-boundary', 447: 'http-error-boundary'},
'JobTrackerApi/Controllers/ProfileCvController.Pipeline.cs': {383: 'upstream-error-forwarding',
439: 'processing-outcome-message', 440: 'processing-outcome-message',
564: 'persisted-error-forwarding', 571: 'persisted-error-forwarding'},
}
LABELS = {'Certifications', 'Projects', 'Languages', 'Profile', 'Experience', 'Skills', 'Education'}
PROTECTED = 'expectedVersion jobApplicationId jobId ThreadId CompanyName JobTitle threadId messageId clientRequestId linked rejected review suggested pending_review Pro Premium Stripe Gmail NAV SMTP PDF DOCX MB UUID'.split()
CONTEXT = {
'http-error-boundary': 'HTTP validation/conflict/failure display boundary; preserve status and code identities, inspect client masking or code mapping before choosing translation owner.',
'message-field': 'Display/response field literal; containing declaration and exact expression identify its producer. Do not treat adjacent data fields as interface text.',
'synthetic-preview-example': 'Repository preview/sample CV content, not real career facts; distinguish sample schema keys, labels and example facts before any locale-specific fixture adaptation.',
'exception-boundary': 'Exception or typed failure producer; exposure and forwarding domain need caller review. An exception string is not automatically a UI catalogue entry.',
'generation-prompt-frame': 'Model-instruction/output-language context, distinct from UI locale. Preserve factual source boundaries; no model execution or semantic qualification performed.',
'diagnostic-template': 'Operator diagnostic template; private identifier/path/provider parameters are excluded. Translation/exposure policy unresolved; not user-notification prose by default.',
'upstream-error-forwarding': 'Unbounded exception message forwarded to persisted processing history; inspect upstream code/controlled prose/private-value domains before export.',
'processing-outcome-message': 'Controlled processing admission/outcome prose; preserve machine status/category and distinguish user opt-out from entitlement failure.',
'persisted-error-forwarding': 'Failure text enters persisted history; locale-at-creation versus live-display policy and upstream message domains remain unresolved.',
'processing-email-template': 'Completion/failure email frame; run trigger/status/id, error and ISO completion time are runtime parameters, not translated source content.',
'owned-payload-or-empty-status': 'Owned payload or empty HTTP status boundary with no captured message literal; does not constitute a translatable message.',
'private-content-boundary': 'Subject/data reference without controlled literal; runtime user/provider/document values must retain original content and remain outside translation exports.',
'validation-producer': 'Controlled draft validation producer returned by multiple HTTP callers; numeric limits and protocol field names remain bound to validation semantics.',
'mail-subject-fallback': 'Controlled no-subject fallback only; real provider subject remains original private message content.',
'security-email-template': 'Security mail source frame; preserve token-bearing URL as an opaque private parameter and obtain recipient-language/security wording review.',
'test-email-template': 'Admin test-mail default/wrapper; preserve explicitly supplied subject/body. Timestamp formatting and recipient locale are independent contracts.',
'document-label-or-filename': 'Physical document label/ARIA/filename contract; document settings own language independently of UI. Filename policy remains unresolved.',
}
TRACE_SPEC = {
'discovery-masks-provider-detail': [('JobTrackerApi/Controllers/JobDiscoveryController.cs', [46, 76, 125]),
('job-tracker-ui/src/views/JobDiscoveryPage.tsx', [121, 140, 156])],
'draft-validation-to-presenter': [('JobTrackerApi/Controllers/EmailDraftsController.cs', [92, 128, 220, 222, 224]),
('job-tracker-ui/src/views/CorrespondenceInboxPage.tsx', [122, 404]), ('job-tracker-ui/src/api.ts', [18, 19, 20, 21, 22, 23, 24])],
'cv-revision-code-client-display': [('JobTrackerApi/Controllers/CvVariantController.cs', [140, 202, 212]),
('job-tracker-ui/src/views/CvBuilderEditor.tsx', [226, 227, 229, 315, 548])],
'processing-outcome-mail-and-history': [('JobTrackerApi/Controllers/ProfileCvController.Pipeline.cs', [383, 428, 537, 564, 571, 655, 659]),
('job-tracker-ui/src/views/CareerProfilePage.tsx', [506, 890, 892])],
'letter-output-language-and-aria': [('JobTrackerApi/Services/CoverLetterDocumentRenderer.cs', [36, 37, 42, 56, 59]),
('JobTrackerApi/Services/ApplicationAssetsService.cs', [300, 309])],
'admin-security-mail-transport': [('JobTrackerApi/Controllers/UsersController.cs', [235, 237, 238, 263, 265, 270, 273]),
('JobTrackerApi/Services/SmtpEmailSender.cs', [8, 22])],
'imap-send-failure-boundary': [('JobTrackerApi/Services/EmailProviders/ImapProvider.cs', [60, 61]),
('JobTrackerApi/Controllers/EmailController.cs', [196, 209, 210, 214])],
'admin-error-versus-diagnostic-display': [('JobTrackerApi/Controllers/AdminSystemController.cs', [126, 127, 155, 164]),
('job-tracker-ui/src/views/AdminSystemPage.tsx', [334, 447, 451])],
'processing-stable-code-and-private-labels': [('JobTrackerApi/Controllers/AdminProcessingController.cs', [183, 191, 200, 203, 206]),
('job-tracker-ui/src/views/AdminProcessingPage.tsx', [181, 183, 214, 215, 219])],
'persisted-notification-client-boundary': [('job-tracker-ui/src/components/NotificationsPopover.tsx', [185, 204, 206])],
}
TRACE_PATHS = {path for entries in TRACE_SPEC.values() for path, _ in entries}
PRIOR = {batch: json.loads((HERE / f'BACKEND_MESSAGE_CENSUS_{batch}.json').read_text(encoding='utf-8'))
for batch in ['B01', 'B02']}
PRIOR_PATHS = {binding['path'] for payload in PRIOR.values() for binding in payload['sourceBindings']}
def digest(value):
return hashlib.sha256(value).hexdigest()
@lru_cache(maxsize=None)
def blob(path):
assert path in SCOPE or path in TRACE_PATHS or path in PRIOR_PATHS, path
return subprocess.check_output(['git', 'show', f'{SOURCE}:{path}'], cwd=ROOT)
def selected(line, path):
text = line.strip()
if text.startswith('//'):
return None
if path.endswith('CoverLetterDocumentRenderer.cs'):
return 'document-label-or-filename' if re.search(r'var (letterLabel|bodyLabel) =|return new\(html', text) else None
if 'LogWarning(' in text or 'LogError(' in text or 'LogInformation(' in text:
return 'diagnostic-template' if '"' in text else None
if 'var subject =' in text:
if path.endswith('ProfileCvController.cs'):
return 'generation-prompt-frame'
if path.endswith('GmailController.cs'):
return 'mail-subject-fallback'
if path.endswith('UsersController.cs'):
return 'test-email-template'
if '"' not in text:
return 'private-content-boundary'
return 'processing-email-template'
if re.search(r'\? \$"Your CV|: \$"Your CV', text):
return 'processing-email-template'
if BOUNDARY.search(text) and ('"' in text or re.search(r'\b(?:ex|error|validation|result|outcome)\b', text)):
return 'http-error-boundary'
if ('throw new ' in text or 'new EmailProviderDeliveryException(' in text) and '"' in text:
return 'exception-boundary'
if re.search(r'\b(?:message|Message|Detail|Title|ErrorMessage)\s*=', text) and '"' in text:
return 'message-field'
if re.search(r'\berror\s*=\s*"', text):
return 'validation-producer'
return None
def partition(expression, reason):
raw = LITERAL.findall(expression)
# C# interpolation/raw-string parsing is outside this lexical census.
ambiguous = '"""' in expression or (('$"' in expression) and len(raw) > 2)
candidates, invariants, unresolved = [], [], []
for value in raw:
if ambiguous or reason in {'diagnostic-template', 'synthetic-preview-example', 'generation-prompt-frame'}:
unresolved.append(value)
elif (reason == 'document-label-or-filename' and value.endswith('.pdf')) or (('<' in value or '>' in value) and reason != 'http-error-boundary'):
unresolved.append(value)
elif value in LABELS or (len(value.split()) > 1 and re.search(r'[A-Za-zÀ-ž]', value)
and not value.startswith(('http', '/', 'data:'))) or value in ['Søknadsbrev', 'Brevtekst']:
candidates.append(value)
else:
invariants.append(value)
return raw, candidates, invariants, unresolved
def build():
records, bindings, coverage = [], [], []
for path, namespace in SCOPE.items():
data = blob(path)
lines = data.decode('utf-8').splitlines()
bindings.append({'path': path, 'sha256': digest(data), 'lineCount': len(lines)})
selected_lines = []
declaration, declaration_line = 'file scope / multiline declaration unresolved', 1
for number, line in enumerate(lines, 1):
if DECLARATION.match(line):
declaration, declaration_line = line.strip(), number
reason = EXTRA_LINES.get(path, {}).get(number) or selected(line, path)
if path.endswith('CvVariantController.cs') and 334 <= number <= 346 and reason:
reason = 'synthetic-preview-example'
if reason == 'http-error-boundary' and '"' not in line and not re.search(r'\b(?:ex\.Message|error|validation)\b', line):
reason = 'owned-payload-or-empty-status'
if not reason:
continue
selected_lines.append(number)
expression = line.strip()
raw, candidates, invariants, unresolved = partition(expression, reason)
placeholders = sorted(set(value for segment in raw for value in re.findall(r'(?<!\{)\{([^{}]+)\}(?!\})', segment)))
records.append({
'id': f'I00-B03-{len(records) + 1:04}', 'source': path, 'line': number,
'sourceSha256': digest(data), 'sourceLineSha256': digest(line.encode('utf-8')),
'sourceExpression': expression, 'namespace': namespace, 'owner': Path(path).stem,
'declarationContext': declaration, 'declarationLine': declaration_line,
'boundary': reason, 'sourceLiteralSegments': raw,
'translatorContext': CONTEXT[reason],
'sourceLanguage': 'en/nb-NO' if path.endswith('CoverLetterDocumentRenderer.cs') else 'en source templates; upstream/runtime language not inferred',
'translationCandidateSegments': candidates, 'nonTranslatableSegments': invariants,
'unresolvedSegments': unresolved, 'placeholderExpressions': placeholders,
'embeddedProtocolOrProperTokens': [token for token in PROTECTED if any(re.search(r'(?<!\w)' + re.escape(token) + r'(?!\w)', value) for value in candidates)],
'parameterTypeStatus': 'not resolved by lexical census; inspect declaration/caller before translation',
'sensitivity': 'source templates only; runtime user/provider/document/account/exception values excluded',
'contextStatus': 'owner/boundary inventoried; individual translator and upstream-domain context pending',
'classificationStatus': 'lexical suggestions only; not translator-ready or native reviewed',
'nativeReview': 'not-run', 'runtimeReview': 'not-run', 'translator': None,
})
coverage.append({'source': path, 'selectedLines': selected_lines, 'records': len(selected_lines),
'otherQuotedLinesNotClassified': [number for number, line in enumerate(lines, 1)
if '"' in line and number not in selected_lines],
'scope': 'all loci selected by versioned B03 predicate; not every string in the file'})
traces = []
for trace_id, entries in TRACE_SPEC.items():
evidence = []
for path, numbers in entries:
data = blob(path)
lines = data.decode('utf-8').splitlines()
if not any(binding['path'] == path for binding in bindings):
bindings.append({'path': path, 'sha256': digest(data), 'lineCount': len(lines)})
for number in numbers:
assert 1 <= number <= len(lines)
evidence.append({'source': path, 'line': number, 'sourceSha256': digest(data),
'sourceExpression': lines[number - 1].strip(),
'sourceLineSha256': digest(lines[number - 1].encode('utf-8'))})
traces.append({'id': trace_id, 'scope': 'source-level association only; no runtime execution or complete upstream-domain proof', 'evidence': evidence})
prior_comparison = {batch: {'recordedSource': payload['sourceCommit'], 'files': [
{'source': binding['path'], 'recordedSha256': binding['sha256'],
'currentSha256': digest(blob(binding['path'])),
'sourceBytesUnchanged': binding['sha256'] == digest(blob(binding['path']))}
for binding in payload['sourceBindings']]} for batch, payload in PRIOR.items()}
return {'status': 'partial-research-only', 'sourceCommit': SOURCE,
'method': 'B03 versioned line-boundary predicate and C# lexical captures; not an AST or semantic completeness claim',
'sourceBindings': bindings, 'ownerCoverage': coverage, 'records': records, 'traces': traces,
'priorBatchSourceComparison': prior_comparison,
'counts': {'sources': len(bindings), 'records': len(records),
'byNamespace': dict(Counter(record['namespace'] for record in records)),
'byBoundary': dict(Counter(record['boundary'] for record in records)),
'traces': len(traces), 'traceLineOccurrences': sum(len(trace['evidence']) for trace in traces)},
'segmentAccounting': {key: sum(len(record[key]) for record in records) for key in
['sourceLiteralSegments', 'translationCandidateSegments', 'nonTranslatableSegments', 'unresolvedSegments']}}
def validate(payload):
assert payload['status'] == 'partial-research-only'
assert payload['sourceCommit'] == SOURCE
expected = build()
assert payload == expected, 'inventory differs from immutable source, scope, metadata or reconciled accounting'
assert len({record['id'] for record in payload['records']}) == len(payload['records'])
for record in payload['records']:
assert Counter(record['sourceLiteralSegments']) == Counter(record['translationCandidateSegments'] +
record['nonTranslatableSegments'] + record['unresolvedSegments'])
catalogue = json.loads((HERE / 'LOCALE_CATALOG.json').read_text(encoding='utf-8'))
assert digest((HERE / 'LOCALE_CATALOG.json').read_bytes()) == 'e3c4071134b5aa890183d7c11fea793c25e13f19264012faaaa5131061e3a83b'
assert digest((HERE / 'LOCALE_SCHEMA.json').read_bytes()) == 'df189c9b83e52d9244839e80672fcc2e99bb66ee9f35d8afc457f27f676eb0c3'
assert len(catalogue['locales']) == 60
languages = {locale['tag'].split('-')[0] for locale in catalogue['locales']}
eu24 = set('bg hr cs da nl en et fi fr de el hu ga it lv lt mt pl pt ro sk sl es sv'.split())
assert eu24 <= languages
assert all(locale['status'] == 'planned' and set(locale['surfaces'].values()) == {'planned'} for locale in catalogue['locales'])
return {'status': 'inventory-integrity-only', 'sourceCommit': SOURCE, 'counts': payload['counts'],
'segmentAccounting': payload['segmentAccounting'], 'candidateTags': 60, 'euLanguageGroups': 24,
'fileRecordBindings': len(payload['sourceBindings']) + len(payload['records']),
'traceLineBindings': sum(len(trace['evidence']) for trace in payload['traces']),
'otherQuotedLineFrontier': sum(len(owner['otherQuotedLinesNotClassified']) for owner in payload['ownerCoverage']),
'inventorySha256': digest((HERE / 'B03_CENSUS.json').read_bytes()),
'validatorSha256': digest(Path(__file__).read_bytes()),
'testsSha256': digest((HERE / 'B03_tests.py').read_bytes()),
'priorBatchChangedFiles': {batch: [row['source'] for row in comparison['files'] if not row['sourceBytesUnchanged']]
for batch, comparison in payload['priorBatchSourceComparison'].items()},
'productNativeTests': 'NOT RUN', 'localeStageAcceptance': 'NOT GRANTED'}
if __name__ == '__main__':
import sys
if sys.argv[1:] == ['--build']:
(HERE / 'B03_CENSUS.json').write_text(json.dumps(build(), ensure_ascii=False, indent=2) + '\n', encoding='utf-8')
else:
print(json.dumps(validate(json.loads((HERE / 'B03_CENSUS.json').read_text(encoding='utf-8'))), indent=2))
+66
View File
@@ -0,0 +1,66 @@
"""Small mutation checks for source-bound research evidence, not product acceptance."""
from copy import deepcopy
import json
import unittest
import B03_inventory as census
class ResearchIntegrity(unittest.TestCase):
@classmethod
def setUpClass(cls):
cls.payload = json.loads((census.HERE / 'B03_CENSUS.json').read_text(encoding='utf-8'))
def test_current_source_envelope_and_planned_european_scope(self):
report = census.validate(self.payload)
self.assertEqual(60, report['candidateTags'])
self.assertEqual(24, report['euLanguageGroups'])
self.assertEqual(28, len(self.payload['ownerCoverage']))
self.assertEqual(10, len(self.payload['traces']))
self.assertEqual('NOT GRANTED', report['localeStageAcceptance'])
def test_mutations_of_source_scope_semantics_and_approval_are_rejected(self):
mutations = {
'omitted-locus': lambda p: p['records'].pop(),
'omitted-owner-frontier': lambda p: p['ownerCoverage'].pop(),
'omitted-trace': lambda p: p['traces'].pop(),
'changed-expression': lambda p: p['records'][0].update(sourceExpression='forged'),
'changed-file-hash': lambda p: p['sourceBindings'][0].update(sha256='0' * 64),
'changed-line-hash': lambda p: p['records'][0].update(sourceLineSha256='0' * 64),
'changed-line': lambda p: p['records'][0].update(line=1),
'duplicate-locus': lambda p: p['records'].append(deepcopy(p['records'][0])),
'lost-segment': lambda p: p['records'][0].update(translationCandidateSegments=[]),
'forged-placeholder': lambda p: p['records'][0].update(placeholderExpressions=['private-value']),
'forged-native-review': lambda p: p['records'][0].update(nativeReview='approved'),
'forged-release': lambda p: p.update(status='released'),
'changed-trace-line': lambda p: p['traces'][0]['evidence'][0].update(sourceExpression='forged'),
'wrong-count': lambda p: p['counts'].update(records=0),
'forged-prior-source-equivalence': lambda p: p['priorBatchSourceComparison']['B01']['files'][0].update(currentSha256='0' * 64),
}
for name, mutate in mutations.items():
with self.subTest(name=name):
payload = deepcopy(self.payload)
mutate(payload)
with self.assertRaises(AssertionError):
census.validate(payload)
def test_diagnostic_example_and_prompt_fragments_are_not_translation_ready(self):
for record in self.payload['records']:
if record['boundary'] in {'diagnostic-template', 'synthetic-preview-example', 'generation-prompt-frame'}:
self.assertFalse(record['translationCandidateSegments'])
self.assertEqual(record['sourceLiteralSegments'], record['unresolvedSegments'])
def test_producer_and_consumer_evidence_retain_original_language_owners(self):
traces = {trace['id']: trace for trace in self.payload['traces']}
discovery = '\n'.join(line['sourceExpression'] for line in traces['discovery-masks-provider-detail']['evidence'])
self.assertIn('jobDiscoveryUnavailable', discovery)
self.assertIn('jobDiscoverySaveSearchFailed', discovery)
letter = '\n'.join(line['sourceExpression'] for line in traces['letter-output-language-and-aria']['evidence'])
self.assertIn('s.Language', letter)
self.assertIn('aria-label', letter)
imap = '\n'.join(line['sourceExpression'] for line in traces['imap-send-failure-boundary']['evidence'])
self.assertIn('ex.Category', imap)
self.assertIn('unsupported_provider', imap)
if __name__ == '__main__':
unittest.main()