Compare commits
1 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| f1c42a5245 |
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,65 @@
|
||||
# B03: document, mail, discovery and operational message boundaries
|
||||
|
||||
18 November 2026. I18N-00 research only, branch `codex/i18n-message-census-b03`, research base `ea18422a3f94b6bf970cedace4f87d3f8f51eb0a`. Application evidence is pinned to **`1db5f488f6138e4f0f33749452b7d9f47658f4b0`**, read with `git show`, never substituted with the older research checkout or changing A3 files. The full frozen I18N master and I18N-00 prompt, B01/B02 inventories/reports, Q01 and handoff were read. I18N-00 permits inventory/documentation and schema research; it does not authorise runtime stages 01–08.
|
||||
|
||||
The [inventory](B03_CENSUS.json) adds **346 source loci in 28 named backend owners** and ten source-level producer/consumer associations. **38 raw Git blobs** bind the producer owners and additional transport/client evidence. Each locus retains the full expression, line, UTF-8 line hash, file hash, namespace, owner/declaration context, boundary-specific translator context, raw lexical segments, provisional classification, parameter expressions and sensitivity limits. Actual user/provider/account/document values were never loaded or exported.
|
||||
|
||||
## Scope and exact accounting
|
||||
|
||||
| Namespace | Source loci | Bounded owners |
|
||||
| --- | ---: | --- |
|
||||
| Documents | 183 | CV variant/preview, profile/import/pipeline, application assets, document index, public CV, Career Profile, attachments, letter renderer |
|
||||
| Communications | 85 | Send/drafts/correspondence, Gmail/Graph/IMAP controllers and provider facades |
|
||||
| Discovery | 5 | Existing discovery validation/provider-unavailable responses and import-result boundary |
|
||||
| Operations | 16 | Processing metadata boundary, system/maintenance errors, audit Undo results |
|
||||
| Account/billing | 57 | Account lifecycle, billing, two-factor and admin user/mail operations |
|
||||
| **Total** | **346** | The JSON owner coverage table enumerates all 28 files, including two provider facades with zero selected prose/error loci |
|
||||
|
||||
The record unit is a selected source line, not a unique sentence or translation key. Sixteen owned-payload/empty-status loci and two private-content boundaries have no literal message. Duplicated validation sites remain separate source loci; trace lines are not extra messages. The source-level associations have **80 evidence-line occurrences**, bound independently to exact expressions/hashes. Repeated occurrences are not claimed as distinct files or unique lines.
|
||||
|
||||
Raw lexical captures partition exactly: **402 occurrences = 258 provisional prose/label segments + 61 invariant segments + 83 unresolved segments**. Diagnostic templates, synthetic preview facts and model-instruction frames remain unresolved rather than being exported as UI translations. The document filename is also unresolved. Candidate classification is deliberately provisional: C# lexical captures do not resolve nested interpolation, raw strings, external Identity/provider message domains, or actual display reachability. Protected code/property/proper-name tokens are retained separately inside candidate prose. Placeholder expressions are recorded, but their declared types still require focused declaration/caller review.
|
||||
|
||||
All selected loci under the versioned boundary predicate are accounted for. **708 other quote-bearing lines remain explicitly listed per owner** in `otherQuotedLinesNotClassified`. That frontier includes route/schema/format/CSS/prompt literals and additional context; it is not 708 known missing messages. The predicate plus reviewed explicit continuation lines is a reproducible scope definition, not an AST, a whole-file every-string inventory, or proof that all critical surfaces are complete. This batch must not close full I18N-00.
|
||||
|
||||
## Source findings and European expansion implications
|
||||
|
||||
| ID | Bound evidence | Implication for the gated implementation plan |
|
||||
| --- | --- | --- |
|
||||
| B03-F01 | `draft-validation-to-presenter`: draft validation L220/222/224 enters create/update L92/128; the inbox presenter uses `getApiErrorMessage` | Controlled recipient/subject/body validation can bypass translated fallback copy. Preserve numeric limits and field identities, then assign one deliberate server/client display owner; do not bulk-translate arbitrary exception text. |
|
||||
| B03-F02 | `cv-revision-code-client-display`: variant response L140/202/212; editor code checks L227/315/548 | Stable `cv_revision_required`/`cv_revision_conflict` codes already drive client state. Translate display without changing revision/conflict meaning or relying on English text comparison. Other error paths use server-message presentation. |
|
||||
| B03-F03 | `discovery-masks-provider-detail`: backend L46/76/125; frontend catch paths L121/140/156 | Existing Norway discovery deliberately renders local fallback keys rather than provider/validation details. Translating backend strings alone does not establish visible coverage. Source/job-ad content and the future GJ country/provider programme remain separate and gated. |
|
||||
| B03-F04 | `processing-outcome-mail-and-history`: persisted exception/failure assignments L383/428/537/564/571, failure mail L655/659 and profile consumers L506/890/892 | Controlled outcomes and upstream exceptions share persisted string boundaries. Mail embeds run trigger/status/id, error and ISO completion time. Define locale-at-creation/live-display and private-parameter policy before expanding languages; do not translate original CV text or exception payloads as catalogue entries. |
|
||||
| B03-F05 | `admin-security-mail-transport`: user-controller L237/238/263/265/273 and SMTP contract L8/22 | Security and test-mail frames lack a recipient-locale argument in the inspected transport contract. Token-bearing reset URL, supplied subject/body and runtime timestamp remain opaque/private parameters. Native security wording and recipient-language ownership require separate review. |
|
||||
| B03-F06 | `imap-send-failure-boundary`: provider L61 and send controller L209/210 | The IMAP unsupported-send prose is an exception producer, while the controller returns its category in `SendResult`. Do not assume every exception string is displayed. Provider transport errors and private provider data need their own complete domain inventory. |
|
||||
| B03-F07 | `admin-error-versus-diagnostic-display` and `processing-stable-code-and-private-labels` | Admin HTTP errors use server-message presentation; AI `lastError` is displayed separately. Processing task/state/failure labels are mapped through local keys while job/document names remain original data. Keep diagnostics, invariant codes and private labels distinct from translator-owned text. This is classification, not security acceptance. |
|
||||
| B03-F08 | `letter-output-language-and-aria`: renderer L36/37/42/56/59, asset render calls L300/309 | Document settings own HTML language and physical-document ARIA labels. The current explicit Bokmål branch otherwise falls back to English. UI locale must not become the output owner; the fixed `cover-letter.pdf` filename needs a separate future policy. No PDF/font/native rendering was executed. |
|
||||
| B03-F09 | `persisted-notification-client-boundary`: popover L204/206; earlier B02 producer contracts | The client displays persisted title/message strings. Changing interface locale does not itself translate existing rows. A persisted-message strategy is still needed; this batch does not replay notifications or retroactively refresh B02 semantic findings. |
|
||||
| B03-F10 | Preview samples and profile rewrite frames remain unresolved in the segment partition | Sample engineering career facts and AI instruction frames are neither interface chrome nor real career facts to rewrite on a locale switch. European cross-occupation fixtures and independently owned output-language instructions need deliberate later qualification. |
|
||||
|
||||
The newer-source comparison is explicit in the inventory: **B01 7/7 source blobs are byte-identical** to its recorded `07fbd5f7` baseline; **B02 14/15 are identical**. `SavedJobSearchAlertHostedService.cs` has changed. Its older B02 evidence remains historical and is not promoted to current-source acceptance. Source equality alone also does not requalify changed external consumers, native language or runtime behaviour. B01/B02 records, decisions and counts were not modified or added to B03 totals.
|
||||
|
||||
## Locale scope, QA and reproduction
|
||||
|
||||
The target remains **all 24 EU official language groups and the unchanged 60 planned candidate tags**, including wider national/script/regional variants. They are candidate tags, not 60 distinct languages or shipped translations. B03 checks the exact catalogue/schema hashes retained by Q01, candidate count, EU language groups and planned surface statuses. The existing Q01 Ajv validator was also invoked directly: 60 candidates/24 groups passed, all surfaces planned; `cnr-ME` canonicalises to `sr-ME` observationally, with no identity or alias decision. No catalogue/schema was changed.
|
||||
|
||||
From repository root:
|
||||
|
||||
```powershell
|
||||
rtk proxy python docs/i18n/00/B03_inventory.py
|
||||
rtk proxy python docs/i18n/00/B03_tests.py -v
|
||||
# Explicit research regeneration only; writes B03_CENSUS.json, not app/catalogue files:
|
||||
rtk proxy python docs/i18n/00/B03_inventory.py --build
|
||||
```
|
||||
|
||||
The validator requires the recorded Git objects locally and fails rather than fetching missing objects or reading mutable application files. No package installation is needed. It independently re-reads immutable Git blobs and recomputes the declared selector, contexts, raw segments, partitions, placeholders, counts, trace expressions and prior-batch source comparison. This checks faithful deterministic evidence reproduction, not semantic correctness of the selector/classifier. Review the source expression and complete containing declaration before creating translation keys.
|
||||
|
||||
- [Validation receipt](B03_VALIDATION.json): **384 file/record bindings + 80 trace-line occurrence bindings**, 346 segment partitions and owner/frontier/count reconciliation passed. These are integrity checks, not product tests.
|
||||
- [Native Python runner receipt](B03_TEST_RESULTS.txt): **4/4 research-integrity methods passed**, including **15 rejected mutations inside one method**. Mutations cover omitted loci/owners/traces, duplicate loci, source expression/file/line hashes, line number, segments/placeholders, forged native/released status, prior-source equivalence and counts. Do not call this 19 separate tests or add overlapping runs.
|
||||
- Positive evidence checks distinguish server fallback masking, stable revision categories, document-language/ARIA ownership and category-only IMAP result handling. They assert recorded source contracts, not executed application behaviour.
|
||||
- CLI, Q01 direct catalogue validation and `git diff --check` passed. Source/tool/test/input hashes are retained in the receipt. Graph-first navigation used the existing A3 root graph; actual claims were checked against pinned Git lines. No application source changed, and no graph rebuild or heavy application lane ran for this documentation package.
|
||||
|
||||
## True remaining blockers
|
||||
|
||||
Full critical-surface/every-string inventory, translator context and declared parameter typing remain incomplete. The 708-line frontier, unselected controller/service/renderer/middleware surfaces, external Identity/provider domains, frontend JSX/ARIA/non-catalogue text, full prompt/formatter domains and broader Python-sidecar strings need further source review. A regex inventory must never be marketed as completed localisation. Country/legal relationships, reviewer/market priorities, native/device/AT/PDF/font and all per-locale acceptance also remain open.
|
||||
|
||||
No application/unit/browser/backend/database/model/provider/mail/queue/hosted/production or native-language test ran. No translation, locale release, catalogue status change or approval was made. Phase 10 D02, Product Review, accepted I18N-00 and explicit entry/publication decisions remain mandatory for formal stages 01–08. **I18N-00 remains partial; production NO-GO.** Root owns integration/publication and central records.
|
||||
@@ -0,0 +1,9 @@
|
||||
test_current_source_envelope_and_planned_european_scope (__main__.ResearchIntegrity.test_current_source_envelope_and_planned_european_scope) ... ok
|
||||
test_diagnostic_example_and_prompt_fragments_are_not_translation_ready (__main__.ResearchIntegrity.test_diagnostic_example_and_prompt_fragments_are_not_translation_ready) ... ok
|
||||
test_mutations_of_source_scope_semantics_and_approval_are_rejected (__main__.ResearchIntegrity.test_mutations_of_source_scope_semantics_and_approval_are_rejected) ... ok
|
||||
test_producer_and_consumer_evidence_retain_original_language_owners (__main__.ResearchIntegrity.test_producer_and_consumer_evidence_retain_original_language_owners) ... ok
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 4 tests in 3.977s
|
||||
|
||||
OK
|
||||
@@ -0,0 +1,58 @@
|
||||
{
|
||||
"status": "inventory-integrity-only",
|
||||
"sourceCommit": "1db5f488f6138e4f0f33749452b7d9f47658f4b0",
|
||||
"counts": {
|
||||
"sources": 38,
|
||||
"records": 346,
|
||||
"byNamespace": {
|
||||
"documents": 183,
|
||||
"communications": 85,
|
||||
"discovery": 5,
|
||||
"operations": 16,
|
||||
"account-billing": 57
|
||||
},
|
||||
"byBoundary": {
|
||||
"http-error-boundary": 214,
|
||||
"message-field": 16,
|
||||
"synthetic-preview-example": 7,
|
||||
"exception-boundary": 37,
|
||||
"generation-prompt-frame": 8,
|
||||
"diagnostic-template": 26,
|
||||
"upstream-error-forwarding": 1,
|
||||
"processing-outcome-message": 2,
|
||||
"persisted-error-forwarding": 2,
|
||||
"processing-email-template": 3,
|
||||
"owned-payload-or-empty-status": 16,
|
||||
"private-content-boundary": 2,
|
||||
"validation-producer": 3,
|
||||
"mail-subject-fallback": 1,
|
||||
"security-email-template": 2,
|
||||
"test-email-template": 3,
|
||||
"document-label-or-filename": 3
|
||||
},
|
||||
"traces": 10,
|
||||
"traceLineOccurrences": 80
|
||||
},
|
||||
"segmentAccounting": {
|
||||
"sourceLiteralSegments": 402,
|
||||
"translationCandidateSegments": 258,
|
||||
"nonTranslatableSegments": 61,
|
||||
"unresolvedSegments": 83
|
||||
},
|
||||
"candidateTags": 60,
|
||||
"euLanguageGroups": 24,
|
||||
"fileRecordBindings": 384,
|
||||
"traceLineBindings": 80,
|
||||
"otherQuotedLineFrontier": 708,
|
||||
"inventorySha256": "b4e91fb2c4821cfadcf58445dbc358f20abb26c505c219681757c0ae22f5b8e3",
|
||||
"validatorSha256": "3c4f7cce984306766b50a09059e2fda7c5a86ca86309c7ed40c0b2aa003a26e4",
|
||||
"testsSha256": "f8d4019b6ec934cdd34a2f20e1db58dfd87b25b09a251a34d62ff92c10e57c3a",
|
||||
"priorBatchChangedFiles": {
|
||||
"B01": [],
|
||||
"B02": [
|
||||
"JobTrackerApi/Services/SavedJobSearchAlertHostedService.cs"
|
||||
]
|
||||
},
|
||||
"productNativeTests": "NOT RUN",
|
||||
"localeStageAcceptance": "NOT GRANTED"
|
||||
}
|
||||
@@ -0,0 +1,251 @@
|
||||
"""Reproduce bounded I18N-00 lexical evidence; never translate or approve locales."""
|
||||
from collections import Counter
|
||||
from functools import lru_cache
|
||||
import hashlib
|
||||
import json
|
||||
from pathlib import Path
|
||||
import re
|
||||
import subprocess
|
||||
|
||||
HERE = Path(__file__).resolve().parent
|
||||
ROOT = HERE.parents[2]
|
||||
SOURCE = '1db5f488f6138e4f0f33749452b7d9f47658f4b0'
|
||||
OWNERS = {
|
||||
'documents': ['CvVariantController', 'ProfileCvController', 'ProfileCvController.Import',
|
||||
'ProfileCvController.Pipeline', 'ApplicationAssetsController', 'DocumentsController',
|
||||
'PublicCvController', 'CareerProfileController', 'AttachmentsController'],
|
||||
'communications': ['EmailController', 'EmailDraftsController', 'CorrespondenceController',
|
||||
'GmailController', 'MicrosoftGraphController', 'ImapController'],
|
||||
'discovery': ['JobDiscoveryController', 'JobImportController'],
|
||||
'operations': ['AdminProcessingController', 'AdminSystemController', 'AdminAuditController'],
|
||||
'account-billing': ['AccountLifecycleController', 'BillingController', 'TwoFactorController', 'UsersController'],
|
||||
}
|
||||
SCOPE = {f'JobTrackerApi/Controllers/{owner}.cs': namespace
|
||||
for namespace, owners in OWNERS.items() for owner in owners}
|
||||
SCOPE.update({f'JobTrackerApi/Services/EmailProviders/{owner}.cs': 'communications'
|
||||
for owner in ['GmailProvider', 'MicrosoftGraphProvider', 'ImapProvider']})
|
||||
SCOPE['JobTrackerApi/Services/CoverLetterDocumentRenderer.cs'] = 'documents'
|
||||
LITERAL = re.compile(r'"((?:\\.|[^"\\])*)"')
|
||||
DECLARATION = re.compile(r'^\s*(?:public|private|internal|protected)\s+.*?\b(\w+)\s*\(')
|
||||
BOUNDARY = re.compile(r'\b(?:BadRequest|Conflict|NotFound|Unauthorized|Problem|ValidationProblem|StatusCode)\s*\(')
|
||||
EXTRA_LINES = {
|
||||
'JobTrackerApi/Controllers/UsersController.cs': {237: 'security-email-template', 238: 'security-email-template',
|
||||
265: 'test-email-template', 273: 'test-email-template'},
|
||||
'JobTrackerApi/Controllers/ProfileCvController.cs': {422: 'generation-prompt-frame', 424: 'generation-prompt-frame',
|
||||
425: 'generation-prompt-frame', 426: 'generation-prompt-frame', 427: 'generation-prompt-frame',
|
||||
428: 'generation-prompt-frame', 431: 'generation-prompt-frame', 442: 'http-error-boundary',
|
||||
443: 'http-error-boundary', 446: 'http-error-boundary', 447: 'http-error-boundary'},
|
||||
'JobTrackerApi/Controllers/ProfileCvController.Pipeline.cs': {383: 'upstream-error-forwarding',
|
||||
439: 'processing-outcome-message', 440: 'processing-outcome-message',
|
||||
564: 'persisted-error-forwarding', 571: 'persisted-error-forwarding'},
|
||||
}
|
||||
LABELS = {'Certifications', 'Projects', 'Languages', 'Profile', 'Experience', 'Skills', 'Education'}
|
||||
PROTECTED = 'expectedVersion jobApplicationId jobId ThreadId CompanyName JobTitle threadId messageId clientRequestId linked rejected review suggested pending_review Pro Premium Stripe Gmail NAV SMTP PDF DOCX MB UUID'.split()
|
||||
CONTEXT = {
|
||||
'http-error-boundary': 'HTTP validation/conflict/failure display boundary; preserve status and code identities, inspect client masking or code mapping before choosing translation owner.',
|
||||
'message-field': 'Display/response field literal; containing declaration and exact expression identify its producer. Do not treat adjacent data fields as interface text.',
|
||||
'synthetic-preview-example': 'Repository preview/sample CV content, not real career facts; distinguish sample schema keys, labels and example facts before any locale-specific fixture adaptation.',
|
||||
'exception-boundary': 'Exception or typed failure producer; exposure and forwarding domain need caller review. An exception string is not automatically a UI catalogue entry.',
|
||||
'generation-prompt-frame': 'Model-instruction/output-language context, distinct from UI locale. Preserve factual source boundaries; no model execution or semantic qualification performed.',
|
||||
'diagnostic-template': 'Operator diagnostic template; private identifier/path/provider parameters are excluded. Translation/exposure policy unresolved; not user-notification prose by default.',
|
||||
'upstream-error-forwarding': 'Unbounded exception message forwarded to persisted processing history; inspect upstream code/controlled prose/private-value domains before export.',
|
||||
'processing-outcome-message': 'Controlled processing admission/outcome prose; preserve machine status/category and distinguish user opt-out from entitlement failure.',
|
||||
'persisted-error-forwarding': 'Failure text enters persisted history; locale-at-creation versus live-display policy and upstream message domains remain unresolved.',
|
||||
'processing-email-template': 'Completion/failure email frame; run trigger/status/id, error and ISO completion time are runtime parameters, not translated source content.',
|
||||
'owned-payload-or-empty-status': 'Owned payload or empty HTTP status boundary with no captured message literal; does not constitute a translatable message.',
|
||||
'private-content-boundary': 'Subject/data reference without controlled literal; runtime user/provider/document values must retain original content and remain outside translation exports.',
|
||||
'validation-producer': 'Controlled draft validation producer returned by multiple HTTP callers; numeric limits and protocol field names remain bound to validation semantics.',
|
||||
'mail-subject-fallback': 'Controlled no-subject fallback only; real provider subject remains original private message content.',
|
||||
'security-email-template': 'Security mail source frame; preserve token-bearing URL as an opaque private parameter and obtain recipient-language/security wording review.',
|
||||
'test-email-template': 'Admin test-mail default/wrapper; preserve explicitly supplied subject/body. Timestamp formatting and recipient locale are independent contracts.',
|
||||
'document-label-or-filename': 'Physical document label/ARIA/filename contract; document settings own language independently of UI. Filename policy remains unresolved.',
|
||||
}
|
||||
TRACE_SPEC = {
|
||||
'discovery-masks-provider-detail': [('JobTrackerApi/Controllers/JobDiscoveryController.cs', [46, 76, 125]),
|
||||
('job-tracker-ui/src/views/JobDiscoveryPage.tsx', [121, 140, 156])],
|
||||
'draft-validation-to-presenter': [('JobTrackerApi/Controllers/EmailDraftsController.cs', [92, 128, 220, 222, 224]),
|
||||
('job-tracker-ui/src/views/CorrespondenceInboxPage.tsx', [122, 404]), ('job-tracker-ui/src/api.ts', [18, 19, 20, 21, 22, 23, 24])],
|
||||
'cv-revision-code-client-display': [('JobTrackerApi/Controllers/CvVariantController.cs', [140, 202, 212]),
|
||||
('job-tracker-ui/src/views/CvBuilderEditor.tsx', [226, 227, 229, 315, 548])],
|
||||
'processing-outcome-mail-and-history': [('JobTrackerApi/Controllers/ProfileCvController.Pipeline.cs', [383, 428, 537, 564, 571, 655, 659]),
|
||||
('job-tracker-ui/src/views/CareerProfilePage.tsx', [506, 890, 892])],
|
||||
'letter-output-language-and-aria': [('JobTrackerApi/Services/CoverLetterDocumentRenderer.cs', [36, 37, 42, 56, 59]),
|
||||
('JobTrackerApi/Services/ApplicationAssetsService.cs', [300, 309])],
|
||||
'admin-security-mail-transport': [('JobTrackerApi/Controllers/UsersController.cs', [235, 237, 238, 263, 265, 270, 273]),
|
||||
('JobTrackerApi/Services/SmtpEmailSender.cs', [8, 22])],
|
||||
'imap-send-failure-boundary': [('JobTrackerApi/Services/EmailProviders/ImapProvider.cs', [60, 61]),
|
||||
('JobTrackerApi/Controllers/EmailController.cs', [196, 209, 210, 214])],
|
||||
'admin-error-versus-diagnostic-display': [('JobTrackerApi/Controllers/AdminSystemController.cs', [126, 127, 155, 164]),
|
||||
('job-tracker-ui/src/views/AdminSystemPage.tsx', [334, 447, 451])],
|
||||
'processing-stable-code-and-private-labels': [('JobTrackerApi/Controllers/AdminProcessingController.cs', [183, 191, 200, 203, 206]),
|
||||
('job-tracker-ui/src/views/AdminProcessingPage.tsx', [181, 183, 214, 215, 219])],
|
||||
'persisted-notification-client-boundary': [('job-tracker-ui/src/components/NotificationsPopover.tsx', [185, 204, 206])],
|
||||
}
|
||||
TRACE_PATHS = {path for entries in TRACE_SPEC.values() for path, _ in entries}
|
||||
PRIOR = {batch: json.loads((HERE / f'BACKEND_MESSAGE_CENSUS_{batch}.json').read_text(encoding='utf-8'))
|
||||
for batch in ['B01', 'B02']}
|
||||
PRIOR_PATHS = {binding['path'] for payload in PRIOR.values() for binding in payload['sourceBindings']}
|
||||
|
||||
def digest(value):
|
||||
return hashlib.sha256(value).hexdigest()
|
||||
|
||||
@lru_cache(maxsize=None)
|
||||
def blob(path):
|
||||
assert path in SCOPE or path in TRACE_PATHS or path in PRIOR_PATHS, path
|
||||
return subprocess.check_output(['git', 'show', f'{SOURCE}:{path}'], cwd=ROOT)
|
||||
|
||||
def selected(line, path):
|
||||
text = line.strip()
|
||||
if text.startswith('//'):
|
||||
return None
|
||||
if path.endswith('CoverLetterDocumentRenderer.cs'):
|
||||
return 'document-label-or-filename' if re.search(r'var (letterLabel|bodyLabel) =|return new\(html', text) else None
|
||||
if 'LogWarning(' in text or 'LogError(' in text or 'LogInformation(' in text:
|
||||
return 'diagnostic-template' if '"' in text else None
|
||||
if 'var subject =' in text:
|
||||
if path.endswith('ProfileCvController.cs'):
|
||||
return 'generation-prompt-frame'
|
||||
if path.endswith('GmailController.cs'):
|
||||
return 'mail-subject-fallback'
|
||||
if path.endswith('UsersController.cs'):
|
||||
return 'test-email-template'
|
||||
if '"' not in text:
|
||||
return 'private-content-boundary'
|
||||
return 'processing-email-template'
|
||||
if re.search(r'\? \$"Your CV|: \$"Your CV', text):
|
||||
return 'processing-email-template'
|
||||
if BOUNDARY.search(text) and ('"' in text or re.search(r'\b(?:ex|error|validation|result|outcome)\b', text)):
|
||||
return 'http-error-boundary'
|
||||
if ('throw new ' in text or 'new EmailProviderDeliveryException(' in text) and '"' in text:
|
||||
return 'exception-boundary'
|
||||
if re.search(r'\b(?:message|Message|Detail|Title|ErrorMessage)\s*=', text) and '"' in text:
|
||||
return 'message-field'
|
||||
if re.search(r'\berror\s*=\s*"', text):
|
||||
return 'validation-producer'
|
||||
return None
|
||||
|
||||
def partition(expression, reason):
|
||||
raw = LITERAL.findall(expression)
|
||||
# C# interpolation/raw-string parsing is outside this lexical census.
|
||||
ambiguous = '"""' in expression or (('$"' in expression) and len(raw) > 2)
|
||||
candidates, invariants, unresolved = [], [], []
|
||||
for value in raw:
|
||||
if ambiguous or reason in {'diagnostic-template', 'synthetic-preview-example', 'generation-prompt-frame'}:
|
||||
unresolved.append(value)
|
||||
elif (reason == 'document-label-or-filename' and value.endswith('.pdf')) or (('<' in value or '>' in value) and reason != 'http-error-boundary'):
|
||||
unresolved.append(value)
|
||||
elif value in LABELS or (len(value.split()) > 1 and re.search(r'[A-Za-zÀ-ž]', value)
|
||||
and not value.startswith(('http', '/', 'data:'))) or value in ['Søknadsbrev', 'Brevtekst']:
|
||||
candidates.append(value)
|
||||
else:
|
||||
invariants.append(value)
|
||||
return raw, candidates, invariants, unresolved
|
||||
|
||||
def build():
|
||||
records, bindings, coverage = [], [], []
|
||||
for path, namespace in SCOPE.items():
|
||||
data = blob(path)
|
||||
lines = data.decode('utf-8').splitlines()
|
||||
bindings.append({'path': path, 'sha256': digest(data), 'lineCount': len(lines)})
|
||||
selected_lines = []
|
||||
declaration, declaration_line = 'file scope / multiline declaration unresolved', 1
|
||||
for number, line in enumerate(lines, 1):
|
||||
if DECLARATION.match(line):
|
||||
declaration, declaration_line = line.strip(), number
|
||||
reason = EXTRA_LINES.get(path, {}).get(number) or selected(line, path)
|
||||
if path.endswith('CvVariantController.cs') and 334 <= number <= 346 and reason:
|
||||
reason = 'synthetic-preview-example'
|
||||
if reason == 'http-error-boundary' and '"' not in line and not re.search(r'\b(?:ex\.Message|error|validation)\b', line):
|
||||
reason = 'owned-payload-or-empty-status'
|
||||
if not reason:
|
||||
continue
|
||||
selected_lines.append(number)
|
||||
expression = line.strip()
|
||||
raw, candidates, invariants, unresolved = partition(expression, reason)
|
||||
placeholders = sorted(set(value for segment in raw for value in re.findall(r'(?<!\{)\{([^{}]+)\}(?!\})', segment)))
|
||||
records.append({
|
||||
'id': f'I00-B03-{len(records) + 1:04}', 'source': path, 'line': number,
|
||||
'sourceSha256': digest(data), 'sourceLineSha256': digest(line.encode('utf-8')),
|
||||
'sourceExpression': expression, 'namespace': namespace, 'owner': Path(path).stem,
|
||||
'declarationContext': declaration, 'declarationLine': declaration_line,
|
||||
'boundary': reason, 'sourceLiteralSegments': raw,
|
||||
'translatorContext': CONTEXT[reason],
|
||||
'sourceLanguage': 'en/nb-NO' if path.endswith('CoverLetterDocumentRenderer.cs') else 'en source templates; upstream/runtime language not inferred',
|
||||
'translationCandidateSegments': candidates, 'nonTranslatableSegments': invariants,
|
||||
'unresolvedSegments': unresolved, 'placeholderExpressions': placeholders,
|
||||
'embeddedProtocolOrProperTokens': [token for token in PROTECTED if any(re.search(r'(?<!\w)' + re.escape(token) + r'(?!\w)', value) for value in candidates)],
|
||||
'parameterTypeStatus': 'not resolved by lexical census; inspect declaration/caller before translation',
|
||||
'sensitivity': 'source templates only; runtime user/provider/document/account/exception values excluded',
|
||||
'contextStatus': 'owner/boundary inventoried; individual translator and upstream-domain context pending',
|
||||
'classificationStatus': 'lexical suggestions only; not translator-ready or native reviewed',
|
||||
'nativeReview': 'not-run', 'runtimeReview': 'not-run', 'translator': None,
|
||||
})
|
||||
coverage.append({'source': path, 'selectedLines': selected_lines, 'records': len(selected_lines),
|
||||
'otherQuotedLinesNotClassified': [number for number, line in enumerate(lines, 1)
|
||||
if '"' in line and number not in selected_lines],
|
||||
'scope': 'all loci selected by versioned B03 predicate; not every string in the file'})
|
||||
traces = []
|
||||
for trace_id, entries in TRACE_SPEC.items():
|
||||
evidence = []
|
||||
for path, numbers in entries:
|
||||
data = blob(path)
|
||||
lines = data.decode('utf-8').splitlines()
|
||||
if not any(binding['path'] == path for binding in bindings):
|
||||
bindings.append({'path': path, 'sha256': digest(data), 'lineCount': len(lines)})
|
||||
for number in numbers:
|
||||
assert 1 <= number <= len(lines)
|
||||
evidence.append({'source': path, 'line': number, 'sourceSha256': digest(data),
|
||||
'sourceExpression': lines[number - 1].strip(),
|
||||
'sourceLineSha256': digest(lines[number - 1].encode('utf-8'))})
|
||||
traces.append({'id': trace_id, 'scope': 'source-level association only; no runtime execution or complete upstream-domain proof', 'evidence': evidence})
|
||||
prior_comparison = {batch: {'recordedSource': payload['sourceCommit'], 'files': [
|
||||
{'source': binding['path'], 'recordedSha256': binding['sha256'],
|
||||
'currentSha256': digest(blob(binding['path'])),
|
||||
'sourceBytesUnchanged': binding['sha256'] == digest(blob(binding['path']))}
|
||||
for binding in payload['sourceBindings']]} for batch, payload in PRIOR.items()}
|
||||
return {'status': 'partial-research-only', 'sourceCommit': SOURCE,
|
||||
'method': 'B03 versioned line-boundary predicate and C# lexical captures; not an AST or semantic completeness claim',
|
||||
'sourceBindings': bindings, 'ownerCoverage': coverage, 'records': records, 'traces': traces,
|
||||
'priorBatchSourceComparison': prior_comparison,
|
||||
'counts': {'sources': len(bindings), 'records': len(records),
|
||||
'byNamespace': dict(Counter(record['namespace'] for record in records)),
|
||||
'byBoundary': dict(Counter(record['boundary'] for record in records)),
|
||||
'traces': len(traces), 'traceLineOccurrences': sum(len(trace['evidence']) for trace in traces)},
|
||||
'segmentAccounting': {key: sum(len(record[key]) for record in records) for key in
|
||||
['sourceLiteralSegments', 'translationCandidateSegments', 'nonTranslatableSegments', 'unresolvedSegments']}}
|
||||
|
||||
def validate(payload):
|
||||
assert payload['status'] == 'partial-research-only'
|
||||
assert payload['sourceCommit'] == SOURCE
|
||||
expected = build()
|
||||
assert payload == expected, 'inventory differs from immutable source, scope, metadata or reconciled accounting'
|
||||
assert len({record['id'] for record in payload['records']}) == len(payload['records'])
|
||||
for record in payload['records']:
|
||||
assert Counter(record['sourceLiteralSegments']) == Counter(record['translationCandidateSegments'] +
|
||||
record['nonTranslatableSegments'] + record['unresolvedSegments'])
|
||||
catalogue = json.loads((HERE / 'LOCALE_CATALOG.json').read_text(encoding='utf-8'))
|
||||
assert digest((HERE / 'LOCALE_CATALOG.json').read_bytes()) == 'e3c4071134b5aa890183d7c11fea793c25e13f19264012faaaa5131061e3a83b'
|
||||
assert digest((HERE / 'LOCALE_SCHEMA.json').read_bytes()) == 'df189c9b83e52d9244839e80672fcc2e99bb66ee9f35d8afc457f27f676eb0c3'
|
||||
assert len(catalogue['locales']) == 60
|
||||
languages = {locale['tag'].split('-')[0] for locale in catalogue['locales']}
|
||||
eu24 = set('bg hr cs da nl en et fi fr de el hu ga it lv lt mt pl pt ro sk sl es sv'.split())
|
||||
assert eu24 <= languages
|
||||
assert all(locale['status'] == 'planned' and set(locale['surfaces'].values()) == {'planned'} for locale in catalogue['locales'])
|
||||
return {'status': 'inventory-integrity-only', 'sourceCommit': SOURCE, 'counts': payload['counts'],
|
||||
'segmentAccounting': payload['segmentAccounting'], 'candidateTags': 60, 'euLanguageGroups': 24,
|
||||
'fileRecordBindings': len(payload['sourceBindings']) + len(payload['records']),
|
||||
'traceLineBindings': sum(len(trace['evidence']) for trace in payload['traces']),
|
||||
'otherQuotedLineFrontier': sum(len(owner['otherQuotedLinesNotClassified']) for owner in payload['ownerCoverage']),
|
||||
'inventorySha256': digest((HERE / 'B03_CENSUS.json').read_bytes()),
|
||||
'validatorSha256': digest(Path(__file__).read_bytes()),
|
||||
'testsSha256': digest((HERE / 'B03_tests.py').read_bytes()),
|
||||
'priorBatchChangedFiles': {batch: [row['source'] for row in comparison['files'] if not row['sourceBytesUnchanged']]
|
||||
for batch, comparison in payload['priorBatchSourceComparison'].items()},
|
||||
'productNativeTests': 'NOT RUN', 'localeStageAcceptance': 'NOT GRANTED'}
|
||||
|
||||
if __name__ == '__main__':
|
||||
import sys
|
||||
if sys.argv[1:] == ['--build']:
|
||||
(HERE / 'B03_CENSUS.json').write_text(json.dumps(build(), ensure_ascii=False, indent=2) + '\n', encoding='utf-8')
|
||||
else:
|
||||
print(json.dumps(validate(json.loads((HERE / 'B03_CENSUS.json').read_text(encoding='utf-8'))), indent=2))
|
||||
@@ -0,0 +1,66 @@
|
||||
"""Small mutation checks for source-bound research evidence, not product acceptance."""
|
||||
from copy import deepcopy
|
||||
import json
|
||||
import unittest
|
||||
import B03_inventory as census
|
||||
|
||||
|
||||
class ResearchIntegrity(unittest.TestCase):
|
||||
@classmethod
|
||||
def setUpClass(cls):
|
||||
cls.payload = json.loads((census.HERE / 'B03_CENSUS.json').read_text(encoding='utf-8'))
|
||||
|
||||
def test_current_source_envelope_and_planned_european_scope(self):
|
||||
report = census.validate(self.payload)
|
||||
self.assertEqual(60, report['candidateTags'])
|
||||
self.assertEqual(24, report['euLanguageGroups'])
|
||||
self.assertEqual(28, len(self.payload['ownerCoverage']))
|
||||
self.assertEqual(10, len(self.payload['traces']))
|
||||
self.assertEqual('NOT GRANTED', report['localeStageAcceptance'])
|
||||
|
||||
def test_mutations_of_source_scope_semantics_and_approval_are_rejected(self):
|
||||
mutations = {
|
||||
'omitted-locus': lambda p: p['records'].pop(),
|
||||
'omitted-owner-frontier': lambda p: p['ownerCoverage'].pop(),
|
||||
'omitted-trace': lambda p: p['traces'].pop(),
|
||||
'changed-expression': lambda p: p['records'][0].update(sourceExpression='forged'),
|
||||
'changed-file-hash': lambda p: p['sourceBindings'][0].update(sha256='0' * 64),
|
||||
'changed-line-hash': lambda p: p['records'][0].update(sourceLineSha256='0' * 64),
|
||||
'changed-line': lambda p: p['records'][0].update(line=1),
|
||||
'duplicate-locus': lambda p: p['records'].append(deepcopy(p['records'][0])),
|
||||
'lost-segment': lambda p: p['records'][0].update(translationCandidateSegments=[]),
|
||||
'forged-placeholder': lambda p: p['records'][0].update(placeholderExpressions=['private-value']),
|
||||
'forged-native-review': lambda p: p['records'][0].update(nativeReview='approved'),
|
||||
'forged-release': lambda p: p.update(status='released'),
|
||||
'changed-trace-line': lambda p: p['traces'][0]['evidence'][0].update(sourceExpression='forged'),
|
||||
'wrong-count': lambda p: p['counts'].update(records=0),
|
||||
'forged-prior-source-equivalence': lambda p: p['priorBatchSourceComparison']['B01']['files'][0].update(currentSha256='0' * 64),
|
||||
}
|
||||
for name, mutate in mutations.items():
|
||||
with self.subTest(name=name):
|
||||
payload = deepcopy(self.payload)
|
||||
mutate(payload)
|
||||
with self.assertRaises(AssertionError):
|
||||
census.validate(payload)
|
||||
|
||||
def test_diagnostic_example_and_prompt_fragments_are_not_translation_ready(self):
|
||||
for record in self.payload['records']:
|
||||
if record['boundary'] in {'diagnostic-template', 'synthetic-preview-example', 'generation-prompt-frame'}:
|
||||
self.assertFalse(record['translationCandidateSegments'])
|
||||
self.assertEqual(record['sourceLiteralSegments'], record['unresolvedSegments'])
|
||||
|
||||
def test_producer_and_consumer_evidence_retain_original_language_owners(self):
|
||||
traces = {trace['id']: trace for trace in self.payload['traces']}
|
||||
discovery = '\n'.join(line['sourceExpression'] for line in traces['discovery-masks-provider-detail']['evidence'])
|
||||
self.assertIn('jobDiscoveryUnavailable', discovery)
|
||||
self.assertIn('jobDiscoverySaveSearchFailed', discovery)
|
||||
letter = '\n'.join(line['sourceExpression'] for line in traces['letter-output-language-and-aria']['evidence'])
|
||||
self.assertIn('s.Language', letter)
|
||||
self.assertIn('aria-label', letter)
|
||||
imap = '\n'.join(line['sourceExpression'] for line in traces['imap-send-failure-boundary']['evidence'])
|
||||
self.assertIn('ex.Category', imap)
|
||||
self.assertIn('unsupported_provider', imap)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
Reference in New Issue
Block a user