Repository navigation
Release Notes: ctxfw v4.2.1 — Regulatory/OCR Precision, Full-Line Neutralization & Percent-Encoding Hardening
ctxfw v4.2.1 is a deterministic security and structural precision hardening release built on top of v4.2.0 (Sovereign Pentad + Wave 2 Blocks A–E). It introduces the SARA-RegDoc-60 offline Spanish regulatory/legal/OCR benchmark (100.00% Recall, 0.00% FPR, -62.52% OCR noise reduction, zero benchmark gaming), eliminates Red-Team neutralization leaks under STRIP_COMMANDS, expands 1..4-round percent-encoding detection with exact raw span mapping, hardens multilingual rule/policy override coverage across ES/EN/FR/DE/PT, and seals pure LF repository hygiene and cwd-independent CI tests.
For full empirical benchmarks, see docs/benchmarks/REGULATORY_LEGAL_OCR_BENCHMARK.md and docs/benchmarks/WAVE2_V42_SECURITY_LATENCY_BENCHMARK.md.
Highlights in v4.2.1
1. Zero-FPR Regulatory, Legal & Noisy OCR Disambiguation (ctxfw.sara — Zero Benchmark Gaming)
- Contractual Legal Repeal Disambiguation (
is_declarative_legal_repeal): Recognizes legitimate 3rd-person Spanish contractual repeal clauses (up to5intermediate legal locution tokens in[-128:start], e.g."El presente convenio deja sin efecto y anula las instrucciones previas...") atALLOW (0.00), while rejecting exemption if any agentic/system noun or imperative/modal hijack verb appears anywhere in[end : end + 180]across punctuation, prepositions, or sentence breaks. - POSIX Lowercase
idRestriction (_IMPERATIVE_RE): Prevents false positives on uppercase regulatory document headers (ID <folio>,ID: ...) while strictly blockingid,id -u,echo ok | id,`id`, and$(id). - Localized Metadata Header Lookahead (
LOCALIZED_ROLE_DIRECTIVE_LOOKAHEAD): Disambiguates passive metadata labels (\nAsistente:,\nSistema:) atALLOW (0.00)while catching localized role-override directives in ES/EN/FR/DE/PT (including French/Portuguese plural and formal imperatives:ignore[zr]?,montre*,envoi*,envoy*,mostre*,envie*,exiba*,imprima*) across up to2bounded lines (<= 240chars total, strictly linear time). - Numeric Folio Separation (
_PATH_RE): Excludes pure numeric document folios/dates (/2025/0819) while preserving full blocking of/etc/*,/var/*,../, and sensitive dotfiles (/.env,.env.*,.ssh/*,.aws/*).
2. Red-Team LexicalSanitizer.neutralize() & Homoglyph Command Hardening
- Complete Directive Line Neutralization (
STRIP_COMMANDS): Strips the entire malicious directive line ([^\n]*, or 2-line span when triggered on line 2) for\b(?:System|Assistant|Developer):and lookahead-validated localized headers (Sistema:,Asistente:,Desarrollador:,Système:,Entwickler:,Assistente:,Desenvolvedor:, etc.) while keepingSanitizationResult.role_override_markerscompact and backwards-compatible. - 1:1 Homoglyph Span Projection & Pure Cyrillic Command Detection: Uses exact length-preserving
fold_confusables(cleaned)(len(fold_confusables(s)) == len(s)) to strip homoglyph-spoofed role blocks (<ѕуѕтем>...</ѕуѕтем>,<\u0455ystem>) and folded directive lines (\nSiste\u043ca: ...), strips intra-token confusable tokens, adds"rm"to_CONFUSABLE_SECURITY_KEYWORDS, and inspectsfold_confusables(cmd_scan_text)insideInductionParser.parseso both mixed (r\u043c -rf *) and pure Cyrillic (\u0433\u043c -rf *) destructive commands escalate toQUARANTINE_TOOL. - Direct Rule/Policy/Guideline Overrides (ES/EN/FR/DE/PT): Covers direct overrides targeting rules, restrictions, policies, and guidelines (
ignora/olvida/omite/descarta/anula ... reglas/restricciones/políticas/directrices,ignore/disregard/forget/override ... rules/restrictions/policies/guidelines, and FR/DE/PT equivalents) identically inROLE_OVERRIDE_REand_ROLE_IGNORE_RE.
3. Bounded 1..4-Pass %XX Path Decoding & Raw Span Mapping (ctxfw.sara.induction)
- Any
%[0-9a-fA-F]{2}ASCII Escape Detection: Expands_ENCODED_PATH_HINT_REtor"%[0-9a-fA-F]{2}"so single-byte hex escapes (/%65tc/passwd,/.%65nv) and nested1..4-round percent-encodings (%2e%2e%2f,%252e%252e%252f,%25252e%25252e%25252f) converge cleanly via boundedurllib.parse.unquote(max 4 rounds, early stop on convergence). - Real
(raw_start, raw_end)Offset Mapping: Maps every decoded_PATH_REfinding back to its exact raw%XXtoken offsets inclean_text(never(0, 0)) and neutralizes full raw encoded tokens inLexicalSanitizer.neutralize(), while maintainingALLOW (0.00)on benign percentages (IVA 16%,100% verificado).
4. Reproducible LF Hygiene, cwd-Independent CI Tests & Standalone Packaging Parity
- Zero-
CRLFEnforcement: Enforcesnewline="\n"on report emitters (benchmarks/run_regulatory_ocr_benchmark.py,tests/conftest.py), pins-textin.gitattributesacrossv4.2.1artifacts, and normalizes repository text files to pureLF. - Working-Directory Isolation (
tests/test_ci_gatekeeper.py): EliminatesPath.cwd()dependency so runningpytestfrom outside the repository root passes reliably in< 2s. - Clean Distribution Artifacts: Clean
ctxfw-4.2.1-py3-none-any.whl,ctxfw-4.2.1.tar.gz, and standalonedist/ctxfw.exebundling all19.sqland.jsonschema/blueprint assets (13/13SQLite DDL checks verified).
Verification & Upgrade
# Install or upgrade from PyPI
pip install --upgrade ctxfw
# Check version, self-diagnostics, and specification axioms
ctxfw --version # ctxfw 4.2.1
ctxfw doctor # Verdict: [HEALTHY] [ATTESTED]
ctxfw spec verify SPEC.axioms.md # ACI Score: 1.0000 (VERIFIED)
pytest tests/ -q # 667/667 passed
# Run reproducible offline security & OCR benchmarks (zero network egress)
python benchmarks/run_regulatory_ocr_benchmark.py --json
python benchmarks/run_sara_security_benchmark.py --json
python benchmarks/run_wave2_v42_benchmark.py --json