CodecCat v0.1.1 — Stable

CodecCat v0.1.1 — Opt-in decoding ambiguity diagnostics

This is a backward-compatible patch to the native codeccat Runtime. The released default inference model remains byte-for-byte identical to v0.1.0 (SHA256 16073306b5027bf93acc2883f57fbda0cb0cf2d77b9a7bb62cf1b07fcbb9748b). The established detect(), decode(), Detection fields, CLI and BOM/L3 policies are unchanged.

New: Optional diagnostic for potentially ambiguous C1 controls

from codeccat import detect
from codeccat.diagnostics import inspect_ambiguity

raw = b"Price: \x80; total."
result = detect(raw)
diagnostic = inspect_ambiguity(raw, result)
if diagnostic.warning:
    print(diagnostic.reason, diagnostic.alternative_encodings)

inspect_ambiguity(raw, result) returns immutable metadata (warning, reason, alternative_encodings). It checks whether the already-selected decoding includes Unicode C1 controls (U+0080–U+009F), and if one of the already-ranked codecs can strictly decode the original bytes into different, C1-free text. It does not change or substitute the selected codec, decoded string, candidates, Detection.ambiguous, BOM decision or opt-in L3 state. It has no new ML dependencies.

Interpretation: a warning is a request to consult a trusted encoding or file metadata, not proof of wrong decoding and not a model confidence probability. Legal Latin-1 data may intentionally contain C1 controls. No warning does not imply correctness.

Validation and caveats

• Clean-installed Wheel/API/model identity checks passed on Linux, macOS and Windows for x86-64 and ARM64; Python 3.10–3.14 supported.
• Examined 1,886 historical original HTML files after provenance and training-overlap screening. Of the 572 files whose Unicode text differed from the archive-directory codec's strict decoding, 17 were flagged; none of 1,314 directory-label-agreeing results were flagged. This is only source-label agreement, not manually verified ground truth or an estimated population false-positive rate.
• Replaced per-character Python C1 scan with compiled regex, reducing incremental diagnostic call p50 approximately 10–12× on this original HTML test set. Times are Linux CI sample measurements, not universal latency guarantees.
• See research protocol and validation record. No original HTML or copyrighted training corpus is bundled.

Distribution

This release adds the tested source distribution (sdist) to the six platform-specific Wheels: manylinux_2_28 x86-64/ARM64, macOS x86-64/ARM64, and Windows x86-64/ARM64. The frozen default CCM model and optional L3 artifact are verified in every asset. Official PyPI Trusted Publishing is a separate action after GitHub release and asset audits.

codeccat-train==0.1.0 is a separate, already published training package and is not changed by this Runtime patch.

The project's historical training-data rights caveat remains described in RIGHTS_AND_PROVENANCE.md.

CodecCat v0.1.0 — Stable

CodecCat v0.1.0 — First Stable Release

CodecCat is a native Python character-encoding detector and strict Unicode decoder. This release formalizes its 0.1.x API compatibility contract and improves distribution portability without retraining the model or changing the proven v0.1.0a5 decoder logic.

Stable behavior and compatibility

• Supports Python 3.10–3.14 on released Linux, macOS, and Windows targets.
• Public CodecCat, detect, decode, Detection fields, BOM policy and CLI flags have a documented compatibility contract: docs/STABILITY.md.
• Invalid decoding is not silently replaced. Encoding inference itself cannot guarantee unique correctness for arbitrary bytes.
• Native inference has no NumPy/scikit-learn dependencies; legacy API compatibility remains available as an optional extra.

Portability and release engineering

• Genuine manylinux_2_28 Linux Wheels for x86-64 and ARM64, built and tested in corresponding policy containers.
• macOS x86-64/ARM64 and Windows x86-64/ARM64 Wheels tested on their own GitHub-hosted runners.
• Frozen default model SHA256 is identical across six Wheels. Every asset has a checksum in SHA256SUMS and metadata in MODEL_PROVENANCE.json.
• Wheel build uses verified PEP-517 packaging to include the model files; avoids the previous bug where a direct maturin build omitted its bundled model.

Accuracy evidence and limitations

The decoder model and inference algorithm are unchanged from v0.1.0a5. That version scored 826/843 (97.98%) correctly decoded text on a fixed engineering benchmark, compared with chardet's 829/843 (98.34%). The tested 843 examples include development and training sources, so this is not an independently representative real-world accuracy rate.

Codec identification of sufficiently ambiguous bytes cannot be universally perfect. The inference model was derived from third-party text with mixed upstream rights; rights and provenance remain documented and are not an assertion of legal clearance.

Install: select the Wheel for your OS and CPU from the assets below. On supported Linux/Python versions, the manylinux_2_28 Wheels are preferable to the earlier plain linux_* Alpha files. The release is distributed on GitHub; a PyPI upload is a separate action and not claimed here.

CodecCat v0.1.0a5 — Native Alpha

Native Alpha: more reliable UTF-8 decoding of text containing only a few non-ASCII characters.

• Conservative evidence threshold reduced from 16 to 3 non-ASCII Unicode characters, while retaining strict UTF-8 validation, BOM handling, 64-byte minimum, control-character guard and L3 opt-in behavior.
• Exact frozen model bytes unchanged from v0.1.0a4. No new runtime dependencies or public API changes.
• Six platform Wheels (Linux/macOS/Windows, x86-64/ARM64) passed built-and-installed CI: https://github.com/Johnny-Kao/CodecCat/actions/runs/42157726285
• Python 3.10-3.14, Root Wheel, public API all PASS: #12
• Final post-release 843-case comparison installed the actual release Wheel: CodecCat 826/843 (97.98%) correct Unicode text, chardet 829/843 (98.34%), charset-normalizer 698/843 (82.80%). https://github.com/Johnny-Kao/CodecCat/actions/runs/42158308518
• Before/after held-out source check (same bundled model): GNOME Software 198/225 -> 216/225 correctly decoded, 18 helps and 0 observed harms. https://github.com/Johnny-Kao/CodecCat/actions/runs/42157052821
• Model provenance and SHA256SUMS included for all six released Wheels.

Caveat: The 843-case corpus includes training/development data and is NOT independent population accuracy. Encoding recognition of arbitrary bytes can be ambiguous; new sources cannot prove all legacy codec behavior. CPU/latency rankings are single-run observations. This release is experimental Alpha, not Stable. No PyPI publication.

Methods: https://github.com/Johnny-Kao/CodecCat/blob/main/docs/BENCHMARKS.md

CodecCat v0.1.0a4 — Native Alpha

Native Alpha: improved UTF-8 text recovery.

New in v0.1.0a4:

• Conservative full-string UTF-8 evidence can correct some legacy-charset misclassifications when bytes provide strong Unicode evidence.
• BOM always takes precedence. Strict decoding and optional L3 behavior remain unchanged.
• No model retraining, extra runtime dependencies, or public API signature changes.

Official published-artifact validation:

• 6 platforms, including Linux, macOS, and Windows x86-64 / ARM64, all PASS: https://github.com/Johnny-Kao/CodecCat/actions/runs/42150439965
• Five source-stratified corpora, 843 samples, published v0.1.0a4: 819/843 correctly decoded Unicode text (97.15%). chardet 829/843 (98.34%); charset-normalizer 698/843 (82.80%): https://github.com/Johnny-Kao/CodecCat/actions/runs/42151051728
• Published v0.1.0a3 vs v0.1.0a4 on 285 GLib translation windows: 227 correct -> 271 correct, 44 helps and zero observed harms. Both release wheels contain identical default model SHA256 16073306b5027bf93acc2883f57fbda0cb0cf2d77b9a7bb62cf1b07fcbb9748b, so this is a paired decoder-only improvement: https://github.com/Johnny-Kao/CodecCat/actions/runs/42151536276

Install the Wheel matching your operating system and CPU; Python 3.10+. The release includes SHA256SUMS and MODEL_PROVENANCE.json. Linux wheels are platform-tagged rather than repaired manylinux wheels.

These benchmarks contain training/development overlap or cover one external domain, so they are not independent representative real-world accuracy claims. CPU and latency are single-run observations. Complete caveats and reproduction:
https://github.com/Johnny-Kao/CodecCat/blob/main/research/NATIVE_UTF8_FIX_2026-11-06.md

GitHub Alpha release only; no PyPI upload.

CodecCat v0.1.0a3 — Native Alpha

First Rust-native CodecCat Alpha.

Download the wheel for your operating system and CPU (Linux, macOS, Windows; x86-64 or ARM64), then install the downloaded .whl file with python -m pip install ./filename.whl. Python 3.10+.

What changed:

• Rust/PyO3 inference with a bundled CCM2 model; no NumPy, SciPy or scikit-learn required for standard inference.
• Existing detect(), decode(), CodecCat and CLI workflows retained.
• Optional legacy compatibility and separately installable offline trainer.
• Six platform wheels passed clean installation, API parity and reliability checks.

Validation: https://github.com/Johnny-Kao/CodecCat/actions/runs/42119685359

The release includes SHA256SUMS and MODEL_PROVENANCE.json. This is an experimental Alpha, not a general accuracy guarantee. Linux wheels are tagged for the build Linux platform rather than repaired manylinux wheels; use compatible systems. No PyPI publication.

CodecCat v0.1.0a2 (Alpha)

Experimental alpha release with a model-bearing installable wheel.

Includes: default stable23 inference model, opt-in L3, Python API and CLI.

Verified: source unit tests, clean wheel installation, real Python/CLI decoding, UTF BOM regression tests.

Not a production release. Accuracy varies by source; trained-model redistribution provenance is still under review. No PyPI distribution is implied.

CodecCat v0.1.0a1 (Alpha)

Experimental alpha release with a model-bearing installable wheel.

Includes: default stable23 inference model, opt-in L3, Python API and CLI.

Verified: source unit tests, clean wheel installation, real Python/CLI decoding, UTF BOM regression tests.

Not a production release. Accuracy varies by source; trained-model redistribution provenance is still under review. No PyPI distribution is implied.