Skip to content

v0.4.0

Latest
Latest

Choose a tag to compare

View all tags
@markusmobius markusmobius released this 12 Sep 20:37
v0.4.0
1119e89

The initial Rust release, v0.4.0, tracks go-py3langid v0.4.0 and adbar/py3langid 0.4.0.

A pure-Rust language identification library with an embedded model for 139 languages plus zxx (non-linguistic content). No runtime Go, Python, NumPy, native C library, or model download is needed.

  • Raw and normalized scores, full rankings, language restrictions, confidence-based abstention, and custom model loading.
  • Thread-safe identifiers with portable, single-threaded inference optimizations. No internal classification threads or CPU-specific build flags are required.
  • Validated against 42 pinned Python fixtures and 68 Go reference cases, with exact Go score parity on those cases. Optimized and serial inference also match across all scores on the original 1,000-sample benchmark corpus.

Requires Rust 1.98.1. Install from the release tag:

[dependencies]
rust-py3langid = { git = "https://github.com/markusmobius/rust-py3langid", tag = "v0.4.0" }

This is a source-only GitHub release, matching the Go release format. No crates.io publication or prebuilt binaries are included. See the README for benchmarks, compatibility details, and platform validation.

Assets 2
Loading