3.7.0
Changes
-
Native (C++) token/terminal matching — parsing is up to 2.2x faster.
The majority of token/terminal match decisions are now made natively in the
C++ Earley core instead of via Python callbacks, roughly doubling parsing
speed on fresh text (~89% of matching callbacks eliminated). Matching
results are bit-exact with the previous Python matcher, verified by a
query-level parity mode (GREYNIR_MATCHING_PARITY=1) and an extensive
test corpus. Derived parsers that override token wrapping (such as
GreynirCorrect) automatically continue to use Python matching. -
Python >= 3.10 is now required. Python 3.9 is end-of-life; users on
3.9 should stay on GreynirEngine 3.6.x. Binary wheels are now built as
cp310abi3 for CPython, plus PyPy 3.11 wheels. -
Robust interprocess locking and atomic grammar writes. The binary
grammar file is written atomically (temp file + rename), so an
interrupted process can never leave a truncated grammar behind. The
hand-rolled lock inglock.pyhas been replaced by the cross-platform
filelockpackage; the lock file now lives next to the binary grammar
instead of in/tmp, lock acquisition times out with a clear error
instead of hanging, and no lock is taken at all when the binary grammar
is already up to date. -
Hardened parser core. The C++ core now validates the binary grammar
file fully on load, catches out-of-memory conditions at the C ABI
boundary instead of crashing, uses race-free diagnostic counters, and
bounds the token matching cache (~125 MB) in long-running processes. -
Greynir()now raisesTypeErroron unrecognized keyword options
instead of silently ignoring them. Note that e.g.max_sent_tokensis a
parameter of the parse methods, not of the constructor. -
Modernized tooling. The project now uses
uvwith a committed
lockfile and PEP 735 dependency groups; PEP 639 license metadata;
mypy-clean type checking as a CI gate; and CI testing on Windows and
macOS in addition to Linux (CPython 3.10-3.14 and PyPy 3.11).