You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Fix keys >= 2**63 when built with Cython 3.3, release v3.0.15
Cython 3.3 compiles a __getitem__/__setitem__/__delitem__ whose key is a
C integer into the sequence slots, so the key is converted through
Py_ssize_t. preshed 3.0.14 was built with Cython 3.3.0, and on every
Python version PreshMap raised 'OverflowError: Python int too large to
convert to C ssize_t' for any key >= 2**63, i.e. about half of all
64-bit hashes. This broke spaCy's tokenizer cache and serialization.
Take the key as an object and convert it to key_t explicitly, which
behaves the same under every Cython version. Out-of-range keys
(negative or >= 2**64) still raise OverflowError.