StringStore-related optimizations
#10938
Conversation
…types to `hash_t`
…`id` with `hash`
Add comment on early return
|
@explosion-bot please test_slow |
URL: https://buildkite.com/explosion-ai/spacy-slow-tests/builds/124 |
…ing-store-optimizations
|
@explosion-bot please test_slow_gpu |
URL: https://buildkite.com/explosion-ai/spacy-slow-gpu-tests/builds/49 |
|
I don't understand why the slow GPU tests succeed and the slow tests fail while building? Is there some difference in the setup that leads to this? (Accidentally not testing the right branches or something like that?) |
Yeah, I find that rather curious too. The error does point to some branch mismatch between the spacy and thinc codebases, but the version numbers ostensibly seem like they ought to line up? I'll take a closer look at it. |
…ing-store-optimizations
|
@explosion-bot please test_slow |
URL: https://buildkite.com/explosion-ai/spacy-slow-tests/builds/164 |
|
@explosion-bot please test_slow_gpu |
URL: https://buildkite.com/explosion-ai/spacy-slow-gpu-tests/builds/61 |
* `strings`: More roubust type checking of keys/IDs, coerce `int`-like types to `hash_t` * Preserve existing public API behaviour * Fix return type * Replace `bool` with `bint`, rename to `_try_coerce_to_hash`, replace `id` with `hash` * Avoid unnecessary re-encoding and re-calculation of strings and hashs respectively * Rename variables named `hash` Add comment on early return
Description
numpyinteger types to native hash/integer type to avoid comparison overhead.StringStore.__getitem__/__contains__for readability.StringStore.__contains__that calledstr.encodeon a non-strobject.The error handling changes have been deferred to a future PR that will target the
v4branch, allowing us to break backward-compatibility.Types of change
Performance enhancement
Checklist