You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When running in a thread, the Blosc compressor encodes / decodes slower than normal. As discovered in pangeo-data/rechunker#35 (comment), this only affects numcodecs.Blosc. The relative slowdown does not appear when using blosc directly, which makes me think that numcodecs.Blosc may be holding the GIL (hopefully unnecessarily?)
Version and installation information
Please provide the following:
Value of numcodecs.__version__: 0.6.4
Version of Python interpreter: 3.8.5
Operating system (Linux/Windows/Mac): Linux / Mac
How NumCodecs was installed (e.g., "using pip into virtual environment", or "using conda"): conda
The text was updated successfully, but these errors were encountered:
Minimal, reproducible code sample, a copy-pastable example if possible
This example times an encode / decode cycle on some random bytes. It's slower by a factor of 2x - 3x when run in a thread pool.
Problem description
When running in a thread, the Blosc compressor encodes / decodes slower than normal. As discovered in pangeo-data/rechunker#35 (comment), this only affects
numcodecs.Blosc
. The relative slowdown does not appear when using blosc directly, which makes me think that numcodecs.Blosc may be holding the GIL (hopefully unnecessarily?)Version and installation information
Please provide the following:
numcodecs.__version__
: 0.6.4The text was updated successfully, but these errors were encountered: