Blosc compressor slower when running in a thread

[TomAugspurger]

Minimal, reproducible code sample, a copy-pastable example if possible

This example times an encode / decode cycle on some random bytes. It's slower by a factor of 2x - 3x when run in a thread pool.

import numcodecs
import time
import numpy as np
import concurrent.futures

def encdec():
    times = []
    for i in range(5):
        x = np.random.RandomState(0).random(10_000_0000).tobytes()    
        compressor = numcodecs.Blosc()
        t0 = time.time()
        a = compressor.encode(x)
        t1 = time.time()
        b = compressor.decode(a)
        t2 = time.time()
        times.append((t1 - t0, t2 - t1))

    return times


>>> local = encdec()
>>> tpool = concurrent.futures.ThreadPoolExecutor(max_workers=1)
>>> thread  = tpool.submit(encdec).result()

>>> np.mean(local, 0) / np.mean(thread, 0)
array([0.43173625, 0.34887189])

Problem description

When running in a thread, the Blosc compressor encodes / decodes slower than normal. As discovered in pangeo-data/rechunker#35 (comment), this only affects numcodecs.Blosc. The relative slowdown does not appear when using blosc directly, which makes me think that numcodecs.Blosc may be holding the GIL (hopefully unnecessarily?)

Version and installation information

Please provide the following:

• Value of numcodecs.__version__: 0.6.4
• Version of Python interpreter: 3.8.5
• Operating system (Linux/Windows/Mac): Linux / Mac
• How NumCodecs was installed (e.g., "using pip into virtual environment", or "using conda"): conda