DWT of read-only numpy arrays fails

[jhossbach]

I have a project that uses joblib, reading numpy arrays and applying a DWT on them.
This results in the error message ValueError: buffer source array is read-only. See below for the full error message

The issue stems from joblib using memmap on large numpy arrays, which makes their buffer read-only. This leads to errors on the Cython side if the function argument is not const-qualified.
This issue is known, see for example with pandas or in scikit-learn.

I am not too familiar with cython but based on the other PRs the fix should be simply to const-qualify

pywt/pywt/_extensions/_dwt.pyx

Line 26 in 587b8f0

cpdef dwt_single(cdata_t[::1] data, Wavelet wavelet, MODE mode):

i.e.

-cpdef dwt_single(cdata_t[::1] data, Wavelet wavelet, MODE mode):
+cpdef dwt_single(const cdata_t[::1] data, Wavelet wavelet, MODE mode):

joblib.externals.loky.process_executor._RemoteTraceback:
"""
Traceback (most recent call last):
  File "/work/jhossbach/test_project/.venv/lib/python3.13/site-packages/joblib/externals/loky/process_executor.py", line 490, in _process_worker
    r = call_item()
  File "/work/jhossbach/test_project/.venv/lib/python3.13/site-packages/joblib/externals/loky/process_executor.py", line 291, in __call__
    return self.fn(*self.args, **self.kwargs)
           ~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/jhossbach/test_project/.venv/lib/python3.13/site-packages/joblib/parallel.py", line 607, in __call__
    return [func(*args, **kwargs) for func, args, kwargs in self.items]
            ~~~~^^^^^^^^^^^^^^^^^
  File "/work/jhossbach/test_project/.venv/lib/python3.13/site-packages/okapi/detection/detectors.py", line 215, in _detect_chunk_parallel
    filtered_data = filter_obj(chunk) if filter_obj else chunk
                    ~~~~~~~~~~^^^^^^^
  File "/work/jhossbach/test_project/.venv/lib/python3.13/site-packages/okapi/filters/wavelet.py", line 124, in __call__
    coeffs = pywt.wavedec(data, self.wavelet, level=self.maxlev)
  File "/work/jhossbach/test_project/.venv/lib/python3.13/site-packages/pywt/_multilevel.py", line 103, in wavedec
    a, d = dwt(a, wavelet, mode, axis)
           ~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/jhossbach/test_project/.venv/lib/python3.13/site-packages/pywt/_dwt.py", line 182, in dwt
    cA, cD = dwt_single(data, wavelet, mode)
             ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
  File "pywt/_extensions/_dwt.pyx", line 26, in pywt._extensions._dwt.__pyx_fuse_0dwt_single
  File "<stringsource>", line 663, in View.MemoryView.memoryview_cwrapper
  File "<stringsource>", line 351, in View.MemoryView.memoryview.__cinit__
ValueError: buffer source array is read-only
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/work/jhossbach/test_project/test.py", line 20, in <module>
    events, rel_edges, abs_edges = detector.detect(time_series)
                                   ~~~~~~~~~~~~~~~^^^^^^^^^^^^^
  File "/work/jhossbach/test_project/.venv/lib/python3.13/site-packages/okapi/detection/detectors.py", line 366, in detect
    results = Parallel(n_jobs=self.n_jobs, backend="loky")(
        delayed(_detect_chunk_parallel)(
    ...<8 lines>...
        for chunk, offset in zip(chunks, offsets)
    )
  File "/work/jhossbach/test_project/.venv/lib/python3.13/site-packages/joblib/parallel.py", line 2072, in __call__
    return output if self.return_generator else list(output)
                                                ~~~~^^^^^^^^
  File "/work/jhossbach/test_project/.venv/lib/python3.13/site-packages/joblib/parallel.py", line 1682, in _get_outputs
    yield from self._retrieve()
  File "/work/jhossbach/test_project/.venv/lib/python3.13/site-packages/joblib/parallel.py", line 1784, in _retrieve
    self._raise_error_fast()
    ~~~~~~~~~~~~~~~~~~~~~~^^
  File "/work/jhossbach/test_project/.venv/lib/python3.13/site-packages/joblib/parallel.py", line 1859, in _raise_error_fast
    error_job.get_result(self.timeout)
    ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
  File "/work/jhossbach/test_project/.venv/lib/python3.13/site-packages/joblib/parallel.py", line 758, in get_result
    return self._return_or_raise()
           ~~~~~~~~~~~~~~~~~~~~~^^
  File "/work/jhossbach/test_project/.venv/lib/python3.13/site-packages/joblib/parallel.py", line 773, in _return_or_raise
    raise self._result
ValueError: buffer source array is read-only

[rgommers]

Thanks for the report @jhossbach. That assessment sounds right, unfortunately Cython makes this difficult. I suspect this issue with read-only arrays is not limited to that one location in the code, it'd need const additions in quite a few places probably.

[jhossbach]

I can take a look at it and open a PR if something comes out :)

[jhossbach]

MWE to reproduce this issue:

import numpy as np
from joblib import Parallel, delayed
import pywt

def worker_function(arr_slice):
    pywt.dwt(arr_slice, "db4")
    pywt.wavedec(arr_slice, "db4", level=2)

if __name__ == "__main__":
    data = np.random.randn(2_000_000).astype(np.float64)

    results = Parallel(n_jobs=2, backend="loky")(
        delayed(worker_function)(data[i : i + 1_000_000])
        for i in range(0, 2_000_000, 1_000_000)
    )

EDIT: Better version without joblib

import numpy as np
import pywt

data = np.random.randn(1000).astype(np.float64)

data.setflags(write=False)

pywt.dwt(data, "db4")
pywt.wavedec(data, "db4", level=2)

[rgommers]

Thanks @jhossbach, a PR with a test and the fix would be great 🙏🏼

[jhossbach]

Sure thing! I will restrict myself to solving the issue at hand, I was not able to reproduce this error with the other methods (dwt2, cwt, swt). I will do some more digging and submit a PR.