[Core] Read-only buffer error in some scikit-learn models

[wingkitlee0]

What happened + What you expected to happen

I got some "buffer source array is read-only" error when using certain scikit-learn models.

• related to Cython / serialization? it seems to affect when the sklearn model is a C library.
• If I pickled the model explicity it works (see example)
• ray.put does not help
• multiprocessing.Pool would work; but Ray's Pool does not

ray.exceptions.RayTaskError(ValueError): ray::func() (pid=45108, ip=172.25.209.242)
  File "repro1.py", line 23, in func
    y_pred = model.predict_proba(X_eval)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "lib/python3.12/site-packages/sklearn/svm/_base.py", line 869, in predict_proba
    return pred_proba(X)
           ^^^^^^^^^^^^^
  File "lib/python3.12/site-packages/sklearn/svm/_base.py", line 909, in _dense_predict_proba
    pprob = libsvm.predict_proba(
            ^^^^^^^^^^^^^^^^^^^^^
  File "sklearn/svm/_libsvm.pyx", line 475, in sklearn.svm._libsvm.predict_proba
  File "stringsource", line 660, in View.MemoryView.memoryview_cwrapper
  File "stringsource", line 350, in View.MemoryView.memoryview.__cinit__
ValueError: buffer source array is read-only

So there is a workaround (pickle). but I am not sure if it's a Ray problem or scikit-learn. I cannot reproduce the same error in pure scikit-learn despite there were similar (now fixed) bugs in sklearn.

Versions / Dependencies

master

Reproduction script

The following script works fine when the pickling option is used.

import argparse
import pickle
import ray
from sklearn.datasets import make_classification
from sklearn.svm import SVC


def create_model():
    X, y = make_classification(n_samples=1000)

    model = SVC(kernel="linear", probability=True)
    model.fit(X, y)

    return model

@ray.remote
def func(model: SVC | bytes):
    if isinstance(model, bytes):
        model = pickle.loads(model)

    X_eval, _ = make_classification(n_samples=1000)

    y_pred = model.predict_proba(X_eval)

    return y_pred


def main():
    parser = argparse.ArgumentParser()
    parser.add_argument("-p", action="store_true", help="use pickle to load model")
    args = parser.parse_args()

    model = create_model()

    if args.p:
        model = pickle.dumps(model)

    refs = [
        func.remote(model)
        for _ in range(2)
    ]

    ray.get(refs)


if __name__ == "__main__":
    main()

Issue Severity

Medium: It is a significant difficulty but I can work around it.

[chuang0221]

I think the issue occurs because Ray's serialization system intentionally makes all buffers immutable (read-only) for distributed computing safety as shown in the python/ray/includes/serialization.pxi:

def __getbuffer__(self, Py_buffer* buffer, int flags):
    if flags & cpython.PyBUF_WRITABLE:
        # Ray ensures all buffers are immutable.
        raise BufferError
    buffer.readonly = self.readonly

[wingkitlee0]

Thanks for the info. Is it correct to say that it's a scikit-learn bug that those variables were not declared as read-only?

[chuang0221]

Well, I don't think so. I think it's an intentional design choice on both sides that happens to conflict:

• Ray intentionally makes all shared buffers read-only for memory safety in distributed computing.
• scikit-learn's libsvm implementation in Cython requires writable buffers for its internal operations during predict_proba. I think it's because it needs to perform calculations in-place for efficiency.

This type of conflict could occur with other libraries that use Cython or C extensions that expect writable buffers for in-place operations.

[jjyao]

@chuang0221 is right. If you need a writable one, you need to make a copy of it. Ray objects are immutable by default.