[PROPOSAL] Adopt SPEC 0 and add scikit-learn dependency

[csadorf]

Overview

This proposal recommends to adopt SPEC 0 to inform cuml's dependencies and as part of this effort declare scikit-learn as a build and runtime dependency with a version constraint that is compatible with those outlined in the spec.

Motivation

Adopting SPEC 0 and properly declaring scikit-learn as a dependency will:

• Ensure better compatibility and integration with the broader scientific Python ecosystem
• Improve user experience by making dependency management more predictable
• Enable better integration with scikit-learn's ecosystem tools and utilities
• Reduce potential version conflicts and compatibility issues

Implementation Details

Compatibility with the declared version dependencies on scikit-learn would concretely mean:

• cuml can be installed into an environment with a compatible scikit-learn version
• cuml can be loaded and executed within a runtime environment together with a compatible scikit-learn version
• cuml's estimators follow the estimator guidelines and are thus compatible with scikit-learn's meta estimators, such as pipelines
• the cuml.accel mode can be used to accelerate all compatible scikit-learn versions
• accelerated estimators can be serialized such that they are executable by scikit-learn with a version that is identical to the one used to serialize it

[betatim]

Getting an overview of which tests fail/pass with different scikit-learn versions.

With scikit-learn 1.4.2 we get the following failures after making a small adjustment for an import statement (sklearn.utils.resample). There are two kinds of errors: assertion errors about two arrays not being close enough. This probably needs a bit of investigation about why this happens.

______________________________________________________________________ test_quantile_transform[numba-100-True-normal-100-0] ______________________________________________________________________

failure_logger = None
nan_filled_positive = (array([[       nan, 0.7656295 , 0.97988446, ..., 0.50598514, 0.28186977,
        0.78271047],
       [1.07918608, 0.6...       nan,        nan,
        0.40068517]]), <numba.cuda.cudadrv.devicearray.DeviceNDArray object at 0x7eb61c1b9310>)
axis = 0, n_quantiles = 100, output_distribution = 'normal', ignore_implicit_zeros = True, subsample = 100

    @pytest.mark.parametrize("axis", [0, 1])
    @pytest.mark.parametrize("n_quantiles", [30, 100])
    @pytest.mark.parametrize("output_distribution", ["uniform", "normal"])
    @pytest.mark.parametrize("ignore_implicit_zeros", [False, True])
    @pytest.mark.parametrize("subsample", [100])
    def test_quantile_transform(
        failure_logger,
        nan_filled_positive,  # noqa: F811
        axis,
        n_quantiles,
        output_distribution,
        ignore_implicit_zeros,
        subsample,
    ):
        X_np, X = nan_filled_positive

        t_X = cu_quantile_transform(
            X,
            axis=axis,
            n_quantiles=n_quantiles,
            output_distribution=output_distribution,
            ignore_implicit_zeros=ignore_implicit_zeros,
            subsample=subsample,
            random_state=42,
            copy=True,
        )
        assert type(t_X) is type(X)

        sk_t_X = sk_quantile_transform(
            X_np,
            axis=axis,
            n_quantiles=n_quantiles,
            output_distribution=output_distribution,
            ignore_implicit_zeros=ignore_implicit_zeros,
            subsample=subsample,
            random_state=42,
            copy=True,
        )

>       assert_allclose(t_X, sk_t_X)

test_preprocessing.py:1079:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/home/thead/miniforge3/envs/cuml-20250326/lib/python3.12/site-packages/cuml/testing/test_preproc_utils.py:344: in assert_allclose
    return np_assert_allclose(actual, desired, rtol=rtol, atol=atol)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

args = (<function assert_allclose.<locals>.compare at 0x7eb855c41bc0>, array([[        nan,  0.64847541,  1.45808954, ..., -0... [-3.51725010e+00,  5.93200768e-02,             nan, ...,
                    nan,             nan, -5.71744774e-01]]))
kwds = {'equal_nan': True, 'err_msg': '', 'header': 'Not equal to tolerance rtol=1e-05, atol=1e-05', 'verbose': True}

    @wraps(func)
    def inner(*args, **kwds):
        with self._recreate_cm():
>           return func(*args, **kwds)
E           AssertionError:
E           Not equal to tolerance rtol=1e-05, atol=1e-05
E
E           Mismatched elements: 6570 / 10000 (65.7%)
E           Max absolute difference: 3.73770219
E           Max relative difference: 114773.87490057
E            x: array([[      nan,  0.648475,  1.45809 , ..., -0.430727, -1.044634,
E                    0.644604],
E                  [ 1.813974,  0.268446,  1.820561, ...,       nan,       nan,...
E            y: array([[          nan,  3.261473e-01,  1.373025e+00, ..., -2.686549e-01,
E                   -1.164197e+00,  6.081318e-01],
E                  [ 1.813974e+00, -9.497520e-05,  2.480569e+00, ...,           nan,...

/home/thead/miniforge3/envs/cuml-20250326/lib/python3.12/contextlib.py:81: AssertionError

The only other kind of error is one about hyper-parameter value. This should be fairly straightforward to fix

______________________________________________________________________________ test_pca_svd_solver[covariance_eigh] ______________________________________________________________________________

pca_data = (array([[-2.07757468,  0.91264787,  0.02919547, ...,  1.34516818,
        -1.55860589,  0.89064467],
       [ 0.040361...   0, 0, 1, 0, 1, 1, 0, 0, 0, 1, 0, 1, 1, 1, 0, 0, 0, 0, 1, 1, 0, 1,
       0, 0, 1, 0, 0, 1, 1, 0, 0, 1, 0, 0, 1, 1]))
svd_solver = 'covariance_eigh'

    @pytest.mark.parametrize(
        "svd_solver", ["auto", "full", "arpack", "randomized", "covariance_eigh"]
    )
    def test_pca_svd_solver(pca_data, svd_solver):
        X, _ = pca_data
>       pca = PCA(n_components=5, svd_solver=svd_solver, random_state=42).fit(X)

accel/estimators_hyperparams/test_accel_pca.py:64:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/home/thead/miniforge3/envs/cuml-20250326/lib/python3.12/site-packages/sklearn/base.py:1467: in wrapper
    estimator._validate_params()
/home/thead/miniforge3/envs/cuml-20250326/lib/python3.12/site-packages/sklearn/base.py:666: in _validate_params
    validate_parameter_constraints(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

parameter_constraints = {'copy': ['boolean'], 'iterated_power': [<sklearn.utils._param_validation.StrOptions object at 0x7eb6db1a95e0>, <sklea...at 0x7eb6db1a94f0>, None], 'n_oversamples': [<sklearn.utils._param_validation.Interval object at 0x7eb6db1a9670>], ...}
params = {'copy': True, 'iterated_power': 'auto', 'n_components': 5, 'n_oversamples': 10, ...}, caller_name = 'PCA'

    def validate_parameter_constraints(parameter_constraints, params, caller_name):
        """Validate types and values of given parameters.

        Parameters
        ----------
        parameter_constraints : dict or {"no_validation"}
            If "no_validation", validation is skipped for this parameter.

            If a dict, it must be a dictionary `param_name: list of constraints`.
            A parameter is valid if it satisfies one of the constraints from the list.
            Constraints can be:
            - an Interval object, representing a continuous or discrete range of numbers
            - the string "array-like"
            - the string "sparse matrix"
            - the string "random_state"
            - callable
            - None, meaning that None is a valid value for the parameter
            - any type, meaning that any instance of this type is valid
            - an Options object, representing a set of elements of a given type
            - a StrOptions object, representing a set of strings
            - the string "boolean"
            - the string "verbose"
            - the string "cv_object"
            - the string "nan"
            - a MissingValues object representing markers for missing values
            - a HasMethods object, representing method(s) an object must have
            - a Hidden object, representing a constraint not meant to be exposed to the user

        params : dict
            A dictionary `param_name: param_value`. The parameters to validate against the
            constraints.

        caller_name : str
            The name of the estimator or function or method that called this function.
        """
        for param_name, param_val in params.items():
            # We allow parameters to not have a constraint so that third party estimators
            # can inherit from sklearn estimators without having to necessarily use the
            # validation tools.
            if param_name not in parameter_constraints:
                continue

            constraints = parameter_constraints[param_name]

            if constraints == "no_validation":
                continue

            constraints = [make_constraint(constraint) for constraint in constraints]

            for constraint in constraints:
                if constraint.is_satisfied_by(param_val):
                    # this constraint is satisfied, no need to check further.
                    break
            else:
                # No constraint is satisfied, raise with an informative message.

                # Ignore constraints that we don't want to expose in the error message,
                # i.e. options that are for internal purpose or not officially supported.
                constraints = [
                    constraint for constraint in constraints if not constraint.hidden
                ]

                if len(constraints) == 1:
                    constraints_str = f"{constraints[0]}"
                else:
                    constraints_str = (
                        f"{', '.join([str(c) for c in constraints[:-1]])} or"
                        f" {constraints[-1]}"
                    )

>               raise InvalidParameterError(
                    f"The {param_name!r} parameter of {caller_name} must be"
                    f" {constraints_str}. Got {param_val!r} instead."
                )
E               sklearn.utils._param_validation.InvalidParameterError: The 'svd_solver' parameter of PCA must be a str among {'randomized', 'auto', 'arpack', 'full'}. Got 'covariance_eigh' instead.

/home/thead/miniforge3/envs/cuml-20250326/lib/python3.12/site-packages/sklearn/utils/_param_validation.py:95: InvalidParameterError

Full list of failed tests from running pytest -sv --tb=auto --ignore=dask -m "not memleak"

Details

``` FAILED test_preprocessing.py::test_quantile_transformer[numpy-100-False-uniform-30] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[numpy-100-False-uniform-100] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[numpy-100-False-normal-30] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[numpy-100-False-normal-100] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[numpy-100-True-uniform-30] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[numpy-100-True-uniform-100] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[numpy-100-True-normal-30] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[numpy-100-True-normal-100] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[numpy-100-False-uniform-30-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[numpy-100-False-uniform-100-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[numpy-100-False-normal-30-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[numpy-100-False-normal-100-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[numpy-100-True-uniform-30-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[numpy-100-True-uniform-100-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[numpy-100-True-normal-30-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[numpy-100-True-normal-100-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[dataframe-100-False-uniform-30] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[dataframe-100-False-uniform-100] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[dataframe-100-False-normal-30] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[dataframe-100-False-normal-100] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[dataframe-100-True-uniform-30] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[dataframe-100-True-uniform-100] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[dataframe-100-True-normal-30] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[dataframe-100-True-normal-100] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[dataframe-100-False-uniform-30-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[dataframe-100-False-uniform-100-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[dataframe-100-False-normal-30-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[dataframe-100-False-normal-100-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[dataframe-100-True-uniform-30-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[dataframe-100-True-uniform-100-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[dataframe-100-True-normal-30-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[dataframe-100-True-normal-100-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[cupy-100-False-uniform-30] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[cupy-100-False-uniform-100] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[cupy-100-False-normal-30] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[cupy-100-False-normal-100] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[cupy-100-True-uniform-30] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[cupy-100-True-uniform-100] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[cupy-100-True-normal-30] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[cupy-100-True-normal-100] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[cupy-100-False-uniform-30-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[cupy-100-False-uniform-100-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[cupy-100-False-normal-30-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[cupy-100-False-normal-100-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[cupy-100-True-uniform-30-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[cupy-100-True-uniform-100-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[cupy-100-True-normal-30-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[cupy-100-True-normal-100-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[cudf-100-False-uniform-30] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[cudf-100-False-uniform-100] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[cudf-100-False-normal-30] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[cudf-100-False-normal-100] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[cudf-100-True-uniform-30] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[cudf-100-True-uniform-100] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[cudf-100-True-normal-30] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[cudf-100-True-normal-100] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[cudf-100-False-uniform-30-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[cudf-100-False-uniform-100-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[cudf-100-False-normal-30-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[cudf-100-False-normal-100-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[cudf-100-True-uniform-30-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[cudf-100-True-uniform-100-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[cudf-100-True-normal-30-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[cudf-100-True-normal-100-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[numba-100-False-uniform-30] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[numba-100-False-uniform-100] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[numba-100-False-normal-30] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[numba-100-False-normal-100] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[numba-100-True-uniform-30] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[numba-100-True-uniform-100] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[numba-100-True-normal-30] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[numba-100-True-normal-100] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[numba-100-False-uniform-30-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[numba-100-False-uniform-100-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[numba-100-False-normal-30-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[numba-100-False-normal-100-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[numba-100-True-uniform-30-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[numba-100-True-uniform-100-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[numba-100-True-normal-30-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[numba-100-True-normal-100-0] - AssertionError: FAILED accel/estimators_hyperparams/test_accel_pca.py::test_pca_svd_solver[covariance_eigh] - sklearn.utils._param_validation.InvalidParameterError: The 'svd_solver' parameter of PCA must be a str among {'randomized', 'auto', 'arpack', 'full'}. Got 'covariance_eigh' instead. ```

[betatim]

Running the tests with scikit-learn 1.3.2 and the resample fix gives a bigger list of failures.

Assertion errors are like for 1.4, and the PCA svd_solver error occurs here as well.

There is a new group of errors like the following which is related to sparse matrices. Maybe this is related to changes in scipy's sparse matrix handling (aka which version of scipy should we be using?)

__________________________________________________________________________ test_min_max_axis_extremes[cupy-csc-False-1] __________________________________________________________________________

sparse_extremes = (<Compressed Sparse Column sparse matrix of dtype 'float64'
	with 19 stored elements and shape (9, 3)>, <cupyx.scipy.sparse._csc.csc_matrix object at 0x7ba42fe38ce0>), axis = 1
ignore_nan = False

    @pytest.mark.parametrize("axis", [None, 0, 1])
    @pytest.mark.parametrize("ignore_nan", [False, True])
    # ignore warning about changing sparsity in both cupy and scipy
    @pytest.mark.filterwarnings("ignore:(.*)expensive(.*)::")
    # ignore warning about all nan row in sparse_extremes
    @pytest.mark.filterwarnings("ignore:All-NaN(.*)::")
    def test_min_max_axis_extremes(sparse_extremes, axis, ignore_nan):
        X_sparse_np, X_sparse = sparse_extremes

        cu_min, cu_max = cu_min_max_axis(
            X_sparse, axis=axis, ignore_nan=ignore_nan
        )
>       sk_min, sk_max = sk_min_max_axis(
            X_sparse_np, axis=axis, ignore_nan=ignore_nan
        )

test_thirdparty.py:226:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/home/thead/miniforge3/envs/cuml-20250326/lib/python3.12/site-packages/sklearn/utils/sparsefuncs.py:512: in min_max_axis
    return _sparse_min_max(X, axis=axis)
/home/thead/miniforge3/envs/cuml-20250326/lib/python3.12/site-packages/sklearn/utils/sparsefuncs.py:472: in _sparse_min_max
    _sparse_min_or_max(X, axis, np.minimum),
/home/thead/miniforge3/envs/cuml-20250326/lib/python3.12/site-packages/sklearn/utils/sparsefuncs.py:465: in _sparse_min_or_max
    return _min_or_max_axis(X, axis, min_or_max)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

X = <Compressed Sparse Column sparse matrix of dtype 'float64'
	with 19 stored elements and shape (9, 3)>, axis = 1, min_or_max = <ufunc 'minimum'>

    def _min_or_max_axis(X, axis, min_or_max):
        N = X.shape[axis]
        if N == 0:
            raise ValueError("zero-size array to reduction operation")
        M = X.shape[1 - axis]
        mat = X.tocsc() if axis == 0 else X.tocsr()
        mat.sum_duplicates()
        major_index, value = _minor_reduce(mat, min_or_max)
        not_full = np.diff(mat.indptr)[major_index] < N
        value[not_full] = min_or_max(value[not_full], 0)
        mask = value != 0
        major_index = np.compress(mask, major_index)
        value = np.compress(mask, value)

        if axis == 0:
            res = sp.coo_matrix(
                (value, (np.zeros(len(value)), major_index)), dtype=X.dtype, shape=(1, M)
            )
        else:
            res = sp.coo_matrix(
                (value, (major_index, np.zeros(len(value)))), dtype=X.dtype, shape=(M, 1)
            )
>       return res.A.ravel()
E       AttributeError: 'coo_matrix' object has no attribute 'A'

/home/thead/miniforge3/envs/cuml-20250326/lib/python3.12/site-packages/sklearn/utils/sparsefuncs.py:448: AttributeError

Details

``` ==================================================================================== short test summary info ===================================================================================== FAILED test_thirdparty.py::test_min_max_axis[cupy-csr-False-0] - AttributeError: 'coo_matrix' object has no attribute 'A' FAILED test_thirdparty.py::test_min_max_axis[cupy-csr-False-1] - AttributeError: 'coo_matrix' object has no attribute 'A' FAILED test_thirdparty.py::test_min_max_axis[cupy-csr-True-0] - AttributeError: 'coo_matrix' object has no attribute 'A' FAILED test_thirdparty.py::test_min_max_axis[cupy-csr-True-1] - AttributeError: 'coo_matrix' object has no attribute 'A' FAILED test_thirdparty.py::test_min_max_axis[cupy-csc-False-0] - AttributeError: 'coo_matrix' object has no attribute 'A' FAILED test_thirdparty.py::test_min_max_axis[cupy-csc-False-1] - AttributeError: 'coo_matrix' object has no attribute 'A' FAILED test_thirdparty.py::test_min_max_axis[cupy-csc-True-0] - AttributeError: 'coo_matrix' object has no attribute 'A' FAILED test_thirdparty.py::test_min_max_axis[cupy-csc-True-1] - AttributeError: 'coo_matrix' object has no attribute 'A' FAILED test_preprocessing.py::test_maxabs_scaler_sparse[scipy-csr] - AttributeError: 'coo_matrix' object has no attribute 'A' FAILED test_preprocessing.py::test_normalizer_sparse[scipy-csr-max] - AttributeError: 'coo_matrix' object has no attribute 'A' FAILED test_preprocessing.py::test_normalize_sparse[scipy-csr-max] - AttributeError: 'coo_matrix' object has no attribute 'A' FAILED test_preprocessing.py::test_maxabs_scaler_sparse[scipy-csc] - AttributeError: 'coo_matrix' object has no attribute 'A' FAILED test_preprocessing.py::test_normalize_sparse[scipy-csc-max] - AttributeError: 'coo_matrix' object has no attribute 'A' FAILED test_preprocessing.py::test_maxabs_scaler_sparse[cupy-csr] - AttributeError: 'coo_matrix' object has no attribute 'A' FAILED test_preprocessing.py::test_normalizer_sparse[cupy-csr-max] - AttributeError: 'coo_matrix' object has no attribute 'A' FAILED test_preprocessing.py::test_normalize_sparse[cupy-csr-max] - AttributeError: 'coo_matrix' object has no attribute 'A' FAILED test_preprocessing.py::test_maxabs_scaler_sparse[cupy-csc] - AttributeError: 'coo_matrix' object has no attribute 'A' FAILED test_preprocessing.py::test_normalize_sparse[cupy-csc-max] - AttributeError: 'coo_matrix' object has no attribute 'A' FAILED test_device_selection.py::test_umap_methods[cpu] - AttributeError: 'coo_matrix' object has no attribute 'A' FAILED test_device_selection.py::test_umap_methods[gpu] - AttributeError: 'coo_matrix' object has no attribute 'A' FAILED test_preprocessing.py::test_quantile_transformer[numpy-100-False-uniform-30] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[numpy-100-False-uniform-100] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[numpy-100-False-normal-30] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[numpy-100-False-normal-100] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[numpy-100-True-uniform-30] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[numpy-100-True-uniform-100] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[numpy-100-True-normal-30] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[numpy-100-True-normal-100] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[numpy-100-False-uniform-30-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[numpy-100-False-uniform-100-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[numpy-100-False-normal-30-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[numpy-100-False-normal-100-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[numpy-100-True-uniform-30-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[numpy-100-True-uniform-100-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[numpy-100-True-normal-30-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[numpy-100-True-normal-100-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[dataframe-100-False-uniform-30] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[dataframe-100-False-uniform-100] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[dataframe-100-False-normal-30] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[dataframe-100-False-normal-100] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[dataframe-100-True-uniform-30] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[dataframe-100-True-uniform-100] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[dataframe-100-True-normal-30] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[dataframe-100-True-normal-100] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[dataframe-100-False-uniform-30-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[dataframe-100-False-uniform-100-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[dataframe-100-False-normal-30-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[dataframe-100-False-normal-100-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[dataframe-100-True-uniform-30-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[dataframe-100-True-uniform-100-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[dataframe-100-True-normal-30-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[dataframe-100-True-normal-100-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[cupy-100-False-uniform-30] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[cupy-100-False-uniform-100] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[cupy-100-False-normal-30] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[cupy-100-False-normal-100] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[cupy-100-True-uniform-30] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[cupy-100-True-uniform-100] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[cupy-100-True-normal-30] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[cupy-100-True-normal-100] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[cupy-100-False-uniform-30-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[cupy-100-False-uniform-100-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[cupy-100-False-normal-30-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[cupy-100-False-normal-100-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[cupy-100-True-uniform-30-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[cupy-100-True-uniform-100-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[cupy-100-True-normal-30-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[cupy-100-True-normal-100-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[cudf-100-False-uniform-30] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[cudf-100-False-uniform-100] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[cudf-100-False-normal-30] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[cudf-100-False-normal-100] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[cudf-100-True-uniform-30] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[cudf-100-True-uniform-100] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[cudf-100-True-normal-30] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[cudf-100-True-normal-100] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[cudf-100-False-uniform-30-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[cudf-100-False-uniform-100-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[cudf-100-False-normal-30-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[cudf-100-False-normal-100-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[cudf-100-True-uniform-30-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[cudf-100-True-uniform-100-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[cudf-100-True-normal-30-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[cudf-100-True-normal-100-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[numba-100-False-uniform-30] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[numba-100-False-uniform-100] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[numba-100-False-normal-30] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[numba-100-False-normal-100] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[numba-100-True-uniform-30] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[numba-100-True-uniform-100] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[numba-100-True-normal-30] - AssertionError: FAILED test_preprocessing.py::test_quantile_transformer[numba-100-True-normal-100] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[numba-100-False-uniform-30-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[numba-100-False-uniform-100-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[numba-100-False-normal-30-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[numba-100-False-normal-100-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[numba-100-True-uniform-30-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[numba-100-True-uniform-100-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[numba-100-True-normal-30-0] - AssertionError: FAILED test_preprocessing.py::test_quantile_transform[numba-100-True-normal-100-0] - AssertionError: FAILED test_thirdparty.py::test_min_max_axis_extremes[cupy-csr-False-0] - AttributeError: 'coo_matrix' object has no attribute 'A' FAILED test_thirdparty.py::test_min_max_axis_extremes[cupy-csr-False-1] - AttributeError: 'coo_matrix' object has no attribute 'A' FAILED test_thirdparty.py::test_min_max_axis_extremes[cupy-csr-True-0] - AttributeError: 'coo_matrix' object has no attribute 'A' FAILED test_thirdparty.py::test_min_max_axis_extremes[cupy-csr-True-1] - AttributeError: 'coo_matrix' object has no attribute 'A' FAILED test_thirdparty.py::test_min_max_axis_extremes[cupy-csc-False-0] - AttributeError: 'coo_matrix' object has no attribute 'A' FAILED test_thirdparty.py::test_min_max_axis_extremes[cupy-csc-False-1] - AttributeError: 'coo_matrix' object has no attribute 'A' FAILED test_thirdparty.py::test_min_max_axis_extremes[cupy-csc-True-0] - AttributeError: 'coo_matrix' object has no attribute 'A' FAILED test_thirdparty.py::test_min_max_axis_extremes[cupy-csc-True-1] - AttributeError: 'coo_matrix' object has no attribute 'A' FAILED accel/test_pipeline.py::test_umap_with_logistic_regression - AttributeError: 'coo_matrix' object has no attribute 'A' FAILED accel/estimators_hyperparams/test_accel_pca.py::test_pca_svd_solver[covariance_eigh] - sklearn.utils._param_validation.InvalidParameterError: The 'svd_solver' parameter of PCA must be a str among {'full', 'randomized', 'auto', 'arpack'}. Got 'covariance_eigh' instead. ERROR test_device_selection.py::test_train_cpu_infer_cpu[/umap_test_data-spectral-cupy-2] - AttributeError: 'coo_matrix' object has no attribute 'A' ERROR test_device_selection.py::test_train_gpu_infer_cpu[/umap_test_data-spectral-cupy-2] - AttributeError: 'coo_matrix' object has no attribute 'A' ERROR test_device_selection.py::test_train_cpu_infer_gpu[/umap_test_data-spectral-cupy-2] - AttributeError: 'coo_matrix' object has no attribute 'A' ERROR test_device_selection.py::test_train_gpu_infer_gpu[/umap_test_data-spectral-cupy-2] - AttributeError: 'coo_matrix' object has no attribute 'A' ERROR test_device_selection.py::test_pickle_interop[/umap_test_data-spectral-cupy-2] - AttributeError: 'coo_matrix' object has no attribute 'A' ERROR test_device_selection.py::test_train_cpu_infer_cpu[/umap_test_data-spectral-cupy-16] - AttributeError: 'coo_matrix' object has no attribute 'A' ERROR test_device_selection.py::test_train_gpu_infer_cpu[/umap_test_data-spectral-cupy-16] - AttributeError: 'coo_matrix' object has no attribute 'A' ERROR test_device_selection.py::test_train_cpu_infer_gpu[/umap_test_data-spectral-cupy-16] - AttributeError: 'coo_matrix' object has no attribute 'A' ERROR test_device_selection.py::test_train_gpu_infer_gpu[/umap_test_data-spectral-cupy-16] - AttributeError: 'coo_matrix' object has no attribute 'A' ERROR test_device_selection.py::test_pickle_interop[/umap_test_data-spectral-cupy-16] - AttributeError: 'coo_matrix' object has no attribute 'A' ERROR test_device_selection.py::test_train_cpu_infer_cpu[/umap_test_data-random-cupy-2] - AttributeError: 'coo_matrix' object has no attribute 'A' ERROR test_device_selection.py::test_train_gpu_infer_cpu[/umap_test_data-random-cupy-2] - AttributeError: 'coo_matrix' object has no attribute 'A' ERROR test_device_selection.py::test_train_cpu_infer_gpu[/umap_test_data-random-cupy-2] - AttributeError: 'coo_matrix' object has no attribute 'A' ERROR test_device_selection.py::test_train_gpu_infer_gpu[/umap_test_data-random-cupy-2] - AttributeError: 'coo_matrix' object has no attribute 'A' ERROR test_device_selection.py::test_pickle_interop[/umap_test_data-random-cupy-2] - AttributeError: 'coo_matrix' object has no attribute 'A' ERROR test_device_selection.py::test_train_cpu_infer_cpu[/umap_test_data-random-cupy-16] - AttributeError: 'coo_matrix' object has no attribute 'A' ERROR test_device_selection.py::test_train_gpu_infer_cpu[/umap_test_data-random-cupy-16] - AttributeError: 'coo_matrix' object has no attribute 'A' ERROR test_device_selection.py::test_train_cpu_infer_gpu[/umap_test_data-random-cupy-16] - AttributeError: 'coo_matrix' object has no attribute 'A' ERROR test_device_selection.py::test_train_gpu_infer_gpu[/umap_test_data-random-cupy-16] - AttributeError: 'coo_matrix' object has no attribute 'A' ERROR test_device_selection.py::test_pickle_interop[/umap_test_data-random-cupy-16] - AttributeError: 'coo_matrix' object has no attribute 'A' ```

[betatim]

Another note: if we follow SPEC0 we should support 1.3 until about June 2025. The next release of cuml will be in June. So we should think about whether it is worth the effort to support 1.3 given that we (strictly speaking) could drop support for it shortly after.

[csadorf]

Another note: if we follow SPEC0 we should support 1.3 until about June 2025. The next release of cuml will be in June. So we should think about whether it is worth the effort to support 1.3 given that we (strictly speaking) could drop support for it shortly after.

I think that depends on whether it would require significant effort.

[betatim]

The sparse matrix related errors go away if you use a scipy version "of the same vintage" as scikit-learn 1.3, namely 1.11.4. Earlier versions don't have support for Python 3.12 which is required by raft.

[csadorf]

For version 25.08 RAPIDS supports Python 3.10,<3.14.
For version 25.08 cuML supports numpy >1.23,<3.0a0.
There are no version constraints on the matplotlib and pandas (test) dependencies.

cuDF has a runtime dependency on pandas >=2.0,<2.4.0dev0

I believe that we can close this issue for now since we are in compliance with at least the support window outlined by SPEC 0. We should re-evaluate dropping support for cuML-specific dependencies following the Drop Schedule outlined by SPEC 0 with each new release.