[MNT] Python 3.13 compatibility and upstream dependency maintenance by WilliamJudge94 · Pull Request #30 · sktime/pycaret

WilliamJudge94 · 2026-08-08T05:45:32Z

Fully passing CI (https://github.com/WilliamJudge94/sktime-pycaret/actions/runs/31238435952)

Summary

This PR brings the fork's Python 3.13 compatibility work together with upstream dependency maintenance. Local main now contains sktime/pycaret:main through 7e766c1a, then adds the fork-specific compatibility, CI, test, and dependency constraints required to keep the project working on Python 3.10–3.13.

What changed and why

Python 3.13 and dependency compatibility

Added Python 3.13 support across package metadata, runtime guards, and CI.
Added Python-version dependency markers for NumPy, pandas, SciPy, scikit-learn, LightGBM, Numba, Trio, Matplotlib, SHAP, XGBoost, pmdarima, and scikit-learn-intelex. These constraints select dependency versions with Python 3.13 wheels and compatible APIs rather than using one range for all supported Python releases.
Added compatibility fixes for NumPy 2, current scikit-learn validation/metric APIs, SHAP output shapes, Moto, MLflow/cloudpickle serialization, Joblib cache behavior, and optional dependencies.
Disabled BATS/TBATS where NumPy 2 compatibility is unavailable, preventing model discovery from failing during setup.

Runtime and time-series compatibility

Added a scikit-learn compatibility layer and updated preprocessing, metrics, caching, experiment setup, and parallel/logging integrations to support newer dependency APIs.
Restored legacy forecasting-horizon split behavior and supports both current and legacy sktime exogenous-variable tags. This preserves the fork's expected train/test semantics while remaining compatible with newer sktime tag names.
Added or updated tests for cache persistence, MLflow artifacts, model behavior, Python 3.13 clustering-engine availability, BATS/TBATS handling, forecasting tags, and pipeline tag propagation.

CI and upstream synchronization

Kept the fork's Python 3.13 CI matrix, notebook coverage, and platform safeguards, including the macOS/Python-3.13 daal4py guard.
Merged upstream through 7e766c1a, establishing shared ancestry so these dependency-maintenance commits will not need to be re-integrated as a future divergence.
Retained upstream removals of direct markupsafe, importlib_metadata, nbformat, and deprecation requirements.

Why selected upstream changes were rolled back locally

feature/merge_deps_removal merged upstream/main at merge commit 58c9cff1. This establishes shared ancestry through upstream commit 7e766c1a, preventing the same six upstream commits from being re-integrated as a future divergence.

The merge preserved this branch's existing Python 3.13 CI matrices and daal4py guard. Two deliberate follow-up commits restore behavior that upstream removed but this fork requires:

e638bea7 — bug: retain numba test safeguard
5f45a6bf — bug: retain wurlitzer logging

Upstream commit disposition

Upstream commit	Subject	Decision	Local outcome and rationale
`805629fb`	`[MNT] remove stale.yml workflow which automatically closes issues or PR (#17)`	Kept	The stale issue/PR workflow was removed as upstream intended.
`635416f0`	`[MNT] fix CI - use uv, remove legacy builds (#15)`	Adapted	Its upstream ancestry is retained. The branch kept its already-working Python 3.13 CI matrix and stronger `pytest.importorskip("daal4py")`/macOS guard rather than replacing them with the older workflow and import changes. The upstream test extra addition (`pytest-timeout`) is retained through the merge.
`345f9581`	`[MNT] remove unnecessary dependencies - numba (#23)`	Rewritten	The upstream removal was accepted by the merge, then `e638bea7` restored Numba's Python-version dependency declarations, version reporting, and the `disable_numba` test fixture. Docker Python 3.13 testing showed that deleting this fixture causes PyOD ABOD to raise `numba.core.errors.TypingError`; keeping it prevents that regression.
`8ad0d2a0`	`[MNT] remove unnecessary deps - notebook/colab related packages (#22)`	Kept	Removed direct `markupsafe`, `importlib_metadata`, and `nbformat` requirements while retaining the branch's Python 3.13 dependency markers. Targeted Docker tests passed.
`b1727ca1`	`[MNT] remove unnecessary deps - wurlitzer (#21)`	Rewritten	The upstream removal was accepted by the merge, then `5f45a6bf` restored `wurlitzer`, its version report entry, and `redirect_output`'s C/child-process stdout/stderr capture. This fork intentionally keeps that logging capability.
`7e766c1a`	`[MNT] Remove unused packages - deprecations (#25)`	Kept	Removed the direct `deprecation` requirement. Targeted Docker install/import checks passed.

Comparison context and remaining risk

At the time of comparison, local main was 65 commits ahead and 0 commits behind sktime/pycaret:main; all remaining changes are fork-specific. The highest-risk areas are the broad Python-version dependency matrix and the custom legacy sktime forecasting split. Full CI across Python 3.10–3.13 remains important to validate the permitted dependency combinations and platform behavior.

Conflict resolutions

File	Resolution
`.github/workflows/test.yml`	Kept the local workflow because it already uses `uv` and contains the branch's Python 3.13-focused CI configuration.
`tests/test_clustering_engines.py`	Kept the local `pytest.importorskip("daal4py")`, macOS/Python-3.13 guard, and module-based assertions.
`pyproject.toml`	Retained local Python-version markers. Applied safe upstream removals; restored Numba and Wurlitzer through the separate override commits.

Validation

The integrated branch was tested in Docker using python:3.13-slim with libgomp1 installed for LightGBM:

Python 3.13.14
uv pip install --system --no-cache ".[test]"  # passed
python -m pip check                           # passed
redirect_output() context smoke test          # passed
python -m pytest -q tests/test_models.py      # 5 passed

The full CI matrix was not run locally. The Docker validation specifically confirms the two protected regressions (Numba fixture and Wurlitzer logging) alongside the dependency merge. CI should validate the full supported Python 3.10–3.13 matrix before merge.

- Update requires-python to >=3.9,<3.14 and add 3.13 classifier - Replace deprecated distutils.version.LooseVersion with packaging.version.Version - Fix NumPy 2.0 deprecations (np.NaN -> np.nan, np.product -> np.prod) - Fix sklearn 1.6+ API changes (validate_data, root_mean_squared_error, _check_reg_targets signature, IterativeImputer._validate_limit) - Fix joblib Memory API change (bytes_limit moved to reduce_size()) - Add Python version markers for dependencies requiring newer versions on 3.13 - Add Python 3.13 to GitHub Actions test matrix - Update README.md with Python 3.13 support Closes #4121

…mpatibility The yellowbrick library's CooksDistance visualizer uses the deprecated 'use_line_collection' parameter in ax.stem() which was removed in matplotlib 3.8. This adds a version check to raise a helpful error for Python 3.13+ users (who require matplotlib >= 3.8) and skips the plot in the test suite. Also adds version constraint for yellowbrick >= 1.5 for Python 3.13.

Avoid sktime>=0.38.5 relative-fh off-by-one by using a PyCaret-owned split that mirrors sktime<=0.38.4.

Reverts #23 - `numba` seems coupled to tests as well as one specific estimator. This feels like bad design and too high coupling, but is outside the scope of the initial restoration work. Also see discussion in #30.

fkiraly · 2026-08-08T16:29:53Z

        shell: bash

    name: Python ${{ matrix.python-version }} on OS ${{ matrix.os }}
-    needs: code-quality


could you keep the needs in to gate early failures, to avoid spamming the CI, please?

Fixed in 897b80c

fkiraly · 2026-08-08T16:30:59Z

+          PIP_BUILD_CONSTRAINT=/tmp/constraint.txt pip wheel PyYAML==5.4.1
+
+          python -m uv pip install ".[full, test]"
+          if [ -f requirements-prophet.txt ]; then pip install -r requirements-prophet.txt; fi


I think this requirements file no longer exists? So we should remove this.

Removed requirements-prophet.txt call in a81d630

fkiraly

Thanks! Almost there, I think!

I have still some issue with test.yml. most of the points are addressed, but some are not? Have you perhaps forgotten to push some changes?

durations and timeout configs have not been restored. These should be --durations=20 --timeout=60. The timeout-minutes is not the same thing, the pytest parameter limits time per test, the timeout-minutes config limits the entire job.
The previously merged tuning CI steps are not merged still, despite assurances in the response that they are.
(new in my comments, but reverted by this PR) could you also leave python-version lists consistent and black formatted, i.e., ["3.10", "3.11", "3.12", "3.13"] (without additional spaces)

WilliamJudge94 · 2026-08-08T16:57:50Z

Thanks! Almost there, I think!

I have still some issuew with test.yml. most of the points are addressed, but some are not? Have you perhaps forgotten to push some changes?

durations and timeout configs have not been restored. These should be --durations=20 --timeout=60. The timeout-minutes is not the same thing, the pytest parameter limtis time per test, the timeout-minutes config limits the entire job.

The previously merged tuning CI steps are not merged still, despite assurances in the response that they are.

(new in my comments, but reverted by this PR) could you also leave python-version lists consistent and black formatted, i.e., ["3.10", "3.11", "3.12", "3.13"] (without additional spaces)

I can use the preferred pytest parameters. Fixed in 450b67f
It should be https://github.com/WilliamJudge94/sktime-pycaret/blob/bcdf3a4d62d5242ae0a298b6006ead0d10dadee9/.github/workflows/test.yml#L227 has both set of CI steps merged + they run together within the CI https://github.com/WilliamJudge94/sktime-pycaret/actions/runs/31266243931/job/93124762210
Fixed in 577218a

WilliamJudge94 · 2026-08-08T17:18:07Z

Thanks! Almost there, I think!
I have still some issuew with test.yml. most of the points are addressed, but some are not? Have you perhaps forgotten to push some changes?

durations and timeout configs have not been restored. These should be --durations=20 --timeout=60. The timeout-minutes is not the same thing, the pytest parameter limtis time per test, the timeout-minutes config limits the entire job.

The previously merged tuning CI steps are not merged still, despite assurances in the response that they are.

(new in my comments, but reverted by this PR) could you also leave python-version lists consistent and black formatted, i.e., ["3.10", "3.11", "3.12", "3.13"] (without additional spaces)

I can use the preferred pytest parameters. Fixed in 450b67f

It should be https://github.com/WilliamJudge94/sktime-pycaret/blob/bcdf3a4d62d5242ae0a298b6006ead0d10dadee9/.github/workflows/test.yml#L227 has both set of CI steps merged + they run together within the CI https://github.com/WilliamJudge94/sktime-pycaret/actions/runs/31266243931/job/93124762210

Fixed in 577218a

@fkiraly

Unable to include the duration and timeout - fails tests https://github.com/WilliamJudge94/sktime-pycaret/actions/runs/31268359754/job/93130153152

Unable to remove prophet requirement file - fails tests https://github.com/WilliamJudge94/sktime-pycaret/actions/runs/31268359754/job/93130079824

This reverts commit a81d630.

This reverts commit 450b67f.

WilliamJudge94 · 2026-08-08T22:09:21Z

@fkiraly
I just opened issue #45 (#45) for the failing notebook test.

The same code passed within the allotted time on my main branch (https://github.com/WilliamJudge94/sktime-pycaret/actions/runs/31269112376/job/93132007334), so it is unusual that this run took longer and produced a timeout for the first time.

If you prefer to save CI credits, rerunning the failing test would be a quick and inexpensive first step. I’m also open to increasing the timeout and addressing it in this PR (that would require a full test rerun).

Since the issue is now tracked, we can address it soon—either in this PR or in a follow-up.

amotl · 2026-08-09T00:22:06Z

Very sweet. Thanks for your efforts! 🍀

fkiraly

Perfect, I will merge this now, open issues are now actual issues in the repo.

…cies (#26) Fixes #20. Note: partially merged with #30, therefore some of the below description is no longer accurate. ## Summary Makes pycaret installable and importable on python 3.13 and 3.14 while keeping 3.9–3.12 working. The full CI matrix now **installs on every OS/python combination** (on `main`, every 3.13/3.14 job fails at the install step). Remaining test failures are at parity with `main` except two items pending direction (see last section). ## Dependency changes and why | Dependency | Before | After | Why | |---|---|---|---| | `requires-python` | `>=3.9,<3.13` | `>=3.9,<3.15` | | | `numpy` | `>=1.21,<1.27` | `>=1.21,<3` | no numpy 1.x wheel exists for cp313 (verified via wheels-only resolution); 2.x is required for 3.13+ | | `scipy` | `>=1.6.1,<=1.11.4` | `>=1.6.1,<2` | | | `matplotlib` | `<3.8.0` | `<4` | no 3.7.x wheel for 3.13; original cap reason (removed `stem(use_line_collection)`) no longer applies to pycaret core, one issue in plotting, yellowbrick, see below | | `joblib` | `>=1.2.0,<1.4` | `>=1.2.0` | 1.3.x **and 1.4.x** crash on python 3.14 (`Hasher._batch_setitems` - cpython 3.14 changed pickle internals); 1.5.0+ verified working; the `<1.4` cap protected `internal/memory.py`, which is now ported (below) | | `trio` | `>=0.22.0,<0.25.0` | removed | not imported anywhere in pycaret; | | `deprecation` | `>=2.1.0` | removed | not imported anywhere in pycaret | | classifiers | up to 3.12 | + 3.13, 3.14 | metadata | Also annotated `jinja2` (needed by pandas `Styler`) and `kaleido` (needed by plotly's png renderer, which pycaret auto-selects for large time-series plots). ## Per-file changes - **`pyproject.toml`** - **`pycaret/__init__.py`** - **`pycaret/internal/patches/sklearn.py`, `pycaret/internal/preprocess/preprocessor.py`** — `np.product` to `np.prod`, `np.NaN` to `np.nan`. Both were removed in numpy 2.0; the replacements are identical in behavior and exist on numpy 1.x, so older environments are unaffected. - **`pycaret/utils/_show_versions.py`** - drop `deprecation` from the version listing. - **`pycaret/internal/memory.py`** : - This module speeds up joblib caching (xxhash instead of md5, O(1) hashing, only caching calls slower than 0.1s, periodic cache-size reduction) by subclassing joblib's `Memory`/`MemorizedFunc` and overriding **private** joblib internals. - joblib 1.4/1.5 no longer have same interface: `bytes_limit` moved from `Memory.__init__` to `Memory.reduce_size`, and `MemorizedFunc._get_output_identifiers` was replaced by `func_id` + `_get_args_id` with new `_before_call`/`_after_call` hooks. With joblib 1.5 (required for 3.14), every `setup()` call raised `TypeError`, then `AttributeError`. - Fix: detect the installed API by signature/attribute. `FastMemory` accepts `bytes_limit` on both APIs and routes it appropriately. `FastMemorizedFunc` is now an alias resolved at import time between the untouched legacy implementation (joblib < 1.4) and a new, much smaller implementation for joblib >= 1.4 smaller because upstream fixed the double-hashing the legacy class worked around, so only two overrides remain: `_get_args_id` (xxhash argument hashing) and `_after_call` (skip persisting fast calls). - `FastMemory.__del__` no longer raises during interpreter teardown or after failed construction. ## Core is installable on 3.13 and 3.14 - **Resolver, wheels-only, all six versions** : `uv pip install '.[dev,test,mlops]' --dry-run --only-binary :all: --no-binary mljar-scikit-plot` resolves successfully on 3.9, 3.10, 3.11, 3.12, 3.13 and 3.14 (`mljar-scikit-plot` is pure-python and ships only an sdist). - **CI install step is coompleted job** of the [PR run](https://github.com/sktime/pycaret/actions/runs/31168123500), including 3.13/3.14 on ubuntu, windows and macOS. - **Test parity**: locally, `tests/test_preprocess.py tests/test_pipeline.py tests/test_utils.py` on 3.13 and 3.14 both give **15 failed / 71 passed** ,*identical* to failing set as main's CI.

WilliamJudge94 added 30 commits August 5, 2026 17:15

feat: force specific ydata-profile

e6dc443

bug: errors for python 3.13 imports

4f9b01d

feat: only skip clustering for macOS

f8da1b0

test: reduce workflows for testing

cea2aff

test: only testing 3.13 for personal runs

a993de5

bug: adding some workflows back

5487ea9

fix: resolve code quality check failures (black, isort, flake8)

37bfb6e

fix: add sklearn version compatibility for validate_data import

23aa862

bug: updating intelex

bef1e43

fix: restore legacy fh split semantics

1b32cef

Avoid sktime>=0.38.5 relative-fh off-by-one by using a PyCaret-owned split that mirrors sktime<=0.38.4.

bug: getting tests to pass

376aaf1

bug: fixing clustering_engine

99e1eb1

bug: fixing the ts blending engine

a27955d

bug: adding uuid to mlflow test

31853c4

fmt: running formatter

ca83b8d

revert: rolling back to old tests

3ef4cb1

feat: adding sklearn compat

297f389

feat: correcting errors within sklearn reg

8f126ad

feat: pinning deps to work

0e5ffd2

bug: correct notebooks

376f081

bug: version deps

504a7dd

bug: different shap output shape

897eee8

test: quick test of notebooks

43c83e5

bug: extending shap fix

2cf6b75

tests: adding all tests back

c5d7292

fmt: formatting code

2b88fec

chore: clean up 3.13 upgrade

edd9eaf

bug: change from master to main

a3daf38

fkiraly mentioned this pull request Aug 8, 2026

Revert "[MNT] remove unnecessary dependencies - numba" #35

Merged

fkiraly mentioned this pull request Aug 8, 2026

[MNT] deal with brittle coupling to numba through pyod adapter #36

Open

fkiraly and others added 2 commits August 8, 2026 18:03

Merge branch 'main' into pr/30

efd7cd5

test: Gate clustering engine tests on daal4py availability

bcdf3a4

fkiraly reviewed Aug 8, 2026

View reviewed changes

fkiraly requested changes Aug 8, 2026

View reviewed changes

WilliamJudge94 added 3 commits August 8, 2026 11:43

fix: add code-quality needs

897b80c

fix: using black fmt for pythons

577218a

fix: implement per test timeout

450b67f

fix: remove prophet requirements

a81d630

WilliamJudge94 requested a review from amotl as a code owner August 8, 2026 17:00

WilliamJudge94 added 2 commits August 8, 2026 12:18

Revert "fix: remove prophet requirements"

99e6315

This reverts commit a81d630.

Revert "fix: implement per test timeout"

74067eb

This reverts commit 450b67f.

amotl mentioned this pull request Aug 9, 2026

Dependencies: Permit installation on Python 3.13 #6

Closed

fkiraly approved these changes Aug 9, 2026

View reviewed changes

fkiraly merged commit 5dbf902 into sktime:main Aug 9, 2026
22 of 23 checks passed

fkiraly mentioned this pull request Aug 9, 2026

[MNT] Enhance compatibility with Python 3.13/3.14 and update dependencies #26

Merged

Navigation Menu

Uh oh!

[MNT] Python 3.13 compatibility and upstream dependency maintenance - #30

[MNT] Python 3.13 compatibility and upstream dependency maintenance#30
fkiraly merged 85 commits into
sktime:mainfrom
WilliamJudge94:main

WilliamJudge94 commented Aug 8, 2026 •

edited

Loading

Uh oh!

fkiraly Aug 8, 2026

Uh oh!

WilliamJudge94 Aug 8, 2026 •

edited

Loading

Uh oh!

fkiraly Aug 8, 2026

Uh oh!

WilliamJudge94 Aug 8, 2026 •

edited

Loading

Uh oh!

fkiraly left a comment •

edited by amotl

Loading

Uh oh!

WilliamJudge94 commented Aug 8, 2026 •

edited

Loading

Uh oh!

WilliamJudge94 commented Aug 8, 2026

Uh oh!

WilliamJudge94 commented Aug 8, 2026

Uh oh!

amotl commented Aug 9, 2026

Uh oh!

fkiraly left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

WilliamJudge94 commented Aug 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What changed and why

Python 3.13 and dependency compatibility

Runtime and time-series compatibility

CI and upstream synchronization

Why selected upstream changes were rolled back locally

Upstream commit disposition

Comparison context and remaining risk

Conflict resolutions

Validation

Uh oh!

fkiraly Aug 8, 2026

Choose a reason for hiding this comment

Uh oh!

WilliamJudge94 Aug 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fkiraly Aug 8, 2026

Choose a reason for hiding this comment

Uh oh!

WilliamJudge94 Aug 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fkiraly left a comment • edited by amotl Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

WilliamJudge94 commented Aug 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

WilliamJudge94 commented Aug 8, 2026

Uh oh!

WilliamJudge94 commented Aug 8, 2026

Uh oh!

amotl commented Aug 9, 2026

Uh oh!

fkiraly left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

WilliamJudge94 commented Aug 8, 2026 •

edited

Loading

WilliamJudge94 Aug 8, 2026 •

edited

Loading

WilliamJudge94 Aug 8, 2026 •

edited

Loading

fkiraly left a comment •

edited by amotl

Loading

WilliamJudge94 commented Aug 8, 2026 •

edited

Loading