Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ENH] python 3.11 support #3963

Open
fkiraly opened this issue Dec 20, 2022 · 4 comments
Open

[ENH] python 3.11 support #3963

fkiraly opened this issue Dec 20, 2022 · 4 comments
Labels
enhancement Adding new functionality maintenance Continuous integration, unit testing & package distribution

Comments

@fkiraly
Copy link
Collaborator

fkiraly commented Dec 20, 2022

update, Feb 23, 2023: numba is now fully isolated, and sktime release 0.16.1 has been released with experimental 3.11 support.
Issues that remain to fix before full support in 0.17.0:


Umbrella issue for discussing python 3.11 support, to direct frequent questions to.

Current blockers to the "naive" solution of releasing sktime for 3.11 as-is, as far as I know:

  • numba is a core dependency and not python 3.11 compatible
  • a number of soft dependencies do not have python 3.11 support

Potential solutions:

  • soft dependencies are all isolated, so conditional install in pyproject.toml would fix the issue. This would require us to check python 3.11 support for soft dependencies periodically and potentially remove the condition at a later point in time.
  • numba could be solved by "wait for python 3.11 support". When that happens is unclear.
  • or, we can turn numba into a soft dependency right now, and deal with it as with the other soft dependencies. This is entirely feasible - the PR here could be merged and enable python 3.11 EXPERIMENTAL: [MNT] python 3.11 compatibility with numba as soft dependency #3856. But, it is a very contentious issue with some core developers.
  • a third option keeps numba as core dependency, but makes it optional only on python 3.11, until it becomes compatible. This option is implemented here, ready for merge and enable: [MNT] python 3.11 compatibility, with numba as core dependency on 3.7-3.10 #4000. As it turns out, this is also very contentious.
@fkiraly fkiraly added maintenance Continuous integration, unit testing & package distribution enhancement Adding new functionality labels Dec 20, 2022
@fkiraly
Copy link
Collaborator Author

fkiraly commented Dec 20, 2022

FYI @alexanderreschjedox, I tried to summarize the situation above.

fkiraly added a commit that referenced this issue Feb 13, 2023
…3631)

This PR isolates `numba` throughout `sktime`.

Note: does *not* turn `numba` into a soft dependency.

This is required for all approaches to making `sktime` python 3.11 compatible as long as `numba` is not python 3.11 compatible, because that requires being able to run the framework components of `sktime` without `numba` imports.

See also #3963.

The general recipe to isolate `numba`:

* move `numba` imports into functions and classes, where possible
* where loose functions are decorated with `jit` and `njit`, move them to a private module `_orig_modulename_numba`, and move imports of that into functions or classes
* add the package dependency tag to `numba` dependent estimators to isolate them
* use `pytest.skipif` to conditionally skip `numba` dependent non-suite tests if `numba` is not installed

Makes the following changes:
* changes the `test_module_softdeps` test to ignore private modules (starting with `_`), i.e., to not require a user warning. This is ok since users are not expected to import private modules.

Depends on #3857
@jplatipus
Copy link

The github page should be updated: currently https://github.com/sktime/sktime says sktime works with python 3.11, but I get a conflict report:

(sktimeTests) C:\Users\jacqu>conda install -c conda-forge sktime
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: |
Found conflicts! Looking for incompatible packages.
This can take several minutes. Press CTRL-C to abort.
failed

UnsatisfiableError: The following specifications were found
to be incompatible with the existing python installation in your environment:

Specifications:

  • sktime -> python[version='>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.8,<3.9.0a0|>=3.9,<3.10.0a0']

Your python: python=3.11

If python is on the left-most side of the chain, that's the version you've asked for.
When python appears to the right, that indicates that the thing on the left is somehow
not available for the python version you are constrained to. Note that conda will not
change your python version to a different minor version unless you explicitly specify
that.

@fkiraly
Copy link
Collaborator Author

fkiraly commented Feb 21, 2023

@jplatipus, as far as I know sktime works on 3.11 from version 0.16.1 onwards. Support is currently experimental.

3.11 compatibility is tested in the CI/CD pipeline, and I just checked pip install from conda and I do not get the message.
pip (e.g., from conda) is how the majority of users are installing sktime, and this works as intended.

Now you are using conda-forge, i.e., conda install -c conda-forge sktime.
I just tried in a python 3.11 environment, on Windows OS, and it seems to get stuck (not getting an UnsatisfiableError)

I will open an issue to track this.

@fkiraly
Copy link
Collaborator Author

fkiraly commented Feb 21, 2023

Issue is here: #4254

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Adding new functionality maintenance Continuous integration, unit testing & package distribution
Projects
None yet
Development

No branches or pull requests

2 participants