Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

drop support for python=3.9 #8937

Merged
merged 15 commits into from
Aug 14, 2024
Merged

drop support for python=3.9 #8937

merged 15 commits into from
Aug 14, 2024

Conversation

keewis
Copy link
Collaborator

@keewis keewis commented Apr 13, 2024

According to our policy (and NEP-29) we can drop support for python=3.9 since about a week ago. Interestingly, SPEC0 says we could have started doing this about half a year ago (Q4 2023).

We could delay this until we have a release that is compatible with numpy>=2.0, though (numpy>=2.1 will drop support for python=3.9).

  • User visible changes (including notable bug fixes) are documented in whats-new.rst

Comment on lines 131 to 134
# We require a "hack" to tell type checkers that e.g. Variable + DataArray = DataArray
# In reality this returns NotImplementes, but this is not a valid type in python 3.9.
# In reality this returns NotImplemented, but this is not a valid type in python 3.9.
# Therefore, we return DataArray. In reality this would call DataArray.__add__(Variable)
# TODO: change once python 3.10 is the minimum.
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@headtr1ck, can you explain (or directly push a fix, if possible) what you meant by this?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if I will be able to code anytime soon, so here an explanation first:

If you do operators, python checks the left obj First. Since DataArray is an array-like (and therefore part of VariableCompatible), the type checker thinks that Variable + DataArray = Variable, which is incorrect.
If you want to follow the runtime behavior, you should return NotImplemented, unfortunately it is not a valid type in Python 3.9. Therefore we return DataArray directly.

Now the "fix" would be to change the return type to NotImplemented and check if it works in Python >=3.10 (I have not actually confirmed that, in case just leave things like they are since it is technically working).

@TomNicholas
Copy link
Member

This means we could replace Union[] with | everywhere in the type hints right?

@headtr1ck
Copy link
Collaborator

This means we could replace Union[] with | everywhere in the type hints right?

Shouldn't pyupgrade / ruff do that for us?

@keewis
Copy link
Collaborator Author

keewis commented Apr 15, 2024

it does, but only parts of it (other fixes it deems unsafe so they require manual intervention). I don't have time to go through each of them this week, but feel to push any of these directly to this PR.

basnijholt added a commit to pipefunc/pipefunc that referenced this pull request Jun 2, 2024
Because of [NEP29](https://numpy.org/neps/nep-0029-deprecation_policy) NumPy only supports ≥3.10 now.

Also, Zarr, NetworkX, and ([soon](pydata/xarray#8937)) Xarray will support only ≥3.10.
@headtr1ck
Copy link
Collaborator

I think delaying this until we have at least one release that is fully compatible with numpy2 and Python 3.9 is a good idea.

Do we have an estimate on how much this would delay this merge?

@keewis
Copy link
Collaborator Author

keewis commented Jun 18, 2024

we have a numpy 2 + python 3.9 compatible release now, so once the merge conflicts are fixed this should be good to go

@TomNicholas TomNicholas added the dependencies Pull requests that update a dependency file label Jun 18, 2024
@headtr1ck
Copy link
Collaborator

we have a numpy 2 + python 3.9 compatible release now, so once the merge conflicts are fixed this should be good to go

It seems that there are still a few PRs open with fixes for numpy 2 and maybe we should wait a month or two to see if we get any bug reports related to that?

I know that we are all eager to have more nice python things :)

@dcherian
Copy link
Contributor

dcherian commented Aug 6, 2024

I think we should move forward here

@keewis
Copy link
Collaborator Author

keewis commented Aug 7, 2024

it looks like UP007 will have to be converted manually

@dcherian
Copy link
Contributor

dcherian commented Aug 7, 2024

The tests failures are present on main too.

@keewis
Copy link
Collaborator Author

keewis commented Aug 14, 2024

thanks for finishing this, @dcherian. Let's merge this now, and I'll update the list of required checks after that.

@keewis keewis merged commit 16b53ac into pydata:main Aug 14, 2024
27 checks passed
@keewis keewis deleted the drop-python3.9 branch August 14, 2024 10:07
dcherian added a commit to dcherian/xarray that referenced this pull request Aug 14, 2024
* main:
  Revise (pydata#9366)
  Fix rechunking to a frequency with empty bins. (pydata#9364)
  whats-new entry for dropping python 3.9 (pydata#9359)
  drop support for `python=3.9` (pydata#8937)
  Revise (pydata#9357)
  try to fix scheduled hypothesis test (pydata#9358)
dcherian added a commit to TomNicholas/xarray that referenced this pull request Aug 22, 2024
* main: (214 commits)
  Adds copy parameter to __array__ for numpy 2.0 (pydata#9393)
  `numpy 2` compatibility in the `pydap` backend (pydata#9391)
  pyarrow dependency added to doc environment (pydata#9394)
  Extend padding functionalities (pydata#9353)
  refactor GroupBy internals (pydata#9389)
  Combine `UnsignedIntegerCoder` and `CFMaskCoder` (pydata#9274)
  passing missing parameters to ZarrStore.open_store when opening a datatree (pydata#9377)
  Fix tests on big-endian systems (pydata#9380)
  Improve error message on `ds['x', 'y']` (pydata#9375)
  Improve error message for missing coordinate index (pydata#9370)
  Add flaky to TestNetCDF4ViaDaskData (pydata#9373)
  Make chunk manager an option in `set_options` (pydata#9362)
  Revise (pydata#9371)
  Remove duplicate word from docs (pydata#9367)
  Adding open_groups to BackendEntryPointEngine, NetCDF4BackendEntrypoint, and H5netcdfBackendEntrypoint (pydata#9243)
  Revise (pydata#9366)
  Fix rechunking to a frequency with empty bins. (pydata#9364)
  whats-new entry for dropping python 3.9 (pydata#9359)
  drop support for `python=3.9` (pydata#8937)
  Revise (pydata#9357)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dependencies Pull requests that update a dependency file
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants