Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrades: scipy 1.11 and dependencies #36112

Merged
merged 17 commits into from
Sep 10, 2023
Merged

Conversation

mkoeppe
Copy link
Member

@mkoeppe mkoeppe commented Aug 22, 2023

This includes updates of meson, meson_python, ninja_build, pybind11, pyproject_metadata, pythran, gast.

Resolves #35703

📝 Checklist

  • The title is concise, informative, and self-explanatory.
  • The description explains in detail what this PR is about.
  • I have linked a relevant issue or discussion.
  • I have created tests covering the changes.
  • I have updated the documentation accordingly.

⌛ Dependencies

@mkoeppe
Copy link
Member Author

mkoeppe commented Aug 22, 2023

Portability tests running at https://github.com/mkoeppe/sage/actions/runs/5934767113

@mkoeppe
Copy link
Member Author

mkoeppe commented Aug 22, 2023

@mkoeppe mkoeppe requested a review from kiwifb August 31, 2023 16:54
@kiwifb
Copy link
Member

kiwifb commented Sep 1, 2023

CI is failing, I see sagelib failed to build. Do you know why?

@mkoeppe
Copy link
Member Author

mkoeppe commented Sep 1, 2023

First time I see this failure; I think it can only be seen in incremental builds

   Building interpreters for fast_callable
[sagelib-10.2.beta0]     -> First build of interpreters
[sagelib-10.2.beta0]     Exception while cythonizing source files: AttributeError("type object 'PatternTransform' has no attribute 'typePatterns'")
[sagelib-10.2.beta0]     ************************************************************************
[sagelib-10.2.beta0]     Traceback (most recent call last):
[sagelib-10.2.beta0]       File "<string>", line 2, in <module>
[sagelib-10.2.beta0]       File "<pip-setuptools-caller>", line 34, in <module>
[sagelib-10.2.beta0]       File "/sage/src/setup.py", line 108, in <module>
[sagelib-10.2.beta0]         extensions = cythonize(
[sagelib-10.2.beta0]                      ^^^^^^^^^^
[sagelib-10.2.beta0]       File "/sage/local/var/lib/sage/venv-python3.11.1/lib/python3.11/site-packages/Cython/Build/Dependencies.py", line 971, in cythonize
[sagelib-10.2.beta0]         ctx = c_options.create_context()
[sagelib-10.2.beta0]               ^^^^^^^^^^^^^^^^^^^^^^^^^^
[sagelib-10.2.beta0]       File "/sage/local/var/lib/sage/venv-python3.11.1/lib/python3.11/site-packages/Cython/Compiler/Main.py", line 600, in create_context
[sagelib-10.2.beta0]         return Context(self.include_path, self.compiler_directives,
[sagelib-10.2.beta0]                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[sagelib-10.2.beta0]       File "/sage/local/var/lib/sage/venv-python3.11.1/lib/python3.11/site-packages/Cython/Compiler/Main.py", line 80, in __init__
[sagelib-10.2.beta0]         from . import Builtin, CythonScope
[sagelib-10.2.beta0]       File "/sage/local/var/lib/sage/venv-python3.11.1/lib/python3.11/site-packages/Cython/Compiler/CythonScope.py", line 5, in <module>
[sagelib-10.2.beta0]         from .UtilityCode import CythonUtilityCode
[sagelib-10.2.beta0]       File "/sage/local/var/lib/sage/venv-python3.11.1/lib/python3.11/site-packages/Cython/Compiler/UtilityCode.py", line 3, in <module>
[sagelib-10.2.beta0]         from .TreeFragment import parse_from_strings, StringParseContext
[sagelib-10.2.beta0]       File "/sage/local/var/lib/sage/venv-python3.11.1/lib/python3.11/site-packages/Cython/Compiler/TreeFragment.py", line 17, in <module>
[sagelib-10.2.beta0]         from .Visitor import VisitorTransform
[sagelib-10.2.beta0]       File "Cython/Compiler/Visitor.py", line 16, in init Cython.Compiler.Visitor
[sagelib-10.2.beta0]       File "/sage/local/var/lib/sage/venv-python3.11.1/lib/python3.11/site-packages/Cython/Compiler/Nodes.py", line 31, in <module>
[sagelib-10.2.beta0]         from .Pythran import has_np_pythran, pythran_type, is_pythran_buffer
[sagelib-10.2.beta0]       File "/sage/local/var/lib/sage/venv-python3.11.1/lib/python3.11/site-packages/Cython/Compiler/Pythran.py", line 10, in <module>
[sagelib-10.2.beta0]         import pythran
[sagelib-10.2.beta0]       File "/sage/local/var/lib/sage/venv-python3.11.1/lib/python3.11/site-packages/pythran/__init__.py", line 42, in <module>
[sagelib-10.2.beta0]         from pythran.toolchain import (generate_cxx, compile_cxxfile, compile_cxxcode,
[sagelib-10.2.beta0]       File "/sage/local/var/lib/sage/venv-python3.11.1/lib/python3.11/site-packages/pythran/toolchain.py", line 12, in <module>
[sagelib-10.2.beta0]         from pythran.middlend import refine, mark_unexported_functions
[sagelib-10.2.beta0]       File "/sage/local/var/lib/sage/venv-python3.11.1/lib/python3.11/site-packages/pythran/middlend.py", line 4, in <module>
[sagelib-10.2.beta0]         from pythran.optimizations import (ComprehensionPatterns, ListCompToGenexp,
[sagelib-10.2.beta0]       File "/sage/local/var/lib/sage/venv-python3.11.1/lib/python3.11/site-packages/pythran/optimizations/__init__.py", line 21, in <module>
[sagelib-10.2.beta0]         from .pattern_transform import PatternTransform
[sagelib-10.2.beta0]       File "/sage/local/var/lib/sage/venv-python3.11.1/lib/python3.11/site-packages/pythran/optimizations/pattern_transform.py", line 346, in <module>
[sagelib-10.2.beta0]         getattr(PatternTransform, attr_name) + (known_pattern,))
[sagelib-10.2.beta0]         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[sagelib-10.2.beta0]     AttributeError: type object 'PatternTransform' has no attribute 'typePatterns'

@github-actions
Copy link

github-actions bot commented Sep 1, 2023

Documentation preview for this PR (built with commit 5b11b9b; changes) is ready! 🎉

@kiwifb
Copy link
Member

kiwifb commented Sep 1, 2023

OK, now 3 tests failing in CI. One maxima execution timing out - probably something unrelated. Two very strange results in sage/schemes/curves but I cannot reproduce them in sage-on-gentoo with scipy-1.11.1. I'll try 1.11.2 but it seems unlikely to be related.

@mkoeppe
Copy link
Member Author

mkoeppe commented Sep 1, 2023

Two very strange results in sage/schemes/curves

Unrelated and fixed in #35376

Copy link
Member

@kiwifb kiwifb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mkoeppe
Copy link
Member Author

mkoeppe commented Sep 1, 2023

Thank you!

vbraun pushed a commit to vbraun/sage that referenced this pull request Sep 2, 2023
    
<!-- ^^^^^
Please provide a concise, informative and self-explanatory title.
Don't put issue numbers in there, do this in the PR body below.
For example, instead of "Fixes sagemath#1234" use "Introduce new method to
calculate 1+1"
-->
<!-- Describe your changes here in detail -->

This includes updates of `meson`, `meson_python`, `ninja_build`,
`pybind11`, `pyproject_metadata`, `pythran`, `gast`.

<!-- Why is this change required? What problem does it solve? -->
<!-- If this PR resolves an open issue, please link to it here. For
example "Fixes sagemath#12345". -->
Resolves sagemath#35703
<!-- If your change requires a documentation PR, please link it
appropriately. -->

### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->
<!-- If your change requires a documentation PR, please link it
appropriately -->
<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
<!-- Feel free to remove irrelevant items. -->

- [x] The title is concise, informative, and self-explanatory.
- [ ] The description explains in detail what this PR is about.
- [x] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation accordingly.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on
- sagemath#12345: short description why this is a dependency
- sagemath#34567: ...
-->
- Depends on sagemath#35404 (merged here)
- Depends on sagemath#35810 (merged here)
- Depends on sagemath#35084 (merged here)


<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
    
URL: sagemath#36112
Reported by: Matthias Köppe
Reviewer(s): François Bissey
@mkoeppe mkoeppe mentioned this pull request Sep 3, 2023
5 tasks
vbraun pushed a commit to vbraun/sage that referenced this pull request Sep 4, 2023
    
<!-- ^^^^^
Please provide a concise, informative and self-explanatory title.
Don't put issue numbers in there, do this in the PR body below.
For example, instead of "Fixes sagemath#1234" use "Introduce new method to
calculate 1+1"
-->
<!-- Describe your changes here in detail -->

This includes updates of `meson`, `meson_python`, `ninja_build`,
`pybind11`, `pyproject_metadata`, `pythran`, `gast`.

<!-- Why is this change required? What problem does it solve? -->
<!-- If this PR resolves an open issue, please link to it here. For
example "Fixes sagemath#12345". -->
Resolves sagemath#35703
<!-- If your change requires a documentation PR, please link it
appropriately. -->

### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->
<!-- If your change requires a documentation PR, please link it
appropriately -->
<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
<!-- Feel free to remove irrelevant items. -->

- [x] The title is concise, informative, and self-explanatory.
- [ ] The description explains in detail what this PR is about.
- [x] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation accordingly.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on
- sagemath#12345: short description why this is a dependency
- sagemath#34567: ...
-->
- Depends on sagemath#35404 (merged here)
- Depends on sagemath#35810 (merged here)
- Depends on sagemath#35084 (merged here)


<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
    
URL: sagemath#36112
Reported by: Matthias Köppe
Reviewer(s): François Bissey
vbraun pushed a commit to vbraun/sage that referenced this pull request Sep 5, 2023
    
<!-- ^^^^^
Please provide a concise, informative and self-explanatory title.
Don't put issue numbers in there, do this in the PR body below.
For example, instead of "Fixes sagemath#1234" use "Introduce new method to
calculate 1+1"
-->
<!-- Describe your changes here in detail -->

This includes updates of `meson`, `meson_python`, `ninja_build`,
`pybind11`, `pyproject_metadata`, `pythran`, `gast`.

<!-- Why is this change required? What problem does it solve? -->
<!-- If this PR resolves an open issue, please link to it here. For
example "Fixes sagemath#12345". -->
Resolves sagemath#35703
<!-- If your change requires a documentation PR, please link it
appropriately. -->

### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->
<!-- If your change requires a documentation PR, please link it
appropriately -->
<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
<!-- Feel free to remove irrelevant items. -->

- [x] The title is concise, informative, and self-explanatory.
- [ ] The description explains in detail what this PR is about.
- [x] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation accordingly.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on
- sagemath#12345: short description why this is a dependency
- sagemath#34567: ...
-->
- Depends on sagemath#35404 (merged here)
- Depends on sagemath#35810 (merged here)
- Depends on sagemath#35084 (merged here)


<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
    
URL: sagemath#36112
Reported by: Matthias Köppe
Reviewer(s): François Bissey
@mkoeppe mkoeppe mentioned this pull request Sep 5, 2023
3 tasks
@vbraun vbraun merged commit 7ed5b8b into sagemath:develop Sep 10, 2023
10 of 13 checks passed
vbraun pushed a commit to vbraun/sage that referenced this pull request Sep 27, 2023
    
<!-- ^^^^^
Please provide a concise, informative and self-explanatory title.
Don't put issue numbers in there, do this in the PR body below.
For example, instead of "Fixes sagemath#1234" use "Introduce new method to
calculate 1+1"
-->
<!-- Describe your changes here in detail -->

This PR builds on sagemath#36109 and contains the rest of changes needed to
update Cython to 3.0.0. The actual changes needed for building Sage are
a632238 for the code,
67ad2e4 for the build system, and
7195b8f for the doctests. The rest of
commits are either from sagemath#36109 or for patching dependencies.

Closes sagemath#29863.

Ideally we should wait until numpy and scipy are upgraded to 1.25 and
1.11 respectively before merging this, so we can drop the patches for
them.

<!-- Why is this change required? What problem does it solve? -->
<!-- If this PR resolves an open issue, please link to it here. For
example "Fixes sagemath#12345". -->
<!-- If your change requires a documentation PR, please link it
appropriately. -->

### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->
<!-- If your change requires a documentation PR, please link it
appropriately -->
<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
<!-- Feel free to remove irrelevant items. -->

- [x] The title is concise, informative, and self-explanatory.
- [x] The description explains in detail what this PR is about.
- [x] I have linked a relevant issue or discussion.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on
- sagemath#12345: short description why this is a dependency
- sagemath#34567: ...
-->

- Depends on sagemath#36109
- Depends on sagemath#36112 (merged here)

<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
    
URL: sagemath#36110
Reported by: Yutao Yuan
Reviewer(s): Gonzalo Tornaría, Kwankyu Lee, Matthias Köppe, Tobias Diez
vbraun pushed a commit to vbraun/sage that referenced this pull request Oct 1, 2023
    
<!-- ^^^^^
Please provide a concise, informative and self-explanatory title.
Don't put issue numbers in there, do this in the PR body below.
For example, instead of "Fixes sagemath#1234" use "Introduce new method to
calculate 1+1"
-->
<!-- Describe your changes here in detail -->

NumPy 1.26 provides Python 3.12 support and Cython 3 compatibility.

1.26.0 was released 2023-09-16.

As NumPy has changed its build system from setuptools-pinned-to-an-
ancient-version to meson_python, we get rid of our ancient version of
`setuptools`. The SPKGs `setuptools` and `setuptools_wheel` now ship the
same (current) version.

- [x] Check portability run:
https://github.com/mkoeppe/sage/actions/runs/5959239800
- [ ] Check SAGE_FAT_BINARY

<!-- Why is this change required? What problem does it solve? -->
<!-- If this PR resolves an open issue, please link to it here. For
example "Fixes sagemath#12345". -->
Resolves sagemath#34816
<!-- If your change requires a documentation PR, please link it
appropriately. -->

### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->
<!-- If your change requires a documentation PR, please link it
appropriately -->
<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
<!-- Feel free to remove irrelevant items. -->

- [x] The title is concise, informative, and self-explanatory.
- [x] The description explains in detail what this PR is about.
- [x] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation accordingly.

### ⌛ Dependencies
- Depends on sagemath#36112 (for the `meson_python` upgrades, merged here)
- Depends on sagemath#35404 (merged here as
part of above)
- Depends on sagemath#35810 (merged here as part of above)
- Depends on sagemath#36110 (merged here)
- Depends on sagemath#36263 (merged here)
- Depends on sagemath#36238 (merged here)
- Depends on sagemath#36255 (merged here)
<!-- List all open PRs that this PR logically depends on
- sagemath#12345: short description why this is a dependency
- sagemath#34567: ...
-->

<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
    
URL: sagemath#36123
Reported by: Matthias Köppe
Reviewer(s):
vbraun pushed a commit that referenced this pull request Oct 8, 2023
    
<!-- ^^^^^
Please provide a concise, informative and self-explanatory title.
Don't put issue numbers in there, do this in the PR body below.
For example, instead of "Fixes #1234" use "Introduce new method to
calculate 1+1"
-->
<!-- Describe your changes here in detail -->

NumPy 1.26 provides Python 3.12 support and Cython 3 compatibility.

1.26.0 was released 2023-09-16.

As NumPy has changed its build system from setuptools-pinned-to-an-
ancient-version to meson_python, we get rid of our ancient version of
`setuptools`. The SPKGs `setuptools` and `setuptools_wheel` now ship the
same (current) version.

- [x] Check portability run:
https://github.com/mkoeppe/sage/actions/runs/5959239800
- [ ] Check SAGE_FAT_BINARY

<!-- Why is this change required? What problem does it solve? -->
<!-- If this PR resolves an open issue, please link to it here. For
example "Fixes #12345". -->
Resolves #34816
<!-- If your change requires a documentation PR, please link it
appropriately. -->

### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->
<!-- If your change requires a documentation PR, please link it
appropriately -->
<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
<!-- Feel free to remove irrelevant items. -->

- [x] The title is concise, informative, and self-explanatory.
- [x] The description explains in detail what this PR is about.
- [x] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation accordingly.

### ⌛ Dependencies
- Depends on #36112 (for the `meson_python` upgrades, merged here)
- Depends on #35404 (merged here as
part of above)
- Depends on #35810 (merged here as part of above)
- Depends on #36110 (merged here)
- Depends on #36263 (merged here)
- Depends on #36238 (merged here)
- Depends on #36255 (merged here)
<!-- List all open PRs that this PR logically depends on
- #12345: short description why this is a dependency
- #34567: ...
-->

<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
    
URL: #36123
Reported by: Matthias Köppe
Reviewer(s):
@mkoeppe mkoeppe deleted the scipy_1_11 branch October 31, 2023 20:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

SciPy 1.11.x
3 participants