Replace relative imports by absolute ones in `sage.{algebras,arith,categories,cpython,data_structures,misc,modular,rings,sat,symbolic}` #36666

mkoeppe · 2023-11-06T17:26:56Z

Final chunk of the cherry-pick from #35095 for #36228, see also #36572 (comment)

This PR overlaps with #36572, #36588, #36589, but it does not touch .all* files (no changes there are needed for #36228), thus avoiding conflicts with #35095 (see also #36524 (comment))

📝 Checklist

The title is concise, informative, and self-explanatory.
The description explains in detail what this PR is about.
I have linked a relevant issue or discussion.
I have created tests covering the changes.
I have updated the documentation accordingly.

⌛ Dependencies

tobiasdiez · 2023-11-07T03:37:40Z

So you don't want to resolve merge conflicts with your PR, but instead force me to do it?!

mkoeppe · 2023-11-07T16:24:24Z

So you don't want to resolve merge conflicts with your PR, but instead force me to do it?!

Tobias, I know that you know all of the following -- but I'll explain it for the audience:
This is a "common denominator PR", which I prepared carefully so that it avoids conflicts.
Indeed, I have just rebased your #36572, #36588, #36589 on top of it, simply by typing gh pr checkout 36572, then git rebase absolute_cimport_sage_arith, then git push --force. The rebase happened automatically, with no need for conflict resolution.

dcoudert

LGTM and passes tests on Fedora 35.

tobiasdiez · 2023-11-07T17:24:00Z

So you don't want to resolve merge conflicts with your PR, but instead force me to do it?!

Tobias, I know that you know all of the following -- but I'll explain it for the audience: This is a "common denominator PR", which I prepared carefully so that it avoids conflicts. Indeed, I have just rebased your #36572, #36588, #36589 on top of it, simply by typing gh pr checkout 36572, then git rebase absolute_cimport_sage_arith, then git push --force. The rebase happened automatically, with no need for conflict resolution.

And how is that helping with these PRs being in conflict with #35095? You have to resolve these conflicts eventually so why not now?

dcoudert · 2023-11-07T17:27:02Z

I'm not setting this to positive review yet as the priority is to clarify the situation with respect to #36572, #36588, and #36589.

mkoeppe · 2023-11-07T17:41:18Z

Tobias, again you are abusing your privileges as a member of the Triage group.

jhpalmieri · 2023-11-07T18:33:10Z

I'm a little puzzled by the goal here. If I do git grep 'from [.]' src/sage/categories, I get some hits (in algebra_ideals.py, for instance), so you have not replaced all relative imports. As far as I can tell, you have not touched any all.py files, which are filled with relative imports. Can you clarify?

jhpalmieri · 2023-11-07T18:34:12Z

In the meantime, is the main disagreement about which changes get merged first? The other branches have been rebased on top of this one, so why not proceed with this one?

tobiasdiez · 2023-11-07T18:35:05Z

Tobias, again you are abusing your privileges as a member of the Triage group.

Sorry, just clicked the wrong button. Didn't meant to close the pr

mkoeppe · 2023-11-07T20:35:59Z

I'm a little puzzled by the goal here. If I do git grep 'from [.]' src/sage/categories, I get some hits (in algebra_ideals.py, for instance), so you have not replaced all relative imports. As far as I can tell, you have not touched any all.py files, which are filled with relative imports. Can you clarify?

Only relative imports and cimports in Cython files are relevant for the purpose of resolving the problems of current Cython 3.0.x with namespace packages. I made these changes back in September or so in #35095 because of an acute need for this in the modularization. I opened PR #36228 for the task of upstreaming this into mainline Sage.

Changes to all.py files are irrelevant for the Cython issue. Moreover, in #35095, many all.py files are completely restructured for modularization purposes, so changes there have to be done separately from changes in Sage develop.

This PR is a cherry-pick from my branch in #35095, the three commits indicated #36572 (comment), with any conflicts in cherry picking resolved to "no change".

jhpalmieri · 2023-11-07T20:43:35Z

I'm a little puzzled by the goal here. If I do git grep 'from [.]' src/sage/categories, I get some hits (in algebra_ideals.py, for instance), so you have not replaced all relative imports. As far as I can tell, you have not touched any all.py files, which are filled with relative imports. Can you clarify?

Only relative imports and cimports in Cython files are relevant for the purpose of resolving the problems of current Cython 3.0.x with namespace packages. I made these changes back in September or so in #35095 because of an acute need for this in the modularization. I opened PR #36228 for the task of upstreaming this into mainline Sage.

Changes to all.py files are irrelevant for the Cython issue. Moreover, in #35095, many all.py files are completely restructured for modularization purposes, so changes there have to be done separately from changes in Sage develop.

This PR is a cherry-pick from my branch in #35095, the three commits indicated #36572 (comment), with any conflicts in cherry picking resolved to "no change".

Got it, thank you. The changes look good to me, too, so I will echo the approval from @dcoudert.

tobiasdiez · 2023-11-07T21:31:38Z

In the meantime, is the main disagreement about which changes get merged first? The other branches have been rebased on top of this one, so why not proceed with this one?

Of course, I can always open a PR that contains various subsets of changes proposed in other PRs. But there is only value in doing this if there is a problem with the original PRs (otherwise these other PRs could be directly merged). The only problem that has been pointed out so far is that Matthias doesn't want to resolve the merge conflicts in 'all.py' files introduced in these PRs. But someone has to resolve this conflicts at some point, so why not do it now?

mkoeppe · 2023-11-15T21:43:51Z

Thanks!

tobiasdiez · 2023-11-16T00:42:14Z

That is only one of the points in #36666 (comment).

mkoeppe · 2023-11-16T02:36:54Z

That is only one of the points in #36666 (comment).

Would you clarify for us: Do you mean that you are also doubling down on your false claim of authorship (the 2nd point in your list)?

mkoeppe · 2023-11-29T19:46:59Z

That is only one of the points in #36666 (comment).

Would you clarify for us: Do you mean that you are also doubling down on your false claim of authorship (the 2nd point in your list)?

@tobiasdiez My question does require an answer.

tobiasdiez · 2023-12-04T16:05:25Z

That is only one of the points in #36666 (comment).

Would you clarify for us: Do you mean that you are also doubling down on your false claim of authorship (the 2nd point in your list)?

@tobiasdiez My question does require an answer.

Sorry, I read that as a rhetoric question. We both know that I didn't authored these commits, and I never claimed I did. Your little rebase experiment should have convinced you that (all?/most of?) your changes here were already contained in my PRs. So I think a bit of attribution does not hurt.

mkoeppe · 2023-12-05T01:23:49Z

That is only one of the points in #36666 (comment).

Would you clarify for us: Do you mean that you are also doubling down on your false claim of authorship (the 2nd point in your list)?

@tobiasdiez My question does require an answer.

Sorry, I read that as a rhetoric question. We both know that I didn't authored these commits,

Yes, we both do know that.
The problem is that you have tried to mislead the public about it.

and I never claimed I did.

As part of setting the status back to "needs work", you demanded in #36666 (comment):

Add me as coauthor for proper attribution

Using the word "proper" here means that you were accusing me of improperly withholding attribution.

Your little rebase experiment

That's patronizing language, unwelcome here.

should have convinced you that (all?/most of?) your changes here were already contained in my PRs. So I think a bit of attribution does not hurt.

Here again you are writing to mislead the public.

When you opened #36572 on Oct 29, the same day I informed you in #36572 (comment) that I made the changes to the Cython files already in #35095 for the whole Sage library. (On Oct 30, I added the precise commits to this comment for reference.)

On Oct 30, nevertheless you opened #36588 and #36589, which overlap with my changes. (Apparently you were unconcerned about attribution at that time!)

On Nov 3, I informed you in #36588 (comment) and #36589 (comment) that "The only thing that is needed to solve the problem with Cython is to take care of relative cimports, not imports. Doing this in one PR creates lots of gratuitous merge conflicts with #35095." (Note that I did not demand any changes.)

You responded in #36588 (comment): "my main motivation is making it work with pytest. For this the normal imports have to be changed as well." (You seemed unconcerned about my workload of resolving the resulting merge conflicts: "I'm sorry for the merge conflicts, but they should be easy to resolve.")

I opened the present PR, which is cherry-picked carefully from #35095 and does not create merge conflicts. The rest can be read above.

…_sage_arith

github-actions · 2023-12-06T04:19:23Z

Documentation preview for this PR (built with commit b2222c3; changes) is ready! 🎉

mkoeppe · 2023-12-06T07:56:27Z

Regarding point 4 of #36666 (comment),

Revert your recent force pushes to my PRs

... last week Tobias undid the rebase of the branches of #36572, #36588, #36589, and today his original branches have been merged into develop.

This makes the CI conda pass by marking affected files as "known baseline failures", thus reducing the noise that this workflow makes on PRs (including sagemath#36666 (comment)) Less intrusive version of sagemath#36372. In addition to the failure marked there, here we collect a number of more failures observed over the course of a week in https://github.com/sagemath/sage/actions/workflows/ci- conda.yml?query=is%3Acompleted   ### 📝 Checklist     - [x] The title is concise, informative, and self-explanatory. - [ ] The description explains in detail what this PR is about. - [x] I have linked a relevant issue or discussion. - [ ] I have created tests covering the changes. - [ ] I have updated the documentation accordingly. ### ⌛ Dependencies   URL: sagemath#36678 Reported by: Matthias Köppe Reviewer(s): Kwankyu Lee, Tobias Diez

…e relative by absolute imports   We restructure the `all.py` files for modularization. Guided by the technical constraints outlined in https://groups.google.com/g/sage-devel/c/ozh-7ZZ848s, sagemath#35095 defines distribution packages (pip-installable packages) **sagemath-{brial, combinat, eclib, flint, gap, giac, glpk, graphs, groups, homfly, lcalc, libbraiding, libecm, linbox, modules, mpmath, ntl, pari, plot, polyhedra, schemes, singular, standard-no-symbolics, symbolics}**. When a namespace package such as `sage.misc` is filled by modules from several distribution packages, we create modules named: - `src/sage/misc/all__sagemath_environment.py` - `src/sage/misc/all__sagemath_objects.py` - `src/sage/misc/all__sagemath_repl.py` Import statements are moved from `src/sage/misc/all.py` to these files as appropriate, and `src/sage/misc/all.py` imports `*` from there. Also some imports are replaced by lazy imports. The new files provide the top level namespaces for the modularized distribution packages, thus [enabling modularized testing](https://doc.s agemath.org/html/en/developer/packaging_sage_library.html#testing- distribution-packages). This design was introduced in sagemath#29865 (merged in Jan 2022, early in the Sage 9.6 development cycle).   - Copied from sagemath#35095. - Part of sagemath#29705 Moreover, applied a one-line command to replace relative by absolute imports, thus complementing sagemath#36666, which does not touch `.all*` files. The changes to other files in `sage.modules` etc. come from the PRs sagemath#36597, sagemath#36572, sagemath#36588, sagemath#36589 merged here and do not need review.  ### 📝 Checklist     - [x] The title is concise, informative, and self-explanatory. - [x] The description explains in detail what this PR is about. - [x] I have linked a relevant issue or discussion. - [ ] I have created tests covering the changes. - [ ] I have updated the documentation accordingly. ### ⌛ Dependencies  - Depends on sagemath#36597 (merged here to resolve merge conflict) - Depends on sagemath#36572 (merged here to resolve merge conflict) - Depends on sagemath#36588 (merged here to resolve merge conflict) - Depends on sagemath#36589 (merged here to resolve merge conflict) - Depends on sagemath#36449 (merged here to resolve merge conflict)  URL: sagemath#36676 Reported by: Matthias Köppe Reviewer(s): David Coudert, John H. Palmieri, Matthias Köppe, Tobias Diez

@mkoeppe

Author: @mkoeppe, based on @tobiasdiez's config in sagemath#36503. Adding a configuration for the ruff linter as proposed in sagemath#36503 is timely and uncontroversial. However, sagemath#36503 bundled this gratuitously with the controversial design of creating a `pyproject.toml` file in SAGE_ROOT. `pyproject.toml` -- by definition in PEP 518, PEP 621 -- marks a directory as a Python project, i.e., the source directory of a Python distribution package (sagemath#36503 (comment)). Generalizing the use of `pyproject.toml` to "[non-package projects](https://peps.python.org/pep-0735/#motivation)" is still subject to discussion in the Python community in [PEP 735](https://peps.python.org/pep-0735/) (Nov 2023). **The scope of PRs should be chosen to minimize friction, not to maximize friction.** sagemath#36726 (comment) Here we remove the artificial friction from the gratuitous bundling, by configuring ruff in `ruff.toml` instead. Configuration in ruff.toml and pyproject.toml has equal validity / standing per [ruff documentation](https://docs.astral.sh/ruff/configuration/#config-file- discovery). And this is the location of most of our other linter configuration files. Reference on previous common denominator PRs: sagemath#36666 (comment), sagemath#36666 (comment), sagemath#36572 (comment), sagemath#36960 (comment), sagemath#36960 (comment), ...    ### 📝 Checklist     - [x] The title is concise, informative, and self-explanatory. - [x] The description explains in detail what this PR is about. - [x] I have linked a relevant issue or discussion. - [ ] I have created tests covering the changes. - [ ] I have updated the documentation accordingly. ### ⌛ Dependencies   URL: sagemath#37452 Reported by: Matthias Köppe Reviewer(s): Giacomo Pope, Matthias Köppe

…e relative by absolute imports   We restructure the `all.py` files for modularization. Guided by the technical constraints outlined in https://groups.google.com/g/sage-devel/c/ozh-7ZZ848s, sagemath#35095 defines distribution packages (pip-installable packages) **sagemath-{brial, combinat, eclib, flint, gap, giac, glpk, graphs, groups, homfly, lcalc, libbraiding, libecm, linbox, modules, mpmath, ntl, pari, plot, polyhedra, schemes, singular, standard-no-symbolics, symbolics}**. When a namespace package such as `sage.misc` is filled by modules from several distribution packages, we create modules named: - `src/sage/misc/all__sagemath_environment.py` - `src/sage/misc/all__sagemath_objects.py` - `src/sage/misc/all__sagemath_repl.py` Import statements are moved from `src/sage/misc/all.py` to these files as appropriate, and `src/sage/misc/all.py` imports `*` from there. Also some imports are replaced by lazy imports. The new files provide the top level namespaces for the modularized distribution packages, thus [enabling modularized testing](https://doc.s agemath.org/html/en/developer/packaging_sage_library.html#testing- distribution-packages). This design was introduced in sagemath#29865 (merged in Jan 2022, early in the Sage 9.6 development cycle).   - Copied from sagemath#35095. - Part of sagemath#29705 Moreover, applied a one-line command to replace relative by absolute imports, thus complementing sagemath#36666, which does not touch `.all*` files. The changes to other files in `sage.modules` etc. come from the PRs sagemath#36597, sagemath#36572, sagemath#36588, sagemath#36589 merged here and do not need review.  ### 📝 Checklist     - [x] The title is concise, informative, and self-explanatory. - [x] The description explains in detail what this PR is about. - [x] I have linked a relevant issue or discussion. - [ ] I have created tests covering the changes. - [ ] I have updated the documentation accordingly. ### ⌛ Dependencies  - Depends on sagemath#36597 (merged here to resolve merge conflict) - Depends on sagemath#36572 (merged here to resolve merge conflict) - Depends on sagemath#36588 (merged here to resolve merge conflict) - Depends on sagemath#36589 (merged here to resolve merge conflict) - Depends on sagemath#36449 (merged here to resolve merge conflict)  URL: sagemath#36676 Reported by: Matthias Köppe Reviewer(s): David Coudert, John H. Palmieri, Kwankyu Lee, Matthias Köppe, Tobias Diez

mkoeppe added the t: refactoring label Nov 6, 2023

mkoeppe mentioned this pull request Nov 6, 2023

Use Cython 3 implicit namespace package support #36228

Draft

5 tasks

mkoeppe self-assigned this Nov 6, 2023

mkoeppe added the s: needs review label Nov 6, 2023

tobiasdiez added pending and removed s: needs review labels Nov 7, 2023

mkoeppe added s: needs review and removed pending labels Nov 7, 2023

mkoeppe requested a review from dcoudert November 7, 2023 16:03

This was referenced Nov 7, 2023

Replace relative imports by absolute ones in categories #36572

Merged

Replace relative imports by absolute ones in rings #36588

Merged

Replace relative imports by absolute ones in a few packages #36589

Merged

dcoudert approved these changes Nov 7, 2023

View reviewed changes

tobiasdiez closed this Nov 7, 2023

github-actions bot removed the s: needs review label Nov 7, 2023

mkoeppe reopened this Nov 7, 2023

mkoeppe added the s: needs review label Nov 7, 2023

mkoeppe requested a review from jhpalmieri November 7, 2023 17:45

jhpalmieri approved these changes Nov 7, 2023

View reviewed changes

jhpalmieri added s: positive review and removed s: needs review labels Nov 15, 2023

tobiasdiez added s: needs work and removed s: positive review labels Nov 16, 2023

mkoeppe added s: needs review and removed s: needs work labels Nov 16, 2023

mkoeppe requested a review from roed314 November 25, 2023 17:25

Merge remote-tracking branch 'upstream/develop' into absolute_cimport…

b2222c3

…_sage_arith

mkoeppe added s: positive review and removed s: needs review labels Dec 6, 2023

vbraun merged commit a45062c into sagemath:develop Dec 10, 2023
15 of 19 checks passed

github-actions bot removed the s: positive review label Dec 10, 2023

mkoeppe added this to the sage-10.3 milestone Dec 10, 2023

mkoeppe deleted the absolute_cimport_sage_arith branch December 10, 2023 22:28

mkoeppe added the disputed PR is waiting for community vote, see https://groups.google.com/g/sage-devel/c/IgBYUJl33SQ label Dec 23, 2023

mkoeppe mentioned this pull request Dec 31, 2023

Meta-ticket: Modularize sagelib into separate distributions (pip-installable packages) sagemath-... for Sage 10.x #29705

Open

mkoeppe mentioned this pull request Feb 24, 2024

Add config for ruff (#36503, unbundled) #37452

Merged

5 tasks

saraedum mentioned this pull request Apr 29, 2024

Restructure sage.*.all for modularization, replace relative by absolute imports #37900

Open

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Replace relative imports by absolute ones in `sage.{algebras,arith,categories,cpython,data_structures,misc,modular,rings,sat,symbolic}` #36666

Replace relative imports by absolute ones in `sage.{algebras,arith,categories,cpython,data_structures,misc,modular,rings,sat,symbolic}` #36666

mkoeppe commented Nov 6, 2023 •

edited

Loading

tobiasdiez commented Nov 7, 2023

mkoeppe commented Nov 7, 2023

dcoudert left a comment •

edited

Loading

tobiasdiez commented Nov 7, 2023

dcoudert commented Nov 7, 2023

mkoeppe commented Nov 7, 2023

jhpalmieri commented Nov 7, 2023

jhpalmieri commented Nov 7, 2023 •

edited

Loading

tobiasdiez commented Nov 7, 2023

mkoeppe commented Nov 7, 2023

jhpalmieri commented Nov 7, 2023

tobiasdiez commented Nov 7, 2023

mkoeppe commented Nov 15, 2023

tobiasdiez commented Nov 16, 2023

mkoeppe commented Nov 16, 2023

mkoeppe commented Nov 29, 2023

tobiasdiez commented Dec 4, 2023

mkoeppe commented Dec 5, 2023 •

edited

Loading

github-actions bot commented Dec 6, 2023

mkoeppe commented Dec 6, 2023

Replace relative imports by absolute ones in sage.{algebras,arith,categories,cpython,data_structures,misc,modular,rings,sat,symbolic} #36666

Replace relative imports by absolute ones in sage.{algebras,arith,categories,cpython,data_structures,misc,modular,rings,sat,symbolic} #36666

Conversation

mkoeppe commented Nov 6, 2023 • edited Loading

📝 Checklist

⌛ Dependencies

tobiasdiez commented Nov 7, 2023

mkoeppe commented Nov 7, 2023

dcoudert left a comment • edited Loading

Choose a reason for hiding this comment

tobiasdiez commented Nov 7, 2023

dcoudert commented Nov 7, 2023

mkoeppe commented Nov 7, 2023

jhpalmieri commented Nov 7, 2023

jhpalmieri commented Nov 7, 2023 • edited Loading

tobiasdiez commented Nov 7, 2023

mkoeppe commented Nov 7, 2023

jhpalmieri commented Nov 7, 2023

tobiasdiez commented Nov 7, 2023

mkoeppe commented Nov 15, 2023

tobiasdiez commented Nov 16, 2023

mkoeppe commented Nov 16, 2023

mkoeppe commented Nov 29, 2023

tobiasdiez commented Dec 4, 2023

mkoeppe commented Dec 5, 2023 • edited Loading

github-actions bot commented Dec 6, 2023

mkoeppe commented Dec 6, 2023

Replace relative imports by absolute ones in `sage.{algebras,arith,categories,cpython,data_structures,misc,modular,rings,sat,symbolic}` #36666

Replace relative imports by absolute ones in `sage.{algebras,arith,categories,cpython,data_structures,misc,modular,rings,sat,symbolic}` #36666

mkoeppe commented Nov 6, 2023 •

edited

Loading

dcoudert left a comment •

edited

Loading

jhpalmieri commented Nov 7, 2023 •

edited

Loading

mkoeppe commented Dec 5, 2023 •

edited

Loading