pip 24, setuptools 69.5.1, hatchling 1.22.5, hatch_fancy_pypi_readme 24.1.0, platformdirs 4.2.0, packaging 24.0, trove_classifiers 2024.4.10, wheel 0.43.0 #37277

mkoeppe · 2024-02-10T04:27:18Z

📝 Checklist

The title is concise, informative, and self-explanatory.
The description explains in detail what this PR is about.
I have linked a relevant issue or discussion.
I have created tests covering the changes.
I have updated the documentation accordingly.

⌛ Dependencies

mkoeppe · 2024-02-11T08:02:38Z

There is a strange malfunction on ubuntu-mantic-standard
https://github.com/sagemath/sage/actions/runs/7852863047/job/21431835515?pr=37277#step:11:10319

sage -t --random-seed=293799319091173329856561880001578538606 src/sage/misc/package.py
**********************************************************************
File "src/sage/misc/package.py", line 186, in sage.misc.package.pip_installed_packages
Failed example:
    d['prompt-toolkit']
Exception raised:
    Traceback (most recent call last):
      File "/sage/src/sage/doctest/forker.py", line 712, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/sage/src/sage/doctest/forker.py", line 1147, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.misc.package.pip_installed_packages[3]>", line 1, in <module>
        d['prompt-toolkit']
        ~^^^^^^^^^^^^^^^^^^
    KeyError: 'prompt-toolkit'
**********************************************************************

pip list sees prompt_toolkit (underscore) instead prompt-toolkit (dash)

But this build is incremental from an ancient version:

### Incremental build from SageMath version 10.1.beta6, Release Date: 2023-07-09

so can probably ignore.

github-actions · 2024-04-01T10:06:38Z

Documentation preview for this PR (built with commit d2bee8e; changes) is ready! 🎉
This preview will update shortly after each push to this PR.

mkoeppe · 2024-04-15T06:04:43Z

The build failures in CI Linux are unrelated (#37786)

kwankyu · 2024-04-19T03:40:55Z

There is a strange malfunction on ubuntu-mantic-standard https://github.com/sagemath/sage/actions/runs/7852863047/job/21431835515?pr=37277#step:11:10319

sage -t --random-seed=293799319091173329856561880001578538606 src/sage/misc/package.py
**********************************************************************
File "src/sage/misc/package.py", line 186, in sage.misc.package.pip_installed_packages
Failed example:
    d['prompt-toolkit']
Exception raised:
    Traceback (most recent call last):
      File "/sage/src/sage/doctest/forker.py", line 712, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/sage/src/sage/doctest/forker.py", line 1147, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.misc.package.pip_installed_packages[3]>", line 1, in <module>
        d['prompt-toolkit']
        ~^^^^^^^^^^^^^^^^^^
    KeyError: 'prompt-toolkit'
**********************************************************************

pip list sees prompt_toolkit (underscore) instead prompt-toolkit (dash)

But this build is incremental from an ancient version:

### Incremental build from SageMath version 10.1.beta6, Release Date: 2023-07-09

so can probably ignore.

I get the same failure with sage built from scratch on mac.

kwankyu · 2024-04-19T04:01:52Z

Both pip install prompt_toolkit and pip install prompt-toolkit work, and we have

upstream_url=https://pypi.io/packages/source/p/prompt_toolkit/prompt_toolkit-VERSION.tar.gz

Why prompt-tookit (with dash) is expected?

mkoeppe · 2024-04-24T06:12:05Z

Both pip install prompt_toolkit and pip install prompt-toolkit work

Yes, these are all equivalent.

I see the failure now on various platforms. e.g. https://github.com/mkoeppe/sage/actions/runs/8761823524/job/24053507902#step:11:12409

I'll investigate.

src/sage/misc/package.py

… tool versions (variant)

kwankyu

LGTM.

kwankyu · 2024-04-25T05:29:49Z

Good. Thanks.

mkoeppe · 2024-04-25T05:30:54Z

Thanks for the review!

…fancy_pypi_readme 24.1.0, platformdirs 4.2.0, packaging 24.0, trove_classifiers 2024.4.10, wheel 0.43.0      ### 📝 Checklist     - [x] The title is concise, informative, and self-explanatory. - [ ] The description explains in detail what this PR is about. - [ ] I have linked a relevant issue or discussion. - [ ] I have created tests covering the changes. - [ ] I have updated the documentation accordingly. ### ⌛ Dependencies   URL: sagemath#37277 Reported by: Matthias Köppe Reviewer(s): Kwankyu Lee, Matthias Köppe

@kwankyu

…arate jobs for pyright, build, modularized tests, long tests    Running the containers explicitly, instead of using the declarative `container:` feature of GH Actions gives us more control: - we can create more space on the host if necessary; we just scraped by an out of space condition in sagemath#36473 / sagemath#36469 (comment) - we can run some operations outside of the container but in the same job; this will make the separate "Get CI fixes" jobs unnecessary, addressing the cosmetic concerns from sagemath#36338 (comment), sagemath#36349 - it enables caching between the various workflows (as first discussed in sagemath#36446). We split out static type checking with pyright into its separate workflow: - **pyright.yml**: As static checking does not need a build of the Sage library, for PRs that do not make any changes to packages, there's nothing to build, and the workflow gives a fast turnaround just after 10 minutes. For applying the CI fixes from blocker tickets, this workflow uses the technique favored in sagemath#36349. The workflow **build.yml** first launches a job: - **test-new:** It builds incrementally (using a tox-generated `Dockerfile` and https://github.com/marketplace/actions/build-and-push- docker-images) and does the quick incremental test. This completes within 10 to 20 minutes when there's no change. After this is completed, more jobs are launched: - **test-mod:** It again builds incrementally and tests a modularized distribution. Later (with more from sagemath#35095), more jobs will be added to this matrix job for other distributions. - **test-long:** It again builds incrementally and runs the long test. The workflows **doc-build.yml** and **doc-build-pdf.yml** again build incrementally and then build the documentation. The diffing code for the HTML documentation now runs in the host, not the container, which simplifies things. (To show that diffing still works, we include a small change to the Sage library.) Splitting the workflow jobs implements @kwankyu's suggestion from: - sagemath#35652 (comment) (Fixes sagemath#35814) The steps "again builds incrementally" actually use the GH Actions cache, https://docs.docker.com/build/ci/github-actions/cache/#cache- backend-api. When nothing is cached and the 3 workflows are launched in parallel, they may each build the same thing. But when there's congestion that leads to the workflows to be run serially, the 2nd and 3rd workflow will be able to use the cache from the 1st workflow. This elasticity may be helpful in reducing congestion. There is a rather small per-project limit of 10 GB for this cache, so we'll have to see how effectively caching works in practice. See https://github.com/sagemath/sage/actions/caches   ### 📝 Checklist     - [x] The title is concise, informative, and self-explanatory. - [x] The description explains in detail what this PR is about. - [x] I have linked a relevant issue or discussion. - [ ] I have created tests covering the changes. - [ ] I have updated the documentation accordingly. ### ⌛ Dependencies  - Depends on sagemath#36938 (merged here) - Depends on sagemath#36951 (merged here) - Depends on sagemath#37351 (merged here) - Depends on sagemath#37750 (merged here) - Depends on sagemath#37277 (merged here)  URL: sagemath#36498 Reported by: Matthias Köppe Reviewer(s): Kwankyu Lee, Matthias Köppe

mkoeppe added the c: packages: standard label Feb 10, 2024

mkoeppe self-assigned this Feb 10, 2024

mkoeppe added the s: needs review label Feb 11, 2024

mkoeppe changed the title ~~pip 24, setuptools 69.0.3, hatchling 1.21.1, hatch_fancy_pypi_readme 24.1.0~~ pip 24, setuptools 69.1.0, hatchling 1.21.1, hatch_fancy_pypi_readme 24.1.0 Feb 12, 2024

mkoeppe force-pushed the pip-24 branch from 8fefc03 to bc2db49 Compare February 14, 2024 00:32

mkoeppe changed the title ~~pip 24, setuptools 69.1.0, hatchling 1.21.1, hatch_fancy_pypi_readme 24.1.0~~ pip 24, setuptools 69.1.1, hatchling 1.21.1, hatch_fancy_pypi_readme 24.1.0 Feb 24, 2024

mkoeppe force-pushed the pip-24 branch from 869bebb to 9e585eb Compare February 25, 2024 17:44

mkoeppe force-pushed the pip-24 branch from 9e585eb to f578629 Compare March 26, 2024 01:58

mkoeppe changed the title ~~pip 24, setuptools 69.1.1, hatchling 1.21.1, hatch_fancy_pypi_readme 24.1.0~~ pip 24, setuptools 69.2.0, hatchling 1.21.1, hatch_fancy_pypi_readme 24.1.0, platformdirs 4.2.0, packaging 24.0, wheel 0.43.0 Mar 27, 2024

mkoeppe force-pushed the pip-24 branch from 77578fd to c28795f Compare April 1, 2024 00:44

mkoeppe added 11 commits April 14, 2024 10:36

build/pkgs/pip: Update to 24.0

36ce98c

build/pkgs/setuptools: Update to 69.0.3

dd24158

build/pkgs/hatchling: Update to 1.21.1

8bfd3ff

build/pkgs/hatch_fancy_pypi_readme: Update to 24.1.0

fc139cb

build/pkgs/setuptools: Update to 69.1.0

f51abda

build/pkgs/setuptools: Update to 69.1.1

451db04

build/pkgs/setuptools: Update to 69.2.0

b5cc0ac

build/pkgs/wheel: Update to 0.43.0

d4481b8

build/pkgs/packaging: Update to 24.0

8cac22c

build/pkgs/platformdirs: Update to 4.2.0

05f9b74

build/pkgs/hatchling: Update to 1.22.5

1788024

mkoeppe force-pushed the pip-24 branch from 54fc350 to 1788024 Compare April 14, 2024 17:37

mkoeppe changed the title ~~pip 24, setuptools 69.2.0, hatchling 1.21.1, hatch_fancy_pypi_readme 24.1.0, platformdirs 4.2.0, packaging 24.0, wheel 0.43.0~~ pip 24, setuptools 69.2.0, hatchling 1.22.5, hatch_fancy_pypi_readme 24.1.0, platformdirs 4.2.0, packaging 24.0, wheel 0.43.0 Apr 14, 2024

build/pkgs/trove_classifiers: Update to 2024.4.10

02e2ddc

build/pkgs/setuptools: Update to 69.5.1

fce35c8

kwankyu reviewed Apr 25, 2024

View reviewed changes

src/sage/misc/package.py Show resolved Hide resolved

src/sage/misc/package.py: Make 'prompt-toolkit' doctest pass with all…

a177ef2

… tool versions (variant)

mkoeppe force-pushed the pip-24 branch from 2cf664d to a177ef2 Compare April 25, 2024 05:13

kwankyu approved these changes Apr 25, 2024

View reviewed changes

kwankyu added s: positive review and removed s: needs review labels Apr 25, 2024

src/sage/misc/package.py: Make comment more specific

d2bee8e

This was referenced Apr 28, 2024

sage-bootstrap: update cryptographic hashes #37726

Open

CI build, doc-build: Run containers explicitly, separate jobs for pyright, build, modularized tests, long tests #36498

Merged

vbraun merged commit 2d77ac5 into sagemath:develop May 2, 2024
12 of 32 checks passed

github-actions bot removed the s: positive review label May 2, 2024

mkoeppe added this to the sage-10.4 milestone May 2, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

pip 24, setuptools 69.5.1, hatchling 1.22.5, hatch_fancy_pypi_readme 24.1.0, platformdirs 4.2.0, packaging 24.0, trove_classifiers 2024.4.10, wheel 0.43.0 #37277

pip 24, setuptools 69.5.1, hatchling 1.22.5, hatch_fancy_pypi_readme 24.1.0, platformdirs 4.2.0, packaging 24.0, trove_classifiers 2024.4.10, wheel 0.43.0 #37277

mkoeppe commented Feb 10, 2024 •

edited

mkoeppe commented Feb 11, 2024 •

edited

github-actions bot commented Apr 1, 2024 •

edited

mkoeppe commented Apr 15, 2024

kwankyu commented Apr 19, 2024

kwankyu commented Apr 19, 2024

mkoeppe commented Apr 24, 2024

kwankyu left a comment

kwankyu commented Apr 25, 2024

mkoeppe commented Apr 25, 2024

pip 24, setuptools 69.5.1, hatchling 1.22.5, hatch_fancy_pypi_readme 24.1.0, platformdirs 4.2.0, packaging 24.0, trove_classifiers 2024.4.10, wheel 0.43.0 #37277

pip 24, setuptools 69.5.1, hatchling 1.22.5, hatch_fancy_pypi_readme 24.1.0, platformdirs 4.2.0, packaging 24.0, trove_classifiers 2024.4.10, wheel 0.43.0 #37277

Conversation

mkoeppe commented Feb 10, 2024 • edited

📝 Checklist

⌛ Dependencies

mkoeppe commented Feb 11, 2024 • edited

github-actions bot commented Apr 1, 2024 • edited

mkoeppe commented Apr 15, 2024

kwankyu commented Apr 19, 2024

kwankyu commented Apr 19, 2024

mkoeppe commented Apr 24, 2024

kwankyu left a comment

Choose a reason for hiding this comment

kwankyu commented Apr 25, 2024

mkoeppe commented Apr 25, 2024

mkoeppe commented Feb 10, 2024 •

edited

mkoeppe commented Feb 11, 2024 •

edited

github-actions bot commented Apr 1, 2024 •

edited