Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

sagemath: update to 10.0. #43659

Merged
merged 5 commits into from May 31, 2023
Merged

Conversation

tornaria
Copy link
Contributor

This builds on top of #43225 for convenience.

Testing the changes

  • I tested the changes in this PR: briefly

I've been following beta releases and now rc0 includes all of our patches 馃挭

All doctests pass with the version of packages in void except for python-Babel which must be updated (as in #43497) and is included here as part of #43225.

I've switched my system sagemath to this version as everything seems to be working ok. The testsuite is 10-15% faster.

@dkwo: enjoy.

@tornaria tornaria marked this pull request as draft April 28, 2023 20:31
@icp1994
Copy link
Contributor

icp1994 commented Apr 28, 2023

I really hoped they would include updated networkx in this release. I have been meaning to update python3-networkx here to v3 for months but I can't confirm if that version works with sage.

@tornaria
Copy link
Contributor Author

I really hoped they would include updated networkx in this release. I have been meaning to update python3-networkx here to v3 for months but I can't confirm if that version works with sage.

Do you have a draft PR for updating python3-networkx? I can try it.

@tornaria
Copy link
Contributor Author

I really hoped they would include updated networkx in this release. I have been meaning to update python3-networkx here to v3 for months but I can't confirm if that version works with sage.

I think I have a reasonable fix for 10.0.rc0 and it may even work for 9.8. Still testing.
If everything works I will submit upstream and include the patch in #43225 so sagemath in void supports networkx 3.1. After #43225 is merged we can update networkx to 3.1.

Hopefully I will be able to get this fix merged in time for sagemath 10.0, but if not we can patch our sagemath.

@tornaria
Copy link
Contributor Author

See sagemath/sage#35584 for support of networkx 3.1 in sagemath.

@tornaria tornaria force-pushed the sagemath-next branch 2 times, most recently from c04d7c6 to 7506551 Compare May 5, 2023 01:46
@tornaria tornaria changed the title [RFC] sagemath: update to 10.0.rc0. [RFC] sagemath: update to 10.0.rc1. May 5, 2023
@tornaria
Copy link
Contributor Author

tornaria commented May 5, 2023

As of now we need just two patches coming from upstream:

Hopefully both will be merged upstream for the upcoming 10.0 release.

@tornaria
Copy link
Contributor Author

tornaria commented May 5, 2023

CI failure on i686, when checking linbox:

...
 FAIL: test-minpoly
...
============================================================================
Testsuite summary for LinBox 1.7.0
============================================================================
# TOTAL: 40
# PASS:  39
# SKIP:  0
# XFAIL: 0
# FAIL:  1
# XPASS: 0
# ERROR: 0
============================================================================
See tests/test-suite.log
Please report to linbox-use@googlegroups.com
============================================================================```

The exact same thing run in #42326 without any failure...

@tornaria
Copy link
Contributor Author

tornaria commented May 6, 2023

Now CI failure on x86_64-musl, when checking linbox:

...
FAIL: test-solve
...
============================================================================
Testsuite summary for LinBox 1.7.0
============================================================================
# TOTAL: 40
# PASS:  39
# SKIP:  0
# XFAIL: 0
# FAIL:  1
# XPASS: 0
# ERROR: 0
============================================================================
See tests/test-suite.log
Please report to linbox-use@googlegroups.com
============================================================================

I've run check linbox many times and I haven't been lucky (unlucky?) to trigger any failure locally 馃し

@dkwo
Copy link
Contributor

dkwo commented May 6, 2023

Maybe these ci failures are not too serious? I will test locally built pr on x86_64-musl and aarch64-glibc soon.

@tornaria
Copy link
Contributor Author

tornaria commented May 6, 2023

Maybe these ci failures are not too serious? I will test locally built pr on x86_64-musl and aarch64-glibc soon.

They could be random failures that depend on the seed -- unfortunately linbox check doesn't print the seed used for each test.

Or else, this could be some weird race condition type of heisenbug which triggers under heavy load (i.e. github actions) but not in my box.

@dkwo
Copy link
Contributor

dkwo commented May 9, 2023

On aarch64-glibc, this looks quite good: only a couple of timed outs with test=yes

parallel/map_reduce.py
sets/recursively_enumerated_sets.pyx

I can test more completely from within installed sage.

@tornaria
Copy link
Contributor Author

I've added maxima update from #36711 since sagemath is about to support it (sagemath/sage#35619 is moving and the patch they have works for me in x86_64 -- testing now in musl and i686).

@jbgg are you still interested on maxima? I'll adopt the void package if you don't mind, so it's easier for me to track it.

@tornaria tornaria changed the title [RFC] sagemath: update to 10.0.rc1. [RFC] sagemath: update to 10.0.rc2. May 10, 2023
@tornaria tornaria changed the title [RFC] sagemath: update to 10.0.rc2. [RFC] sagemath: update to 10.0.rc3. May 13, 2023
@dkwo
Copy link
Contributor

dkwo commented May 13, 2023

thanks! I'll test soon.

@dkwo
Copy link
Contributor

dkwo commented May 15, 2023

maxima and maxima-emacs conflict for: should we also remove the maxima-emacs pkg?

@tornaria
Copy link
Contributor Author

maxima and maxima-emacs conflict for: should we also remove the maxima-emacs pkg?

On the contrary. I removed maxima-emacs package thus the conflict (since these files are in the maxima-emacs package in the repo and in the maxima package in the update).

I think the simplest solution is to not remove the maxima-emacs package, instead make it into an empty dummy transitional package. Then there shouldn't be file conflicts for the moved files. Then in the next update we can remove maxima-emacs.

@dkwo
Copy link
Contributor

dkwo commented May 15, 2023

Hmm, what I meant is adding maxima-emacs to removed-pkgs: could that work?
Other than that, maxima builds fine. And sage as well: on top of #43659 (comment) , I only have structure/coerce_actions.pyx # 3 doctest failed.

@tornaria
Copy link
Contributor Author

Hmm, what I meant is adding maxima-emacs to removed-pkgs: could that work?

I don't know, maybe adding to conflicts, etc... but I think the empty transition pkg should be ok and the easiest solution, also so that we "prune" the old maxima-emacs from the repo (later we just remove and add to removed-pkgs)

And sage as well: on top of #43659 (comment) , I only have structure/coerce_actions.pyx # 3 doctest failed.

Can you copy the full output for all the tests that fail? Have you tried running tests with --long as well? (./xbps-src check -K will run tests in --long mode, while -Q will run tests in normal mode -- the total cpu time should be about double with --long).

@dkwo
Copy link
Contributor

dkwo commented May 15, 2023

With sage -t --long, there are only two failures:
fail1.log
fail2.log
while structure/coerce_actions.pyx passes all tests
fail3.log

@tornaria
Copy link
Contributor Author

With sage -t --long, there are only two failures: fail1.log fail2.log while structure/coerce_actions.pyx passes all tests fail3.log

Can you try building python3 natively? When built in cross, the configure step gives:

checking for broken sem_getvalue... yes

which then causes this NotImplementedError.

Please have a look at this configure check --- if you get "no" there I expect your natively built python3 will work ok.

If this works then the fix for cross building python3 is adding ac_cv_broken_sem_getvalue=no to configure_args (you could try building python3 cross on a x86_64 with this change, then see if the resulting pkg works, then submit a PR for python3 so the pkg in void repos works out of the box). The problem is that this is checked by compiling and running a program which won't work in cross -- but if we know sem_gevalue is ok in all of our arches, we can always force it (otherwise, force it in the arches that are known to work).


What about the failures in structure/coerce_actions.pyx? Is this passing with --long and failing without it? Is it failing only for a particular value of --random-seed?

BTW: you can use xbps-src to run checks in selected files and/or with a chosen random seed. Just create a file sagemath-check in your void-packages root directory. You can add filenames and options for the sage check step. E.g.

--random-seed=278209243980132307357977126905193403766
src/sage/parallel/map_reduce.py
src/sage/sets/recursively_enumerated_sets.pyx
src/sage/structure/coerce_actions.pyx

Add --long if you want, etc. You can use # to comment. If the file is "empty" (as in not existent or everything is commented out) then checks run normally.

@dkwo
Copy link
Contributor

dkwo commented May 16, 2023

Rebuilding python3 natively, I see checking for broken sem_getvalue... no, and then both tests

parallel/map_reduce.py
sets/recursively_enumerated_sets.pyx

pass with just sage -t. The structure/coerce_actions.pyx is also passing both with and without --long, maybe it was an issue with that particular random seed?

@dkwo
Copy link
Contributor

dkwo commented May 16, 2023

opened a pr for python3. everything looks good to me :)
i can perhaps run again the full testsuite once the 10 is out.
thanks a lot!

@dkwo
Copy link
Contributor

dkwo commented May 18, 2023

Do you mind explicitly adding to the sagemath template a line like
nocross="due to ntl (flintlib), fflas-ffpack, givaro, linbox, cysignals, sympow, maxima"
so that we keep track of packages blocking cross-compile? All build fine natively.

@tornaria tornaria changed the title [RFC] sagemath: update to 10.0.rc3. [RFC] sagemath: update to 10.0. May 21, 2023
@tornaria
Copy link
Contributor Author

10.0 is released.

Before merging, I want to have a look at sagemath/sage#35658 (support sphinx 7). Presumably it won't affect us since we don't build the documentation.

Also:
 - disable sbcl on aarch64
 - don't makedepends on emacs (avoid a nocross build dep)
 - move maxima-emacs files to main pkg
@tornaria
Copy link
Contributor Author

All checks passed.

It turns out that we don't really need sagemath/sage#35658 so I'm removing it.

@dkwo: maybe you want to test it on aarch64.

Also:
 - support networkx 3.1 (sagemath#35584)
 - support linbox 1.7.0 (sagemath#35612)
 - support maxima 5.46.0 (sagemath#35619)
 - support sympy 1.12 (sagemath#35635)
@tornaria tornaria changed the title [RFC] sagemath: update to 10.0. sagemath: update to 10.0. May 21, 2023
@tornaria tornaria marked this pull request as ready for review May 22, 2023 02:07
@dkwo
Copy link
Contributor

dkwo commented May 23, 2023

Looks great! I expect all tests to pass also on aarch64, but won't be able to confirm until next week.

@tornaria
Copy link
Contributor Author

@dkwo : did you get a chance to test? Shall we merge this as is?

@dkwo
Copy link
Contributor

dkwo commented May 29, 2023

I will test tomorrow (I forgot that today is a holiday in the US).
I'm fine either way, I can always open a PR if anything comes up :)

@dkwo
Copy link
Contributor

dkwo commented May 30, 2023

all tests are passing out of the box on aarch64 :)

@dkwo
Copy link
Contributor

dkwo commented May 30, 2023

@leahneukirchen shall we merge this?

@leahneukirchen leahneukirchen merged commit faa5bf4 into void-linux:master May 31, 2023
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants