Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tox.ini, GitHub CI workflow - minor fixes, more homebrew configuration factors, improve docker interface #29417

Closed
mkoeppe opened this issue Mar 27, 2020 · 106 comments

Comments

@mkoeppe
Copy link
Contributor

mkoeppe commented Mar 27, 2020

This makes the following changes to tox.ini:

  • tox -e local-homebrew-macos-standard now uses python3 from homebrew.

    This attempts to reproduce the environment of https://groups.google.com/d/msg/sage-release/kU5M1QVuQQY/Zu10Q-CKAwAJ

  • tox -e local-homebrew-macos-standard-python3_xcode uses /usr/bin/python3 (was the old default)

  • tox -e local-homebrew-macos-standard-python3_pythonorg uses the Python 3.7 installation from the binary package from python.org

New toolchain factors: gcc_spkg, gcc_9.

Improved docker interface.

CC: @jhpalmieri @dimpase @orlitzky @kliem @vbraun

Component: porting

Author: Matthias Koeppe

Branch/Commit: b7bb418

Reviewer: Dima Pasechnik

Issue created by migration from https://trac.sagemath.org/ticket/29417

@mkoeppe mkoeppe added this to the sage-9.1 milestone Mar 27, 2020
@mkoeppe
Copy link
Contributor Author

mkoeppe commented Mar 27, 2020

Dependencies: #29087, #29354

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Mar 27, 2020

Branch: u/mkoeppe/tox-ci-minor-fixes

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Mar 28, 2020

Changed dependencies from #29087, #29354 to #29087, #29354, #29418

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Mar 28, 2020

Last 10 new commits:

fbb7393Fixup packages
cfcd8f2build/bin/sage-guess-package-system, build/bin/sage-print-system-package-command: Handle slackware too
136feebbuild/bin/sage-spkg: Show equivalent system packages for ALL distros
9939051Merge branch 't/29354/tox_ini__add_slackware_environments' into t/29087/CI-gh-actions-tox-configure-build
4ac9e24Reduce SAGE_NUM_THREADS to 3
1cb8f98build/pkgs/curl/distros/slackware.txt: Add openldap-client
eee2877build/pkgs/slackware.txt: Add ca-certificates
c1931f5build/pkgs/curl/distros/slackware.txt: Add libssh2
3145467Merge branch 't/29354/tox_ini__add_slackware_environments' into t/29087/CI-gh-actions-tox-configure-build
b259671[tox-local-homebrew-macos] Repair make parallelization

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Mar 28, 2020

Commit: b259671

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Mar 28, 2020

Author: Matthias Koeppe

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Mar 28, 2020

Branch pushed to git repo; I updated commit sha1. New commits:

bd24641build/bin/sage-download-file: Add option --no-check-certificate
e143939build/bin/sage-spkg: Append to SAGE_DOWNLOAD_FILE_OPTIONS instead of overwriting it
710b52dMerge branch 't/29418/sage_download_file__fix_certificate_problems_with_https_downloads_on_macos_python3' into t/29417/tox-ci-minor-fixes
b1489e7tox.ini: Set SAGE_DOWNLOAD_FILE_OPTIONS=--no-check-certificate

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Mar 28, 2020

Changed commit from b259671 to b1489e7

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Mar 29, 2020

Changed dependencies from #29087, #29354, #29418 to #29087, #29354, #29418, #29415

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Mar 30, 2020

Changed commit from b1489e7 to 846f14e

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Mar 30, 2020

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

354cf90Use conda-install --yes --quiet
df609e0Merge branch 't/29415/tox_ini__add_environment_local_conda' into t/29417/tox-ci-minor-fixes
cf27321build/bin/sage-download-file: Add option --no-check-certificate
90ea00bbuild/bin/sage-spkg: Append to SAGE_DOWNLOAD_FILE_OPTIONS instead of overwriting it
bccc2c2Merge branch 't/29418/sage_download_file__fix_certificate_problems_with_https_downloads_on_macos_python3' into t/29417/tox-ci-minor-fixes
87b7351tox.ini: Add configuration factors python3, macos-python3_xcode, macos-python3_pythonorg
329ea99tox.yml: Test more homebrew configs
de1bce2tox.yml: Add local-ubuntu-conda test
86d5c84tox.ini: Add toolchain factors
846f14etox.yml: Test more Fedora, no conda

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Mar 30, 2020

Changed dependencies from #29087, #29354, #29418, #29415 to #29087, #29418, #29415

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Mar 30, 2020

Changed commit from 846f14e to 9838fd7

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Mar 30, 2020

Branch pushed to git repo; I updated commit sha1. New commits:

9838fd7fixup

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Mar 30, 2020

Branch pushed to git repo; I updated commit sha1. New commits:

a3ccd46build/pkgs/python3/distros/homebrew.txt: New

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Mar 30, 2020

Changed commit from 9838fd7 to a3ccd46

@mkoeppe

This comment has been minimized.

@mkoeppe mkoeppe changed the title tox.ini, GitHub CI workflow - minor fixes tox.ini, GitHub CI workflow - minor fixes, more homebrew configuration factors Mar 30, 2020
@mkoeppe
Copy link
Contributor Author

mkoeppe commented Mar 30, 2020

Changed dependencies from #29087, #29418, #29415 to #29087, #29418, #29415, #29401

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Mar 30, 2020

Changed commit from a3ccd46 to 2bc1243

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Mar 30, 2020

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

6153945merge the sections on tox factors
ab11ebdelide some details from terminal output
c7fcf91change 'ls -l' to 'ls -la' and elide some details
e73bc8dElide more details, shorten directory names more
a31b2a0Reviewer patch
b7b45feReviewer patch
f7d2dc9Remove old section head
99aca39Reformat list, remove unfinished commented out section
2c53ae2Merge branch 't/29401/add_documentation_of_tox_and_github_actions_workflow_to_developer_s_manual' into t/29417/tox-ci-minor-fixes
2bc1243src/doc/en/developer/portability_testing.rst: Add documentation on new local-conda and local-homebrew environments

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Mar 31, 2020

Branch pushed to git repo; I updated commit sha1. New commits:

5dd7652tox.ini: Add more linuxmint versions
c6cc486tox.ini: IGNORE_MISSING_SYSTEM_PACKAGES=yes for old linuxmint versions
cd72da9tox.ini: Add linuxmint-i386
7986ab3tox.yml: Add more linuxmint
a54ab32tox.ini: Remove linuxmint-20 (no docker image)

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Mar 31, 2020

Changed commit from 2bc1243 to a54ab32

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Apr 2, 2020

Changed commit from a54ab32 to 4281c92

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Apr 2, 2020

Branch pushed to git repo; I updated commit sha1. New commits:

3757212tox.ini: Switch local-conda-forge to miniforge
a61c48etox.ini [local-conda]: Activate conda before calling conda install
8f58411Merge branch 't/29415/tox_ini__add_environment_local_conda' into t/29417/tox-ci-minor-fixes
4281c92Replace commands_pre by commands, to support tox < 3.4

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Apr 2, 2020

Changed commit from 4281c92 to 49aeac3

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Apr 13, 2020

comment:55

Test for 9.1.rc0 running at https://github.com/mkoeppe/sage/actions/runs/76923542

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Apr 13, 2020

Branch pushed to git repo; I updated commit sha1. New commits:

faf0ed6tox.ini: Another quoting fix

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Apr 13, 2020

Changed commit from d94969c to faf0ed6

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Apr 15, 2020

Changed commit from faf0ed6 to aed3fb8

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Apr 15, 2020

Branch pushed to git repo; I updated commit sha1. New commits:

aed3fb8src/doc/en/developer/portability_testing.rst: Explain how to read [GitHub](../wiki/GitHub) Actions page

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Apr 15, 2020

comment:58

Can we finalize the review of this one please to get it into 9.1?

@dimpase
Copy link
Member

dimpase commented Apr 15, 2020

comment:59

running GH Actions tests on this here: dimpase#3

@dimpase
Copy link
Member

dimpase commented Apr 15, 2020

comment:60

Replying to @dimpase:

running GH Actions tests on this here: dimpase#3

I see

/Users/runner/runners/2.168.0/work/sage/sage/build/bin/sage-venv "/Users/runner/runners/2.168.0/work/sage/sage/.tox/local-homebrew-macos-python3_pythonorg-standard/local"
2020-04-15T16:31:11.9205990Z /bin/sh: /Library/Frameworks/Python.framework/Versions/3.7/bin/python3: No such file or directory
2020-04-15T16:31:11.9206560Z make[1]: *** [/Users/runner/runners/2.168.0/work/sage/sage/.tox/local-homebrew-macos-python3_pythonorg-standard/local/pyvenv.cfg] Error 127
2020-04-15T16:31:11.9207060Z make[1]: Target `doc-html' not remade because of errors.

there. Isn't this ticked supposed to be fixing this?

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Apr 15, 2020

comment:61

Thanks for catching this. Turns out I forgot to install the python.org python3 in tox.yml.
(This cannot be done in tox.ini because it is not relocatable, see comment there)

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Apr 15, 2020

Changed commit from aed3fb8 to d39d627

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Apr 15, 2020

Branch pushed to git repo; I updated commit sha1. New commits:

d39d627tox.yml [python3_xcode]: Install the package

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Apr 15, 2020

Changed commit from d39d627 to ba0c266

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Apr 15, 2020

Branch pushed to git repo; I updated commit sha1. New commits:

ba0c266fixup

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Apr 15, 2020

Branch pushed to git repo; I updated commit sha1. New commits:

9060d23fixup

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Apr 15, 2020

Changed commit from ba0c266 to 9060d23

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Apr 16, 2020

Changed commit from 9060d23 to b7bb418

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Apr 16, 2020

Branch pushed to git repo; I updated commit sha1. New commits:

b7bb418tox.yml [macos-python3_pythonorg]: Use sudo with installer

@dimpase
Copy link
Member

dimpase commented Apr 16, 2020

comment:66

next try here: dimpase#4

@dimpase
Copy link
Member

dimpase commented Apr 17, 2020

comment:67

homebrew-macos-python3_pythonorg, standard (https://github.com/dimpase/sage/runs/591414002) crashes at [dochtml] stage - not sure whether one can get complete logs/dochtml.log file from GitHub run above

@dimpase
Copy link
Member

dimpase commented Apr 17, 2020

comment:68

I presume that various errors on local-macos standard runs are to be debugged individually.

e.g. on https://github.com/dimpase/sage/runs/591413919 (homebrew-macos-python3_xcode, minimal) I see ##[error] ==== ERROR IN LOG FILE artifacts/logs-commit-8bb28ec8ffd6869c6b86e3f5224333decb049d09-tox-local-homebrew-macos-python3_xcode-minimal/log/pkgs/gc-7.6.4.p0.log ==== and
Warning: Error testing package gc-7.6.4.p0 (ignored)

@dimpase
Copy link
Member

dimpase commented Apr 17, 2020

Reviewer: Dima Pasechnik

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Apr 17, 2020

comment:69

Thanks!

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Apr 17, 2020

comment:70

Yes, errors are to be fixed on their own tickets.

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Apr 17, 2020

comment:71

Some of the errors are actually "only" testsuite failures. Tickets such as #29501 clean up some of the messages.

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Apr 17, 2020

comment:72

Replying to @dimpase:

homebrew-macos-python3_pythonorg, standard (https://github.com/dimpase/sage/runs/591414002) crashes at [dochtml] stage - not sure whether one can get complete logs/dochtml.log file from GitHub run above

Yes, download the log artifact. But it only says

------------------------------------------------------------------------
(no backtrace available)
------------------------------------------------------------------------
Unhandled SIGSEGV: A segmentation fault occurred.
This probably occurred because a *compiled* module has a bug
in it and is not properly wrapped with sig_on(), sig_off().
Python will now terminate.
------------------------------------------------------------------------
/Users/runner/runners/2.169.0/work/sage/sage/src/bin/sage-python: line 2: 14003 Segmentation fault: 11  sage -python "$@"

@vbraun
Copy link
Member

vbraun commented Apr 19, 2020

Changed branch from u/mkoeppe/tox-ci-minor-fixes to b7bb418

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants