tox.ini, GitHub CI workflow - minor fixes, more homebrew configuration factors, improve docker interface #29417

mkoeppe · 2020-03-27T17:06:56Z

This makes the following changes to tox.ini:

tox -e local-homebrew-macos-standard now uses python3 from homebrew.

This attempts to reproduce the environment of https://groups.google.com/d/msg/sage-release/kU5M1QVuQQY/Zu10Q-CKAwAJ
tox -e local-homebrew-macos-standard-python3_xcode uses /usr/bin/python3 (was the old default)
tox -e local-homebrew-macos-standard-python3_pythonorg uses the Python 3.7 installation from the binary package from python.org

New toolchain factors: gcc_spkg, gcc_9.

Improved docker interface.

CC: @jhpalmieri @dimpase @orlitzky @kliem @vbraun

Component: porting

Author: Matthias Koeppe

Branch/Commit: b7bb418

Reviewer: Dima Pasechnik

Issue created by migration from https://trac.sagemath.org/ticket/29417

The text was updated successfully, but these errors were encountered:

mkoeppe · 2020-03-27T17:10:03Z

Dependencies: #29087, #29354

mkoeppe · 2020-03-27T17:11:19Z

Branch: u/mkoeppe/tox-ci-minor-fixes

mkoeppe · 2020-03-28T15:47:46Z

Changed dependencies from #29087, #29354 to #29087, #29354, #29418

mkoeppe · 2020-03-28T15:47:46Z

Last 10 new commits:

`fbb7393`	`Fixup packages`
`cfcd8f2`	`build/bin/sage-guess-package-system, build/bin/sage-print-system-package-command: Handle slackware too`
`136feeb`	`build/bin/sage-spkg: Show equivalent system packages for ALL distros`
`9939051`	`Merge branch 't/29354/tox_ini__add_slackware_environments' into t/29087/CI-gh-actions-tox-configure-build`
`4ac9e24`	`Reduce SAGE_NUM_THREADS to 3`
`1cb8f98`	`build/pkgs/curl/distros/slackware.txt: Add openldap-client`
`eee2877`	`build/pkgs/slackware.txt: Add ca-certificates`
`c1931f5`	`build/pkgs/curl/distros/slackware.txt: Add libssh2`
`3145467`	`Merge branch 't/29354/tox_ini__add_slackware_environments' into t/29087/CI-gh-actions-tox-configure-build`
`b259671`	`[tox-local-homebrew-macos] Repair make parallelization`

mkoeppe · 2020-03-28T15:47:46Z

Commit: b259671

mkoeppe · 2020-03-28T15:47:46Z

Author: Matthias Koeppe

sagetrac-git · 2020-03-28T15:59:46Z

Branch pushed to git repo; I updated commit sha1. New commits:

`bd24641`	`build/bin/sage-download-file: Add option --no-check-certificate`
`e143939`	`build/bin/sage-spkg: Append to SAGE_DOWNLOAD_FILE_OPTIONS instead of overwriting it`
`710b52d`	`Merge branch 't/29418/sage_download_file__fix_certificate_problems_with_https_downloads_on_macos_python3' into t/29417/tox-ci-minor-fixes`
`b1489e7`	`tox.ini: Set SAGE_DOWNLOAD_FILE_OPTIONS=--no-check-certificate`

sagetrac-git · 2020-03-28T15:59:46Z

Changed commit from b259671 to b1489e7

mkoeppe · 2020-03-29T20:27:18Z

Changed dependencies from #29087, #29354, #29418 to #29087, #29354, #29418, #29415

sagetrac-git · 2020-03-30T19:44:52Z

Changed commit from b1489e7 to 846f14e

sagetrac-git · 2020-03-30T19:44:52Z

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

`354cf90`	`Use conda-install --yes --quiet`
`df609e0`	`Merge branch 't/29415/tox_ini__add_environment_local_conda' into t/29417/tox-ci-minor-fixes`
`cf27321`	`build/bin/sage-download-file: Add option --no-check-certificate`
`90ea00b`	`build/bin/sage-spkg: Append to SAGE_DOWNLOAD_FILE_OPTIONS instead of overwriting it`
`bccc2c2`	`Merge branch 't/29418/sage_download_file__fix_certificate_problems_with_https_downloads_on_macos_python3' into t/29417/tox-ci-minor-fixes`
`87b7351`	`tox.ini: Add configuration factors python3, macos-python3_xcode, macos-python3_pythonorg`
`329ea99`	`tox.yml: Test more homebrew configs`
`de1bce2`	`tox.yml: Add local-ubuntu-conda test`
`86d5c84`	`tox.ini: Add toolchain factors`
`846f14e`	`tox.yml: Test more Fedora, no conda`

mkoeppe · 2020-03-30T19:45:20Z

Changed dependencies from #29087, #29354, #29418, #29415 to #29087, #29418, #29415

sagetrac-git · 2020-03-30T20:01:03Z

Changed commit from 846f14e to 9838fd7

sagetrac-git · 2020-03-30T20:01:03Z

Branch pushed to git repo; I updated commit sha1. New commits:

`9838fd7`	`fixup`

sagetrac-git · 2020-03-30T20:09:56Z

Branch pushed to git repo; I updated commit sha1. New commits:

`a3ccd46`	`build/pkgs/python3/distros/homebrew.txt: New`

sagetrac-git · 2020-03-30T20:09:56Z

Changed commit from 9838fd7 to a3ccd46

mkoeppe · 2020-03-30T20:50:54Z

Changed dependencies from #29087, #29418, #29415 to #29087, #29418, #29415, #29401

sagetrac-git · 2020-03-30T22:07:56Z

Changed commit from a3ccd46 to 2bc1243

sagetrac-git · 2020-03-30T22:07:56Z

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

`6153945`	`merge the sections on tox factors`
`ab11ebd`	`elide some details from terminal output`
`c7fcf91`	`change 'ls -l' to 'ls -la' and elide some details`
`e73bc8d`	`Elide more details, shorten directory names more`
`a31b2a0`	`Reviewer patch`
`b7b45fe`	`Reviewer patch`
`f7d2dc9`	`Remove old section head`
`99aca39`	`Reformat list, remove unfinished commented out section`
`2c53ae2`	`Merge branch 't/29401/add_documentation_of_tox_and_github_actions_workflow_to_developer_s_manual' into t/29417/tox-ci-minor-fixes`
`2bc1243`	`src/doc/en/developer/portability_testing.rst: Add documentation on new local-conda and local-homebrew environments`

sagetrac-git · 2020-03-31T13:46:49Z

Branch pushed to git repo; I updated commit sha1. New commits:

`5dd7652`	`tox.ini: Add more linuxmint versions`
`c6cc486`	`tox.ini: IGNORE_MISSING_SYSTEM_PACKAGES=yes for old linuxmint versions`
`cd72da9`	`tox.ini: Add linuxmint-i386`
`7986ab3`	`tox.yml: Add more linuxmint`
`a54ab32`	`tox.ini: Remove linuxmint-20 (no docker image)`

sagetrac-git · 2020-03-31T13:46:49Z

Changed commit from 2bc1243 to a54ab32

sagetrac-git · 2020-04-02T16:13:46Z

Changed commit from a54ab32 to 4281c92

sagetrac-git · 2020-04-02T16:13:46Z

Branch pushed to git repo; I updated commit sha1. New commits:

`3757212`	`tox.ini: Switch local-conda-forge to miniforge`
`a61c48e`	`tox.ini [local-conda]: Activate conda before calling conda install`
`8f58411`	`Merge branch 't/29415/tox_ini__add_environment_local_conda' into t/29417/tox-ci-minor-fixes`
`4281c92`	`Replace commands_pre by commands, to support tox < 3.4`

sagetrac-git · 2020-04-02T16:39:38Z

Changed commit from 4281c92 to 49aeac3

mkoeppe · 2020-04-13T15:44:00Z

comment:55

Test for 9.1.rc0 running at https://github.com/mkoeppe/sage/actions/runs/76923542

sagetrac-git · 2020-04-13T17:20:29Z

Branch pushed to git repo; I updated commit sha1. New commits:

`faf0ed6`	`tox.ini: Another quoting fix`

sagetrac-git · 2020-04-13T17:20:29Z

Changed commit from d94969c to faf0ed6

sagetrac-git · 2020-04-15T01:24:07Z

Changed commit from faf0ed6 to aed3fb8

sagetrac-git · 2020-04-15T01:24:07Z

Branch pushed to git repo; I updated commit sha1. New commits:

`aed3fb8`	`src/doc/en/developer/portability_testing.rst: Explain how to read [GitHub](../wiki/GitHub) Actions page`

mkoeppe · 2020-04-15T01:24:39Z

comment:58

Can we finalize the review of this one please to get it into 9.1?

dimpase · 2020-04-15T01:55:58Z

comment:59

running GH Actions tests on this here: dimpase#3

dimpase · 2020-04-15T17:44:37Z

comment:60

Replying to @dimpase:

running GH Actions tests on this here: dimpase#3

I see

/Users/runner/runners/2.168.0/work/sage/sage/build/bin/sage-venv "/Users/runner/runners/2.168.0/work/sage/sage/.tox/local-homebrew-macos-python3_pythonorg-standard/local"
2020-04-15T16:31:11.9205990Z /bin/sh: /Library/Frameworks/Python.framework/Versions/3.7/bin/python3: No such file or directory
2020-04-15T16:31:11.9206560Z make[1]: *** [/Users/runner/runners/2.168.0/work/sage/sage/.tox/local-homebrew-macos-python3_pythonorg-standard/local/pyvenv.cfg] Error 127
2020-04-15T16:31:11.9207060Z make[1]: Target `doc-html' not remade because of errors.

there. Isn't this ticked supposed to be fixing this?

mkoeppe · 2020-04-15T18:17:49Z

comment:61

Thanks for catching this. Turns out I forgot to install the python.org python3 in tox.yml.
(This cannot be done in tox.ini because it is not relocatable, see comment there)

sagetrac-git · 2020-04-15T18:38:32Z

Changed commit from aed3fb8 to d39d627

sagetrac-git · 2020-04-15T18:38:32Z

Branch pushed to git repo; I updated commit sha1. New commits:

`d39d627`	`tox.yml [python3_xcode]: Install the package`

sagetrac-git · 2020-04-15T18:42:51Z

Changed commit from d39d627 to ba0c266

sagetrac-git · 2020-04-15T18:42:51Z

Branch pushed to git repo; I updated commit sha1. New commits:

`ba0c266`	`fixup`

sagetrac-git · 2020-04-15T21:10:16Z

Branch pushed to git repo; I updated commit sha1. New commits:

`9060d23`	`fixup`

sagetrac-git · 2020-04-15T21:10:16Z

Changed commit from ba0c266 to 9060d23

sagetrac-git · 2020-04-16T02:13:22Z

Changed commit from 9060d23 to b7bb418

sagetrac-git · 2020-04-16T02:13:22Z

Branch pushed to git repo; I updated commit sha1. New commits:

`b7bb418`	`tox.yml [macos-python3_pythonorg]: Use sudo with installer`

dimpase · 2020-04-16T08:02:47Z

comment:66

next try here: dimpase#4

dimpase · 2020-04-17T15:55:26Z

comment:67

homebrew-macos-python3_pythonorg, standard (https://github.com/dimpase/sage/runs/591414002) crashes at [dochtml] stage - not sure whether one can get complete logs/dochtml.log file from GitHub run above

dimpase · 2020-04-17T16:03:33Z

comment:68

I presume that various errors on local-macos standard runs are to be debugged individually.

e.g. on https://github.com/dimpase/sage/runs/591413919 (homebrew-macos-python3_xcode, minimal) I see ##[error] ==== ERROR IN LOG FILE artifacts/logs-commit-8bb28ec8ffd6869c6b86e3f5224333decb049d09-tox-local-homebrew-macos-python3_xcode-minimal/log/pkgs/gc-7.6.4.p0.log ==== and
Warning: Error testing package gc-7.6.4.p0 (ignored)

dimpase · 2020-04-17T16:03:33Z

Reviewer: Dima Pasechnik

mkoeppe · 2020-04-17T16:21:18Z

comment:69

Thanks!

mkoeppe · 2020-04-17T16:21:39Z

comment:70

Yes, errors are to be fixed on their own tickets.

mkoeppe · 2020-04-17T16:23:23Z

comment:71

Some of the errors are actually "only" testsuite failures. Tickets such as #29501 clean up some of the messages.

mkoeppe · 2020-04-17T16:27:15Z

comment:72

Replying to @dimpase:

homebrew-macos-python3_pythonorg, standard (https://github.com/dimpase/sage/runs/591414002) crashes at [dochtml] stage - not sure whether one can get complete logs/dochtml.log file from GitHub run above

Yes, download the log artifact. But it only says

------------------------------------------------------------------------
(no backtrace available)
------------------------------------------------------------------------
Unhandled SIGSEGV: A segmentation fault occurred.
This probably occurred because a *compiled* module has a bug
in it and is not properly wrapped with sig_on(), sig_off().
Python will now terminate.
------------------------------------------------------------------------
/Users/runner/runners/2.169.0/work/sage/sage/src/bin/sage-python: line 2: 14003 Segmentation fault: 11  sage -python "$@"

vbraun · 2020-04-19T19:25:17Z

Changed branch from u/mkoeppe/tox-ci-minor-fixes to b7bb418

mkoeppe added this to the sage-9.1 milestone Mar 27, 2020

mkoeppe added c: porting labels Mar 27, 2020

This comment has been minimized.

Sign in to view

mkoeppe changed the title ~~tox.ini, GitHub CI workflow - minor fixes~~ tox.ini, GitHub CI workflow - minor fixes, more homebrew configuration factors Mar 30, 2020

dimpase added s: positive review and removed s: needs review labels Apr 17, 2020

vbraun removed the s: positive review label Apr 19, 2020

vbraun closed this as completed in 1007d3c Apr 19, 2020

tox.ini, GitHub CI workflow - minor fixes, more homebrew configuration factors, improve docker interface #29417

tox.ini, GitHub CI workflow - minor fixes, more homebrew configuration factors, improve docker interface #29417

Comments

mkoeppe commented Mar 27, 2020

mkoeppe commented Mar 27, 2020

mkoeppe commented Mar 27, 2020

mkoeppe commented Mar 28, 2020

mkoeppe commented Mar 28, 2020

mkoeppe commented Mar 28, 2020

mkoeppe commented Mar 28, 2020

sagetrac-git mannequin commented Mar 28, 2020

sagetrac-git mannequin commented Mar 28, 2020

mkoeppe commented Mar 29, 2020

sagetrac-git mannequin commented Mar 30, 2020

sagetrac-git mannequin commented Mar 30, 2020

mkoeppe commented Mar 30, 2020

sagetrac-git mannequin commented Mar 30, 2020

sagetrac-git mannequin commented Mar 30, 2020

sagetrac-git mannequin commented Mar 30, 2020

sagetrac-git mannequin commented Mar 30, 2020

This comment has been minimized.

mkoeppe commented Mar 30, 2020

sagetrac-git mannequin commented Mar 30, 2020

sagetrac-git mannequin commented Mar 30, 2020

sagetrac-git mannequin commented Mar 31, 2020

sagetrac-git mannequin commented Mar 31, 2020

sagetrac-git mannequin commented Apr 2, 2020

sagetrac-git mannequin commented Apr 2, 2020

sagetrac-git mannequin commented Apr 2, 2020

mkoeppe commented Apr 13, 2020

sagetrac-git mannequin commented Apr 13, 2020

sagetrac-git mannequin commented Apr 13, 2020

sagetrac-git mannequin commented Apr 15, 2020

sagetrac-git mannequin commented Apr 15, 2020

mkoeppe commented Apr 15, 2020

dimpase commented Apr 15, 2020

dimpase commented Apr 15, 2020

mkoeppe commented Apr 15, 2020

sagetrac-git mannequin commented Apr 15, 2020

sagetrac-git mannequin commented Apr 15, 2020

sagetrac-git mannequin commented Apr 15, 2020

sagetrac-git mannequin commented Apr 15, 2020

sagetrac-git mannequin commented Apr 15, 2020

sagetrac-git mannequin commented Apr 15, 2020

sagetrac-git mannequin commented Apr 16, 2020

sagetrac-git mannequin commented Apr 16, 2020

dimpase commented Apr 16, 2020

dimpase commented Apr 17, 2020

dimpase commented Apr 17, 2020

dimpase commented Apr 17, 2020

mkoeppe commented Apr 17, 2020

mkoeppe commented Apr 17, 2020

mkoeppe commented Apr 17, 2020

mkoeppe commented Apr 17, 2020

vbraun commented Apr 19, 2020