Add to tox.ini some homebrew toxenvs (with a fresh install not in /usr/local), add homebrew packages

[mkoeppe]

#29053 adds tox.ini to support testing the sage distribution in isolated environments via docker.

This ticket adds toxenvs local-homebrew-macos-{minimal,standard} for testing with (best-effort) isolated installations of homebrew on macOS (not using docker).

To test:

    brew install tox
    tox -e local-homebrew-macos-standard

See comments in tox.ini for more examples.

An example run of that is here: https://github.com/mkoeppe/sage/actions/runs/32812271

Depends on #29335
Depends on #29361

CC: @dimpase @jhpalmieri @mwageringel @vbraun @tscrim

Component: porting

Author: Matthias Koeppe

Branch/Commit: dfce7d4

Reviewer: Dima Pasechnik, John Palmieri

Issue created by migration from https://trac.sagemath.org/ticket/29104

[mkoeppe]

Branch: u/mkoeppe/tox-homebrew

[mkoeppe]

Last 10 new commits:

7e59a6f	build/bin/write-dockerfile.sh: Add commands to activate conda env
2ad9a97	Merge tag '9.1.beta2' into t/29053/installation_manual__generate__apt_get_install__and__yum_install__lines_from_build_pkgs_at___bootstrap_time
9fca230	add doc
f6cbb8a	add centos-{7,8}
c651058	on centos, IGNORE_MISSING_SYSTEM_PACKAGES=yes
98edff6	Add build/pkgs/gfortran/distros/fedora.txt
843ca76	build/pkgs/fedora.txt: Add python3
bd6a545	tox.ini: Add debian stretch, put sid last
23c6334	build/pkgs/openblas/distros/arch.txt: Add lapack, cblas
fca6654	tox.ini: Add toxenvs local-homebrew-macos-minimal, local-homebrew-macos-standard

[mkoeppe]

Commit: fca6654

[sagetrac-git]

Changed commit from fca6654 to e0e8a4c

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

e0e8a4c

tox.ini [local-homebrew]: Fixup PATH, PKG_CONFIG_PATH, set LDFLAGS, CPPFLAGS; [local-direct]: New

[sagetrac-git]

Changed commit from e0e8a4c to 16ca88f

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

16ca88f

tox.ini: Use {envlogdir}; use ln -sf

[sagetrac-git]

Changed commit from 16ca88f to 6a1803d

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

6a1803d

tox.ini [local]: First build the base-toolchain

[sagetrac-git]

Changed commit from 6a1803d to 4b72987

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

4b72987

tox.ini [local-homebrew]: Set PATH earlier and set ACLOCAL_PATH for bootstrapping

[mkoeppe]

Author: Matthias Koeppe

[dimpase]

comment:10

Is this ticket MacOS-specific?

[mkoeppe]

comment:11

Hmm, I haven’t tried linuxbrew. Might be worth checking

[dimpase]

comment:12

I'll fire up my vintage OSX 10.13 macbookair...

[sagetrac-git]

Changed commit from 4b72987 to 2141b75

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

2141b75

Merge tag '9.1.beta3' into t/29104/tox-homebrew

[dimpase]

comment:14

I tried this:

tox -p auto -e local-homebrew-macos-standard

and got

...
[sagelib-9.1.beta2] user	0m10.244s
[sagelib-9.1.beta2] sys	0m3.812s

real	0m16.288s
user	0m10.956s
sys	0m4.047s
Sage build/upgrade complete!
✔ OK local-homebrew-macos-standard in 1 minute, 14.362 seconds
_________________________________________________________________________________________________ summary __________________________________________________________________________________________________
  local-homebrew-macos-standard: commands succeeded
  congratulations :)

not tests were run, it seems, is this normal?

[mkoeppe]

comment:15

The default packages set in tox.ini are only fflas_ffpack scipy r, and those (and their dependencies) are checked.
I don't know why it's actually building sagelib, but I hope to fix bizarre build behavior like this in #29113.

[mkoeppe]

Changed dependencies from #29053 to none

[sagetrac-git]

Changed commit from 2141b75 to 4eb201b

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

4eb201b

Merge tag '9.1.beta4' into t/29104/tox-homebrew

[dimpase]

comment:126

I'm 8 timezones away from my MacOS machine.

I'd like more details/pointers on how the Github Actions testing is done - for this looks to me like "all sufficiently advanced technology is indistinguishable from magic" kind of thing :-)

[mkoeppe]

comment:127

This ticket does no GitHub Actions testing. This is all introduced in #29087.

[dimpase]

comment:128

how does one get the data I linked in comment:126 then?

[mkoeppe]

comment:129

By merging #29087 and pushing the branch to a GitHub repository.

[mkoeppe]

comment:130

If the GitHub Actions interface is overwhelming, just download the "Artifacts". They are tar.gz files of build logs.

[dimpase]

comment:131

Replying to @mkoeppe:

By merging #29087 and pushing the branch to a GitHub repository.

yes,but that's how one triggers this - I'm asking rather about how stuff works, inside. Locally and then somehow pushing the logs to GitHub?

[mkoeppe]

comment:132

I don't push logs to GitHub.

#29087 adds .github/workflows/tox.yml (https://github.com/sagemath/sagetrac-mirror/blob/576a18834a16ff05cd6894172a55e86bcc95d407/.github/workflows/tox.yml&id2=be1e22c441d9332fd0bf4a3066e1964deddaf8c7) which gets run on every commit pushed to GitHub and on every pull request.

The job tox-local-homebrew-macos defined in that file provisions a macOS host, checks out the branch and runs tox -e local-homebrew-macos-standard there. Then it packages up logs and makes them available as an artifact.

[dimpase]

comment:133

and where is that MacOS host getting provisioned?

[mkoeppe]

comment:134

GitHub takes care of that, I think it's using Azure.

[mwageringel]

comment:135

Replying to @jhpalmieri:

@mwageringel, @dimpase: do you think more needs to be done here, or can we proceed with a positive review? I'm happy with it.

I have not tested this locally, but as you have done that, I am happy with this too.

One last question though: As far as I can see, readline is not installed into /usr/local. Why do you not run into conflicts with macOS's system "readline"/libedit, as in #29000?

[mkoeppe]

comment:136

These tests use their own homebrew installation in a prefix .tox/local-homebrew-macos-standard/homebrew/, and sets up environment variables using the new script .homebrew-build-env. This sets LDFLAGS and CPPFLAGS for the readline installed into opt under the homebrew prefix.

[mwageringel]

comment:137

The problem in #29000 occured despite passing LDFLAGS and CPPFLAGS to configure, so I guess setting these also during make would avoid that problem, as a workaround.

[mkoeppe]

comment:138

I think the problem description of #29000 needs some more precision.

[dimpase]

comment:139

OK, looks good to me.

[dimpase]

Reviewer: Dima Pasechnik, John Palmieri

[mkoeppe]

comment:140

Thanks everyone!

[vbraun]

Changed branch from u/mkoeppe/tox-homebrew to dfce7d4