tox.ini, build/bin/write-dockerfile.sh: Simplify by using new options of "sage -package list"

[mkoeppe]

On this ticket, we simplify the code for portability testing using the new options of sage -package list introduced in #30865.

The simplifications to build/bin/write-dockerfile.sh are similar to those done in #30947 for src/doc/bootstrap.

The changes to tox.ini are to redefine the package configurations -minimal, -standard, -maximal using a variable SAGE_PACKAGE_LIST_ARGS - which is passed to sage -package list to obtain a list of packages. (For -minimal, we use SAGE_PACKAGE_LIST_ARGS=_prereq, which produces the empty package list; _prereq is in anticipation of #29124, which creates a script package by this name.)

To test, use for example

    tox -e docker-archlinux-minimal -- config.status
    tox -e docker-ubuntu-bionic-standard -- config.status
    tox -e docker-fedora-32-maximal -- config.status

Depends on #30865
Depends on #30923

CC: @tobiasdiez @seblabbe @kliem

Component: porting

Author: Matthias Koeppe

Branch/Commit: 8a575d4

Reviewer: Sébastien Labbé

Issue created by migration from https://trac.sagemath.org/ticket/30951

[mkoeppe]

Branch: u/mkoeppe/tox_ini__build_bin_write_dockerfile_sh__simplify_by_using_new_options_of__sage__package_list_

[mkoeppe]

Commit: ef2ce1a

[mkoeppe]

Author: Matthias Koeppe

[mkoeppe]

Last 10 new commits:

b762c3d	sage_bootstrap.expand_class: Use .append, .extend instead of += for lists
5efa1e7	Merge branch 't/30865/sage_bootstrap__update_extend_system_package_tools' into t/30951/tox_ini__build_bin_write_dockerfile_sh__simplify_by_using_new_options_of__sage__package_list_
30e624a	tox.ini: Add local-sudo
75ecd11	tox.ini (local-sudo): Also use ...-bootstrap.txt
0dc79eb	build/bin/sage-print-system-package-command: Handle --no-install-recommends, --yes for systems for which write-dockerfile.sh knows these flags
050dcb8	tox.ini (local-sudo): Use --yes --no-install-recommends
c8fbe0b	tox.ini (local-sudo): Ignore errors when IGNORE_MISSING_SYSTEM_PACKAGES=yes
ff34897	tox.ini (local): Guess the package system if it is not provided as a factor
0db7963	Merge branch 't/30923/tox_ini__add_environments_local_sudo_ubuntu_standard__etc_' into t/30951/tox_ini__build_bin_write_dockerfile_sh__simplify_by_using_new_options_of__sage__package_list_
ef2ce1a	tox.ini: build/bin/write-dockerfile.sh: Instead matching TYPE_PATTERN, use "sage -package list $SAGE_PACKAGE_LIST_ARGS"

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

b0459c3	Merge tag '9.3.beta2' into t/30951/tox_ini__build_bin_write_dockerfile_sh__simplify_by_using_new_options_of__sage__package_list_
a8708dd	build/bin/write-dockerfile.sh: Change bash -x back to bash

[sagetrac-git]

Changed commit from ef2ce1a to a8708dd

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

8a575d4

tox.ini (minimal): Use SAGE_PACKAGE_LIST_ARGS=_prereq

[sagetrac-git]

Changed commit from a8708dd to 8a575d4

[mkoeppe]

comment:7

Ready for review

[seblabbe]

comment:9

Something is weird, I can not get the documentation to build, I get:

[dochtml] [manifolds] ImportError: /home/slabbe/GitBox/sage/local/lib/python3.8/site-packages/sage/matrix/matrix_symbolic_dense.cpython-38-x86_64-linux-gnu.so: fichier trop court
[dochtml] [manifolds] The inventory files are in local/share/doc/sage/inventory/en/reference/manifolds.
[dochtml] Error building the documentation.
[dochtml] Traceback (most recent call last):
[dochtml]   File "/usr/lib/python3.8/runpy.py", line 192, in _run_module_as_main
[dochtml]     return _run_code(code, main_globals, None,
[dochtml]   File "/usr/lib/python3.8/runpy.py", line 85, in _run_code
[dochtml]     exec(code, run_globals)
[dochtml]   File "/home/slabbe/GitBox/sage/local/lib/python3.8/site-packages/sage_setup/docbuild/__main__.py", line 2, in <module>
[dochtml]     main()                                              
[dochtml]   File "/home/slabbe/GitBox/sage/local/lib/python3.8/site-packages/sage_setup/docbuild/__init__.py", line 1730, in main
[dochtml]     builder()
[dochtml]   File "/home/slabbe/GitBox/sage/local/lib/python3.8/site-packages/sage_setup/docbuild/__init__.py", line 343, in _wrapper
[dochtml]     getattr(get_builder(document), 'inventory')(*args, **kwds)
[dochtml]   File "/home/slabbe/GitBox/sage/local/lib/python3.8/site-packages/sage_setup/docbuild/__init__.py", line 569, in _wrapper
[dochtml]     self._build_everything_except_bibliography(lang, format, *args, **kwds)
[dochtml]   File "/home/slabbe/GitBox/sage/local/lib/python3.8/site-packages/sage_setup/docbuild/__init__.py", line 555, in _build_everything_except_bibliography
[dochtml]     build_many(build_ref_doc, non_references)          
[dochtml]   File "/home/slabbe/GitBox/sage/local/lib/python3.8/site-packages/sage_setup/docbuild/__init__.py", line 295, in build_many
[dochtml]     _build_many(target, args, processes=NUM_THREADS)   
[dochtml]   File "/home/slabbe/GitBox/sage/local/lib/python3.8/site-packages/sage_setup/docbuild/utils.py", line 289, in build_many
[dochtml]     raise worker_exc.original_exception
[dochtml] OSError: /home/slabbe/GitBox/sage/local/lib/python3.8/site-packages/sage/manifolds/differentiable/integrated_curve.py:docstring of sage.manifolds.differentiable.integrated_curve:5
1: WARNING: Exception occurred in plotting integrated_curve-1
Makefile:1888: recipe for target 'doc-html' failed            
make[3]: *** [doc-html] Error 1            
Makefile:1778: recipe for target 'all-start' failed

probably it has nothing to do with the current ticket, but the error stays after make doc-clean. Is there a new way of cleaning the documentation?

[mkoeppe]

comment:10

I don't think that this ticket should affect docbuilding in any way.

[seblabbe]

comment:11

make ptestlong works on Ubuntu 18.04.

I also tested archlinux and fedora with the command given in the current ticket description:

$ tox -e docker-archlinux-minimal -- config.status
...
___________________________________ summary ____________________________________
  docker-archlinux-minimal: commands succeeded
  congratulations :)

and

$ tox -e docker-fedora-32-maximal -- config.status
...
___________________________________ summary ____________________________________
  docker-fedora-32-maximal: commands succeeded
  congratulations :)

[mkoeppe]

comment:12

Thank you!

[mkoeppe]

Reviewer: Sébastien Labbé

[vbraun]

Changed branch from u/mkoeppe/tox_ini__build_bin_write_dockerfile_sh__simplify_by_using_new_options_of__sage__package_list_ to 8a575d4