Make docker images from tox/GitHub CI workflow, regular Sage Docker images, gitpod, CoCalc interoperable

[mkoeppe]

1. Adjust directory names to something that works for both cases ...

   • docker/Dockerfile currently has:
     • SAGE_ROOT=/home/sage/sage
     • SAGE_LOCAL=$SAGE_ROOT/local
   • Docker images built with tox/GH Actions (including Gitpod) and also computop/sage have:
     • SAGE_ROOT=/sage
     • SAGE_LOCAL=/sage/local
   • After this ticket:
     • SAGE_ROOT=/sage
     • SAGE_LOCAL=/opt/sage
   • CoCalc Docker image has:
     • SAGE_ROOT=/usr/local/sage
     • SAGE_LOCAL=/usr/local/sage/local

2. Script package _develop; improve installation instructions in the manual #33851: Define a script package representing "useful stuff to have on a Docker image"

3. Replace all apt-get in docker/Dockerfile by using a Docker image built using build/bin/write-dockerfile as the base image.

4. By default, docker/Dockerfile would use configure --without-system-... a lot so that CI is done against "the reference distribution"

5. Move build/bin/write-dockerfile.sh into docker/

6. Add a mechanism (environment variable) to tox -e docker... to copy SAGE_LOCAL from a given Docker image.

CC: @kliem @dimpase @embray @saraedum @slel @fchapoton @tobiasdiez @williamstein @culler @NathanDunfield

Component: docker

Keywords: ci, ContinuousIntegration, sd109

Issue created by migration from https://trac.sagemath.org/ticket/29536

[dimpase]

comment:2

by the way, we don't seem to have in docs an explanation regarding file shared between docker images and regular file system of the host system.

[mkoeppe]

Changed keywords from ci, ContinuousIntegration to ci, ContinuousIntegration, sd109

[embray]

comment:5

I'm confused--should this say "GitHub" in the title or "GitLab"? I don't know of any GitHub CI docker images.

[dimpase]

comment:6

Replying to @embray:

I'm confused--should this say "GitHub" in the title or "GitLab"? I don't know of any GitHub CI docker images.

please see
https://doc.sagemath.org/html/en/developer/index.html#testing-on-multiple-platforms

we now have a builder of docker images for multiple platforms, configurable, and by default hosted on github.

[mkoeppe]

Dependencies: #30960

[mkoeppe]

comment:14

Sage development has entered the release candidate phase for 9.3. Setting a new milestone for this ticket based on a cursory review of ticket status, priority, and last modification date.

[dimpase]

comment:17

Given that GitLab Docker build is broken for ages, can we arrange for GitHub-built docker images to be pushed to the main Docker repo,
or at least create some sort of advertised Docker repo on GitHub?

[mkoeppe]

comment:22

Bikeshedding time! Let's pick SAGE_ROOT, SAGE_LOCAL. My proposal is in the ticket description.

(The only important constraint for me is: Don't nest SAGE_LOCAL within SAGE_ROOT - for flexibility such as bind-mounting a new version of the source tree over SAGE_ROOT without hiding SAGE_LOCAL.)

[mkoeppe]

Changed dependencies from #30960 to #33740

[mkoeppe]

Branch: u/mkoeppe/make_docker_images_from_tox_github_ci_workflow__regular_sage_docker_images__and_gitpod_interoperable

[sagetrac-git]

Commit: 58e112a

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

10338a7	Change ci as well
0cf7334	Add sage-setup as install-requires dep
6d2b149	Use relative paths to not install src package
1dba3ba	Maybe thats the correct shebang?
c1b2fbf	Revert install_requires change
ad56ba3	Go back to two pip installs for now
b42f5f5	Readd how to use specific python version
87f360c	Readd name arg
01eb374	Merge #33740
58e112a	bootstrap-conda: Use script package _develop instead of hardcoded list of dev tools; remove unused RECOMMENDED

[mkoeppe]

Changed commit from 58e112a to none

[mkoeppe]

Changed dependencies from #33740 to none

[mkoeppe]

Changed branch from u/mkoeppe/make_docker_images_from_tox_github_ci_workflow__regular_sage_docker_images__and_gitpod_interoperable to none