Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tox.ini (local): Add environment variables to skip system package installs and other steps, add mechanism for a local interactive shell #31216

Closed
mkoeppe opened this issue Jan 9, 2021 · 31 comments

Comments

@mkoeppe
Copy link
Contributor

mkoeppe commented Jan 9, 2021

(from #31064)

We add the following to the local-... environments:

  • an environment variable that can be passed to tox to skip system package installs, directly reusing a previously set up system
SKIP_SYSTEM_PKG_INSTALL=yes tox -e local-homebrew-macos-standard -- config.status
SKIP_SYSTEM_PKG_INSTALL=yes tox -e local-homebrew-macos-standard -- config.status

This can save time and also give developers more control for experiments with system packages.

  • a target that gives an interactive shell in the tox environment:
tox -e local-homebrew-macos-standard -- bash
tox -e local-direct -- bash
SKIP_SYSTEM_PKG_INSTALL=yes SKIP_BOOTSTRAP=1 SKIP_CONFIGURE=1 tox -e local-homebrew-macos-standard -- bash

Depends on #30944

CC: @tobiasdiez @kliem @dimpase

Component: porting

Author: Matthias Koeppe

Branch/Commit: 2d84b5a

Reviewer: Tobias Diez

Issue created by migration from https://trac.sagemath.org/ticket/31216

@mkoeppe mkoeppe added this to the sage-9.3 milestone Jan 9, 2021
@mkoeppe
Copy link
Contributor Author

mkoeppe commented Jan 9, 2021

Dependencies: #30944

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Jan 9, 2021

Commit: 13f58fc

@mkoeppe

This comment has been minimized.

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Jan 9, 2021

Last 10 new commits:

e5fe752Merge tag '9.3.beta4' into t/30940/src_bin_sage_list_packages__make_it_work_if_sage_root_is_not_available
78ff9d5src/sage/misc/package.py: Add one more # optional - build
a44042fMerge branch 't/30940/src_bin_sage_list_packages__make_it_work_if_sage_root_is_not_available' into t/29124/script-packages-prereq-toolchain-bootstrap
64bde5fMerge tag '9.3.beta5' into t/30940/src_bin_sage_list_packages__make_it_work_if_sage_root_is_not_available
e9a7572src/sage/misc/package.py: Improve source formatting
c7bcda9Merge branch 't/30940/src_bin_sage_list_packages__make_it_work_if_sage_root_is_not_available' into t/29124/script-packages-prereq-toolchain-bootstrap
9988c5fci-cygwin*.yml: Adjust to new script packages _bootstrap, _prereq
ab19133Merge branch 't/29124/script-packages-prereq-toolchain-bootstrap' into t/30944/tox__improve_local_sudo_ubuntu_standard
c6ad06eMerge branch 't/30944/tox__improve_local_sudo_ubuntu_standard' into t/31216/tox_ini__local___add_environment_variables_to_skip_system_package_installs__mechanism_for_a_local_interactive_shell
13f58fctox.ini (local): Run an interactive shell if posargs are bash

@mkoeppe

This comment has been minimized.

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Jan 10, 2021

Branch pushed to git repo; I updated commit sha1. New commits:

ace4ef6tox.ini (local): Add option SKIP_SYSTEM_PKG_INSTALL=1

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Jan 10, 2021

Changed commit from 13f58fc to ace4ef6

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Jan 10, 2021

Author: Matthias Koeppe

@mkoeppe

This comment has been minimized.

@tobiasdiez
Copy link
Contributor

comment:8

On a first glance this looks good. Thanks! Could you please also add documentation for these tox commands / switches. Also I was wondering if one could add a tox -e local-homebrew-macos-standard -- install-system-packages that only installs the system packages?

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Jan 10, 2021

Changed commit from ace4ef6 to ddc4f0b

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Jan 10, 2021

Branch pushed to git repo; I updated commit sha1. New commits:

ddc4f0btox.ini (local): Handle environment variables SKIP_BOOTSTRAP, SKIP_CONFIGURE

@mkoeppe

This comment has been minimized.

@mkoeppe mkoeppe changed the title tox.ini (local): Add environment variables to skip system package installs, mechanism for a local interactive shell tox.ini (local): Add environment variables to skip system package installs and other steps, add mechanism for a local interactive shell Jan 10, 2021
@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Jan 11, 2021

Branch pushed to git repo; I updated commit sha1. New commits:

bb6aec4src/doc/en/developer/portability_testing.rst: Updates
6253bd8src/doc/en/developer/portability_testing.rst: New section on SKIP_... environment variables

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Jan 11, 2021

Changed commit from ddc4f0b to 6253bd8

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Jan 11, 2021

comment:12

Replying to @tobiasdiez:

Could you please also add documentation for these tox commands / switches.

Done

Also I was wondering if one could add a tox -e local-homebrew-macos-standard -- install-system-packages that only installs the system packages?

I have instead added more SKIP_... options.

@tobiasdiez
Copy link
Contributor

comment:13

Replying to @mkoeppe:

I have instead added more SKIP_... options.

Thanks! Would it also make sense to add one to skip make? (I remember you once told me how to do this, I think, it was via config.status or something - but maybe the version using skip is closer in spirit anyway).

I'll review this once #30944 is merged in develop.

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Jan 11, 2021

comment:14

Replying to @tobiasdiez:

Would it also make sense to add one to skip make?

You can just pass a make target that does nothing - such as Makefile

@tobiasdiez
Copy link
Contributor

comment:15

local-sudo-standard run-test: commands[1] | bash -c 'case "" in 1|y*|Y*);; *) eval $(build/bin/sage-print-system-package-command $(build/bin/sage-guess-package-system) --sudo update) ;;'/usr/bin/bash: -c: line 1: syntax error: unexpected end of file

I think there is a "esac" missing at the end of these two lines:


+    local-{root,sudo}:    bash -c 'case "{env:SKIP_SYSTEM_PKG_INSTALL:}" in 1|y*|Y*);; *) eval $(build/bin/sage-print-system-package-command {env:SYSTEM} {env:__SUDO:} update) ;;'
+    local-{root,sudo}:    bash -c 'case "{env:SKIP_SYSTEM_PKG_INSTALL:}" in 1|y*|Y*);; *) PACKAGES=$(build/bin/sage-get-system-packages {env:SYSTEM} $(PATH=build/bin:$PATH build/bin/sage-package list {env:SAGE_PACKAGE_LIST_ARGS}) _bootstrap); eval $(build/bin/sage-print-system-package-command {env:SYSTEM} {env:__SUDO:} --yes --no-install-recommends install $PACKAGES) || [ "$IGNORE_MISSING_SYSTEM_PACKAGES" = yes ] && echo "(ignoring errors)" ;;'

@tobiasdiez
Copy link
Contributor

comment:16

Replying to @mkoeppe:

Replying to @tobiasdiez:

Would it also make sense to add one to skip make?

You can just pass a make target that does nothing - such as Makefile

tox -e local-sudo-standard Makefile with SKIP_BOOTSTRAP: yes and SKIP_CONFIGURE: yes still runs bootstrap and base-toolchain in the make command, which then fails since configure was not called before.

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Jan 12, 2021

comment:17

Ah, of course. Make that configure.ac instead of Makefile...

@tobiasdiez
Copy link
Contributor

comment:18

That worked, thanks! Apart from the missing "esac" mentioned above, this looks good to me.

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Jan 18, 2021

Branch pushed to git repo; I updated commit sha1. New commits:

2d84b5atox.ini: Add missing esac

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Jan 18, 2021

Changed commit from 6253bd8 to 2d84b5a

@tobiasdiez
Copy link
Contributor

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Jan 19, 2021

comment:22

Replying to @tobiasdiez:

Is the prefix [mkoeppe@sage sage]$ really needed / what's its purpose?

It's a typical shell prompt on posix systems. The second sage indicates the current directory. This is consistent with other parts of the Sage developer guide, for example https://doc.sagemath.org/html/en/developer/doctesting.html

@tobiasdiez
Copy link
Contributor

comment:23

Thanks, then it looks good to me!

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Jan 19, 2021

comment:24

Thanks!

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Jan 19, 2021

Reviewer: Tobias Diez

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants