Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[WIP] Re-install python libraries rather than upgrading #1939

Draft
wants to merge 3 commits into
base: dev
Choose a base branch
from
Draft

[WIP] Re-install python libraries rather than upgrading #1939

wants to merge 3 commits into from

Commits on Apr 13, 2023

  1. install-pn-test-server: make /physionet/python-env/physionet a link. 

    In order to upgrade Python libraries atomically, we want to be able to
change /physionet/python-env/physionet to point to a new location,
while leaving the old tree in place.

When setting up the test environment, create 'physionet' as a link
pointing to the real directory 'initial', so that subsequent scripts
can overwrite the link atomically.
    Benjamin Moody committed Apr 13, 2023
    Configuration menu
    Copy the full SHA
    3e1e03d View commit details
    Browse the repository at this point in the history

  2. update: install python libraries atomically. 

    We want to avoid "upgrading" python libraries using pip, because this
may not give predictable results, and if installation fails it can
leave the system in a broken half-installed state.

Consequently, instead of installing packages into
/physionet/python-env/physionet, install them into a directory named
for the MD5 hash of requirements.txt.

Installation may produce a lot of output (e.g. when compiling
extension modules from source), so store the log in the installation
directory rather than further cluttering update.log.

After the pip command is successful, copy requirements.txt to a new
file installed.txt.  This serves as a flag to indicate that the
packages were installed successfully, and also provides documentation
of what was supposedly installed.

If requirements.txt matches installed.txt, then we know that we don't
need to reinstall anything.

After the packages have been successfully installed and we have tested
that they're working (at least well enough to invoke
getmigrationtargets), then update the symlink
/physionet/python-env/physionet to point to the new prefix.
    Benjamin Moody committed Apr 13, 2023
    Configuration menu
    Copy the full SHA
    33d3689 View commit details
    Browse the repository at this point in the history

  3. test-upgrade.sh: install python libraries without upgrading. 

    We want to avoid "upgrading" python libraries using pip, because this
may not give predictable results, and if installation fails it can
leave the system in a broken half-installed state.  Instead, we want
to install packages from scratch whenever it is required.

To be consistent with the new behavior of the git update hook, do the
same thing when running upgrade tests: rather than "upgrading" the
virtualenv directory, delete the old one and create a new one.  This
makes caching simpler, too, since the directory contents should be
reproducible just from requirements.txt.
    Benjamin Moody committed Apr 13, 2023
    Configuration menu
    Copy the full SHA
    f674ac1 View commit details
    Browse the repository at this point in the history