standardizing compute environments

[soniapolt]

• common data

• general path function

• reproducible virtual environments based on requirements.txt

  • Should be able to pip install -r requirements.txt to fully reproduce virtualenv

[cjnani16]

Thus far I can only anticipate running this on Nyla and 'Amoxicillin' so I guess some sort of switch which sets paths to data on the fs based on hostname is going to be very achievable.

Also generating the requirements.txt will be hugely helpful!

[eshedmargalit]

Just so it's recorded somewhere, here's how I reproduce virtual environments across machines:

1. Enter the virtual environment you want to clone, however you normally do (whether conda or virtualenv or pyenv)

pyenv activate some_environment

2. Save the installed pip packages to a requirements file

pip freeze > requirements.txt

3. Commit the new requirements file

git add requirements.txt
git commit -m "adds requirements file for virtual environment"

4. Clone or pull the repo on other machine via git pull

5. Create a new virtual environment on the other machine with conda, virtualenv, pyenv, etc

pyenv virtualenv some_environment

Enter the new virtualenv if needed.

6. Install the requirements cloned from the repo

pip install -r requirements.txt

And done!

[cjnani16]

Awesome thank you for these instructions!!

…

-- Sent from mobile: 510.940.3695

________________________________ From: Eshed Margalit <notifications@github.com> Sent: Friday, November 8, 2019 6:25:42 PM To: cjnani16/deeprfs_tf <deeprfs_tf@noreply.github.com> Cc: Jñani Crawford <crawford.jnani@gmail.com>; Assign <assign@noreply.github.com> Subject: Re: [cjnani16/deeprfs_tf] standardizing compute environments (#3) Just so it's recorded somewhere, here's how I reproduce virtual environments across machines: 1. Enter the virtual environment you want to clone, however you normally do (whether conda or virtualenv or pyenv) pyenv activate some_environment 1. Save the installed pip packages to a requirements file pip freeze > requirements.txt 1. Commit the new requirements file git add requirements.txt git commit -m "adds requirements file for virtual environment" 1. Clone or pull the repo on other machine via git pull 2. Create a new virtual environment on the other machine with conda, virtualenv, pyenv, etc pyenv virtualenv some_environment Enter the new virtualenv if needed. 1. Install the requirements cloned from the repo pip install -r requirements.txt And done! — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub<#3?email_source=notifications&email_token=ABGNN2WUMS5P6T6U3DQPYMDQSYNSNA5CNFSM4JK75RVKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDT3QGY#issuecomment-552056859>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ABGNN2Q3JVAFDJXEQYPJGWDQSYNSNANCNFSM4JK75RVA>.

[eshedmargalit]

To-dos 2019115

• Add to general path function a checkpoints path