Python Tips

I am provided a "requirements.txt" file, what do I do?

python -m pip install -r requirements.txt

or

pip install -r requirements.txt

will install all the packages listed in the requirements.txt file.

I had to install a few more packages that were not listed. How do I document that?

If the author provided a requirements.txt, simply overwrite it, git will track the changes. You can generate the file after you're done testing with

python -m pip freeze > requirements.txt

I have a Jupyter notebook, how do I quickly view its contents?

There are two ways to do this. You will need Python installed. Then

• To view a HTML version, run jupyter-notebook. This should open a browser window, in which you can navigate to the Jupyter notebook. You should now be able to view it, as well as the output from the last run.
• To create a PDF version (which can be committed to the repository), run jupyter-nbconvert name.ipynb --to pdf, which requires LaTeX to be installed. This should generate a PDF. [Tested on Linux only]

I have a Jupyter notebook, how do I execute it without using interactive tools?

jupyter nbconvert --execute --to notebook --inplace your-notebook.ipynb

will run the code in the notebook, and save everything back to the original notebook. If the code does not explicitly write out figures or tables, you may have to run the above command to convert it to PDF.

Best practice to reproduce a Python paper (Python environments)

You should create a Python environment that is dedicated to the project. See Anaconda instructions as one possible method, venv as another one, though others exist.

Here's venv version in a nutshell (full guide)

• Ensure venv exists:

pip3 install pyenv

• Create a new environment

python3 -m venv /path/to/new/virtual/environment

or if using relative paths

python3 -m venv env

which will create /path/to/new/virtual/environment or (relative to your current working directory) env. That directory will now contain all of your project-related Python packages.

To activate:

source env/bin/activate

On Windows Bash (depends on install)

python -m venv env
source env/Scripts/activate

To deactivate:

deactivate

Jupyter Notebook/JupyterLab on CCSS systems (or Windows)

Using standard Python

Ideally, create an environment first, see above!

Install Jupyter Python packages:

pip install jupyter
jupyter lab

which will start a browser window with the Jupyter Lab interface.

Using Anaconda

This is known to work on CISER (CCSS-Classic).

This may not be the way it works on CCSS-Cloud.

If using the default "Jupyter" link in the Start Menu, the working directory won't be right. Assuming that you have set your Workspace to U:\Documents\Workspace, the following will create a Jupyter Notebook in the right location (thanks to Louis Liu for creating this Howto)

Search "anaconda prompt" from the start menu. right click on the app when it appears and pin it to the taskbar.

Right click on anaconda prompt in the taskbar (looks like a black window, similar to command line or terminal). Right click on "anaconda powershell prompt" in the tasks menu that pops up, and then properties.

In the properties window, go to the shortcut tab and change the "Start in:" field to U:\Documents\Workspace or whichever directory you keep your bitbucket repos in. Click apply.

Next, click on the anaconda prompt shortcut in the taskbar. When anaconda prompt opens, enter the command "Jupyter notebook"

Conda on BioHPC

If a replication package uses conda for package management, rather than pip, follow instructions at BioHPC on how to install miniconda in your home directory, then add the line

source $HOME/miniconda3/bin/activate

at an appropriate location in the code (for instance, replacing module load conda).

Random

Dynamically setting hard-coded directories

## Windows System Path
FolderList = ["D:\\Dropbox\\Research Projects\\ProjectMe\\", \
              "B:\\Dropbox\\Research Projects\\ProjectHim\\", \
              "/home/user/project/hpc"]
for Folder in FolderList:
    if os.path.exists(Folder):
        os.chdir(Folder)