Python tools for submm astronomy instrumentation started at the University of Colorado Boulder
This project was started to make tools, used by everyone in the submm lab, available to everyone via version control on GitHub. This project continues to grow and expand in scope as University of Colorado graduates and collaborators continue to contribute to the project. Contributions and feature requests are not only welcome but are a fundamental part of the culture of submm astronomy.
Git and GitHub are standard tools across astronomy, computer science, computer engineering, and other fields. This is a toolset that everyone should learn for code management. It is always useful and often expected for contributors to a project of any size.
This installation will include gitbash on Windows machines. This is a terminal application that simulates a unix terminal on Windows. It is useful for managing code and working with git and github. If this is the first terminal/shell you have installed on your Windows computer, you will be able to use this to enter all of the terminal commands present in the rest of this README.md file.
A 15-minute introduction video for git is available at https://www.youtube.com/watch?v=USjZcfj8yxE
Get the latest version of Python3
https://www.python.org/downloads/
This project was tested with Python 3.8.5 to 3.10.x. It probably works with older versions of Python, but it is not guaranteed.
Some python
installations will need to call python3 instead of python from the terminal.
Check your python version with python --version if it comes back with
2.7.x, try python3 --version and expect to get 3.x.x.
Modern Python installations require you to sign an SSL certificate after installation. This is done by simply running a script in the installation directory. Programs like, Homebrew will do this step for you. If you see SSL errors in Python, it is probably because you skip the certificate step.
From a command line interface (terminal, gitbash, cmd, or powershell) Clone this Repository (https://github.com/Wheeler1711/submm_python_routines) using the following:
git clone https://github.com/Wheeler1711/submm_python_routines
then cd into the directory using:
cd submm_python_routines
Remember to update the repository periodically with:
git pull
Configure the virtual environment for DetMap or from the terminal.
This step is done one time to initialize the virtual environment.
Window and Linux Tested in Windows PowerShell and OSX Terminal. Some Python
installations will need to call python3 instead of python. Check your Python version with
python --version if it comes back with 2.7.x, try python3 --version and expect to get 3.x.x.
If you have many Python installations, or if you just downloaded the latest version
of python then you should replace python with the full path to the python executable.
python -m pip install --upgrade pip
pip install virtualenv
virtualenv venv
This step needs to be done every time you want to use the virtual environment.
For unix-like environment activation:
source venv/bin/activate
Windows CMD environment activation:
.\venv\Scripts\activate
for Windows powershell environment activation:
.\venv\Scripts\activate.ps1
After activation, the term ail will add a (venv) to the command prompt. At this point
you can drop the full paths to the Python and pip executables into the terminal,
and the python3 in place of python commands.
I like test that everything went as expected with the following:
python --version
pip list
This step needs to be done every time you want to install new Python packages.
The requirements.txt has the list of all packages needed for submm_python_routines. From a terminal with the virtual environment activated, do:
pip install -r requirements.txt
Packages can all be installed individually with pip install <package_name>
If you are a user of the submm_python_routines but are unlikely to modify (develop)
the code in this project, then it is recommended that you install the
submm_python_routines as a Python package. This can be done within a
virtual environment, or for a user, (or global version of Python which makes a
mess https://xkcd.com/1987/).
If you intend to use a virtual environment, activate the environment before this step.
You can always check on the version of Python you have installed with python --version
and your current package list with pip list.
Run setup.py in the usual way:
python setup.py install
check the installation by starting a Python console session with:
python
Then test with:
from KIDs.find_resonances_interactive import InteractiveFilterPlot
Exit the console session with:
quit()
To create a Jupyter notebook from the code in this repository without installing python.
A Docker image is built with the code in this repository. From that image, a new container is launched with the permissions and settings so that the full repository can be demonstrated. This container reads/writes the code in the directory submm_python_routines; edits in the Jupyter Server window will propagate to the local repository, and local changes are visible in the Jupyter Server. With git, this is all a user needs to test and contribute to this repository.
Tested and designed to work on any platform.
-
Docker and docker-compose must be installed
-
get/install docker for your computer at https://docs.docker.com/engine/install/
-
get/install docker-compose at: https://docs.docker.com/compose/install/
-
-
Start the docker engine
- Check Apps for Windows, OSX
- Use a command-line call in unix
- It is possible to configure the docker engine to start on start-up for your machine.
-
Testing Docker Engine and Client The docker engine/Server, not only the client must both be running. Check for them in terminal with:
docker version
- Go into submm_python_routines
cd submm_python_routines
- One command to build and deploy the docker container with a Jupyter server.
- Depending on your permissions and operating system you may need to use
sudo - If you have issues, make sure the docker engine is running (step 3).
- The first time build takes about 10 minutes.
- Subsequent builds will take 2-3 seconds using cached data.
- The notebook is accessible on any browser (I like Chrome) at: http://127.0.0.1:8888/lab?token=.
- Use default password/token:
docker
- Depending on your permissions and operating system you may need to use
docker-compose up --build
- Edit and run code
Changes made in the browser to files in submm_python_routines directory in the container are saved in the local computer at the same path, submm_python_routines. You can shut down the server and not lose any work.
-
Shut Down the Jupyter server
- Shut Down using the browser's drop-down menu.
- or by using
ctrl-cin the terminal.
-
Clean up the container network built with docker-compose with one terminal command
- This is a best practice operation.
docker-compose down
This method works and is simple. However, is not recommended for most users. Simply put, it teaches bad habits that can cause very hard-to-diagnose problems years later. The main issue is the pollution of the Python name space. It is easy to imagine a laboratory computer with many users and a lot of code. If everyone adds to the Python name spaces by appending to the PYTHONPATH, then it is easy to get into a situation where there are multiple copies of modules with the same name (KIDs, utils, instruments, etc.), even multiple copies of the same repository from different users and versions/branches of the code.
To modify your PYTHONPATH environment variable, add the following line to your ~/.bashrc (linux) or
~/.bash_profile (mac) or for windows see creating-and-modifying-environment-variables-on-windows.
export PYTHONPATH="${PYTHONPATH}:/Users/jordan/submm_python_routines
Where you replace /Users/jordan/submm_python_routines with the local installation path.
Maybe the most useful part of this code is the fitting of non-linear resonators. Care has been taken to make sure that resonance fitting is very fast so you can fit many resonators in real-time. You can try it out by doing the following.
cd submm_python_routines
python
from submm.demo import res_fit_demo
res_fit_demo.run()
For extracting TLS loss You can try it out by doing the following.
cd submm_python_routines
python
from submm.demo import fit_tls_demo
fit_tls_demo.run()
Real systems are never ideal. At least not anything I make. So when you have a lot of resonators, some will have collided or some will just be weird. We try to make our automated code very good, but when all else fails, some human intervention will be needed. As such, we have made some interactive plotting tools using matplotlib that allow for more sophisticated interaction with your data to better facilitate human interaction. In particular, there are interactive tools for identifying resonators and interactive tools for plotting resonators for examination. You can try it out by running the below.
cd submm_python_routines
python3 submm/demo/find_res_and_fit.py
or
python3
from submm.demo import find_res_and_fit
find_res_and_fit.main()