-
Notifications
You must be signed in to change notification settings - Fork 0
Setting up a development environment (Linux)
Brian Wheeler edited this page Apr 8, 2026
·
3 revisions
This has been tested on this environment
- Linux (Fedora 44, should work on others)
- Python 3.12
- Visual Studio Code
Create a directory that will hold all of the repositories that we're working with as well as the python virtual environment:
mkdir AMPAV
cd AMPAV
python3.12 -mvenv .venv
source .venv/bin/activate
We'll need a few tools for managing the installation. The build module is used to build the python packages and twine is used to publish those packages to a repository
pip install -U pip
pip install build twine
``
## Clone and install the repositories
Clone the repositories you'll be working on. `ampav-core` will likely always be included. The example below is for working on the
`ampav-whisper` repository
git clone https://github.com/AMPAV/ampav-core.git git clone https://github.com/AMPAV/ampav-whisper.git
The repositories should be installed in "editable" mode – that allows the packages to be modified and tested without the need to reinstall them every time a change is made.
pip install -e ./ampav-core pip install -e ./ampav-whisper
## Do development
If you aren't in the venv you'll need to activate it (if you just came from the instructions above you can skip this):
cd AMPAV
source .venv/bin/activate
Start Visual Studio Code in the current directory:
code .
At this point you should have access to both of the repositories and changes within them are tracked with their respective repository
Fixing import errors on ampav.*
The language tools will sometimes get confused because we're actually editing two different packages which reside in the same namespace. This will manifest itself as an import error in the editor, even though the code will work as expected. We can solve this by creating/modifying the .vscode/settings.json file in the AMPAV directory:
{
"python.analysis.extraPaths": [
"./ampav-core",
"./ampav-whisper"
]
}
Building and uploading packages
Build package
Python packages are built in the repo directory by running:
python -mbuild
It will output a bunch of stuff and should end up looking like this:
((.venv) ) bdwheele@trogdor:~/work_projects/AMPAV/ampav-whisper$ python -mbuild
* Creating isolated environment: venv+pip...
* Installing packages in isolated environment:
- setuptools
* Getting build dependencies for sdist...
running egg_info
creating ampav_whisper.egg-info
writing ampav_whisper.egg-info/PKG-INFO
writing dependency_links to ampav_whisper.egg-info/dependency_links.txt
writing requirements to ampav_whisper.egg-info/requires.txt
writing top-level names to ampav_whisper.egg-info/top_level.txt
writing manifest file 'ampav_whisper.egg-info/SOURCES.txt'
reading manifest file 'ampav_whisper.egg-info/SOURCES.txt'
adding license file 'LICENSE'
writing manifest file 'ampav_whisper.egg-info/SOURCES.txt'
* Building sdist...
running sdist
running egg_info
writing ampav_whisper.egg-info/PKG-INFO
writing dependency_links to ampav_whisper.egg-info/dependency_links.txt
writing requirements to ampav_whisper.egg-info/requires.txt
writing top-level names to ampav_whisper.egg-info/top_level.txt
reading manifest file 'ampav_whisper.egg-info/SOURCES.txt'
adding license file 'LICENSE'
writing manifest file 'ampav_whisper.egg-info/SOURCES.txt'
running check
creating ampav_whisper-0.0.1
creating ampav_whisper-0.0.1/ampav/whisper
creating ampav_whisper-0.0.1/ampav_whisper.egg-info
copying files to ampav_whisper-0.0.1...
copying LICENSE -> ampav_whisper-0.0.1
copying README.md -> ampav_whisper-0.0.1
copying pyproject.toml -> ampav_whisper-0.0.1
copying ampav/whisper/__init__.py -> ampav_whisper-0.0.1/ampav/whisper
copying ampav/whisper/transcribe.py -> ampav_whisper-0.0.1/ampav/whisper
copying ampav_whisper.egg-info/PKG-INFO -> ampav_whisper-0.0.1/ampav_whisper.egg-info
copying ampav_whisper.egg-info/SOURCES.txt -> ampav_whisper-0.0.1/ampav_whisper.egg-info
copying ampav_whisper.egg-info/dependency_links.txt -> ampav_whisper-0.0.1/ampav_whisper.egg-info
copying ampav_whisper.egg-info/requires.txt -> ampav_whisper-0.0.1/ampav_whisper.egg-info
copying ampav_whisper.egg-info/top_level.txt -> ampav_whisper-0.0.1/ampav_whisper.egg-info
copying ampav_whisper.egg-info/SOURCES.txt -> ampav_whisper-0.0.1/ampav_whisper.egg-info
Writing ampav_whisper-0.0.1/setup.cfg
Creating tar archive
removing 'ampav_whisper-0.0.1' (and everything under it)
* Building wheel from sdist
* Creating isolated environment: venv+pip...
* Installing packages in isolated environment:
- setuptools
* Getting build dependencies for wheel...
running egg_info
writing ampav_whisper.egg-info/PKG-INFO
writing dependency_links to ampav_whisper.egg-info/dependency_links.txt
writing requirements to ampav_whisper.egg-info/requires.txt
writing top-level names to ampav_whisper.egg-info/top_level.txt
reading manifest file 'ampav_whisper.egg-info/SOURCES.txt'
adding license file 'LICENSE'
writing manifest file 'ampav_whisper.egg-info/SOURCES.txt'
* Building wheel...
running bdist_wheel
running build
running build_py
creating build/lib/ampav/whisper
copying ampav/whisper/transcribe.py -> build/lib/ampav/whisper
copying ampav/whisper/__init__.py -> build/lib/ampav/whisper
running egg_info
writing ampav_whisper.egg-info/PKG-INFO
writing dependency_links to ampav_whisper.egg-info/dependency_links.txt
writing requirements to ampav_whisper.egg-info/requires.txt
writing top-level names to ampav_whisper.egg-info/top_level.txt
reading manifest file 'ampav_whisper.egg-info/SOURCES.txt'
adding license file 'LICENSE'
writing manifest file 'ampav_whisper.egg-info/SOURCES.txt'
installing to build/bdist.linux-x86_64/wheel
running install
running install_lib
creating build/bdist.linux-x86_64/wheel
creating build/bdist.linux-x86_64/wheel/ampav
creating build/bdist.linux-x86_64/wheel/ampav/whisper
copying build/lib/ampav/whisper/transcribe.py -> build/bdist.linux-x86_64/wheel/./ampav/whisper
copying build/lib/ampav/whisper/__init__.py -> build/bdist.linux-x86_64/wheel/./ampav/whisper
running install_egg_info
Copying ampav_whisper.egg-info to build/bdist.linux-x86_64/wheel/./ampav_whisper-0.0.1-py3.12.egg-info
running install_scripts
creating build/bdist.linux-x86_64/wheel/ampav_whisper-0.0.1.dist-info/WHEEL
creating '/data/work_projects/AMPAV/ampav-whisper/dist/.tmp-feebrko4/ampav_whisper-0.0.1-py3-none-any.whl' and adding 'build/bdist.linux-x86_64/wheel' to it
adding 'ampav/whisper/__init__.py'
adding 'ampav/whisper/transcribe.py'
adding 'ampav_whisper-0.0.1.dist-info/licenses/LICENSE'
adding 'ampav_whisper-0.0.1.dist-info/METADATA'
adding 'ampav_whisper-0.0.1.dist-info/WHEEL'
adding 'ampav_whisper-0.0.1.dist-info/top_level.txt'
adding 'ampav_whisper-0.0.1.dist-info/RECORD'
removing build/bdist.linux-x86_64/wheel
Successfully built ampav_whisper-0.0.1.tar.gz and ampav_whisper-0.0.1-py3-none-any.whl
Setting up the local repository access (one time only)
We're currently using a local repository rather than publishing to PyPI – so we have to do some minor setup.
First, developers will need to have a username/password in dewberry:/srv/services/pypi-prod-8142/htpasswd . It just uses the standard htpasswd tool to manage the database.
Second, we have to tell the publishing tools about our repository. Add these lines to ~/.pypirc :
[distutils]
index-servers =
local
[local]
repository: http://dewberry.dlib.indiana.edu:8142
username: your_username
password: your_password
Publish package
The package can be pushed to our pypi repository by using twine . This is the push for ampav-core version 0.0.4:
twine upload -r local dist/ampav_core-0.0.4*
If you try to push the same version more than once it will fail. The version is managed in pyproject.toml
You can verify the package upload by checking the repository index at http://dewberry.dlib.indiana.edu:8142/packages/
Test packages in a notebook environment
Generally you want to run the notebooks in a virtual environment so it doesn't interact with the packages you have installed in your user environment.
This will be a different venv than you use for development.
Setting up JupyterLab (one time only)
Create a directory, virtual environment, and install the jupyter lab software:
mkdir AMPAV-JupyterLab
cd AMPAV-JupyterLab
python3.12 -mvenv .venv
source .venv/bin/activate
pip install jupyterlab
Start the jupyter lab
If you're not already in the jupyterlab venv, activate it:
source .venv/bin/activate
Start the jupyter lab server:
jupyter lab
The JupyterLab server will start on port 8888 (most likely) and try to open a web browser pointing to the server.
Installing the ampav python packages
Right now there are just two, but we need to test to make sure dependencies work as advertised. The ampav-whisper package depends on ampav-core so theoretically, we can just install it and it should pull in ampav-core as well as the rest of the dependencies (torch, openai-whisper, pydantic, numpy, etc).
Insert this into a code block and run it:
%pip install -U --extra-index-url https://pypi.dlib.indiana.edu ampav-whisper
It should chug along for a while and eventually drop out something like this:
Successfully installed ampav-core-0.0.3 ampav-whisper-0.0.1 annotated-types-0.7.0 cuda-bindings-13.2.0 cuda-pathfinder-1.5.1 cuda-toolkit-13.0.2 filelock-3.25.2 fsspec-2026.3.0 llvmlite-0.47.0 more-itertools-11.0.1 mpmath-1.3.0 networkx-3.6.1 numba-0.65.0 numpy-2.4.4 nvidia-cublas-13.1.0.3 nvidia-cuda-cupti-13.0.85 nvidia-cuda-nvrtc-13.0.88 nvidia-cuda-runtime-13.0.96 nvidia-cudnn-cu13-9.19.0.56 nvidia-cufft-12.0.0.61 nvidia-cufile-1.15.1.6 nvidia-curand-10.4.0.35 nvidia-cusolver-12.0.4.66 nvidia-cusparse-12.6.3.3 nvidia-cusparselt-cu13-0.8.0 nvidia-nccl-cu13-2.28.9 nvidia-nvjitlink-13.0.88 nvidia-nvshmem-cu13-3.4.5 nvidia-nvtx-13.0.85 openai-whisper-20250625 pydantic-2.12.5 pydantic-core-2.41.5 regex-2026.3.32 setuptools-81.0.0 sympy-1.14.0 tiktoken-0.12.0 torch-2.11.0 tqdm-4.67.3 triton-3.6.0 typing-inspection-0.4.2
Another code block shows that we can call a stupid function in the ampav.whisper package which uses ampav.core.gpu and queries what torch devices are available:
from ampav.whisper.transcribe import hello
hello()
Hello from ampav.whisper.transcribe, ya jerk
['cuda:0', 'cpu']