If you’d like more details about the research behind this project, please check out our papers:
[1] Z. Tian, Y. Jing, and A. Han, "UltraWave: An Open-source Multi-GPU Full-wave Simulator for Acoustic and Elastic Wave Scattering in 3-D Heterogeneous Media," IEEE Transactions on Ultrasonics, 2025. Link: https://ieeexplore.ieee.org/abstract/document/11265733.
[2] Z. Tian, Y. Jing, and A. Han, "An Open-Source GPU-Based Acoustic Simulator for Fast and Accurate Modeling of Acoustic Scattering," 2024 IEEE Ultrasonics, Ferroelectrics, and Frequency Control Joint Symposium (UFFC-JS). IEEE, 2024. Link: https://ieeexplore.ieee.org/abstract/document/10793878.
[3] Z. Tian, Y. Jing, and A. Han, "Accurate and Efficient Modeling of Acoustic and Elastic Absorption in Medical Ultrasound Simulations," 2025 IEEE International Ultrasonics Symposium (IUS). IEEE, 2025. Link: https://ieeexplore.ieee.org/abstract/document/11201807.
You can use the toolbox in a Python virtual environment using virtualenv:
To install virtualenv
pip install --upgrade pip
pip install virtualenv
To create a dedicated virtual env in the ultrawave directory (e.g., ultrawave_venv):
python -m venv ultrawave_venv
To activate the virtual environment
source ultrawave_venv/bin/activate
To exit the vitual env
deactivate
To use the toolbox in a conda environment, you need to install either Anaconda or Miniconda.
To create a new conda environment with the name ultrawave
conda create -n ultrawave python=3.10
And activate this environment
conda activate ultrawave
To exit the environment
conda deactivate
First, you need to clone the folder by running:
git clone https://github.com/zixuant5/UltraWave.git
To build and install the ultrawave package in-place, such that you can edit the files without
re-installing the package, run the following command at the UltraWave directory:
cd UltraWave
pip install --upgrade pip
pip install -r requirements.txt
pip install -e .
For Windows users who use conda environment, if an error shows that 'pip' is not recognized, you can try:
conda install pip
You can run pip show ultrawave to check the version of ultrawave. The latest version now is 0.2.0.
To run the jupyter notebook example, you can go to the UltraWave directory.
jupyter notebook
To use GPU for simulation acceleration, you need to install Nvidia HPC SDK based on your CUDA version.
After the installation, you need to set up the environment such as running this command. Reference can be taken in section 1.3 of this link.
export PATH=/opt/nvidia/hpc_sdk/Linux_x86_64/23.7/compilers/bin/:$PATH
There are two ways to use GPU to accelerate your simulation.
The first way is to set environment variables inside your python script or notebook.
from devito import configuration
configuration['platform'] = 'nvidiaX'
configuration['compiler'] = 'pgcc'
configuration['language'] = 'openacc'
The second way is to set environment variables from the shell.
export DEVITO_PLATFORM=nvidiaX
export DEVITO_ARCH=pgcc
export DEVITO_LANGUAGE=openacc
To use multiple GPUs, you need to set the environment variables like the one-GPU case. Besides that, the OpenMPI environment needs to be set up.
export PATH=/opt/nvidia/hpc_sdk/Linux_x86_64/23.7/comm_libs/mpi/bin/:$PATH
export LD_LIBRARY_PATH=/opt/nvidia/hpc_sdk/Linux_x86_64/23.7/comm_libs/mpi/lib/:$LD_LIBRARY_PATH
Install ipyparallel and mpi4py:
pip install --upgrade pip
pip install ipyparallel
pip install mpi4py
If you met an error similar to this one when installing mpi4py, you can run
export CC=$(which nvc)
export CXX=$(which nvc++)
python3 -m pip install --upgrade pip setuptools wheel
CFLAGS=-noswitcherror pip install mpi4py
After environment setting up, you can run the python script with MPI in shell.
DEVITO_MPI=1 mpirun -n 4 python3 elastic_3d.py