The offical implementation of RF Genesis: Zero-Shot Generalization of mmWave Sensing through Simulation-Based Data Synthesis and Generative Diffusion Models.
Xingyu Chen, Xinyu Zhang, UC San Diego.
In SenSys 2023
📢 22/Jan/24 - Initial Release of RF Genesis!
📢 29/March/24 - Added the code for point-cloud processing and visualization.
- Release the RFLoRA pretrained model.
- Release the RFLoRA training dataset and procedures.
- More documentations.
This code was tested on Ubuntu 20.04.5 LTS and requires:
- Python 3.10
- conda3 or miniconda3
- CUDA capable GPU (one is enough)
Clone the repository
git clone https://github.com/Asixa/RF-Genesis.git
cd RF-Genesis
Create a conda environment.
conda create -n rfgen python=3.10 -y
conda activate rfgen
Install python packages
pip install -r requirements.txt
sh setup.sh
Run a simple example.
python run.py -o "a person walking back and forth" -e "a living room" -n "hello_rfgen"
Rendered SMPL animation and radar point clouds.
The current simulation is based on the model of Texas Instruments AWR 1843 radar, with 3TX 4RX MIMO setup.
The radar configuration is shown in TI1843.json and it can be freely adjusted.
@inproceedings{chen2023rfgenesis,
author = {Chen, Xingyu and Zhang, Xinyu},
title = {RF Genesis: Zero-Shot Generalization of mmWave Sensing through Simulation-Based Data Synthesis and Generative Diffusion Models},
booktitle = {ACM Conference on Embedded Networked Sensor Systems (SenSys ’23)},
year = {2023},
pages = {1-14},
address = {Istanbul, Turkiye},
publisher = {ACM, New York, NY, USA},
url = {https://doi.org/10.1145/3625687.3625798},
doi = {10.1145/3625687.3625798}
}
This code is distributed under an MIT LICENSE. Note that our code depends on other libraries, including CLIP, SMPL, MDM, mmMesh and uses datasets that each have their own respective licenses that must also be followed.