MVDoppler-Pose

Environment

• Python: 3.10.8
• Pytorch: 1.13.1
• CUDA: 11.6
• CuDNN: 8
• Environment can directly be imported through Docker image

Preparing the Dataset

• Download the dataset from this Google Drive link
• Unzip folders Data and Metadata
• Data has all the samples with structure of

dataset
    ├── 2022Jul13-1744
        ├── radar_v2
            ├── 20220713174506.h2
            ├── 20220713174732.h2	
            ...
        ├── 20220713174506
            ├── output_3D
                ├── keypoints.npz
        ├── 20220713174732
        ...
    ├── 2022Jul13-1806
    ...

• des_all.csv includes the metadata for the entire dataset in camma seperated values (CSV) format.
  

Argument configurations

This codebase uses Hydra to manage and configure arguments. Hydra offers more flexibility for running and managing complex configurations, and supports rich hierarchical config structures.

The YAML configuration files are in folder conf. So you can have a set of arguments in your YAML file like

train:
  learning_rate: 1e-4
transforms: 
  win_size: 512

Code Tree

/main_multi_keypoint.py: Main file to run the code
/conf/: Configuration file for adjusting parameters
/model/: Include transformer-based neural network models
/utils_multi/: Utility functions for training and testing models such as dataloader, data transformation, model training/testing

Train the Baseline Model

• Specify data folder and metadata file through data_dir and csv_file in /conf/config_keypoint_adjust.yaml
• Rut it through Python

cd MVDoppler-Pose
python main_multi_keypoint.py

Model Inference and Checkpoints

Trained model parameters(mmWave_ckp.pt) and corresponding settings(mmWave_args.yaml) can be found from this Google Drive foler.

To directly do inference using the trained model, you need to change the config file:

• In /conf/config_inference.yaml, specify the path of mmWave_ckp.pt and mmWave_args.yaml with path_model and path_args, respectively.
• Specify the path where the result will be saved in path_save
• Specify the name of the episode you want to test in test_episode (Default episode: '20220610130106')
• Run it through Python

cd MVDoppler-Pose
python main_inference_keypoint.py

License

The MVDoppler-Pose code and dataset are published under the CC BY-NC-ND License, and all codes are published under the Apache License 2.0.