This repository is part of the GRADE project
The goal of this code is to show how you can convert any SMPL-based animation to a USD-based animation. The script is capable of managing mesh caches and skeletal animations. It can export point-sequence based animations and skeletal-based animations.
Install blender connector from the Omniverse launcher. This code was tested with versions 3.4.0-usd.101.0 (main branch). For the paper work we used 3.1.0-usd.100.1.10.
Some limitations of 3.1.0-usd.100.1.10:
- you might need to use the mesh cache modifier instead of the blendshape. There is a minimal difference that arise when loading the animation in Omniverse's products.
- keep textures with absolute paths. You can replace them whenever you want afterwards with our tool USD_text_replace
Install the necessary dependencies. Locate the blender installation path and run python.exe -m pip install ipdb pyquaternion scipy torch pyyaml chumpy.
e.g. In my case C:\User\ebonetto\AppData\Local\ov\pkg\blender-3.4.0-usd.101.0\Release\3.4\python\bin\python.exe -m pip install ipdb pyquaternion scipy torch pyyaml chumpy
Additionally, you need to follow this to fill up the installation missing files that we cannot redistribute because of licensing.
We are already supporting two datasets. Cloth3D and AMASS.
If you want to add a different dataset for AMASS you need to add it to the data_folder/taxonomy.json file
From the cloned repository main folder
\AppData\Local\ov\pkg\blender-3.4.0-usd.101.0\Release\blender.exe --python-use-system-env --python-exit-code 0 --python start_blend_debug.py -- generate_sequence.py --dataset ... --output_dir ... --samples_dir ... --last_sample ... --parent_path ... --sample_id ...
The parameters are explained in the code or self-explaining.
dataset can be either [cloth3d, amass]. With amass a necessary configuration file needs to be included (e.g. --config_file this_repo\humangenerator\amass.json). We provide a sample config here.
Note that AMASS will process the folder directly (by querying subfolders) differently than Cloth3D for which you need to give the main parent folder (eg. cloth3d/train_t1).
sample_id if is an ID it will process that ID otherwise you can set it to all or leave it empty and it will process the whole set of data.
last_sample is used in case sample_id is empty and will be used to signal where to restart the processing.
If running multiple generations the code will automatically periodically clean the whole simulation environment including textures and materials to avoid crashing.
-
Cloth3D single sample example
--python-use-system-env --python-exit-code 0 --python start_blend_debug.py -- generate_sequence.py --dataset cloth3d --output_dir outdir --samples_dir cloth3d\train --last_sample 01056 --parent_path D:\generate_people\data_folder\ --sample_id 01056 -
AMASS
--python-use-system-env --python-exit-code 0 --python start_blend_debug.py -- generate_sequence.py --dataset amass --output_dir D:\cloth3d\exported_usd --samples_dir D:\AMASS\CMU\ --parent_path D:\Cloth3D_to_usd\data_folder\ --config_file D:\Cloth3D_to_usd\humangenerator\amass.json
The texture of the person is random. In the Cloth3D case the chosen ones are the ones with underwears, with AMASS the ones with clothes.
You have the possibility of exporting the SMPL information, the vertex info, the USD file, the STL trace of the animation and much more.
You can also suppress the output from the shell. However, the exporter in USD forcibly write directly to stdout. I have found no redirect strategy that works.
The system will replicate the input folder structure in the output folder.
You can also select a single frame.
You are encouraged to extend this and create pull requests.
Cloth3D clothes are loaded and exported as MeshCaches.
For the human animations you can chose.
You can create your own processor by creating a new class here, adding your dataset name here and write the else here.
In practice you need to write your own python dataset_gen.py.
That file needs to have a process_sample method which will be then called by the main script.
Within process_sample you want to take care either of the sample (CLOTH3D) or of the whole folder (AMASS). Your choice.
We see the processing from the loading of the animation to writing data.
In the main script then there is a call to get_processor that returns processor, PATH_SAMPLES, processor is the instance of the class you just created.
Few lines below you find res = processor.process_sample(sample, frame, smpl_body_list).
The exported USDs will have 24 fps as default. We did not investigate this much. You can change this by using the usd converter to text and change the 4th line to 30 fps. This value will influence how the mesh will be loaded into the simulation by the scripts used in GRADE. In our work we did NOT change this value.
For licensing information, please refer to the main repository located here.
If you find this work useful please cite our work based on this information
Code based on