PhysNAP: Guiding Diffusion-Based Articulated Object Generation by Partial Point Cloud Alignment and Physical Plausibility Constraints

This repository provides source code for PhysNAP accompanying the following publication:

Jens U. Kreber and Joerg Stueckler, "Guiding Diffusion-Based Articulated Object Generation by Partial Point Cloud Alignment and Physical Plausibility Constraints"
In IEEE/CVF International Conference on Computer Vision (ICCV) 2025, Honolulu, USA, to appear. Preprint: https://arxiv.org/abs/2508.00558

If you use the source code provided in this repository for your research, please cite the corresponding publication as:

@inproceedings{kreber2025_physnap,
  author      = {Jens U. Kreber and Joerg Stueckler},
  title       = {Guiding Diffusion-Based Articulated Object Generation by Partial Point Cloud Alignment and Physical Plausibility Constraints},
  booktitle   = {Proc. of the IEEE/CVF International Conference on Computer Vision (ICCV)},
  year        = {2025},
  note        = {to appear, preprint at https://arxiv.org/abs/2508.00558},
  doi={}
}

The described method is mainly implemented in core/models/arti_ddpm_v2.py.

In the following, we detail all steps from setting up the environment to generating articulated objects and calculating metrics.

Environment Setup

For this setup, you need to have conda installed. Create the conda env

conda create -n physnap gcc_linux-64=9 gxx_linux-64=9 python=3.9 -y
conda activate physnap

Then run bash env.sh, which will install all dependencies. The setup is tested to work on Ubuntu 22.04.

Download Data

Note: We use a slightly adapted implementation of the denoising network architecture, see the supplementary material of the paper. Therefore, our trained model differs from the one provided by the NAP authors, but they are trained on the same dataset.

Still, we use the NAP part shape autoencoder as-is, you can find the NAP pretrained model (checkpoint) here. Put the weights of the part shape encoder under PROJECTROOT/log where PROJECTROOT is the root of this repository.

PROJECTROOT/log
└── s1.5_partshape_ae

We do not use the file v6.1_diffusion which is the original NAP model and also contained in the archive.

Next, download the pre-processed dataset provided by NAP (see here) (same dataset NAP is trained on). Put the contents under data:

PROJECTROOT/data
├── partnet_mobility_graph_mesh
└── partnet_mobility_graph_v4

Model Training

Run

python run.py --config ./configs/nap/v6.1_diffusion_adapted.yaml -f

to train the model, similar to original NAP. Replace the config with v6.1_diffusion_adapted_catembed.yaml to train the category-aware model.

Unconditional generation

You can now generate unconditional (without a point cloud observation) samples by running for example

cd eval
python run_guided.py --test --pen_fac 2 --mob_fac 2 --N 10 --bs 10 --guide_fromto 500,1000 --dps_weight 1 --model adapted --seed 0 --name unconditional_with_plausibility

For the category-aware model, use the flag --usecats and switch to --model adapted_catembed.

A directory is created under log and generated samples are saved and plotted together with error values per sample. Per-sample errors are also saved in stats.json.

If you want to run without any guidance, leave at least one of the loss factors > 0, but set --guide_fromto 1000,1000. If all loss factors are 0, the original postprocessing of NAP will be applied, which will rescale part meshes.

Compute Evaluation References for Generative Metrics

The following steps are required if you want to compute generative metrics for the created samples and follows a similar procedure as for NAP. This requires setting up a reference set, which can be either the val or test split of the dataset. The following code shows the procedure for the test split.

cd eval
python save_gt.py test
python sample_pcl.py --src ../log/GTtest/G/K_8_cate_all_gt --dst ../log/GTtest/PCL
python instantiation_distance.py --gen ../log/GTtest/PCL --ref ../log/GTtest/PCL --save_dir ../log/GTtest/ID_D_matrix --ref_name GTtest

Note that the computation of the instantiation distance can take a long time, both now and later for generated samples.

You can now run for example

python run_guided.py --remove --test  --pen_fac 2 --mob_fac 2 --N 10 --bs 10 --guide_fromto 500,1000 --dps_weight 1  --model adapted --seed 0 --name unconditional_with_plausibility --genfull_metrics

The generative metrics are printed and also saved to the stats.json.

Observation Dataset

To perform sample generation conditioned on point cloud observations, a dataset of such observations is required.

First, obtain the original PartNet-Mobility dataset from here and save it under data

PROJECTROOT/data
├── partnet_mobility_graph_mesh
├── partnet_mobility_graph_v4
└── partnet-mobility-v0

(the _graph_ entries already existed before).

For processing examples from the dataset, we use the GAPartNet pipeline for ease of use.

Therefore, clone GAPartNet into PROJECTROOT/GAPartNet and set up its requirements by running

pip install scikit-learn einops sapien==2.2.2
cd GAPartNet/pointnet2_ops_lib
python setup.py install

Now, you should be ready to generate a dataset of point cloud observations. To obtain the dataset we used in the paper (observations of 30 test set objects), run

python create_cond_ds.py --limit_n_models=30 --n_per_model 1 --seed 0 --split test --dir data/cond_test_n30

Due to seeding, it should be identical to our dataset. You can verify this by running

cd data/cond_test_n30
cat model_ids.txt categories.txt pc/*.ply | sha1sum 

which should output the checksum ca9f16d2d82bd7e6d83eb5c46ad0e71b5ac500a0.

Conditional Generation

Now you can use our full method and generate conditional samples. For example, run

cd eval
python run_guided.py  --test  --cond_fac 45 --pen_fac 2 --mob_fac 2  --cond_temp 1e3 --cond_model=squared --cond_dir ../data/cond_test_n30 --cond_per_metrics --cond_fromto=0,2 --N 5 --bs 50  --guide_fromto 500,1000 --dps_weight 0.333  --model adapted --name some_conditional_with_plausibility --seed 0 --genfull_metrics

Several metrics are computed, see the paper supplementary material for details. Our reported metrics are those without "nap" in the name. Note that in the conditional case, metrics should be computed per observation ( --cond_per_metrics ). They are stored individually under metrics_per.

License

See NOTICE.md for information on copyright and licenses.

Note: This repository is based on the publicly available implementation of NAP at https://github.com/JiahuiLei/NAP commit 1b062cb.