Are Pose Estimators Ready for the Open World? STAGE: A GenAI Toolkit for Auditing 3D Human Pose Estimators

3DV 2026

Nikita Kister¹ , Istvan Sarandi¹, Jiayi Wang³, Anna Khoreva⁴ Gerard Pons-Moll^{1, 2}

¹Real Virtual Human Group @ University of Tübingen & Tübingen AI Center
²Max Planck Institute for Informatics, Saarland Informatics Campus
³ Bosch Center of Artificial Intelligence
⁴ Zalando SE

Prepare environment

conda create -n stage python=3.9 -y
conda activate stage

bash install.sh

Project structure

Download SMPL weights (male, female, neutral) from SMPL and SMPLify websites. Extract and rename the weight files (e.g. hbasicmodel_neutral_lbs_10_207_0_v1.0.0.pkl to SMPL_NEUTRAL.pkl). Download test_poses.npz, sub_test_poses, smpl2openpose_2.npy, and ckpts from here: google drive This is how the directory should look like:

StableDiffusionDataGeneration
├── src
├── pretrained
    └── body_models
        └── smpl
            └── SMPL_NEUTRAL.pkl
            └── SMPL_MALE.pkl
            └── SMPL_FEMALE.pkl
        └── smplx_to_smpl.pkl
    └── ckpts
    └── smpl2openpose_2.npy  # google drive
└── data
    └── test_poses.npz
    └── sub_test_poses.npz

Generating Data and Evaluation

This is a sequence of commands to evaluate the MetrAbs pose estimator. Generated images are found in output/eval_gender. Note that sub_test_poses.npz is used, which is a subset of 10 poses from the full pose set. For accurate evaluation we recommend to run on the full set of 1500 poses. Our ControlNet is available at CN-3DPose.

python src/generate.py --gen-name=eval_gender --output-dir "output" --debug --base-gender "male" --base-ethnicity "caucasian" --base-age "adult" --base-clothing "t-shirt and long pants" --base-location "city center" --base-lighting "daytime" --base-weather "summer day" \
                                                                            --att-gender "female" --gt-pose-path "data/sub_test_poses.npz" --use-pose-filter --pose-filter-th 50 --num-good-images 1 --num-failed-rounds 5 --batch-size 2
python src/run_metrabs.py --project output/eval_gender
python src/compute_metrics.py --project output/eval_gender  --pred-file pose-pred-metrabs.npz  --gt-pose-path "data/sub_test_poses.npz"  --out-name analysis-results-metrabs.pkl

For SMPL-X estimators we provide a script (smplx_to_smpl.py) to convert SMPL-X to SMPL in a fast way.