Skip to content

ExMR v0.1.4: joints, BVH, robots, faster demo

Choose a tag to compare

@twehrbein twehrbein released this 26 Oct 10:01
· 11 commits to main since this release

No new model files: existing installations update with git pull. The dependency pins (requirements.txt, constraints.txt) and the prebuilt nvdiffrast wheel are for fresh installs, see docs/INSTALL.md.

New

  • Joints in the results: results.pt and results.npz contain the 3D joints in the camera and world frame (joints_incam, joints_global), their projection into the image (joints_2d) and joint_names: the 55 SMPL-X joints and 21 surface points (nose, eyes, ears, toes, heels, fingertips). exmr.pipeline.compute_joints computes them for results of earlier versions.
  • BVH export: every run also writes <video name>.bvh, the world-frame motion as a skeleton animation of the 55 SMPL-X joints in centimeters, for Blender, Maya, MotionBuilder and similar.
  • Unity, Unreal and humanoid robots: the AMASS file now stores 16 betas, as loaders such as GMR expect; docs/OUTPUTS.md describes the FBX route through the SMPL-X Blender add-on, retargeting to humanoid robots with GMR, and which output to use for what.
  • --start, --end: process only a part of the video, in seconds or as [hh:]mm:ss, cut frame-accurately and losslessly.
  • --f_mm: the camera's focal length (35 mm equivalent), if known, for the model, SAM 3D Body, the camera motion estimation and rendering.
  • Faster demo on the example video (RTX 4090, -s): fast 1 min 1 s, balanced 1 min 17 s, accurate 2 min 59 s (v0.1.3: 1:09, 1:23, 3:22). On GeForce GPUs the FP16 stages accumulate in FP16 (accuracy unchanged in our evaluation), RTMW's session warms up while the tracker runs, DPVO's input is capped at 1080p size, and the SMPL-X model is loaded once per run.
  • Install without compiling: nvdiffrast comes as a prebuilt wheel for the pinned torch; all dependencies are pinned. HaMeR's files are downloaded from the HaMeR authors' Hugging Face Space (2.7 GB instead of the 6 GB archive). Windows: see the WSL2 note in the install docs.
  • Library use: exmr.pipeline exposes the pieces run_video is built from (hand_boxes, model_inputs, compute_joints, save_results, render), e.g. for pipelines with preloaded models.

Changes

  • compute_joints, exmr.export.save_amass and save_bvh take a SmplxModel instead of the path of SMPLX_NEUTRAL.npz (see the examples in docs/OUTPUTS.md).
  • On GeForce GPUs, results differ from v0.1.3 at the level of run-to-run noise (FP16 accumulation); other GPUs are unchanged.
  • run_video checks for nvdiffrast and for the SAM 3D Body submodule before any work.

Clone with --recursive: the source archives below do not include the git submodules.