Repository navigation
ExMR v0.1.4: joints, BVH, robots, faster demo
No new model files: existing installations update with git pull. The dependency pins (requirements.txt, constraints.txt) and the prebuilt nvdiffrast wheel are for fresh installs, see docs/INSTALL.md.
New
- Joints in the results:
results.ptandresults.npzcontain the 3D joints in the camera and world frame (joints_incam,joints_global), their projection into the image (joints_2d) andjoint_names: the 55 SMPL-X joints and 21 surface points (nose, eyes, ears, toes, heels, fingertips).exmr.pipeline.compute_jointscomputes them for results of earlier versions. - BVH export: every run also writes
<video name>.bvh, the world-frame motion as a skeleton animation of the 55 SMPL-X joints in centimeters, for Blender, Maya, MotionBuilder and similar. - Unity, Unreal and humanoid robots: the AMASS file now stores 16 betas, as loaders such as GMR expect; docs/OUTPUTS.md describes the FBX route through the SMPL-X Blender add-on, retargeting to humanoid robots with GMR, and which output to use for what.
--start,--end: process only a part of the video, in seconds or as[hh:]mm:ss, cut frame-accurately and losslessly.--f_mm: the camera's focal length (35 mm equivalent), if known, for the model, SAM 3D Body, the camera motion estimation and rendering.- Faster demo on the example video (RTX 4090,
-s):fast1 min 1 s,balanced1 min 17 s,accurate2 min 59 s (v0.1.3: 1:09, 1:23, 3:22). On GeForce GPUs the FP16 stages accumulate in FP16 (accuracy unchanged in our evaluation), RTMW's session warms up while the tracker runs, DPVO's input is capped at 1080p size, and the SMPL-X model is loaded once per run. - Install without compiling: nvdiffrast comes as a prebuilt wheel for the pinned torch; all dependencies are pinned. HaMeR's files are downloaded from the HaMeR authors' Hugging Face Space (2.7 GB instead of the 6 GB archive). Windows: see the WSL2 note in the install docs.
- Library use:
exmr.pipelineexposes the piecesrun_videois built from (hand_boxes,model_inputs,compute_joints,save_results,render), e.g. for pipelines with preloaded models.
Changes
compute_joints,exmr.export.save_amassandsave_bvhtake aSmplxModelinstead of the path ofSMPLX_NEUTRAL.npz(see the examples in docs/OUTPUTS.md).- On GeForce GPUs, results differ from v0.1.3 at the level of run-to-run noise (FP16 accumulation); other GPUs are unchanged.
run_videochecks for nvdiffrast and for the SAM 3D Body submodule before any work.
Clone with --recursive: the source archives below do not include the git submodules.