Seeing the World and the Self from Egocentric Video

RESELF: REconstructing the Scene and the sELF.

Kai Guan^1,2,, Minchao Jiang^2,, Liruichen Wang², Wentao Zhu^2,†, Lei Zhang^1,†

¹The Hong Kong Polytechnic University    ²Eastern Institute of Technology, Ningbo
^*Equal contribution    ^†Corresponding authors

Project Website

RESELF is a project for joint metric scene reconstruction and full-body motion estimation from monocular egocentric video.

📌 TODO

• Release the test code
• Release the training code
• Release the dataset

🎬 Overview

RESELF jointly reconstructs metric scene geometry and the wearer's full-body motion from monocular egocentric video. The pipeline follows three steps:

• Metric geometry: reconstruct the surrounding scene and camera trajectory in a shared metric frame.
• Conditioned motion: predict full-body motion conditioned on the recovered geometry.
• Kinematic feedback: refine pose and camera consistency with closed-loop feedback.

📂 Dataset

EE4D-JSM is built from EgoExo4D and EE4D-Motion. It aligns egocentric RGB, sparse metric scene geometry, Project Aria camera trajectories, and SMPL-X motion for training and evaluation.

📧 Contact

If you have any questions, feel free to contact: kai11.guan@connect.polyu.hk

🤝 Acknowledgments

This project is built upon Pi3 and UniEgoMotion.

📝 Citation

@misc{guan2026reself,
      title={Seeing the World and the Self from Egocentric Video}, 
      author={Kai Guan and Minchao Jiang and Ruichen WangLi and Wentao Zhu and Lei Zhang},
      journal={arXiv preprint arXiv:2609.01276},
      year={2026},
}