Our real-time 3D rendering system is targeted for laparoscopic surgery. Currently, our work is in the early stages, focusing on a laparoscopic surgery box trainer environment. The development process follows a coarse-to-fine approach: first, we establish the framework, then we fill in and refine the details.
A demo of our work. We demonstrate real-time 3D reconstruction and rendering for a laparoscopic surgery box trainer, specifically focusing on the bean drop task. The shown version is targeted for an experiment that will be conducted in Spring 2025. We plan to sample data from the operations performed by medical students at UM-Madison. The process achieves a real-time performance of 12.6 ms per frame, from reading 5 cameras to completing 3D rendering.
Demo of real-time 3D visualization when operating the LS tool.
Demo of 3D visualization over meat. The distance between the surgical tool and the tissue can be directly observed through EasyVis, although this distance is invisible from the camera view. (a) captured camera views. (b) rendered novel view.
The workflow of our work.
The dataset is in YOLOv5 format.
- Yung-Hong Sun, J. Chen, T. Lin, H. Jiang, Y.H. Hu, Efficient Undistinguishable Dense 3D Objects Association Based on Multi-Camera System (In preparation).
- Yung-Hong Sun, G. Shen, J. Chen, J. Fernandes, H. Jiang, and Y.H. Hu. EasyVis2: A Real Time Multi-view 3D Visualization System for Laparoscopic Surgery Training Enhanced by a Deep Neural Network YOLOv8-Pose. arXiv preprint arXiv:2412.16742 (2024).
- Yung-Hong Sun, J. Ke, J. Fernandes, J. Chen, H. Jiang, Y.H. Hu, EasyVis: An Interactive 3D Visualization Software System for Laparoscopic Box Trainer. Updates in Surgery, 2025. DOI: 10.1007/s13304-025-02153-w.
- Yung-Hong Sun, J. Ke, J. Fernandes, J. Chen, H. Jiang, Y.H. Hu, EasyVis: A Real-time 3D Rendering System for Laparoscopic Surgery Training Box. Oral presentation, BMES 2024 Annual Meeting. Baltimore, MD, United States, Oct 2024.
This work was supported by the National Institute of Biomedical Imaging and Bioengineering (NIBIB) of the U.S. National Institutes of Health (NIH) under award number R01EB019460.
- Yung-Hong Sun
- Gefei Shen
- Jiangang Chen
- Jayer Fernandes
- Jianwei Ke
- Hongrui Jiang
- Yu Hen Hu