New references

[mxochicale]

Open this one to add new references either via issue or creating branch and doing PRs. Cheers, --Miguel

Detection Transformer: Object Detection with DETR - a minimal implementation;

• https://colab.research.google.com/github/facebookresearch/detr/blob/colab/notebooks/detr_demo.ipynb#scrollTo=kqe_0nc5dyAq
• [ICLR 2023] Official implementation of the paper "DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection" https://github.com/IDEA-Research/DINO?tab=readme-ov-file https://arxiv.org/pdf/2203.03605.pdf https://arxiv.org/pdf/2203.03605.pdf https://arxiv.org/abs/2203.03605
• Google search images of "Detection Transformer" https://www.google.co.uk/search?q=Detection+Transformer+&tbm=isch&
• End-to-End Object Detection with Transformers https://arxiv.org/pdf/2005.12872v3.pdf https://github.com/facebookresearch/detr
• https://github.com/mobarakol/tutorial_notebooks/blob/main/Grounded_SAM_Tool_Instance_Segmentation.ipynb

LSTM-related topics

• https://towardsdatascience.com/how-to-use-transformer-networks-to-build-a-forecasting-model-297f9270e630
• Conv- LSTM https://www.google.co.uk/search?sca_esv=de7a49ba27450786&q=Conv-+LSTM&tbm=isch&source=lnms&sa=X&ved=2ahUKEwjpmMGV_IeEAxWhgv0HHT_pDUkQ0pQJegQIDBAB&biw=1920&bih=1100
• Weakly supervised convolutional LSTM approach for tool tracking in laparoscopic videos https://link.springer.com/article/10.1007/s11548-019-01958-6
• multi-object tracking surgical: https://www.google.co.uk/search?sca_esv=6e8c819a630d1523&q=multi-object+tracking+surgical&tbm=isch&source=lnms&sa=X&ved=2ahUKEwiy6_-3odOEAxV8QkEAHc-KC4EQ0pQJegQIDRAB&biw=1920&bih=1100
• Holoscan Sample App Data for AI-based Endoscopy Tool Tracking https://catalog.ngc.nvidia.com/orgs/nvidia/teams/clara-holoscan/resources/holoscan_endoscopy_sample_data

Venues for publication

• For data https://www.nature.com/sdata/
• Cell protocols is a good references for experiment protocols: https://www.cell.com/star-protocols/home https://twitter.com/STARProtocols

[mxochicale]

We would like to browse the table of content of this book

Sensors for Healthcare and Medical Applications
Formica, Domenico Formica, Domenico Schena, Emiliano
Basel, Switzerland MDPI - Multidisciplinary Digital Publishing Institute, 2021

Other relevant papers:

• A Transformer-based Framework for Multivariate Time Series Representation Learning
  • https://arxiv.org/abs/2010.02803
  • https://www.youtube.com/watch?v=K5U9v2SW7_Q
  • https://github.com/gzerveas/mvts_transformer

[mxochicale]

Thanks Steve for sharing, we will review and discuss it.

Heiliger, Christian, Dorian Andrade, Christian Geister, Alexander Winkler, Khaled Ahmed, Alessandra Deodati, Viktor H. Ehrlich V. Treuenstätt et al. "Tracking and evaluating motion skills in laparoscopy with inertial sensors." Surgical Endoscopy (2023): 1-11.

• DOI https://link.springer.com/article/10.1007/s00464-023-09983-y
• CITED: https://scholar.google.com/scholar?cites=14405593716789183856&as_sdt=2005&sciodt=0,5&hl=en
  • Trac, Jessica, Jonguk Lee, Kai-Ho Fok, Brian Carrillo, and Monica Farcas. "Development of a synchronous motion-tracking and video capture tool for flexible ureteroscopy." Canadian Urological Association Journal 18, no. 4 (2024). https://cuaj.ca/index.php/journal/article/view/8530/5952
  • Fanghella, Pietro, Luca Bruzzone, and Matteo Verotti. "A Low-Cost, Moderately Fast System for Online Motion Tracking in Laparoscopic Surgery Training." In IFToMM World Congress on Mechanism and Machine Science, pp. 651-661. Cham: Springer Nature Switzerland, 2023. https://link.springer.com/chapter/10.1007/978-3-031-45705-0_63

Calibration and accuracy evaluation on a phantom
We evaluated the accuracy of the measurement setup on a
3d-printed phantom. The measurement phantom consist-
ing of a mount for the instrument and a flat plate was 3d
printed in Acrylonitrile Butadiene Styrene (ABS), a rigid
plastic. The instrument was restricted in its translational
movement, like the bearing in a trocar. The measurement
phantom consists of a central starting point and six refer-
ence points. The instrument tip is moved from the center to
one reference point in ten repetitions. Each position is being
hold for approximately three seconds, while each position
change takes one second. The procedure is repeated for the
six reference points. The measured orientation was derived
by the average measurement of one position. For each of
these positions, the measured orientation was compared to
the actual value from the geometry of the 3D model (Fig. 1).

[mxochicale]

Lam, Kyle, Junhong Chen, Zeyu Wang, Fahad M. Iqbal, Ara Darzi, Benny Lo, Sanjay Purkayastha, and James M. Kinross. "Machine learning for technical skill assessment in surgery: a systematic review." NPJ digital medicine 5, no. 1 (2022): 24. https://scholar.google.com/scholar?cites=18255856507017471578&as_sdt=2005&sciodt=0,5&hl=en

Lavanchy, Joël L., Joel Zindel, Kadir Kirtac, Isabell Twick, Enes Hosgor, Daniel Candinas, and Guido Beldi. "Automation of surgical skill assessment using a three-stage machine learning algorithm." Scientific reports 11, no. 1 (2021): 5197. https://scholar.google.com/scholar?cites=14668467219995507872&as_sdt=2005&sciodt=0,5&hl=en

Liu, Daochang, Qiyue Li, Tingting Jiang, Yizhou Wang, Rulin Miao, Fei Shan, and Ziyu Li. "Towards unified surgical skill assessment." In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9522-9531. 2021. https://scholar.google.com/scholar?cites=12208140246219122622&as_sdt=2005&sciodt=0,5&hl=en

Fathollahi, Mona, Mohammad Hasan Sarhan, Ramon Pena, Lela DiMonte, Anshu Gupta, Aishani Ataliwala, and Jocelyn Barker. "Video-based surgical skills assessment using long term tool tracking." In International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 541-550. Cham: Springer Nature Switzerland, 2022. https://scholar.google.com/scholar?cites=14295532720293422183&as_sdt=2005&sciodt=0,5&hl=en

Kasa, Kevin, David Burns, Mitchell G. Goldenberg, Omar Selim, Cari Whyne, and Michael Hardisty. "Multi-Modal Deep Learning for Assessing Surgeon Technical Skill." Sensors 22, no. 19 (2022): 7328. https://scholar.google.com/scholar?cites=5754549589610006787&as_sdt=2005&sciodt=0,5&hl=en

[mxochicale]

A8. VIDEO ANALYTICS, AI, MACHINE LEARNING AND DEEP LEARNING
Such granular, trainee level data are currently not
widely available but platforms such as Medtronic’s
Touch Surgery and Johnson & Johnson’s C-SATS
offering personalised video libraries and surgical
analytics already exist. If we can utilise these datasets
and harness the capability of big data104 to predict error,
this could accelerate training and improve patient safety
(Figure 9). Their widespread adoption has not occurred
yet but undoubtedly will occur in the future given that
as more research is conducted in this area, a shift will
occur towards personalised, automated performance
analysis. Collaboration between surgeons, computer
scientists and engineers is crucial to facilitate adoption,
drive AI literacy among clinicians and promote their
active involvement in this field. Only then can the
benefits of AI and ML be translated into better surgical
training and ultimately, improved patient safety
https://futureofsurgery.rcseng.ac.uk/wp-content/uploads/2022/08/FOS_Test_Report_2022.pdf
(page 69.)

[mxochicale]

Few related references that apply Grey level co-occurrence matrix method

• https://haralick-org.torahcode.us/journals/TexturalFeaturesHaralickShanmugamDinstein.pdf
• https://scikit-image.org/docs/dev/auto_examples/features_detection/plot_glcm.html#sphx-glr-auto-examples-features-detection-plot-glcm-py
• http://www.geo-5.com/pdf/papers/2015_FirstBreak_Eichkitz.pdf
• Figure 3: https://www.researchgate.net/publication/277085093_Haralick_Texture_Analysis_of_prostate_MRI_Utility_for_differentiating_non-cancerous_prostate_from_prostate_cancer_and_differentiating_prostate_cancers_with_different_Gleason_Scores
• Figure 1: https://www.researchgate.net/publication/337940559_Steel_Surface_Defect_Diagnostics_Using_Deep_Convolutional_Neural_Network_and_Class_Activation_Map
• Figure 6: https://www.researchgate.net/publication/350466003_Understanding_Sources_of_Variation_to_Improve_the_Reproducibility_of_Radiomics

[mxochicale]

Fathabadi, Fatemeh Rashidi, Janos L. Grantner, Saad A. Shebrain, and Ikhlas Abdel-Qader. "Autonomous sequential surgical skills assessment for the peg transfer task in a laparoscopic box-trainer system with three cameras." Robotica 41, no. 6 (2023): 1837-1855.
https://www.cambridge.org/core/journals/robotica/article/autonomous-sequential-surgical-skills-assessment-for-the-peg-transfer-task-in-a-laparoscopic-boxtrainer-system-with-three-cameras/DCB01829946A6E22AD3FE68BE5057746
dataset: https://drive.google.com/drive/folders/1F97CvN3GnLj-rqg1tk2rHu8x0J740DpC
github: https://github.com/Atena-Rashidi

[mxochicale]

Lavanchy, J. L., J. Zindel, K. Kirtac, I. Twick, E. Hosgor, D. Candinas, and G. Beldi. "Automation of surgical skill assessment using a three-stage machine learning algorithm. Sci Rep 11: 5197." (2021).
https://scholar.google.com/scholar?cites=6644914990870187003

[mxochicale]

Vasu, Pavan Kumar Anasosalu, James Gabriel, Jeff Zhu, Oncel Tuzel, and Anurag Ranjan. "MobileOne: An Improved One Millisecond Mobile Backbone." In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7907-7917. 2023. https://arxiv.org/abs/2206.04040
https://scholar.google.com/scholar?cites=17349182165322877132

[mxochicale]

Chadebecq, François, Laurence B. Lovat, and Danail Stoyanov. "Artificial intelligence and automation in endoscopy and surgery." Nature Reviews Gastroenterology & Hepatology 20, no. 3 (2023): 171-182. https://scholar.google.com/scholar?cites=1544539509892715943&as_sdt=2005&sciodt=0,5&hl=en

[mxochicale]

"A multi-purpose egocentric dataset created using Project Aria" > https://www.projectaria.com/datasets/apd/

Sensor Data
1 x 110 degree FOV Rolling Shutter RGB camera
2 x 150 degree FOV Global Shutter mono cameras for SLAM and hand tracking
2 x 80 degree FOV Global Shutter mono cameras for eye-tracking with IR illumination
2 x 1KHz IMU + barometer & magnetometer environmental sensors
7 x 48 KHz spatial microphones
GPS 1 Hz

Lv, Zhaoyang, Nickolas Charron, Pierre Moulon, Alexander Gamino, Cheng Peng, Chris Sweeney, Edward Miller et al. "Aria Everyday Activities Dataset." arXiv preprint arXiv:2402.13349 (2024). https://arxiv.org/abs/2402.13349 https://scholar.google.com/scholar?cluster=4737684883844122907&hl=en&as_sdt=0,5

USING RERUN.IO viewer_aria_sensors displays the relative position and orientation of all most of Project Aria glasses sensors (cameras, IMUs, microphones, magnetometer & barometer) in a common reference. https://facebookresearch.github.io/projectaria_tools/docs/data_utilities/visualization/visualization_python#run-aria-sensor-viewer

DOCS: https://facebookresearch.github.io/projectaria_tools/docs/open_datasets/aria_everyday_activities_dataset

[mxochicale]

Smedsrud, Pia H., Vajira Thambawita, Steven A. Hicks, Henrik Gjestang, Oda Olsen Nedrejord, Espen Næss, Hanna Borgli et al. "Kvasir-Capsule, a video capsule endoscopy dataset." Scientific Data 8, no. 1 (2021): 142.

• GITHUB https://github.com/simula/kvasir-capsule?tab=readme-ov-file
• DATASET: https://osf.io/dv2ag/
• CITES: https://scholar.google.com/scholar?cites=4327460710533833743&as_sdt=2005&sciodt=0,5&hl=en
  • Oukdach, Yassine, Zakaria Kerkaou, Mohamed El Ansari, Lahcen Koutti, Ahmed Fouad El Ouafdi, and Thomas De Lange. "ViTCA-Net: a framework for disease detection in video capsule endoscopy images using a vision transformer and convolutional neural network with a specific attention mechanism." Multimedia Tools and Applications (2024): 1-20. https://scholar.google.com/scholar?cites=4860481277279608581&as_sdt=2005&sciodt=0,5&hl=en

[mxochicale]

Iakovidis, Dimitris K., and Anastasios Koulaouzidis. "Software for enhanced video capsule endoscopy: challenges for essential progress." Nature Reviews Gastroenterology & Hepatology 12, no. 3 (2015): 172-186.

• CITES: https://scholar.google.com/scholar?cites=7423581077290491439&as_sdt=2005&sciodt=0,5&hl=en
  • Wang, Guankun, Long Bai, Yanan Wu, Tong Chen, and Hongliang Ren. "Rethinking exemplars for continual semantic segmentation in endoscopy scenes: Entropy-based mini-batch pseudo-replay." Computers in Biology and Medicine 165 (2023): 107412.https://scholar.google.com/scholar?cites=16195845445789654443&as_sdt=2005&sciodt=0,5&hl=en