Skip to content
Automatic alignment of surgical videos using kinematic data
Branch: master
Clone or download
Latest commit 4c445f9 Jun 27, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.idea adding full req Mar 19, 2019
img adding img Mar 19, 2019
src - clean some comments Mar 25, 2019
.gitignore add Mar 19, 2019
LICENSE remove kmeans Mar 19, 2019
README.md Update README.md Jun 27, 2019
requirements.txt adding full req Mar 19, 2019

README.md

Automatic alignment of surgical videos using kinematic data

This is the companion repository for our paper also available on ArXiv, titled "Automatic alignment of surgical videos using kinematic data". This paper has been accepted at the Conference on Artificial Intelligence in Medicine (AIME) 2019.

Approach

The following is an example on how a time series alignment is used to synchronize the videos by duplicating the gray-scale frames.

Video without alignment Video with alignment
unsynched synched

The following is an example of aligning coordinate X’s time series for subject F, when performing three trials of the suturing surgical task.

Time series without alignment Time series with alignment
unsynched synched

Data

You will need the JIGSAWS: The JHU-ISI Gesture and Skill Assessment Working Set to re-run the experiments of the paper.

Suturing Knot-Tying Needle-Passing
st kt np

Prerequisites

To run the code you will also need to download seperatly and install the following dependencies (the full list can be found here):

Code

We have three surgical tasks in JIGSAWS.

Suturing
Knot_Tying
Needle_Passing

Before running the code, you might need to generate the Cython files using the following command:

cd src
./build-cython.sh

To align multiple videos for the Suturing task using the NLTS algorithm, you can run:

python3 main.py Suturing align_videos

To aligne only two videos for the Suturing task using the classic DTW algorithm, you can run:

python3 main.py Suturing align_2_videos

Reference

If you re-use this work, please cite:

@InProceedings{IsmailFawaz2019automatic,
  Title                    = {Automatic alignment of surgical videos using kinematic data},
  Author                   = {Ismail Fawaz, Hassan and Forestier, Germain and Weber, Jonathan and Petitjean, François and Idoumghar, Lhassane and Muller, Pierre-Alain},
  booktitle                = {Artificial Intelligence in Medicine},
  Year                     = {2019}
}
You can’t perform that action at this time.