Discontinuous frames exported from .slp dataset

[STAR-811]

Hi! I would like to compare the performance between SLEAP and my own algorithm. According to the code you offer in #779 (reply in thread) , the labeled images are exported from the .slp file in https://sleap.ai/datasets.html. However, the exported neighboring frames weren't taken at an adjacent time point. So, I can't finish pose tracking through this discontinuous video. The example images are shown below. Looking forward for your reply~
T=1:

T=2:

[roomrys]

Hi @STAR-811,

Since the labels package (.pkg.slp) is primarily exported for training purposes, SLEAP does not attempt to order the labeled frames. Labeled frames are added to the project based on the order in which they were labeled and are exported in that order. Note that even after ordering based on video/frame index, the labeled frames may still not be adjacent as we only export the frames that were labeled in .pkg.slp files to help condense the file.

If you would like the labeled frames sorted, you can try the following code on the dataset:

1. Activate the sleap conda environment
2. Open ipython in terminal
3. Run the following lines in ipython:

import sleap
labels = sleap.Labels.load_file("path_to_labels/slp_project.slp")
labeled_frames = labels.labeled_frames
labeled_frames.sort(key=lambda lf: lf.frame_idx)

Note that the above will only sort based on the frame index, not the video. If the project contains multiple videos, then you would want to sort by video first and then by labeled frame as seen below:

import sleap
labels = sleap.Labels.load_file("path_to_labels/slp_project.slp")
labeled_frames = []
for video in labels.videos:
     lfs_to_extend = labels.find(video)
     lfs_to_extend.sort(key=lambda lf: lf.frame_idx)
     lfs_to_extend.extend(lfs_to_extend)

Let us know if you need any help!

Thanks,
Liezl

[talmo]

If you want to get the adjacent (unlabeled) images to a particular labeled frame, you can do something like:

import sleap
labels = sleap.load_file("path_to_labs/slp_project.slp")

window = np.arange(-5, 5)  # window around each labeled frame
lf_ind = 0  # up to len(labels)

labeled_frame = labels[lf_ind]
imgs = labeled_frame.video[window + labeled_frame.frame_idx]
# imgs.shape == (len(window), height, width, channels)

For the datasets that are posted on that page, we do not have the source videos publicly available as most of them are still being actively used for other studies, but you're welcome to contact the authors associated with each dataset individually for permission to get the raw data.