Skip to content

Fix image map loading logic on desktop#1190

Merged
subdavis merged 8 commits into
mainfrom
bugfix/desktop-CSV-image-mapping
Mar 2, 2022
Merged

Fix image map loading logic on desktop#1190
subdavis merged 8 commits into
mainfrom
bugfix/desktop-CSV-image-mapping

Conversation

@subdavis
Copy link
Copy Markdown
Contributor

@subdavis subdavis commented Mar 1, 2022

Approach is explained in a large comment block in viame.ts

fixes #1167

@BryonLewis
Copy link
Copy Markdown
Collaborator

BryonLewis commented Mar 1, 2022

This PR allows all non-empty invalid strings instead of strictly how it was before with image paths vs blank?
This would allow loading wrongly formatted image paths if they were all incorrect?
I.E:

  • real images: ./folder1/image1.png, ./folder1/image2.png
  • but in column 2 it was: ./folder2/image1.png, ./folder2/image2.png where folder2 doesn't exist.

So your annotations could still be out of order if you mess up the relative/full paths for the images. I'm not proposing that we fix this corner case, just trying to make sure I understand what is going on.

@subdavis
Copy link
Copy Markdown
Contributor Author

subdavis commented Mar 1, 2022

This would allow loading wrongly formatted image paths if they were all incorrect?

Yeah.... I don't really know how to juggle all the conflicting needs here, and this seemed like the best option.

As I recall, the main motivation behind reordering is to un-mangle out of order annotations when someone provides the images, and not really to defend against bad data entering the system. It's more of a "you met a special case that lets us help you" feature.

Because we need to accept so many invalid types of data (input_labels.txt, *whitespace*, basically any invalid string) I think it would be hurting instead of helping to attempt to be any smarter than this. For example, who knows what we'd break if we tried to detect path-like strings.

@BryonLewis
Copy link
Copy Markdown
Collaborator

I'm not suggesting we error or prevent having timestamps in that field. I just wanted to understand the possible failure conditions of this change.

Copy link
Copy Markdown
Collaborator

@BryonLewis BryonLewis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Handle the case of multi-frame tracks that have all missing images. This would fail for multiple detections per frame, or multi-frame tracks with the tracks.length being equal to the number of missing images. I think you really want a frame counter and determine that the number of frames in the CSV is equal to the missingImages.length.

Comment thread server/dive_utils/serializers/viame.py Outdated

trackarr = tracks.items()

if imageMap and len(missingImages) and len(missingImages) != len(trackarr):
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isn't this assuming you are only dealing with detections and not tracks?
add in a test probably for tracks that have all missingImages.

const tracks = Array.from(dataMap.values());

if (imageMap !== undefined) {
if (missingImages.length > 0 && missingImages.length !== tracks.length) {
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as other comment. There is no relationship between the number of tracks and the missingImages, unless there is a single detection per frame. This would fail for image-sequences composed of multiframe tracks that don't have filenames. Like on the python version I'd suggest we modify the tests to account for that as well.

@subdavis subdavis self-assigned this Mar 1, 2022
@subdavis
Copy link
Copy Markdown
Contributor Author

subdavis commented Mar 1, 2022

Warning: I had a locally installed version of Nodejs with a buggy implementation of Stream.pipeline that did error handling wrong. If you have this version, tests will fail. If you use n, sudo n lts will install a working version of 16X, or you can install the newest 14X or 17X.

@subdavis subdavis requested a review from BryonLewis March 1, 2022 20:10
BryonLewis
BryonLewis previously approved these changes Mar 1, 2022
Copy link
Copy Markdown
Collaborator

@BryonLewis BryonLewis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did some testing and looks good, update the documentation and should be good.

Comment on lines +267 to +273
* If missing image count was different than track length, then some number of images
* from column 2 were actually valid and some were not. This indicates that the dataset
* being loaded is probably corrupt.
*
* If their counts match perfectly, then every single image was missing, which indicates
* that the dataset either had all empty values in column 2 or some other type of invalid
* string that should not prevent import.
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update this documentation and it should be good to go.

@BryonLewis BryonLewis self-requested a review March 1, 2022 21:52
@subdavis subdavis merged commit aaf9400 into main Mar 2, 2022
@subdavis subdavis deleted the bugfix/desktop-CSV-image-mapping branch March 2, 2022 15:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

A couple of tracking pipelines are failing on videos

2 participants