[Feature Request] UI debug samples tab - improving the filtering \ navigation

[Alexfinkelshtein]

Hi,

[Use Case]
For some experiment types, image visualization is required.
In the case that you use multiple image summaries types, and during your training a large number of images is accumulated navigating and comparing the different images becomes hard

[Pain Points]

• The default behavior of the auto refresh makes no sense when viewing the debug samples tab, when you try to scroll between the images the refresh is fighting you and it super inconvenient.

• Navigating between images is hard. For example, I want to choose a specific epoch to view, then if it was 100 epochs before the last one, I need to click more then 30 times to find it (and if the auto refresh kicks in its even worse).

• Comparison of two experiments in the debug samples tab is sub-optimal, you can't view a full split screen, so you can only view the small version side by side, which is in itself not convenient in the case that you have a lot of image types because you can't filter them.

[Suggestions]

• disable the default auto-refresh behavior on the debug samples tab
• Create a search bar that lets you both filter on epoch and summary name
• Introduce a nicer way to compare two images from to experiments, and maybe also in the same experiment between epochs

(side note)(shouldn't the debug samples tab be called something else? its not only for debug it gives meaningful insight to the state\quality of some model)
thanks!

[ainoam]

@Alexfinkelshtein

disable the default auto-refresh behavior on the debug samples tab

While this would address the use case in question, there would also be other use cases where you would be interested in this functionality (i.e. keeping the browser on the debug samples tab of a running experiment) - So it would make sense to define the behaviour for your use case using the auto-refresh UI on/off switch. No?

Create a search bar that lets you both filter on epoch and summary name

Finer granularity filters for debug samples sound like a great improvement.

Introduce a nicer way to compare two images from to experiments, and maybe also in the same experiment between epochs

I can definitely relate to your desire for better image comparison - Can elaborate on the limitations you encounter for multiple image types?
What did you have in mind for better comparison? Full screen side-by-side? Something else?

[Alexfinkelshtein]

@ainoam

While this would address the use case in question, there would also be other use cases where you would be interested in this functionality (i.e. keeping the browser on the debug samples tab of a running experiment) - So it would make sense to define the behaviour for your use case using the auto-refresh UI on/off switch. No?
I don't agree.
While I am not familiar with all use cases, the only case that I can think of where it would make sense to have the auto refresh in the debug samples, is when all the debug images you have fit in a "single screen" and you want to only glimpse at the latest preview available.
the thing is, that the auto refresh in the debug samples basically breaks the usability of almost any feature for scrolling or viewing the debug samples. That's why I think that the special case here is actually to want the auto refresh while using this tab.
Or alternatively, which actually makes more sense to me, is changing the auto refresh behavior in the debug samples tab.. instead of breaking the viewing, scrolling experience, enabling the access to the new available samples..

I can definitely relate to your desire for better image comparison - Can elaborate on the limitations you encounter for multiple image types? What did you have in mind for better comparison? Full screen side-by-side? Something else?

about limitations it starts from basic stuff like just having the option to see it in full screen, and compare arbitrary sample types from arbitrary epochs

a bit of a side note,
the display of dataframes in the plots tab is very inconvenient.. although the task class report table enables the epoch argument, it seem to be completely ignored..
then instead of having an option to scroll between versions of some table type across epochs it is just displayed one after the other without the epoch information which makes it very hard to use..

[ainoam]

@Alexfinkelshtein

I don't agree.
While I am not familiar with all use cases, the only case that I can think of where it would make sense to have the auto refresh in the debug samples, is when all the debug images you have fit in a "single screen" and you want to only glimpse at the latest preview available.
the thing is, that the auto refresh in the debug samples basically breaks the usability of almost any feature for scrolling or viewing the debug samples. That's why I think that the special case here is actually to want the auto refresh while using this tab.
Or alternatively, which actually makes more sense to me, is changing the auto refresh behavior in the debug samples tab.. instead of breaking the viewing, scrolling experience, enabling the access to the new available samples..

I see what you mean, and agree completely. Automatic refresh should be limited to "default view". It should not affect user view when explicitly stated (e.g. user changed screen scroll position).

about limitations it starts from basic stuff like just having the option to see it in full screen, and compare arbitrary sample types from arbitrary epochs

Thanks for clarifying. I'll make sure it is on our todo list.

the display of dataframes in the plots tab is very inconvenient.. although the task class report table enables the epoch argument, it seem to be completely ignored..
then instead of having an option to scroll between versions of some table type across epochs it is just displayed one after the other without the epoch information which makes it very hard to use..

Thanks for highlighting this bug - A fix should be coming in an imminent release.

[jkhenning]

Closing this as this was already released. Please reopen if required.