Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bugfix: similarity search only works on default_group_slice #3912

Conversation

lanzhenw
Copy link
Contributor

@lanzhenw lanzhenw commented Dec 11, 2023

What changes are proposed in this pull request?

Pass in the active session group slice as a parameter for similarity sort requests to update samples
Fixes a bug: similarity search only works on default_group_slice

similarity.mov

How is this patch tested? If it is not, please explain why.

import fiftyone as fo
import fiftyone.zoo as foz
import fiftyone.brain as fob

ds = foz.load_zoo_dataset('quickstart-groups',max_samples=50)
print(ds.default_group_slice) # prints 'left'
view = ds.select_group_slices('right')
fob.compute_similarity(view,model='clip-vit-base32-torch',brain_key='sim_right')

# now Open Dataset in FOT App. Select 'right' group slice. 
# Select an image in the grid or type in "mountian/road/anything" in similarity search prompt to run an Image Sim Search. 

# this should be working as expected

Release Notes

Is this a user-facing change that should be mentioned in the release notes?

  • No. You can skip the rest of this section.
  • Yes. Give a description of this change to be included in the release
    notes for FiftyOne users.

(Details in 1-2 sentences. You can just refer to another PR with a description
if this PR is part of a larger change.)

What areas of FiftyOne does this PR affect?

  • App: FiftyOne application changes
  • Build: Build and test infrastructure changes
  • Core: Core fiftyone Python library changes
  • Documentation: FiftyOne documentation changes
  • Other

@lanzhenw lanzhenw added the bug Bug fixes label Dec 11, 2023
@lanzhenw lanzhenw self-assigned this Dec 11, 2023
@lanzhenw lanzhenw changed the base branch from develop to release/v0.23.2 December 11, 2023 19:58
Copy link

codecov bot commented Dec 11, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

❗ No coverage uploaded for pull request base (release/v0.23.2@6870d39). Click here to learn what that means.

Additional details and impacted files
@@                Coverage Diff                 @@
##             release/v0.23.2    #3912   +/-   ##
==================================================
  Coverage                   ?   15.86%           
==================================================
  Files                      ?      731           
  Lines                      ?    81757           
  Branches                   ?     1093           
==================================================
  Hits                       ?    12974           
  Misses                     ?    68783           
  Partials                   ?        0           
Flag Coverage Δ
app 15.86% <0.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@lanzhenw lanzhenw force-pushed the TEAMS-2016-similarity-search-fails-in-a-grouped-dataset-only-works-on-default-group-slice branch from 42cf587 to d6b0f93 Compare December 11, 2023 20:16
benjaminpkane
benjaminpkane previously approved these changes Dec 11, 2023
Copy link
Contributor

@benjaminpkane benjaminpkane left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🚀

Copy link
Contributor

@manivoxel51 manivoxel51 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lanzhenw this is great work! 🥇

I see three issues here and I think you have fixed the main one but two pending.

  1. when a current active slice doesn't have a similarity computed, we still show the "Type anything" input. This causes confusion as the user can type "car" for example and the loading spinner will spin forever and a network error occurs without being surfaced to the client. The reason is that current slice is "left" for example but only "right" slice has similarity computed.

I think I know a fix, will push something for it.

  1. If you compute similarity for "right slice". then search for "car". then switch back to left slice, app crashes.

@lanzhenw Can you look into this when you get a chance. end of video shows the crash

Video is app after your fix and my fix (1) above

Screen.Recording.2023-12-12.at.4.56.31.PM.mov

Comment on lines +387 to +404
let methods = get(datasetAtom)?.brainMethods || [];
const isGroupDataset = get(isGroup);
const activeSlice = get(currentSlice(get(isModalActive)));

if (isGroupDataset && activeSlice) {
methods = methods.filter(({ viewStages }) => {
return viewStages.some((vs) => {
const { _cls, kwargs } = JSON.parse(vs);
if (_cls === "fiftyone.core.stages.SelectGroupSlices") {
const sliceValue = kwargs.filter(
(kwarg: string[]) => kwarg[0] === "slices"
)?.[0]?.[1];
if (sliceValue && sliceValue !== activeSlice) return false;
}
return true;
});
});
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@benjaminpkane @lanzhenw added this filters out a method if the activeSlice is not the one one with the similarity computed.

Shows help message for the active slice without similarity computed

Screen.Recording.2023-12-12.at.4.56.31.PM.mov

Copy link
Contributor

@manivoxel51 manivoxel51 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! 🍨

@lanzhenw lanzhenw merged commit 0a840e9 into release/v0.23.2 Dec 18, 2023
12 of 15 checks passed
@lanzhenw lanzhenw deleted the TEAMS-2016-similarity-search-fails-in-a-grouped-dataset-only-works-on-default-group-slice branch December 18, 2023 22:06
@lanzhenw lanzhenw restored the TEAMS-2016-similarity-search-fails-in-a-grouped-dataset-only-works-on-default-group-slice branch December 18, 2023 22:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Bug fixes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants