Exact duplicates always calculated

[smttsp]

cleanvision/src/cleanvision/issue_managers/duplicate_issue_manager.py

Line 97 in cd8f98b

if SETS not in imagelab_info.get(IssueType.EXACT_DUPLICATES.value, {}):

I was trying to get near duplicates only but the code calculates exact duplicates too. I am not sure if it is required but the following lines seem to have a problem.

SETS = "sets"
...
        if SETS not in imagelab_info.get(IssueType.EXACT_DUPLICATES.value, {}):
            to_compute.append(IssueType.EXACT_DUPLICATES.value)

So, the code always adds the exact_duplicates to to_compute list whenever this function is called. Is this the goal for the function?

[sanjanag]

Hi @smttsp! The near duplicates issue only returns near duplicate sets excluding the exact duplicate sets. This line ensures exact duplicate sets are always present when computing near duplicate sets to avoid double counting.

[smttsp]

@sanjanag,

_get_issue_types_to_compute is called from inside duplicate_issue_manager.DuplicateIssueManager. The only thing it checks is exact duplicates and near duplicates. It doesn't care about any other issue.

Essentially, the code should check if IssueType.EXACT_DUPLICATES.value not in imagelab_info, then you will IssueType.EXACT_DUPLICATES.value to the to_compute, right?

If what I am saying is correct, then this doesn't fully make sense to me:

        if "sets" not in imagelab_info.get(IssueType.EXACT_DUPLICATES.value, {}):
            to_compute.append(IssueType.EXACT_DUPLICATES.value)

please correct me if I am wrong.

It may also cause calculating the exact_duplicates twice. I don't know.

[sanjanag]

@smttsp Sorry, I didn't get your suggested change.

It's not computing twice, if you check the update_info method here, you can see SETS key is only added to this dictionary when you have the exact duplicate issues computed.

[smttsp]

I see, okay. It seems that even if the exact_duplicates return no duplicates, that dict will end up having SETS key. So you won't have double calculation, right? If so, we can close this issue.

[sanjanag]

That's correct.

[smttsp]

sure, then we can close the issue. thanks for clarification!