Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix handling of many graph nodes at the same location #111

Merged
merged 1 commit into from
Aug 6, 2024
Merged

Conversation

cevian
Copy link
Collaborator

@cevian cevian commented Jul 22, 2024

When nodes have the same location, their distance to each other is 0. And, their distance to all their neighbors is the same. In the case with few maximum neighbors this can cause problems because the nodes end up linking to each other and getting cut off from the rest of the graph.

We fix this by introducing a secondary "tie break" distance between nodes at the same location (based on on-disk distance). That's used to create a secondary ordering for equivalent nodes and allows the prune function to then properly cut away equivalent nodes so that your neighbor list doesn't get full of them.

Note with SBQ, equivalence classes are actually not /so/ rare especially with low dimension counts.

@cevian cevian requested a review from a team as a code owner July 22, 2024 20:44
@cevian
Copy link
Collaborator Author

cevian commented Jul 22, 2024

This should Fix #110

Copy link

@adolsalamanca adolsalamanca left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good, but you might want someone with deeper rust expertise to also make a review.

thanks!

adolsalamanca
adolsalamanca previously approved these changes Jul 23, 2024
thedodd
thedodd previously approved these changes Jul 23, 2024
Copy link
Member

@thedodd thedodd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few nits, take 'em or leave 'em.

When nodes have the same location, their distance to each other is 0.
And, their distance to all their neighbors is the same. In the case
with few maximum neighbors this can cause problems because the nodes
end up linking to each other and getting cut off from the rest of
the graph.

We fix this by introducing a secondary "tie break" distance between
nodes at the same location (based on on-disk distance). That's
used to create a secondary ordering for equivalent nodes and allows
the prune function to then properly cut away equivalent nodes so
that your neighbor list doesn't get full of them.

Note with SBQ, equivalence classes are actually not /so/ rare especially
with low dimension counts.
Copy link
Contributor

@alejandrodnm alejandrodnm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A minor thing, feel free to ignore it.

if distance_between_candidate_and_existing_neighbor < 0.0 + f32::EPSILON {
if distance_between_candidate_and_point < 0.0 + f32::EPSILON {
1.0
let factor = if distance_between_candidate_and_existing_neighbor
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT: the prune_neighborgs functions is fairly big. Maybe you can abstract the factor calculation away to its own function. That way you can add docs specific to the factor calculation. Also, you can use guard clauses to remove some of the ifs.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm going to ignore this for now. I think we may refactor this later anyway.

@mgrosso
Copy link

mgrosso commented Aug 3, 2024

@cevian I'm delighted to confirm this fixes issue #110 for me. thanks!

@cevian cevian merged commit e698c59 into main Aug 6, 2024
3 checks passed
@cevian cevian deleted the fix_eq branch August 6, 2024 17:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants