Reduce find_indirect_clusters() complexity from O(exp(n)) to O(n) #2071
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dosubot
bot
added
the
size:M
|
Ran into the exact same issue and indeed made a similar fix, would be nice to get it merged since this limits the current code only to very small graphs :) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I was trying to generate a testset from
KnowledgeGraph(nodes: 1861, relationships: 912103)and it would go on for two days in a row without any progress. So I did some profiling to see what's going on on a smaller KG and found that most of the time was being spent infind_indirect_clusters().Here is an optimised implementation that seem to produce similar results but way faster. On my reduced size KG I'm down from 5 min to a fraction of a second and on the full KG I'm down from never finished to a few seconds.