Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

reduce propagate complexity from O(n2) to O(n) #3528

Merged
merged 2 commits into from Sep 12, 2018

Conversation

Projects
None yet
3 participants
@memsharded
Copy link
Contributor

commented Sep 11, 2018

While working on the CI reproducibility, graph locks, etc, I found very slow graph propagation. This PR will highly alleviate it.

Measured, for a linear graph (A->B->C-> .... ->) of 250 nodes, the propagation (computed by ConanInstaller), reduces from 30 to 3 seconds.

Changelog: Feature: Improved graph propagation performance from O(n2) to O(n).

@ghost ghost assigned memsharded Sep 11, 2018

@ghost ghost added the stage: review label Sep 11, 2018

@memsharded memsharded requested a review from jgsogo Sep 11, 2018

@memsharded memsharded added this to the 1.8 milestone Sep 11, 2018

@jgsogo

This comment has been minimized.

Copy link
Member

commented Sep 11, 2018

Do you have a script to generate that chain of dependencies (or even a complex graph)? I would want to try it and understand the underlying code a little bit more, not just review the dict comprehensions thing. Thanks!

@memsharded

This comment has been minimized.

Copy link
Contributor Author

commented Sep 11, 2018

I am using https://github.com/conan-io/test/tree/master/conan_tests/performance to create a graph, then test manually over the created graph.

@jgsogo

jgsogo approved these changes Sep 11, 2018

Copy link
Member

left a comment

It looks much better than before, and it should be faster without those nested loops. Go ahead!

@lasote lasote merged commit 8157e0e into conan-io:develop Sep 12, 2018

2 checks passed

continuous-integration/jenkins/pr-head This commit looks good
Details
license/cla Contributor License Agreement is signed.
Details

@ghost ghost removed the stage: review label Sep 12, 2018

@memsharded memsharded deleted the memsharded:feature/optimize_propagate branch Sep 20, 2018

grisumbras pushed a commit to grisumbras/conan that referenced this pull request Dec 27, 2018

reduce propagate complexity from O(n2) to O(n) (conan-io#3528)
* reduce propagate complexity from O(n2) to O(n)

* fixed sorted py2to3 issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.