[feat] Improve and simplify the topological sorting of the test cases #934
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Algorithm's complexity is now O(V+E) - Return additional information that will help execution policies in executing the test cases. The nodes per level are returned as well as the nodes arranged per level that can be safely cleaned up.
vkarak
changed the title
Contributor
Author
|
There is a small bug in my algorithm and I want also to change a bit the logic with the levels. I don't think they're needed. Classical topological sorting will do the job and for the cleaning up of resources a reference counting based on the dependencies would solve the issue. I'll come back to that later. |
- The algorithm is just a DFS visit of the nodes. The trick is to mark a node as visited after all of its outcoming edges have been explored. - The algorithm operates directly on the test dependencies and does not construct the reverse dependency graph.
vkarak
changed the title
vkarak
changed the title
vkarak
changed the title
Passing cyclic graphs is a bug, so we just raise and AssertionError
Codecov Report
@@ Coverage Diff @@
## master #934 +/- ##
==========================================
+ Coverage 91.7% 91.71% +0.01%
==========================================
Files 79 79
Lines 10486 10465 -21
==========================================
- Hits 9616 9598 -18
+ Misses 870 867 -3
Continue to review full report at Codecov.
|
teojgo
approved these changes
Sep 17, 2019
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Algorithm's complexity is now O(V+E).
The algorithm is doing a DFS traversal of the nodes and assigns each node a level (i.e., in which step it has been discovered). Then the nodes are arranged by increasing level and the test cases of each node (test) are expanded inline. Tests that are in the same level can be executed in any order (or in parallel).The algorithm is simply doing a DFS traversal of the test graph (the original one, not the reversed) and marks a node as visited after all of its outcoming edges are explored. The visited nodes are kept in an ordered set, essentially providing at the end the topologically sorted tests.
Additional information that will help execution policies in executing the test cases is now returned. More specifically, a dictionary that arranges the tests per level is returned, as well as a dictionary arranging the nodes by the level that their resources can be safely cleaned up. A test's resources can be safely deleted when the maximum level of its immediate children has been processed.Another fixed needed for correct support of pre Python 3.6 dictionaries was to use and ordered dictionary for storing the test case graph. This was needed because we want to keep the order of partitions and environments, in order to keep the final output properly organised.