Tweak in SortPerformanceEstimator (faster + log) #398
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
1. Right now, the estimation takes several minutes. Is that really
necessary? The main culprits are the 100,000,000 tests, so I removed
them. OK?
2. The log had two lines per sort ("Sorting ..." and "done"), it's now
only one line.
TODO: It is confusing that the sort estimation is the very first thing one
sees in the server log upon startup. The server initialization comes
afterwards. Is realize that there are reasons for this in the code.
Nevertheless, is it possible to do this the other way round?
|
@joka921 Did you have a chance to look at this, Johannes? It's a rather tiny PR. BTW, it would be good to adapt the array sizes to the input size. For small input collections, the start-up time is virtually zero without the sort performance estimation. Alternatively, one could make the sort performance estimation optional via a command line argument. |
This allows us, to limit the maximum sample size depending on the Knowledgebase size.
hannahbast
commented
May 28, 2021
| /// Set up the sort estimates. This will take some time. Only samples, that | ||
| /// can be allocated by the allocator and that have less thatn | ||
| /// `maxNumberOfElementsToSort` elements will actually be measured. | ||
| void createEstimatesExpensively( |
There was a problem hiding this comment.
How about computeEstimatesExpensively ?
src/global/Constants.h
Outdated
| @@ -114,6 +114,10 @@ static constexpr size_t NUM_OPERATIONS_HASHSET_LOOKUP = 32; | |||
| // than the remaining time, then the sort is canceled with a timeout exception | |||
| static constexpr double SORT_ESTIMATE_CANCELLATION_FACTOR = 3.0; | |||
|
|
|||
| // When initializing a sort performance estimator, at most this percentage of | |||
| // the index size is being sorted at once. | |||
There was a problem hiding this comment.
of the index size -> of the number of triples in the index
|
Since I started this PR, I can't approve it, please do @joka921 |
joka921
approved these changes
May 29, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Right now, the estimation takes several minutes. Is that really
necessary? The main culprits are the 100,000,000 tests, so I removed
them. OK?
The log had two lines per sort ("Sorting ..." and "done"), it's now
only one line.
TODO: It is confusing that the sort estimation is the very first thing one
sees in the server log upon startup. The server initialization comes
afterwards. Is realize that there are reasons for this in the code.
Nevertheless, is it possible to do this the other way round?