Do not merge 2: Everything Else that is needed for the evaluation paper. #419
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
-The degree of parallelism is configurable at compiletime Since it is task-based it also performs well under heavy load. -Updated Dockerfile to use Ubuntu 20.04. This is required for sufficient OpenMP support.
* Since C++17, the STL supports parallel Algorithms like std::sort. * This commit uses this sort as the new default. * For "dynamic" IdTables (IdTable<0>) we still use the GNU parallel extensions, because the IDTables do not support the STL sorting currently (TODO: figure out why, and remove the reason).
This allows more efficient sorting also for intermediate results with a larger number of columns.
… the input into chunks. TODO: test, and tweak the performance.
…a std::vector (or similar data structure) which can cheaply create new Elements (like ints) without initializing (zeroing) them. This makes e.g. resizing the Vector to a bigger size much cheaper.
This commit allows the export of query results directly in the ID space in a dense binary format. This is very useful for debugging.
(This is actually a separate PR which should be merged before this).
* Integrated the Zstandard compression library. * Compress the PSO and POS relations in Blocks of 8 MBs. * When performing an Index Scan, there is much less access to disk and the decompression of the Blocks is performed in parallel.
This allows using/building a docker container without recompilation in case of a test failure. Also comment out the SortPerformanceEstimatorTest because it has spurious failures which are nasty.
# Conflicts: # Dockerfile
joka921
force-pushed
the
f.evaluationPaper2
branch
from
35b36f0
to
f0b42b9
Compare
TODO: Apply the same fix for the other overload of transformToFirstPossibleBiggerValue
…into f.evaluationPaper2
…Lever needs for building AND running to check if we can make incremental building work like this.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently includes the parallel sort via STL #418 , the parallel Join #417 and the parallel pattern Trick #342