-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
perf(sql): support uuid and long256 in parallel GROUP BY #4140
perf(sql): support uuid and long256 in parallel GROUP BY #4140
Commits on Jan 16, 2024
-
Addition of GroupByLong128HashSet
Currently microbenching at (YMMV): Benchmark Mode Cnt Score Error Units GroupByLong128HashSetBenchmark.baseline avgt 3 0.009 ± 0.002 us/op GroupByLong128HashSetBenchmark.testFastMap avgt 3 0.197 ± 0.056 us/op GroupByLong128HashSetBenchmark.testGroupByLong128HashSet avgt 3 0.146 ± 0.009 us/op
Configuration menu - View commit details
-
Copy full SHA for d177365 - Browse repository at this point
Copy the full SHA d177365View commit details -
Addition of CountDistinctUuidGroupByFunction
Includes port of tests from similar CountDistinctLongGroupByFunctionFactory.
Configuration menu - View commit details
-
Copy full SHA for dd641a5 - Browse repository at this point
Copy the full SHA dd641a5View commit details -
Addition of GroupByLong256HashSet
Currently benches at: Benchmark Mode Cnt Score Error Units GroupByLong256HashSetBenchmark.baseline avgt 3 0.015 ± 0.001 us/op GroupByLong256HashSetBenchmark.testFastMap avgt 3 0.214 ± 0.023 us/op GroupByLong256HashSetBenchmark.testGroupByLong256HashSet avgt 3 0.251 ± 0.039 us/op
Configuration menu - View commit details
-
Copy full SHA for 3a956d8 - Browse repository at this point
Copy the full SHA 3a956d8View commit details
Commits on Jan 17, 2024
-
Addition of CountDistinctLong256GroupByFunction
Currently debugging issues with the tests. Added to_long256 to help to align the tests across the long/long128/long256 count distinct groupby functions. Inner expression aren't parsing as expected.
Configuration menu - View commit details
-
Copy full SHA for ed4f9dd - Browse repository at this point
Copy the full SHA ed4f9ddView commit details -
Addition of QuaternaryFunction initialisation fixes for LongsToLong25…
…6FunctionFactory. Tests now pass.
Configuration menu - View commit details
-
Copy full SHA for 640a329 - Browse repository at this point
Copy the full SHA 640a329View commit details -
Updates to GroupByLong128HashSetBenchmark and GroupByLong256HashSetBe…
…nchmark. Benchmark (size) Mode Cnt Score Error Units GroupByLong128HashSetBenchmark.testGroupByLong128HashSet 5000 avgt 3 133.559 ± 24.516 ns/op GroupByLong128HashSetBenchmark.testGroupByLong128HashSet 50000 avgt 3 184.387 ± 11.863 ns/op GroupByLong128HashSetBenchmark.testGroupByLong128HashSet 500000 avgt 3 180.291 ± 38.545 ns/op GroupByLong128HashSetBenchmark.testGroupByLong128HashSet 5000000 avgt 3 180.389 ± 21.280 ns/op GroupByLong128HashSetBenchmark.testOrderedMap 5000 avgt 3 244.329 ± 29.777 ns/op GroupByLong128HashSetBenchmark.testOrderedMap 50000 avgt 3 194.403 ± 6.899 ns/op GroupByLong128HashSetBenchmark.testOrderedMap 500000 avgt 3 191.533 ± 6.748 ns/op GroupByLong128HashSetBenchmark.testOrderedMap 5000000 avgt 3 191.723 ± 2.794 ns/op Benchmark (size) Mode Cnt Score Error Units GroupByLong256HashSetBenchmark.testGroupByLong256HashSet 5000 avgt 3 310.210 ± 135.663 ns/op GroupByLong256HashSetBenchmark.testGroupByLong256HashSet 50000 avgt 3 256.012 ± 32.347 ns/op GroupByLong256HashSetBenchmark.testGroupByLong256HashSet 500000 avgt 3 258.856 ± 56.371 ns/op GroupByLong256HashSetBenchmark.testGroupByLong256HashSet 5000000 avgt 3 275.527 ± 326.510 ns/op GroupByLong256HashSetBenchmark.testOrderedMap 5000 avgt 3 205.034 ± 38.691 ns/op GroupByLong256HashSetBenchmark.testOrderedMap 50000 avgt 3 205.936 ± 94.791 ns/op GroupByLong256HashSetBenchmark.testOrderedMap 500000 avgt 3 199.510 ± 38.080 ns/op GroupByLong256HashSetBenchmark.testOrderedMap 5000000 avgt 3 215.477 ± 159.120 ns/op
Configuration menu - View commit details
-
Copy full SHA for f8f13ca - Browse repository at this point
Copy the full SHA f8f13caView commit details -
Align GroupByLongHashSetBenchmark with Long128 and Long256.
Benchmark (size) Mode Cnt Score Error Units GroupByLongHashSetBenchmark.testGroupByLongHashSet 5000 avgt 3 10.319 ± 0.965 ns/op GroupByLongHashSetBenchmark.testGroupByLongHashSet 50000 avgt 3 14.073 ± 1.312 ns/op GroupByLongHashSetBenchmark.testGroupByLongHashSet 500000 avgt 3 33.237 ± 1.161 ns/op GroupByLongHashSetBenchmark.testGroupByLongHashSet 5000000 avgt 3 60.773 ± 7.917 ns/op GroupByLongHashSetBenchmark.testOrderedMap 5000 avgt 3 27.105 ± 8.215 ns/op GroupByLongHashSetBenchmark.testOrderedMap 50000 avgt 3 27.707 ± 11.362 ns/op GroupByLongHashSetBenchmark.testOrderedMap 500000 avgt 3 134.037 ± 93.003 ns/op GroupByLongHashSetBenchmark.testOrderedMap 5000000 avgt 3 183.419 ± 32.628 ns/op
Configuration menu - View commit details
-
Copy full SHA for a8f4d77 - Browse repository at this point
Copy the full SHA a8f4d77View commit details
Commits on Jan 18, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 2bfab9e - Browse repository at this point
Copy the full SHA 2bfab9eView commit details -
CountDistinctLong256GroupByFunction/CountDistinctUUIDGroupByFunction: Adjusted zero -> null mapping to only map nulls when key is entirely zero. Swapped isParallelismSupported() to use the superclass version for Long/Long128/Long256 GroupByLong128HashSet/GroupByLong256HashSet Adjusted ascii table comments Swapped keyAt functions to keyAddrAt and inlined the offsets, so keyAddr is only calculated once. Calculate address once in setKeyAt Adjusted benchmark values to half or quarter sizes to account for larger size of inserts (aiming to get cache performance impacting the benchmark) Hash Correct algorithm for hashLong256. Long256Impl Added isNull(LLLL) overload. LongToLong256FunctionFactory Adjust whitespace and remove unnecessary 'this'. Use new Long256Impl.isNull(LLLL) QuaternaryFunction Adjust whitespace and naming. RecordSinkSPI and Unordered16Map Adjust names.
Configuration menu - View commit details
-
Copy full SHA for ee8d00f - Browse repository at this point
Copy the full SHA ee8d00fView commit details -
Merge remote-tracking branch 'fork/group-by-int-128-256-hash-set' int…
…o group-by-int-128-256-hash-set
Configuration menu - View commit details
-
Copy full SHA for 343a777 - Browse repository at this point
Copy the full SHA 343a777View commit details -
Merge updates, add capacity check, and re-bench.
Long128 Benchmark (size) Mode Cnt Score Error Units GroupByLong128HashSetBenchmark.testGroupByLong128HashSet 2500 avgt 3 76.840 ± 3.901 ns/op GroupByLong128HashSetBenchmark.testGroupByLong128HashSet 25000 avgt 3 175.917 ± 55.816 ns/op GroupByLong128HashSetBenchmark.testGroupByLong128HashSet 250000 avgt 3 168.134 ± 78.253 ns/op GroupByLong128HashSetBenchmark.testGroupByLong128HashSet 2500000 avgt 3 164.173 ± 27.133 ns/op GroupByLong128HashSetBenchmark.testOrderedMap 2500 avgt 3 229.978 ± 22.202 ns/op GroupByLong128HashSetBenchmark.testOrderedMap 25000 avgt 3 204.169 ± 4.719 ns/op GroupByLong128HashSetBenchmark.testOrderedMap 250000 avgt 3 196.680 ± 32.682 ns/op GroupByLong128HashSetBenchmark.testOrderedMap 2500000 avgt 3 196.723 ± 27.970 ns/op Long256 Benchmark (size) Mode Cnt Score Error Units GroupByLong256HashSetBenchmark.testGroupByLong256HashSet 1250 avgt 3 231.235 ± 61.326 ns/op GroupByLong256HashSetBenchmark.testGroupByLong256HashSet 12500 avgt 3 232.874 ± 47.436 ns/op GroupByLong256HashSetBenchmark.testGroupByLong256HashSet 125000 avgt 3 227.261 ± 11.838 ns/op GroupByLong256HashSetBenchmark.testGroupByLong256HashSet 1250000 avgt 3 235.956 ± 28.635 ns/op GroupByLong256HashSetBenchmark.testOrderedMap 1250 avgt 3 206.105 ± 34.569 ns/op GroupByLong256HashSetBenchmark.testOrderedMap 12500 avgt 3 203.992 ± 12.676 ns/op GroupByLong256HashSetBenchmark.testOrderedMap 125000 avgt 3 204.173 ± 3.784 ns/op GroupByLong256HashSetBenchmark.testOrderedMap 1250000 avgt 3 203.260 ± 2.665 ns/op
Configuration menu - View commit details
-
Copy full SHA for 6a92668 - Browse repository at this point
Copy the full SHA 6a92668View commit details -
Configuration menu - View commit details
-
Copy full SHA for a5cbe7a - Browse repository at this point
Copy the full SHA a5cbe7aView commit details -
Broken commit to show new tests for UUID/Long256 will fail.
UUID ------------------------------------------------------------------------------- Test set: io.questdb.test.griffin.engine.functions.groupby.CountDistinctUuidGroupByFunctionFactoryTest ------------------------------------------------------------------------------- Tests run: 12, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.419 sec <<< FAILURE! - in io.questdb.test.griffin.engine.functions.groupby.CountDistinctUuidGroupByFunctionFactoryTest testMappingZeroToNulls(io.questdb.test.griffin.engine.functions.groupby.CountDistinctUuidGroupByFunctionFactoryTest) Time elapsed: 0.046 sec <<< FAILURE! java.lang.AssertionError: expected:<a s a 4 > but was:<a s a 3 > Long256 ------------------------------------------------------------------------------- Test set: io.questdb.test.griffin.engine.functions.groupby.CountDistinctLong256GroupByFunctionFactoryTest ------------------------------------------------------------------------------- Tests run: 12, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.288 sec <<< FAILURE! - in io.questdb.test.griffin.engine.functions.groupby.CountDistinctLong256GroupByFunctionFactoryTest testMappingZeroToNulls(io.questdb.test.griffin.engine.functions.groupby.CountDistinctLong256GroupByFunctionFactoryTest) Time elapsed: 0.054 sec <<< FAILURE! java.lang.AssertionError: expected:<a s a 4 > but was:<a s a 3 >
Configuration menu - View commit details
-
Copy full SHA for c3d2e64 - Browse repository at this point
Copy the full SHA c3d2e64View commit details -
New tests for UUID/Long256 should pass.
UUID Tests run: 12, Failures: 0, Errors: 0, Skipped: 0 Long256 Tests run: 12, Failures: 0, Errors: 0, Skipped: 0
Configuration menu - View commit details
-
Copy full SHA for 6be4a18 - Browse repository at this point
Copy the full SHA 6be4a18View commit details
Commits on Jan 19, 2024
-
Added oracles to the hash set test. Added extra 0/null mapping cases to the griffin functions.
3Configuration menu - View commit details
-
Copy full SHA for 31e2dab - Browse repository at this point
Copy the full SHA 31e2dabView commit details -
Configuration menu - View commit details
-
Copy full SHA for 497a992 - Browse repository at this point
Copy the full SHA 497a992View commit details