Add Casts and Date Intervals #2411
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dey4ss
changed the title
|
😍 |
Bensk1
reviewed
Sep 20, 2021
Bensk1
reviewed
Sep 20, 2021
Bensk1
reviewed
Sep 20, 2021
dey4ss
commented
Sep 20, 2021
| // CAST(5.5 AS INT) | ||
| values[chunk_offset] = static_cast<Result>(argument_value); | ||
| } | ||
| values[chunk_offset] = *lossy_variant_cast<Result>({argument_value}); |
There was a problem hiding this comment.
This shortens the code, and forwards any boost::bad_lexical_cast when Strings cannot be casted to the requested data type. Before, this just resulted in a field with value 0. According to the SQL standard, an exception should be thrown in such cases. I don't know if catching the error and explicitly Fail() here would be nicer.
There was a problem hiding this comment.
Taken from DB Fiddle / Postgres 13:
CREATE TABLE t (a TEXT);
INSERT INTO t VALUES('1');
INSERT INTO t VALUES('1.2');
SELECT * FROM t WHERE CAST(a as INT) > 0;--> Query Error: error: invalid input syntax for type integer: "1.2"
There was a problem hiding this comment.
Also something that we might discuss in a larger group.
| std::optional<AllTypeVariant> result; | ||
| resolve_data_type(expression.data_type(), [&](auto type) { | ||
| using TargetDataType = typename decltype(type)::type; | ||
| result = *lossy_variant_cast<TargetDataType>(value_expression.value); |
There was a problem hiding this comment.
Same as above -- or should we catch here and fail or even return nullopt?
There was a problem hiding this comment.
Let's discuss this offline, maybe in the fortnightly.
src/lib/sql/sql_translator.cpp
Outdated
| FailInput("'" + date_string + "' is not a valid date"); | ||
| } | ||
| // We don't know if input actually contains dates, but maybe it is good enough that it cointains strings. | ||
| return left; |
There was a problem hiding this comment.
Is this enough?
Pro: Hyrise doesn't really know dates at all, and, e.g., TPC-DS Q21 requests ...cast(d_date AS date) < CAST ('2000-03-11' AS date)). With assuming that d_date can contain dates when containing strings, we can execute such queries.
Con: Does not actually check if all column values are dates, obviously - but we can't check later, as the Date type does not go beyond this point --> we would need to Fail() here
dey4ss
changed the title
|
(on a mobile) Does the PR include reverting many of the TPC-H adaptions that have been required before this PR? E.g., the validation queries and the actual query generation. |
For now, it does not. Actually, I had a look into |
|
After investingating, moving from pre-calculated dates to |
Bouncner
reviewed
Sep 21, 2021
| * a. use strings as data type for now | ||
| * b. pre-calculate date operation | ||
| * 2. Extract is not supported | ||
| * 2. SQLite does not support extract | ||
| * a. Use SUBSTR instead (because our date columns are strings AND SQLite doesn't support EXTRACT) |
There was a problem hiding this comment.
Bot yours, but I thinks it’s substring in TPC-H.
There was a problem hiding this comment.
TPC-H 3.0 seems to use extract(...) in Q7-Q9 (pp. 39ff)
There was a problem hiding this comment.
Yeah, correct. But also substr/substring in other places and I think they switched to substring years ago.
So one more thing we can get more compliant with.
There was a problem hiding this comment.
Ah okay, I think I got you wrong there 😅
Yeah, reference system seems to use substring, too. Guess who cannot parse substring instead of substr? Again, SQLite.
|
I think we should also add benchmark_all or TPC-DS results to have initial reference performance data for the newly supported queries. |
Bensk1
requested changes
Sep 23, 2021
| // CAST(5.5 AS INT) | ||
| values[chunk_offset] = static_cast<Result>(argument_value); | ||
| } | ||
| values[chunk_offset] = *lossy_variant_cast<Result>({argument_value}); |
There was a problem hiding this comment.
Also something that we might discuss in a larger group.
Bensk1
reviewed
Sep 23, 2021
|
Update: Except for TPC-C, all benchmarks are subject to heavily variating query runtimes. Since none except the additional TPC-DS queries contain casts or dates at all, I'll have to investigate this behavior further. System nemea - click to expand
Commit Info and Build Time
hyriseBenchmarkTPCH - single-threaded, SF 10.0 Sum of avg. item runtimes: +1% || Geometric mean of throughput changes: -2%Configuration Overview - click to expand +Configuration Overview----+---------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | /home/Daniel.Lindner/hyrise2/cmake-build-release/benchmark_all_results/hyriseBenchmarkTPCH_9500af59c75d9d256d9c47dc31a061faf69097d2_st.json | /home/Daniel.Lindner/hyrise2/cmake-build-release/benchmark_all_results/hyriseBenchmarkTPCH_a3afe542b1a2373a57fdc3377a4f06418cc27e41_st.json |
+--------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 9500af59c75d9d256d9c47dc31a061faf69097d2-dirty | a3afe542b1a2373a57fdc3377a4f06418cc27e41 |
| benchmark_mode | Ordered | Ordered |
| build_type | release | release |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| clustering | None | None |
| compiler | clang 11.0.0 | clang 11.0.0 |
| cores | 0 | 0 |
| data_preparation_cores | 0 | 0 |
| date | 2021-09-23 13:11:51 | 2021-09-23 16:53:55 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| indexes | False | False |
| max_duration | 60000000000 | 60000000000 |
| max_runs | 100 | 100 |
| scale_factor | 10.0 | 10.0 |
| time_unit | ns | ns |
| use_prepared_statements | False | False |
| using_scheduler | False | False |
| verify | False | False |
| warmup_duration | 1000000000 | 1000000000 |
+--------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+ +----------++----------+---------+--------++----------+----------+--------+---------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+----------++----------+---------+--------++----------+----------+--------+---------+
-| TPC-H 01 || 5707.9 | 5983.2 | +5% || 0.18 | 0.17 | -5% | 0.0013 |
-| TPC-H 02 || 44.4 | 50.3 | +13%˄ || 22.50 | 19.87 | -12%˄ | 0.0000 |
| TPC-H 03 || 2295.4 | 2324.1 | +1% || 0.44 | 0.43 | -1% | 0.1896 |
| TPC-H 04 || 1455.2 | 1449.7 | -0% || 0.69 | 0.69 | +0% | 0.5849 |
| TPC-H 05 || 3102.2 | 3158.7 | +2% || 0.32 | 0.32 | -2% | 0.0293 |
| TPC-H 06 || 164.6 | 160.0 | -3%˄ || 6.07 | 6.25 | +3%˄ | 0.0001 |
-| TPC-H 07 || 912.1 | 963.5 | +6% || 1.10 | 1.04 | -5% | 0.0000 |
| TPC-H 08 || 732.3 | 726.5 | -1% || 1.37 | 1.38 | +1% | 0.0000 |
| TPC-H 09 || 5221.0 | 5267.9 | +1% || 0.19 | 0.19 | -1% | 0.0014 |
| TPC-H 10 || 3008.0 | 2946.0 | -2% || 0.33 | 0.34 | +2% | 0.0317 |
-| TPC-H 11 || 73.6 | 79.0 | +7%˄ || 13.59 | 12.65 | -7%˄ | 0.0000 |
| TPC-H 12 || 1015.5 | 1011.7 | -0% || 0.98 | 0.99 | +0% | 0.2078 |
| TPC-H 13 || 4681.3 | 4632.1 | -1% || 0.21 | 0.22 | +1% | 0.1418 |
| TPC-H 14 || 421.8 | 425.4 | +1%˄ || 2.37 | 2.35 | -1%˄ | 0.0055 |
| TPC-H 15 || 195.9 | 196.9 | +0%˄ || 5.10 | 5.08 | -0%˄ | 0.0000 |
| TPC-H 16 || 628.6 | 646.7 | +3% || 1.59 | 1.55 | -3% | 0.0000 |
| TPC-H 17 || 218.7 | 222.6 | +2%˄ || 4.57 | 4.49 | -2%˄ | 0.0000 |
-| TPC-H 18 || 1498.7 | 1650.4 | +10% || 0.67 | 0.61 | -9% | 0.0000 |
| TPC-H 19 || 275.9 | 282.8 | +3%˄ || 3.62 | 3.54 | -2%˄ | 0.0000 |
| TPC-H 20 || 403.6 | 415.5 | +3%˄ || 2.48 | 2.41 | -3%˄ | 0.0000 |
| TPC-H 21 || 4881.6 | 4714.7 | -3% || 0.20 | 0.21 | +4% | 0.0007 |
-| TPC-H 22 || 422.2 | 456.1 | +8%˄ || 2.37 | 2.19 | -7%˄ | 0.0000 |
+----------++----------+---------+--------++----------+----------+--------+---------+
| Sum || 37360.7 | 37763.9 | +1% || | | | |
| Geomean || | | || | | -2% | |
+----------++----------+---------+--------++----------+----------+--------+---------+
| Notes || ˄ Execution stopped due to max runs reached |
+----------++----------+---------+--------++----------+----------+--------+---------+hyriseBenchmarkTPCH - single-threaded, SF 0.01 Sum of avg. item runtimes: +0% || Geometric mean of throughput changes: -2%Configuration Overview - click to expand +Configuration Overview----+-------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | /home/Daniel.Lindner/hyrise2/cmake-build-release/benchmark_all_results/hyriseBenchmarkTPCH_9500af59c75d9d256d9c47dc31a061faf69097d2_st_s01.json | /home/Daniel.Lindner/hyrise2/cmake-build-release/benchmark_all_results/hyriseBenchmarkTPCH_a3afe542b1a2373a57fdc3377a4f06418cc27e41_st_s01.json |
+--------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 9500af59c75d9d256d9c47dc31a061faf69097d2-dirty | a3afe542b1a2373a57fdc3377a4f06418cc27e41 |
| benchmark_mode | Ordered | Ordered |
| build_type | release | release |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| clustering | None | None |
| compiler | clang 11.0.0 | clang 11.0.0 |
| cores | 0 | 0 |
| data_preparation_cores | 0 | 0 |
| date | 2021-09-23 13:31:26 | 2021-09-23 17:13:36 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| indexes | False | False |
| max_duration | 60000000000 | 60000000000 |
| max_runs | 100 | 100 |
| scale_factor | 0.009999999776482582 | 0.009999999776482582 |
| time_unit | ns | ns |
| use_prepared_statements | False | False |
| using_scheduler | False | False |
| verify | False | False |
| warmup_duration | 1000000000 | 1000000000 |
+--------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------+ +----------++----------+---------+--------++----------+----------+--------+---------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+----------++----------+---------+--------++----------+----------+--------+---------+
| TPC-H 01 || 4.8 | 4.9 | +2%˄ || 207.51 | 203.29 | -2%˄ | 0.0000 |
-| TPC-H 02 || 4.0 | 4.3 | +7%˄ || 250.63 | 234.72 | -6%˄ | 0.2814 |
-| TPC-H 03 || 0.6 | 0.7 | +9%˄ || 1572.58 | 1436.81 | -9%˄ | 0.0000 |
-| TPC-H 04 || 0.4 | 0.5 | +6%˄ || 2284.80 | 2154.50 | -6%˄ | 0.0000 |
| TPC-H 05 || 1.0 | 1.0 | +3%˄ || 1023.22 | 988.88 | -3%˄ | 0.0001 |
| TPC-H 06 || 0.2 | 0.2 | -2%˄ || 5385.42 | 5489.57 | +2%˄ | 0.0020 |
+| TPC-H 07 || 12.1 | 11.5 | -5%˄ || 82.70 | 86.65 | +5%˄ | 0.5078 |
| TPC-H 08 || 16.2 | 15.7 | -3%˄ || 61.89 | 63.50 | +3%˄ | 0.2888 |
| TPC-H 09 || 3.7 | 3.7 | -1%˄ || 269.33 | 270.88 | +1%˄ | 0.9699 |
-| TPC-H 10 || 0.8 | 0.8 | +5%˄ || 1277.45 | 1213.75 | -5%˄ | 0.0000 |
| TPC-H 11 || 0.2 | 0.2 | -3%˄ || 4577.71 | 4718.16 | +3%˄ | 0.0115 |
| TPC-H 12 || 0.6 | 0.6 | +4%˄ || 1601.53 | 1539.10 | -4%˄ | 0.3057 |
| TPC-H 13 || 2.0 | 2.0 | -0%˄ || 498.68 | 501.04 | +0%˄ | 0.1407 |
| TPC-H 14 || 0.3 | 0.4 | +2%˄ || 2876.90 | 2816.55 | -2%˄ | 0.0317 |
| TPC-H 15 || 1.2 | 1.2 | +2%˄ || 855.09 | 841.49 | -2%˄ | 0.0000 |
| TPC-H 16 || 1.9 | 1.9 | +1%˄ || 530.06 | 524.38 | -1%˄ | 0.0004 |
+| TPC-H 17 || 0.7 | 0.6 | -9%˄ || 1404.75 | 1545.35 | +10%˄ | 0.4276 |
-| TPC-H 18 || 1.1 | 1.2 | +9%˄ || 893.72 | 820.47 | -8%˄ | 0.0000 |
| TPC-H 19 || 5.9 | 6.0 | +1%˄ || 169.27 | 167.40 | -1%˄ | 0.0125 |
-| TPC-H 20 || 2.3 | 2.4 | +5%˄ || 435.22 | 413.17 | -5%˄ | 0.0022 |
| TPC-H 21 || 3.8 | 4.0 | +4%˄ || 260.47 | 250.08 | -4%˄ | 0.0145 |
| TPC-H 22 || 1.0 | 1.1 | +2%˄ || 960.55 | 945.81 | -2%˄ | 0.0000 |
+----------++----------+---------+--------++----------+----------+--------+---------+
| Sum || 64.9 | 64.9 | +0% || | | | |
| Geomean || | | || | | -2% | |
+----------++----------+---------+--------++----------+----------+--------+---------+
| Notes || ˄ Execution stopped due to max runs reached |
+----------++----------+---------+--------++----------+----------+--------+---------+hyriseBenchmarkTPCH - multi-threaded, ordered, 1 client, 28 cores, SF 10.0 Sum of avg. item runtimes: +0% || Geometric mean of throughput changes: -0%Configuration Overview - click to expand +Configuration Overview---------+-----------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | /home/Daniel.Lindner/hyrise2/cmake-build-release/benchmark_all_results/hyriseBenchmarkTPCH_9500af59c75d9d256d9c47dc31a061faf69097d2_mt_ordered.json | /home/Daniel.Lindner/hyrise2/cmake-build-release/benchmark_all_results/hyriseBenchmarkTPCH_a3afe542b1a2373a57fdc3377a4f06418cc27e41_mt_ordered.json |
+-------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 9500af59c75d9d256d9c47dc31a061faf69097d2-dirty | a3afe542b1a2373a57fdc3377a4f06418cc27e41 |
| benchmark_mode | Ordered | Ordered |
| build_type | release | release |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| clustering | None | None |
| compiler | clang 11.0.0 | clang 11.0.0 |
| cores | 28 | 28 |
| data_preparation_cores | 0 | 0 |
| date | 2021-09-23 13:31:55 | 2021-09-23 17:14:05 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| indexes | False | False |
| max_duration | 60000000000 | 60000000000 |
| max_runs | -1 | -1 |
| scale_factor | 10.0 | 10.0 |
| time_unit | ns | ns |
| use_prepared_statements | False | False |
| using_scheduler | True | True |
| utilized_cores_per_numa_node | [0, 28] | [0, 28] |
| verify | False | False |
| warmup_duration | 0 | 0 |
+-------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------+ +----------++----------+---------+--------++----------+----------+--------+----------------------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+----------++----------+---------+--------++----------+----------+--------+----------------------+
| TPC-H 01 || 4745.1 | 4761.2 | +0% || 0.20 | 0.20 | +0% | (run time too short) |
| TPC-H 02 || 67.0 | 68.3 | +2% || 13.95 | 13.77 | -1% | 0.0565 |
| TPC-H 03 || 1137.6 | 1125.9 | -1% || 0.87 | 0.88 | +2% | 0.1510 |
| TPC-H 04 || 607.0 | 611.4 | +1% || 1.63 | 1.62 | -1% | 0.5393 |
| TPC-H 05 || 1022.9 | 1017.5 | -1% || 0.97 | 0.97 | -0% | 0.7107 |
| TPC-H 06 || 85.1 | 84.3 | -1% || 11.05 | 11.17 | +1% | 0.0371 |
| TPC-H 07 || 472.0 | 473.4 | +0% || 2.08 | 2.08 | -0% | 0.6014 |
| TPC-H 08 || 411.4 | 409.7 | -0% || 2.40 | 2.40 | -0% | 0.3877 |
| TPC-H 09 || 2507.9 | 2518.0 | +0% || 0.38 | 0.38 | +0% | (run time too short) |
| TPC-H 10 || 1635.1 | 1615.4 | -1% || 0.60 | 0.62 | +3% | (run time too short) |
| TPC-H 11 || 91.8 | 93.7 | +2% || 10.31 | 10.10 | -2% | 0.0009 |
| TPC-H 12 || 567.2 | 564.8 | -0% || 1.73 | 1.75 | +1% | 0.3480 |
| TPC-H 13 || 3073.8 | 3067.6 | -0% || 0.32 | 0.32 | -0% | (run time too short) |
| TPC-H 14 || 176.9 | 177.9 | +1% || 5.48 | 5.47 | -0% | 0.2088 |
| TPC-H 15 || 171.9 | 172.7 | +0% || 5.65 | 5.62 | -1% | 0.3617 |
| TPC-H 16 || 738.4 | 752.9 | +2% || 1.33 | 1.32 | -1% | 0.0066 |
| TPC-H 17 || 100.6 | 100.5 | -0% || 9.42 | 9.45 | +0% | 0.9122 |
| TPC-H 18 || 2573.4 | 2611.6 | +1% || 0.38 | 0.37 | -4% | (run time too short) |
| TPC-H 19 || 168.0 | 168.7 | +0% || 5.78 | 5.75 | -1% | 0.3674 |
| TPC-H 20 || 255.6 | 256.5 | +0% || 3.83 | 3.82 | -0% | 0.6435 |
| TPC-H 21 || 1205.8 | 1197.2 | -1% || 0.82 | 0.82 | -0% | (run time too short) |
| TPC-H 22 || 154.4 | 158.5 | +3% || 6.25 | 6.10 | -2% | 0.0000 |
+----------++----------+---------+--------++----------+----------+--------+----------------------+
| Sum || 21968.8 | 22007.8 | +0% || | | | |
| Geomean || | | || | | -0% | |
+----------++----------+---------+--------++----------+----------+--------+----------------------+hyriseBenchmarkTPCH - multi-threaded, shuffled, 28 clients, 28 cores, SF 10.0 Sum of avg. item runtimes: +1% || Geometric mean of throughput changes: -0%Configuration Overview - click to expand +Configuration Overview---------+---------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | /home/Daniel.Lindner/hyrise2/cmake-build-release/benchmark_all_results/hyriseBenchmarkTPCH_9500af59c75d9d256d9c47dc31a061faf69097d2_mt.json | /home/Daniel.Lindner/hyrise2/cmake-build-release/benchmark_all_results/hyriseBenchmarkTPCH_a3afe542b1a2373a57fdc3377a4f06418cc27e41_mt.json |
+-------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 9500af59c75d9d256d9c47dc31a061faf69097d2-dirty | a3afe542b1a2373a57fdc3377a4f06418cc27e41 |
| benchmark_mode | Shuffled | Shuffled |
| build_type | release | release |
| chunk_size | 65535 | 65535 |
| clients | 28 | 28 |
| clustering | None | None |
| compiler | clang 11.0.0 | clang 11.0.0 |
| cores | 28 | 28 |
| data_preparation_cores | 0 | 0 |
| date | 2021-09-23 13:55:51 | 2021-09-23 17:37:58 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| indexes | False | False |
| max_duration | 1200000000000 | 1200000000000 |
| max_runs | -1 | -1 |
| scale_factor | 10.0 | 10.0 |
| time_unit | ns | ns |
| use_prepared_statements | False | False |
| using_scheduler | True | True |
| utilized_cores_per_numa_node | [0, 28] | [0, 28] |
| verify | False | False |
| warmup_duration | 0 | 0 |
+-------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+ +----------++----------+---------+--------++----------+----------+--------+---------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+----------++----------+---------+--------++----------+----------+--------+---------+
| TPC-H 01 || 5530.1 | 5666.9 | +2% || 0.46 | 0.46 | -0% | 0.0755 |
+| TPC-H 02 || 272.6 | 243.0 | -11% || 0.47 | 0.46 | -0% | 0.5505 |
| TPC-H 03 || 3586.8 | 3592.7 | +0% || 0.46 | 0.46 | -0% | 0.9758 |
-| TPC-H 04 || 2172.9 | 2297.3 | +6% || 0.46 | 0.46 | -0% | 0.3858 |
-| TPC-H 05 || 4070.7 | 4558.9 | +12% || 0.46 | 0.46 | -0% | 0.0314 |
| TPC-H 06 || 485.8 | 467.4 | -4% || 0.47 | 0.46 | -0% | 0.7910 |
+| TPC-H 07 || 2634.3 | 2507.6 | -5% || 0.46 | 0.46 | -0% | 0.5285 |
+| TPC-H 08 || 2094.4 | 1889.4 | -10% || 0.46 | 0.46 | -0% | 0.2103 |
| TPC-H 09 || 5631.8 | 5880.8 | +4% || 0.46 | 0.46 | -0% | 0.2511 |
| TPC-H 10 || 4776.2 | 4590.9 | -4% || 0.46 | 0.46 | -0% | 0.3550 |
| TPC-H 11 || 491.6 | 503.1 | +2% || 0.47 | 0.46 | -0% | 0.9020 |
| TPC-H 12 || 2052.0 | 2067.6 | +1% || 0.47 | 0.46 | -0% | 0.9095 |
| TPC-H 13 || 5664.0 | 5686.7 | +0% || 0.46 | 0.46 | -0% | 0.8410 |
+| TPC-H 14 || 1141.1 | 991.8 | -13% || 0.47 | 0.46 | -0% | 0.1865 |
+| TPC-H 15 || 508.8 | 430.4 | -15% || 0.46 | 0.46 | -0% | 0.1751 |
-| TPC-H 16 || 1743.0 | 1849.1 | +6% || 0.46 | 0.46 | -0% | 0.3483 |
| TPC-H 17 || 563.0 | 581.3 | +3% || 0.47 | 0.46 | -0% | 0.8163 |
-| TPC-H 18 || 3355.9 | 3543.8 | +6% || 0.46 | 0.46 | -0% | 0.0767 |
| TPC-H 19 || 836.2 | 812.3 | -3% || 0.46 | 0.46 | -0% | 0.8306 |
-| TPC-H 20 || 1163.6 | 1262.2 | +8% || 0.47 | 0.46 | -0% | 0.3733 |
| TPC-H 21 || 5694.8 | 5458.2 | -4% || 0.46 | 0.46 | -0% | 0.3706 |
+| TPC-H 22 || 1064.6 | 955.2 | -10% || 0.46 | 0.46 | -0% | 0.4001 |
+----------++----------+---------+--------++----------+----------+--------+---------+
| Sum || 55534.3 | 55836.7 | +1% || | | | |
| Geomean || | | || | | -0% | |
+----------++----------+---------+--------++----------+----------+--------+---------+hyriseBenchmarkTPCDS - single-threaded Sum of avg. item runtimes: -25% || Geometric mean of throughput changes: +10%Configuration Overview - click to expand +Configuration Overview---+----------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | /home/Daniel.Lindner/hyrise2/cmake-build-release/benchmark_all_results/hyriseBenchmarkTPCDS_9500af59c75d9d256d9c47dc31a061faf69097d2_st.json | /home/Daniel.Lindner/hyrise2/cmake-build-release/benchmark_all_results/hyriseBenchmarkTPCDS_a3afe542b1a2373a57fdc3377a4f06418cc27e41_st.json |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 9500af59c75d9d256d9c47dc31a061faf69097d2-dirty | a3afe542b1a2373a57fdc3377a4f06418cc27e41 |
| benchmark_mode | Ordered | Ordered |
| build_type | release | release |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| compiler | clang 11.0.0 | clang 11.0.0 |
| cores | 0 | 0 |
| data_preparation_cores | 0 | 0 |
| date | 2021-09-23 14:17:43 | 2021-09-23 17:59:49 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| indexes | False | False |
| max_duration | 60000000000 | 60000000000 |
| max_runs | 100 | 100 |
| time_unit | ns | ns |
| using_scheduler | False | False |
| verify | False | False |
| warmup_duration | 1000000000 | 1000000000 |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------+ +-----------++----------+---------+---------++----------+----------+---------+----------------------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+-----------++----------+---------+---------++----------+----------+---------+----------------------+
| 01 || 215.1 | 217.9 | +1%˄ || 4.65 | 4.59 | -1%˄ | 0.0020 |
-| 03 || 70.1 | 81.1 | +16%˄ || 14.27 | 12.34 | -14%˄ | 0.0000 |
-| 06 || 149.8 | 170.0 | +14%˄ || 6.68 | 5.88 | -12%˄ | 0.0000 |
| 07 || 285.1 | 294.6 | +3%˄ || 3.51 | 3.39 | -3%˄ | 0.0000 |
| 09 || 655.5 | 631.4 | -4% || 1.53 | 1.58 | +4% | 0.0000 |
-| 10 || 141.0 | 160.2 | +14%˄ || 7.09 | 6.24 | -12%˄ | 0.0000 |
| 13 || 416.8 | 418.2 | +0%˄ || 2.40 | 2.39 | -0%˄ | 0.0327 |
| 15 || 115.5 | 114.2 | -1%˄ || 8.66 | 8.76 | +1%˄ | 0.0000 |
+| 17 -> 16 || 371.8 | 166.3 | -55%˄ || 2.69 | 6.01 | +124%˄ | 0.0000 |
-| 19 -> 17 || 98.4 | 402.5 | +309%˄ || 10.16 | 2.48 | -76%˄ | 0.0000 |
+| 25 -> 19 || 169.6 | 119.5 | -30%˄ || 5.90 | 8.37 | +42%˄ | 0.0000 |
-| 26 -> 25 || 131.6 | 187.5 | +42%˄ || 7.60 | 5.33 | -30%˄ | 0.0000 |
+| 28 -> 26 || 2808.1 | 132.2 | -95%˄ || 0.36 | 7.56 | +2024%˄ | (run time too short) |
-| 29 -> 28 || 465.9 | 2784.9 | +498%˄ || 2.15 | 0.36 | -83%˄ | (run time too short) |
+| 31 -> 29 || 1256.2 | 486.6 | -61%˄ || 0.80 | 2.06 | +158%˄ | (run time too short) |
-| 34 -> 31 || 155.0 | 1325.2 | +755%˄ || 6.45 | 0.75 | -88%˄ | (run time too short) |
+| 35 -> 32 || 614.7 | 45.3 | -93%˄ || 1.63 | 22.08 | +1257%˄ | (run time too short) |
+| 39a -> 34 || 1888.7 | 170.4 | -91%˄ || 0.53 | 5.87 | +1008%˄ | (run time too short) |
+| 39b -> 35 || 1850.0 | 621.3 | -66% || 0.54 | 1.61 | +198% | 0.0000 |
-| 41 -> 37 || 297.0 | 327.6 | +10%˄ || 3.37 | 3.05 | -9%˄ | 0.0000 |
-| 42 -> 39a || 80.9 | 1801.3 | +2126%˄ || 12.36 | 0.56 | -96%˄ | (run time too short) |
-| 43 -> 39b || 916.3 | 1825.8 | +99% || 1.09 | 0.55 | -50% | 0.0000 |
-| 45 -> 41 || 115.5 | 310.5 | +169%˄ || 8.66 | 3.22 | -63%˄ | 0.0000 |
+| 48 -> 42 || 1089.4 | 100.4 | -91%˄ || 0.92 | 9.96 | +985%˄ | (run time too short) |
-| 50 -> 43 || 112.1 | 920.4 | +721%˄ || 8.92 | 1.09 | -88%˄ | (run time too short) |
-| 52 -> 45 || 81.2 | 120.2 | +48%˄ || 12.32 | 8.32 | -32%˄ | 0.0000 |
-| 55 -> 48 || 77.3 | 1091.8 | +1312%˄ || 12.94 | 0.92 | -93%˄ | (run time too short) |
+| 62 -> 50 || 545.3 | 129.8 | -76%˄ || 1.83 | 7.70 | +320%˄ | 0.0000 |
+| 65 -> 52 || 1753.7 | 97.4 | -94%˄ || 0.57 | 10.26 | +1700%˄ | (run time too short) |
+| 69 -> 55 || 129.6 | 94.2 | -27%˄ || 7.72 | 10.61 | +38%˄ | 0.0000 |
-| 73 -> 62 || 77.2 | 545.4 | +606%˄ || 12.95 | 1.83 | -86%˄ | 0.0000 |
-| 79 -> 65 || 462.0 | 1759.0 | +281%˄ || 2.16 | 0.57 | -74%˄ | (run time too short) |
+| 81 -> 69 || 204.1 | 150.6 | -26%˄ || 4.90 | 6.64 | +36%˄ | 0.0000 |
-| 83 -> 73 || 39.2 | 96.3 | +145%˄ || 25.50 | 10.39 | -59%˄ | 0.0000 |
-| 85 -> 79 || 280.5 | 480.7 | +71%˄ || 3.57 | 2.08 | -42%˄ | 0.0000 |
+| 88 -> 81 || 621.5 | 202.8 | -67%˄ || 1.61 | 4.93 | +206%˄ | (run time too short) |
-| 91 -> 82 || 17.3 | 417.0 | +2311%˄ || 57.81 | 2.40 | -96%˄ | 0.0000 |
+| 93 -> 83 || 3847.8 | 43.5 | -99%˄ || 0.26 | 22.97 | +8737%˄ | (run time too short) |
-| 96 -> 85 || 57.3 | 284.2 | +396%˄ || 17.44 | 3.52 | -80%˄ | 0.0000 |
+| 97 -> 88 || 3280.5 | 773.1 | -76% || 0.30 | 1.29 | +324% | 0.0000 |
+| 99 -> 91 || 1048.6 | 19.8 | -98%˄ || 0.95 | 50.59 | +5205%˄ | (run time too short) |
+-----------++----------+---------+---------++----------+----------+---------+----------------------+
+| Sum || 26993.1 | 20121.0 | -25% || | | | |
+| Geomean || | | || | | +10% | |
+-----------++----------+---------+---------++----------+----------+---------+----------------------+
| Notes || ˄ Execution stopped due to max runs reached |
+-----------++----------+---------+---------++----------+----------+---------+----------------------+hyriseBenchmarkTPCDS - multi-threaded, shuffled, 28 clients, 28 cores Sum of avg. item runtimes: -1% || Geometric mean of throughput changes: -29%Configuration Overview - click to expand +Configuration Overview---------+----------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | /home/Daniel.Lindner/hyrise2/cmake-build-release/benchmark_all_results/hyriseBenchmarkTPCDS_9500af59c75d9d256d9c47dc31a061faf69097d2_mt.json | /home/Daniel.Lindner/hyrise2/cmake-build-release/benchmark_all_results/hyriseBenchmarkTPCDS_a3afe542b1a2373a57fdc3377a4f06418cc27e41_mt.json |
+-------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 9500af59c75d9d256d9c47dc31a061faf69097d2-dirty | a3afe542b1a2373a57fdc3377a4f06418cc27e41 |
| benchmark_mode | Shuffled | Shuffled |
| build_type | release | release |
| chunk_size | 65535 | 65535 |
| clients | 28 | 28 |
| compiler | clang 11.0.0 | clang 11.0.0 |
| cores | 28 | 28 |
| data_preparation_cores | 0 | 0 |
| date | 2021-09-23 14:41:10 | 2021-09-23 18:26:56 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| indexes | False | False |
| max_duration | 1200000000000 | 1200000000000 |
| max_runs | -1 | -1 |
| time_unit | ns | ns |
| using_scheduler | True | True |
| utilized_cores_per_numa_node | [0, 28] | [0, 28] |
| verify | False | False |
| warmup_duration | 0 | 0 |
+-------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------+ +-----------++----------+---------+--------++----------+----------+--------+---------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+-----------++----------+---------+--------++----------+----------+--------+---------+
-| 01 || 551.0 | 662.1 | +20% || 0.61 | 0.44 | -29% | 0.0408 |
-| 03 || 207.9 | 237.6 | +14% || 0.61 | 0.44 | -29% | 0.3980 |
-| 06 || 614.5 | 753.2 | +23% || 0.61 | 0.43 | -29% | 0.0976 |
-| 07 || 753.4 | 840.1 | +12% || 0.61 | 0.44 | -29% | 0.1897 |
-| 09 || 1124.7 | 1234.1 | +10% || 0.61 | 0.44 | -29% | 0.1119 |
-| 10 || 486.1 | 519.8 | +7% || 0.61 | 0.44 | -29% | 0.6645 |
-| 13 || 1471.5 | 1811.9 | +23% || 0.61 | 0.43 | -29% | 0.0076 |
-| 15 || 427.5 | 498.1 | +17% || 0.61 | 0.44 | -29% | 0.2374 |
+| 17 -> 16 || 1052.2 | 772.0 | -27% || 0.61 | 0.44 | -29% | 0.0005 |
-| 19 -> 17 || 385.3 | 1167.3 | +203% || 0.61 | 0.43 | -29% | 0.0000 |
+| 25 -> 19 || 655.7 | 456.5 | -30% || 0.61 | 0.44 | -29% | 0.0021 |
-| 26 -> 25 || 514.2 | 787.0 | +53% || 0.61 | 0.43 | -29% | 0.0019 |
+| 28 -> 26 || 1566.5 | 436.8 | -72% || 0.61 | 0.44 | -29% | 0.0000 |
-| 29 -> 28 || 1290.5 | 1576.0 | +22% || 0.61 | 0.44 | -29% | 0.0009 |
+| 31 -> 29 || 1704.4 | 1317.2 | -23% || 0.61 | 0.43 | -29% | 0.0001 |
-| 34 -> 31 || 405.7 | 1866.5 | +360% || 0.61 | 0.43 | -29% | 0.0000 |
+| 35 -> 32 || 1866.3 | 337.3 | -82% || 0.61 | 0.44 | -29% | 0.0000 |
+| 39a -> 34 || 2028.4 | 405.4 | -80% || 0.61 | 0.44 | -29% | 0.0000 |
-| 39b -> 35 || 2027.9 | 2073.5 | +2% || 0.61 | 0.43 | -29% | 0.6833 |
+| 41 -> 37 || 2502.3 | 608.8 | -76% || 0.61 | 0.43 | -29% | 0.0000 |
-| 42 -> 39a || 324.3 | 2315.5 | +614% || 0.61 | 0.44 | -29% | 0.0000 |
-| 43 -> 39b || 1114.1 | 2252.0 | +102% || 0.61 | 0.44 | -29% | 0.0000 |
-| 45 -> 41 || 469.1 | 2877.3 | +513% || 0.61 | 0.43 | -29% | 0.0000 |
+| 48 -> 42 || 2310.8 | 453.5 | -80% || 0.61 | 0.44 | -29% | 0.0000 |
-| 50 -> 43 || 489.6 | 1327.8 | +171% || 0.61 | 0.44 | -29% | 0.0000 |
-| 52 -> 45 || 312.0 | 573.9 | +84% || 0.61 | 0.43 | -29% | 0.0000 |
-| 55 -> 48 || 277.4 | 2705.0 | +875% || 0.61 | 0.43 | -29% | 0.0000 |
+| 62 -> 50 || 795.7 | 555.2 | -30% || 0.61 | 0.44 | -29% | 0.0000 |
+| 65 -> 52 || 3307.9 | 381.4 | -88% || 0.61 | 0.43 | -29% | 0.0000 |
+| 69 -> 55 || 466.2 | 348.1 | -25% || 0.61 | 0.44 | -29% | 0.0252 |
-| 73 -> 62 || 258.2 | 839.9 | +225% || 0.61 | 0.44 | -29% | 0.0000 |
-| 79 -> 65 || 1009.4 | 3616.7 | +258% || 0.61 | 0.43 | -29% | 0.0000 |
+| 81 -> 69 || 816.1 | 625.9 | -23% || 0.61 | 0.44 | -29% | 0.0204 |
-| 83 -> 73 || 235.2 | 304.8 | +30% || 0.61 | 0.44 | -29% | 0.1487 |
-| 85 -> 79 || 1074.5 | 1190.8 | +11% || 0.61 | 0.44 | -29% | 0.2148 |
-| 88 -> 81 || 644.7 | 774.5 | +20% || 0.61 | 0.44 | -29% | 0.0777 |
-| 91 -> 82 || 163.5 | 840.4 | +414% || 0.61 | 0.44 | -29% | 0.0000 |
+| 93 -> 83 || 2157.7 | 303.0 | -86% || 0.61 | 0.44 | -29% | 0.0000 |
-| 96 -> 85 || 185.6 | 1173.5 | +532% || 0.61 | 0.44 | -29% | 0.0000 |
+| 97 -> 88 || 3735.9 | 822.8 | -78% || 0.61 | 0.44 | -29% | 0.0000 |
+| 99 -> 91 || 1248.2 | 145.6 | -88% || 0.61 | 0.44 | -29% | 0.0000 |
+-----------++----------+---------+--------++----------+----------+--------+---------+
| Sum || 43031.9 | 42789.0 | -1% || | | | |
-| Geomean || | | || | | -29% | |
+-----------++----------+---------+--------++----------+----------+--------+---------+hyriseBenchmarkTPCC - single-threaded Sum of avg. item runtimes: +2% || Geometric mean of throughput changes: -2%Configuration Overview - click to expand +Configuration Overview---+---------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | /home/Daniel.Lindner/hyrise2/cmake-build-release/benchmark_all_results/hyriseBenchmarkTPCC_9500af59c75d9d256d9c47dc31a061faf69097d2_st.json | /home/Daniel.Lindner/hyrise2/cmake-build-release/benchmark_all_results/hyriseBenchmarkTPCC_a3afe542b1a2373a57fdc3377a4f06418cc27e41_st.json |
+-------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 9500af59c75d9d256d9c47dc31a061faf69097d2-dirty | a3afe542b1a2373a57fdc3377a4f06418cc27e41 |
| benchmark_mode | Shuffled | Shuffled |
| build_type | release | release |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| compiler | clang 11.0.0 | clang 11.0.0 |
| cores | 0 | 0 |
| data_preparation_cores | 0 | 0 |
| date | 2021-09-23 15:01:36 | 2021-09-23 18:47:24 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| indexes | False | False |
| max_duration | 60000000000 | 60000000000 |
| max_runs | -1 | -1 |
| scale_factor | 1 | 1 |
| time_unit | ns | ns |
| using_scheduler | False | False |
| verify | False | False |
| warmup_duration | 0 | 0 |
+-------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+ +--------------++----------+---------+--------++----------+----------+--------+---------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+--------------++----------+---------+--------++----------+----------+--------+---------+
| Delivery || 27.7 | 28.4 | +3% || 3.50 | 3.47 | -1% | 0.0000 |
| New-Order || 20.0 | 20.4 | +2% || 39.50 | 38.63 | -2% | 0.0136 |
| Order-Status || 1.2 | 1.2 | +1% || 3.52 | 3.40 | -3% | 0.1453 |
| Payment || 2.5 | 2.5 | +1% || 37.81 | 36.95 | -2% | 0.0104 |
| Stock-Level || 3.5 | 3.6 | +2% || 3.53 | 3.43 | -3% | 0.0482 |
+--------------++----------+---------+--------++----------+----------+--------+---------+
| Sum || 55.0 | 56.2 | +2% || | | | |
| Geomean || | | || | | -2% | |
+--------------++----------+---------+--------++----------+----------+--------+---------+hyriseBenchmarkTPCC - multi-threaded, shuffled, 28 clients, 28 cores Sum of avg. item runtimes: -0% || Geometric mean of throughput changes: -0%Configuration Overview - click to expand +Configuration Overview---------+---------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | /home/Daniel.Lindner/hyrise2/cmake-build-release/benchmark_all_results/hyriseBenchmarkTPCC_9500af59c75d9d256d9c47dc31a061faf69097d2_mt.json | /home/Daniel.Lindner/hyrise2/cmake-build-release/benchmark_all_results/hyriseBenchmarkTPCC_a3afe542b1a2373a57fdc3377a4f06418cc27e41_mt.json |
+-------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 9500af59c75d9d256d9c47dc31a061faf69097d2-dirty | a3afe542b1a2373a57fdc3377a4f06418cc27e41 |
| benchmark_mode | Shuffled | Shuffled |
| build_type | release | release |
| chunk_size | 65535 | 65535 |
| clients | 28 | 28 |
| compiler | clang 11.0.0 | clang 11.0.0 |
| cores | 28 | 28 |
| data_preparation_cores | 0 | 0 |
| date | 2021-09-23 15:02:37 | 2021-09-23 18:48:24 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| indexes | False | False |
| max_duration | 1200000000000 | 1200000000000 |
| max_runs | -1 | -1 |
| scale_factor | 1 | 1 |
| time_unit | ns | ns |
| using_scheduler | True | True |
| utilized_cores_per_numa_node | [0, 28] | [0, 28] |
| verify | False | False |
| warmup_duration | 0 | 0 |
+-------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+ +--------------++----------+---------+--------++----------+----------+--------+---------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+--------------++----------+---------+--------++----------+----------+--------+---------+
| Delivery || 223.0 | 221.4 | -1% || 4.24 | 4.27 | +1% | 0.3695 |
| unsucc.: || 4.5 | 4.4 | -0% || 62.27 | 62.08 | -0% | |
| New-Order || 61.4 | 62.3 | +1% || 126.27 | 124.43 | -1% | 0.0000 |
| unsucc.: || 4.9 | 5.0 | +2% || 621.99 | 622.01 | +0% | |
| Order-Status || 7.8 | 7.8 | +0% || 66.51 | 66.35 | -0% | 0.6080 |
| Payment || 10.3 | 10.6 | +3% || 20.80 | 20.55 | -1% | 0.0001 |
| unsucc.: || 4.4 | 4.5 | +3% || 694.21 | 692.72 | -0% | |
| Stock-Level || 17.7 | 17.6 | -0% || 66.51 | 66.35 | -0% | 0.6008 |
+--------------++----------+---------+--------++----------+----------+--------+---------+
| Sum || 320.2 | 319.7 | -0% || | | | |
| Geomean || | | || | | -0% | |
+--------------++----------+---------+--------++----------+----------+--------+---------+hyriseBenchmarkJoinOrder - single-threaded Sum of avg. item runtimes: +1% || Geometric mean of throughput changes: -7%Configuration Overview - click to expand +Configuration Overview---+--------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | /home/Daniel.Lindner/hyrise2/cmake-build-release/benchmark_all_results/hyriseBenchmarkJoinOrder_9500af59c75d9d256d9c47dc31a061faf69097d2_st.json | /home/Daniel.Lindner/hyrise2/cmake-build-release/benchmark_all_results/hyriseBenchmarkJoinOrder_a3afe542b1a2373a57fdc3377a4f06418cc27e41_st.json |
+-------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 9500af59c75d9d256d9c47dc31a061faf69097d2-dirty | a3afe542b1a2373a57fdc3377a4f06418cc27e41 |
| benchmark_mode | Ordered | Ordered |
| build_type | release | release |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| compiler | clang 11.0.0 | clang 11.0.0 |
| cores | 0 | 0 |
| data_preparation_cores | 0 | 0 |
| date | 2021-09-23 15:22:44 | 2021-09-23 19:08:32 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| indexes | False | False |
| max_duration | 60000000000 | 60000000000 |
| max_runs | 100 | 100 |
| time_unit | ns | ns |
| using_scheduler | False | False |
| verify | False | False |
| warmup_duration | 1000000000 | 1000000000 |
+-------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------+ +---------++----------+---------+--------++----------+----------+--------+---------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+---------++----------+---------+--------++----------+----------+--------+---------+
-| 10a || 167.2 | 180.5 | +8%˄ || 5.98 | 5.54 | -7%˄ | 0.0000 |
-| 10b || 116.8 | 129.7 | +11%˄ || 8.56 | 7.71 | -10%˄ | 0.0000 |
| 10c || 320.4 | 320.8 | +0%˄ || 3.12 | 3.12 | -0%˄ | 0.8066 |
-| 11a || 51.1 | 55.5 | +8%˄ || 19.55 | 18.03 | -8%˄ | 0.0000 |
-| 11b || 50.8 | 55.3 | +9%˄ || 19.67 | 18.07 | -8%˄ | 0.0000 |
-| 11c || 47.2 | 51.7 | +10%˄ || 21.20 | 19.34 | -9%˄ | 0.0000 |
-| 11d || 23.4 | 28.2 | +21%˄ || 42.82 | 35.41 | -17%˄ | 0.0000 |
-| 12a || 51.4 | 54.3 | +6%˄ || 19.44 | 18.40 | -5%˄ | 0.0000 |
-| 12b || 91.7 | 102.2 | +11%˄ || 10.91 | 9.79 | -10%˄ | 0.0000 |
| 12c || 1699.0 | 1708.5 | +1% || 0.59 | 0.59 | -1% | 0.0000 |
| 13a || 342.2 | 348.6 | +2%˄ || 2.92 | 2.87 | -2%˄ | 0.0000 |
-| 13b || 124.8 | 136.8 | +10%˄ || 8.01 | 7.31 | -9%˄ | 0.0000 |
-| 13c || 59.6 | 68.5 | +15%˄ || 16.77 | 14.60 | -13%˄ | 0.0000 |
| 13d || 653.9 | 657.4 | +1% || 1.53 | 1.52 | -1% | 0.0001 |
| 14a || 2387.1 | 2390.8 | +0% || 0.42 | 0.42 | -0% | 0.1614 |
| 14b || 2298.0 | 2307.4 | +0% || 0.44 | 0.43 | -0% | 0.0000 |
| 14c || 2458.3 | 2462.8 | +0% || 0.41 | 0.41 | -0% | 0.0524 |
| 15a || 81.4 | 83.6 | +3%˄ || 12.28 | 11.96 | -3%˄ | 0.0000 |
| 15b || 80.7 | 82.7 | +2%˄ || 12.39 | 12.09 | -2%˄ | 0.0000 |
-| 15c || 41.6 | 47.1 | +13%˄ || 24.06 | 21.24 | -12%˄ | 0.0000 |
| 15d || 85.5 | 86.1 | +1%˄ || 11.70 | 11.62 | -1%˄ | 0.0000 |
| 16a || 2219.5 | 2205.2 | -1% || 0.45 | 0.45 | +1% | 0.3278 |
| 16b || 3261.8 | 3210.3 | -2% || 0.31 | 0.31 | +2% | 0.0002 |
| 16c || 2320.5 | 2314.6 | -0% || 0.43 | 0.43 | +0% | 0.7228 |
| 16d || 2272.4 | 2291.8 | +1% || 0.44 | 0.44 | -1% | 0.2114 |
| 17a || 580.1 | 588.7 | +1%˄ || 1.72 | 1.70 | -1%˄ | 0.0000 |
| 17b || 462.3 | 472.8 | +2%˄ || 2.16 | 2.11 | -2%˄ | 0.0000 |
| 17c || 441.1 | 452.4 | +3%˄ || 2.27 | 2.21 | -2%˄ | 0.0000 |
| 17d || 521.5 | 535.5 | +3%˄ || 1.92 | 1.87 | -3%˄ | 0.0000 |
| 17e || 1572.7 | 1550.0 | -1% || 0.64 | 0.65 | +1% | 0.0000 |
| 17f || 938.9 | 952.1 | +1% || 1.07 | 1.05 | -1% | 0.0000 |
+| 18a || 1666.5 | 1568.9 | -6% || 0.60 | 0.64 | +6% | 0.0000 |
-| 18b || 113.7 | 127.3 | +12%˄ || 8.79 | 7.85 | -11%˄ | 0.0000 |
| 18c || 2245.9 | 2263.0 | +1% || 0.45 | 0.44 | -1% | 0.0000 |
| 19a || 2984.9 | 2963.8 | -1% || 0.34 | 0.34 | +1% | 0.0000 |
| 19b || 1369.6 | 1362.8 | -1% || 0.73 | 0.73 | +1% | 0.0000 |
| 19c || 2867.5 | 2850.5 | -1% || 0.35 | 0.35 | +1% | 0.0000 |
| 19d || 2168.5 | 2167.8 | -0% || 0.46 | 0.46 | +0% | 0.4133 |
| 1a || 27.2 | 28.1 | +3%˄ || 36.74 | 35.59 | -3%˄ | 0.0000 |
-| 1b || 9.4 | 11.4 | +22%˄ || 106.73 | 87.78 | -18%˄ | 0.0000 |
-| 1c || 10.4 | 12.4 | +20%˄ || 96.24 | 80.49 | -16%˄ | 0.0000 |
-| 1d || 9.3 | 11.3 | +22%˄ || 107.38 | 88.11 | -18%˄ | 0.0000 |
| 20a || 776.8 | 773.7 | -0% || 1.29 | 1.29 | +0% | 0.0000 |
| 20b || 664.1 | 663.7 | -0% || 1.51 | 1.51 | +0% | 0.1444 |
| 20c || 422.2 | 440.1 | +4%˄ || 2.37 | 2.27 | -4%˄ | 0.0000 |
-| 21a || 70.4 | 82.9 | +18%˄ || 14.21 | 12.06 | -15%˄ | 0.0000 |
-| 21b || 57.3 | 62.4 | +9%˄ || 17.46 | 16.02 | -8%˄ | 0.0000 |
-| 21c || 70.6 | 83.5 | +18%˄ || 14.17 | 11.97 | -15%˄ | 0.0000 |
| 22a || 1885.7 | 1896.3 | +1% || 0.53 | 0.53 | -1% | 0.0000 |
| 22b || 1782.4 | 1792.4 | +1% || 0.56 | 0.56 | -1% | 0.0000 |
| 22c || 2747.8 | 2754.9 | +0% || 0.36 | 0.36 | -0% | 0.0665 |
| 22d || 2843.6 | 2850.2 | +0% || 0.35 | 0.35 | -0% | 0.0790 |
-| 23a || 47.1 | 52.3 | +11%˄ || 21.24 | 19.10 | -10%˄ | 0.0000 |
-| 23b || 44.9 | 56.6 | +26%˄ || 22.26 | 17.66 | -21%˄ | 0.0000 |
-| 23c || 52.5 | 57.4 | +9%˄ || 19.06 | 17.42 | -9%˄ | 0.0000 |
| 24a || 1355.9 | 1368.1 | +1% || 0.74 | 0.73 | -1% | 0.0000 |
| 24b || 1338.9 | 1345.0 | +0% || 0.75 | 0.74 | -0% | 0.0000 |
| 25a || 281.3 | 292.5 | +4%˄ || 3.55 | 3.42 | -4%˄ | 0.0000 |
-| 25b || 114.0 | 136.9 | +20%˄ || 8.77 | 7.30 | -17%˄ | 0.0000 |
| 25c || 2641.6 | 2651.6 | +0% || 0.38 | 0.38 | -0% | 0.0042 |
-| 26a || 376.0 | 396.3 | +5%˄ || 2.66 | 2.52 | -5%˄ | 0.0000 |
-| 26b || 345.3 | 365.5 | +6%˄ || 2.90 | 2.74 | -6%˄ | 0.0000 |
-| 26c || 435.3 | 454.9 | +5%˄ || 2.30 | 2.20 | -4%˄ | 0.0000 |
| 27a || 1695.6 | 1709.4 | +1% || 0.59 | 0.58 | -1% | 0.0000 |
| 27b || 1696.5 | 1711.4 | +1% || 0.59 | 0.58 | -1% | 0.0000 |
-| 27c || 70.8 | 83.3 | +18%˄ || 14.12 | 12.00 | -15%˄ | 0.0000 |
| 28a || 2333.8 | 2347.9 | +1% || 0.43 | 0.43 | -1% | 0.0000 |
| 28b || 1665.4 | 1683.1 | +1% || 0.60 | 0.59 | -1% | 0.0000 |
| 28c || 2421.7 | 2434.9 | +1% || 0.41 | 0.41 | -1% | 0.0000 |
| 29a || 1338.9 | 1345.0 | +0% || 0.75 | 0.74 | -0% | 0.0000 |
-| 29b || 93.7 | 121.4 | +30%˄ || 10.67 | 8.23 | -23%˄ | 0.0000 |
| 29c || 1369.8 | 1370.2 | +0% || 0.73 | 0.73 | -0% | 0.5936 |
-| 2a || 46.0 | 50.4 | +9%˄ || 21.73 | 19.86 | -9%˄ | 0.0000 |
-| 2b || 37.3 | 41.8 | +12%˄ || 26.78 | 23.93 | -11%˄ | 0.0000 |
-| 2c || 28.5 | 33.2 | +16%˄ || 35.04 | 30.13 | -14%˄ | 0.0000 |
-| 2d || 64.7 | 68.9 | +6%˄ || 15.46 | 14.52 | -6%˄ | 0.0000 |
-| 30a || 151.1 | 166.8 | +10%˄ || 6.62 | 5.99 | -9%˄ | 0.0000 |
-| 30b || 215.6 | 233.2 | +8%˄ || 4.64 | 4.29 | -8%˄ | 0.0000 |
| 30c || 2092.6 | 2112.0 | +1% || 0.48 | 0.47 | -1% | 0.0000 |
-| 31a || 146.8 | 172.2 | +17%˄ || 6.81 | 5.81 | -15%˄ | 0.0000 |
-| 31b || 228.0 | 255.2 | +12%˄ || 4.39 | 3.92 | -11%˄ | 0.0000 |
-| 31c || 171.3 | 197.1 | +15%˄ || 5.84 | 5.07 | -13%˄ | 0.0000 |
-| 32a || 12.8 | 16.9 | +32%˄ || 78.26 | 59.21 | -24%˄ | 0.0000 |
-| 32b || 30.6 | 34.4 | +13%˄ || 32.68 | 29.05 | -11%˄ | 0.0000 |
-| 33a || 19.4 | 26.1 | +34%˄ || 51.45 | 38.34 | -25%˄ | 0.0000 |
-| 33b || 18.7 | 25.4 | +36%˄ || 53.47 | 39.35 | -26%˄ | 0.0000 |
-| 33c || 23.8 | 30.4 | +28%˄ || 42.04 | 32.92 | -22%˄ | 0.0000 |
| 3a || 1955.2 | 1956.4 | +0% || 0.51 | 0.51 | -0% | 0.0773 |
-| 3b || 13.2 | 17.3 | +30%˄ || 75.47 | 57.95 | -23%˄ | 0.0000 |
| 3c || 2396.7 | 2375.4 | -1% || 0.42 | 0.42 | +1% | 0.0000 |
| 4a || 242.4 | 242.1 | -0%˄ || 4.13 | 4.13 | +0%˄ | 0.5075 |
-| 4b || 11.5 | 15.3 | +33%˄ || 87.01 | 65.20 | -25%˄ | 0.0000 |
| 4c || 272.7 | 269.3 | -1%˄ || 3.67 | 3.71 | +1%˄ | 0.0000 |
| 5a || 1886.3 | 1889.4 | +0% || 0.53 | 0.53 | -0% | 0.0000 |
| 5b || 142.8 | 142.8 | -0%˄ || 7.00 | 7.00 | +0%˄ | 0.3297 |
| 5c || 2177.2 | 2180.8 | +0% || 0.46 | 0.46 | -0% | 0.0001 |
-| 6a || 103.9 | 119.2 | +15%˄ || 9.62 | 8.39 | -13%˄ | 0.0000 |
-| 6b || 117.6 | 131.2 | +11%˄ || 8.50 | 7.62 | -10%˄ | 0.0000 |
-| 6c || 98.9 | 113.9 | +15%˄ || 10.11 | 8.78 | -13%˄ | 0.0000 |
| 6d || 531.8 | 539.2 | +1%˄ || 1.88 | 1.85 | -1%˄ | 0.0000 |
-| 6e || 104.0 | 119.0 | +14%˄ || 9.61 | 8.40 | -13%˄ | 0.0000 |
| 6f || 697.4 | 709.0 | +2% || 1.43 | 1.41 | -2% | 0.0000 |
-| 7a || 68.9 | 93.3 | +35%˄ || 14.51 | 10.72 | -26%˄ | 0.0000 |
-| 7b || 49.9 | 75.1 | +50%˄ || 20.03 | 13.31 | -34%˄ | 0.0000 |
| 7c || 620.2 | 632.1 | +2% || 1.61 | 1.58 | -2% | 0.0000 |
-| 8a || 39.2 | 46.1 | +18%˄ || 25.52 | 21.71 | -15%˄ | 0.0000 |
-| 8b || 69.6 | 73.9 | +6%˄ || 14.36 | 13.54 | -6%˄ | 0.0000 |
| 8c || 1864.5 | 1830.4 | -2% || 0.54 | 0.55 | +2% | 0.0000 |
| 8d || 327.2 | 317.6 | -3%˄ || 3.06 | 3.15 | +3%˄ | 0.0000 |
| 9a || 1718.0 | 1727.7 | +1% || 0.58 | 0.58 | -1% | 0.0000 |
-| 9b || 111.9 | 118.0 | +5%˄ || 8.94 | 8.48 | -5%˄ | 0.0000 |
| 9c || 1715.0 | 1725.6 | +1% || 0.58 | 0.58 | -1% | 0.0000 |
| 9d || 1916.4 | 1919.9 | +0% || 0.52 | 0.52 | -0% | 0.1239 |
+---------++----------+---------+--------++----------+----------+--------+---------+
| Sum || 92875.8 | 93526.9 | +1% || | | | |
-| Geomean || | | || | | -7% | |
+---------++----------+---------+--------++----------+----------+--------+---------+
| Notes || ˄ Execution stopped due to max runs reached |
+---------++----------+---------+--------++----------+----------+--------+---------+hyriseBenchmarkJoinOrder - multi-threaded, shuffled, 28 clients, 28 cores Sum of avg. item runtimes: +0% || Geometric mean of throughput changes: -1%Configuration Overview - click to expand +Configuration Overview---------+--------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------+
| Parameter | /home/Daniel.Lindner/hyrise2/cmake-build-release/benchmark_all_results/hyriseBenchmarkJoinOrder_9500af59c75d9d256d9c47dc31a061faf69097d2_mt.json | /home/Daniel.Lindner/hyrise2/cmake-build-release/benchmark_all_results/hyriseBenchmarkJoinOrder_a3afe542b1a2373a57fdc3377a4f06418cc27e41_mt.json |
+-------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 9500af59c75d9d256d9c47dc31a061faf69097d2-dirty | a3afe542b1a2373a57fdc3377a4f06418cc27e41 |
| benchmark_mode | Shuffled | Shuffled |
| build_type | release | release |
| chunk_size | 65535 | 65535 |
| clients | 28 | 28 |
| compiler | clang 11.0.0 | clang 11.0.0 |
| cores | 28 | 28 |
| data_preparation_cores | 0 | 0 |
| date | 2021-09-23 16:27:12 | 2021-09-23 20:14:09 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| indexes | False | False |
| max_duration | 1200000000000 | 1200000000000 |
| max_runs | -1 | -1 |
| time_unit | ns | ns |
| using_scheduler | True | True |
| utilized_cores_per_numa_node | [0, 28] | [0, 28] |
| verify | False | False |
| warmup_duration | 0 | 0 |
+-------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------+ +---------++----------+----------+--------++----------+----------+--------+---------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+---------++----------+----------+--------++----------+----------+--------+---------+
+| 10a || 425.9 | 342.3 | -20% || 0.20 | 0.20 | -1% | 0.1824 |
| 10b || 312.2 | 317.6 | +2% || 0.20 | 0.20 | -1% | 0.9095 |
-| 10c || 782.9 | 867.8 | +11% || 0.20 | 0.20 | -1% | 0.3672 |
+| 11a || 226.6 | 214.1 | -5% || 0.20 | 0.20 | -1% | 0.8265 |
+| 11b || 240.7 | 227.6 | -5% || 0.20 | 0.20 | -1% | 0.8474 |
+| 11c || 364.5 | 210.3 | -42% || 0.20 | 0.20 | -1% | 0.0221 |
-| 11d || 203.6 | 363.1 | +78% || 0.20 | 0.20 | -1% | 0.0174 |
-| 12a || 379.2 | 442.3 | +17% || 0.20 | 0.20 | -1% | 0.4045 |
-| 12b || 131.9 | 194.3 | +47% || 0.20 | 0.20 | -1% | 0.1378 |
-| 12c || 1068.0 | 1248.0 | +17% || 0.20 | 0.20 | -1% | 0.1187 |
-| 13a || 1290.2 | 1469.0 | +14% || 0.20 | 0.20 | -1% | 0.2264 |
-| 13b || 276.9 | 365.2 | +32% || 0.20 | 0.20 | -1% | 0.2364 |
-| 13c || 292.9 | 421.8 | +44% || 0.20 | 0.20 | -1% | 0.3408 |
-| 13d || 1719.7 | 1797.6 | +5% || 0.20 | 0.20 | -1% | 0.5950 |
| 14a || 2045.0 | 2017.2 | -1% || 0.20 | 0.20 | -1% | 0.8672 |
+| 14b || 1553.4 | 1341.8 | -14% || 0.20 | 0.20 | -1% | 0.0516 |
| 14c || 2167.3 | 2212.5 | +2% || 0.20 | 0.20 | -1% | 0.7587 |
+| 15a || 416.3 | 348.1 | -16% || 0.20 | 0.20 | -1% | 0.3137 |
+| 15b || 345.8 | 301.2 | -13% || 0.20 | 0.20 | -1% | 0.3740 |
-| 15c || 236.3 | 264.3 | +12% || 0.20 | 0.20 | -1% | 0.6945 |
-| 15d || 332.2 | 426.4 | +28% || 0.20 | 0.20 | -1% | 0.0798 |
+| 16a || 4335.2 | 3954.0 | -9% || 0.20 | 0.20 | -1% | 0.0503 |
| 16b || 5646.6 | 5881.5 | +4% || 0.20 | 0.20 | -1% | 0.2620 |
| 16c || 4249.5 | 4160.1 | -2% || 0.20 | 0.20 | -1% | 0.6421 |
| 16d || 4115.6 | 4234.0 | +3% || 0.20 | 0.20 | -1% | 0.4873 |
| 17a || 1619.8 | 1681.1 | +4% || 0.20 | 0.20 | -1% | 0.6596 |
-| 17b || 1238.6 | 1418.3 | +15% || 0.20 | 0.20 | -1% | 0.2076 |
+| 17c || 1251.3 | 1046.0 | -16% || 0.20 | 0.20 | -1% | 0.0981 |
| 17d || 1425.2 | 1367.1 | -4% || 0.20 | 0.20 | -1% | 0.7025 |
-| 17e || 2430.3 | 2882.1 | +19% || 0.20 | 0.20 | -1% | 0.0091 |
| 17f || 2042.7 | 2009.0 | -2% || 0.20 | 0.20 | -1% | 0.8175 |
-| 18a || 2632.1 | 2862.5 | +9% || 0.20 | 0.20 | -1% | 0.1598 |
+| 18b || 390.7 | 367.2 | -6% || 0.20 | 0.20 | -1% | 0.6561 |
| 18c || 1545.7 | 1535.5 | -1% || 0.20 | 0.20 | -1% | 0.9326 |
| 19a || 1388.4 | 1360.5 | -2% || 0.20 | 0.20 | -1% | 0.8417 |
+| 19b || 1010.3 | 962.0 | -5% || 0.20 | 0.20 | -1% | 0.7378 |
-| 19c || 1473.2 | 1703.3 | +16% || 0.20 | 0.20 | -1% | 0.1093 |
| 19d || 2664.9 | 2701.3 | +1% || 0.20 | 0.20 | -1% | 0.8310 |
+| 1a || 172.6 | 145.7 | -16% || 0.20 | 0.20 | -1% | 0.6358 |
| 1b || 144.3 | 141.3 | -2% || 0.20 | 0.20 | -1% | 0.9493 |
-| 1c || 100.3 | 178.8 | +78% || 0.20 | 0.20 | -1% | 0.1483 |
-| 1d || 80.5 | 112.5 | +40% || 0.20 | 0.20 | -1% | 0.2040 |
-| 20a || 935.8 | 1025.4 | +10% || 0.20 | 0.20 | -1% | 0.3150 |
| 20b || 852.0 | 837.3 | -2% || 0.20 | 0.20 | -1% | 0.8939 |
-| 20c || 748.2 | 835.2 | +12% || 0.20 | 0.20 | -1% | 0.3418 |
+| 21a || 306.5 | 259.2 | -15% || 0.20 | 0.20 | -1% | 0.4070 |
-| 21b || 266.4 | 318.3 | +19% || 0.20 | 0.20 | -1% | 0.3542 |
+| 21c || 394.5 | 260.3 | -34% || 0.20 | 0.20 | -1% | 0.0997 |
| 22a || 1763.6 | 1827.6 | +4% || 0.20 | 0.20 | -1% | 0.6719 |
+| 22b || 1223.2 | 1135.0 | -7% || 0.20 | 0.20 | -1% | 0.4353 |
| 22c || 2693.1 | 2653.1 | -1% || 0.20 | 0.20 | -1% | 0.8041 |
| 22d || 2777.4 | 2738.5 | -1% || 0.20 | 0.20 | -1% | 0.8291 |
| 23a || 264.0 | 261.7 | -1% || 0.20 | 0.20 | -1% | 0.9532 |
+| 23b || 294.7 | 271.8 | -8% || 0.20 | 0.20 | -1% | 0.6542 |
-| 23c || 398.1 | 490.6 | +23% || 0.20 | 0.20 | -1% | 0.2812 |
+| 24a || 1146.6 | 930.8 | -19% || 0.20 | 0.20 | -1% | 0.1146 |
+| 24b || 758.0 | 688.8 | -9% || 0.20 | 0.20 | -1% | 0.3377 |
+| 25a || 1270.5 | 1094.7 | -14% || 0.20 | 0.20 | -1% | 0.1738 |
+| 25b || 309.8 | 286.2 | -8% || 0.20 | 0.20 | -1% | 0.6978 |
| 25c || 2283.2 | 2265.9 | -1% || 0.20 | 0.20 | -1% | 0.9130 |
-| 26a || 578.1 | 760.8 | +32% || 0.20 | 0.20 | -1% | 0.0481 |
+| 26b || 525.5 | 421.3 | -20% || 0.20 | 0.20 | -1% | 0.2417 |
| 26c || 902.6 | 886.6 | -2% || 0.20 | 0.20 | -1% | 0.8722 |
-| 27a || 922.4 | 993.6 | +8% || 0.20 | 0.20 | -1% | 0.5403 |
-| 27b || 950.0 | 1060.0 | +12% || 0.20 | 0.20 | -1% | 0.3194 |
-| 27c || 239.2 | 313.7 | +31% || 0.20 | 0.20 | -1% | 0.1973 |
| 28a || 2216.2 | 2140.0 | -3% || 0.20 | 0.20 | -1% | 0.6152 |
-| 28b || 863.9 | 1019.2 | +18% || 0.20 | 0.20 | -1% | 0.0713 |
+| 28c || 2331.1 | 2187.5 | -6% || 0.20 | 0.20 | -1% | 0.3802 |
-| 29a || 744.1 | 803.8 | +8% || 0.20 | 0.20 | -1% | 0.5809 |
| 29b || 343.9 | 358.4 | +4% || 0.20 | 0.20 | -1% | 0.8036 |
-| 29c || 931.6 | 1114.5 | +20% || 0.20 | 0.20 | -1% | 0.2216 |
+| 2a || 428.9 | 363.4 | -15% || 0.20 | 0.20 | -1% | 0.3216 |
+| 2b || 321.1 | 268.5 | -16% || 0.20 | 0.20 | -1% | 0.2087 |
-| 2c || 250.6 | 269.5 | +8% || 0.20 | 0.20 | -1% | 0.6802 |
| 2d || 504.9 | 511.1 | +1% || 0.20 | 0.20 | -1% | 0.9499 |
+| 30a || 857.4 | 758.2 | -12% || 0.20 | 0.20 | -1% | 0.4199 |
+| 30b || 492.7 | 418.0 | -15% || 0.20 | 0.20 | -1% | 0.2655 |
+| 30c || 2198.0 | 1991.4 | -9% || 0.20 | 0.20 | -1% | 0.2468 |
+| 31a || 806.6 | 599.0 | -26% || 0.20 | 0.20 | -1% | 0.0894 |
| 31b || 395.8 | 410.5 | +4% || 0.20 | 0.20 | -1% | 0.8116 |
-| 31c || 801.7 | 867.6 | +8% || 0.20 | 0.20 | -1% | 0.5623 |
+| 32a || 116.6 | 102.7 | -12% || 0.20 | 0.20 | -1% | 0.6554 |
-| 32b || 310.1 | 354.1 | +14% || 0.20 | 0.20 | -1% | 0.4872 |
-| 33a || 168.5 | 211.7 | +26% || 0.20 | 0.20 | -1% | 0.3119 |
+| 33b || 271.7 | 232.7 | -14% || 0.20 | 0.20 | -1% | 0.5710 |
-| 33c || 233.0 | 291.3 | +25% || 0.20 | 0.20 | -1% | 0.4422 |
+| 3a || 1252.2 | 1038.0 | -17% || 0.20 | 0.20 | -1% | 0.0471 |
+| 3b || 246.1 | 199.5 | -19% || 0.20 | 0.20 | -1% | 0.4127 |
-| 3c || 1538.8 | 1665.3 | +8% || 0.20 | 0.20 | -1% | 0.3306 |
+| 4a || 756.8 | 695.0 | -8% || 0.20 | 0.20 | -1% | 0.5350 |
| 4b || 163.9 | 170.7 | +4% || 0.20 | 0.20 | -1% | 0.9278 |
-| 4c || 774.3 | 986.5 | +27% || 0.20 | 0.20 | -1% | 0.0501 |
+| 5a || 935.7 | 785.2 | -16% || 0.20 | 0.20 | -1% | 0.2499 |
+| 5b || 501.6 | 467.6 | -7% || 0.20 | 0.20 | -0% | 0.6663 |
-| 5c || 1146.9 | 1351.1 | +18% || 0.20 | 0.20 | -1% | 0.1176 |
+| 6a || 241.2 | 192.3 | -20% || 0.20 | 0.20 | -1% | 0.2492 |
+| 6b || 433.7 | 296.0 | -32% || 0.20 | 0.20 | -1% | 0.1918 |
+| 6c || 263.5 | 231.4 | -12% || 0.20 | 0.20 | -1% | 0.5963 |
| 6d || 1132.7 | 1179.1 | +4% || 0.20 | 0.20 | -1% | 0.6505 |
+| 6e || 344.5 | 219.7 | -36% || 0.20 | 0.20 | -1% | 0.0907 |
| 6f || 1639.3 | 1634.1 | -0% || 0.20 | 0.20 | -1% | 0.9650 |
-| 7a || 286.5 | 354.7 | +24% || 0.20 | 0.20 | -1% | 0.2508 |
| 7b || 285.2 | 276.3 | -3% || 0.20 | 0.20 | -1% | 0.8483 |
+| 7c || 1881.6 | 1680.7 | -11% || 0.20 | 0.20 | -1% | 0.1785 |
| 8a || 339.5 | 351.6 | +4% || 0.20 | 0.20 | -1% | 0.8608 |
-| 8b || 275.6 | 368.4 | +34% || 0.20 | 0.20 | -1% | 0.1327 |
-| 8c || 3467.3 | 3708.5 | +7% || 0.20 | 0.20 | -1% | 0.2327 |
| 8d || 1012.1 | 987.6 | -2% || 0.20 | 0.20 | -1% | 0.8177 |
+| 9a || 1444.1 | 1262.0 | -13% || 0.20 | 0.20 | -1% | 0.1233 |
+| 9b || 856.8 | 611.4 | -29% || 0.20 | 0.20 | -1% | 0.0200 |
| 9c || 1768.1 | 1732.5 | -2% || 0.20 | 0.20 | -1% | 0.8240 |
-| 9d || 1941.5 | 2116.2 | +9% || 0.20 | 0.20 | -1% | 0.2410 |
+---------++----------+----------+--------++----------+----------+--------+---------+
| Sum || 118090.6 | 118546.7 | +0% || | | | |
| Geomean || | | || | | -1% | |
+---------++----------+----------+--------++----------+----------+--------+---------+ |
Nice, thanks! The |
Exactly, I just added objects with empty runs in the JSON file for the new queries. I don't really have an explanation for the improvements in TPC-DS. E.g., I examined Q96. When looking at the PQPs, they are identical, but one SemiJoin shows a little speedup (
Q95 is the one that keeps running extremely slow with OD Join Elimination, as only a rather small join can be optimized (the SemiJoin with |
|
Thanks for the information. Are SemiJoins responsible for the performance differences in the other queries as well? Also, which step of the join is affected, hovering over the operator in the SVG should give more details. |
|
I also looked at Q88, and there the performance improvements stem from SemiJoins, as well. In general, |
Me neither. I also went through your changes twice and cannot explain it. |
|
New benchmark runs show alike results for DS SF 10. Configuration Overview - click to expand +Configuration Overview---+------------------------------------------------+------------------------------------------------+
| Parameter | ds-master.json | ds-head.json |
+-------------------------+------------------------------------------------+------------------------------------------------+
| GIT-HASH | a750ae96866018ca705584a8087bb4cbd0fa72da-dirty | 6c142ad931d8ea12af6574476f0a5d7258f4f83e-dirty |
| benchmark_mode | Ordered | Ordered |
| build_type | release | release |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| compiler | clang 11.0.0 | clang 11.0.0 |
| cores | 0 | 0 |
| data_preparation_cores | 0 | 0 |
| date | 2021-10-06 08:58:30 | 2021-10-06 09:43:24 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| indexes | False | False |
| max_duration | 60000000000 | 60000000000 |
| max_runs | 100 | 100 |
| time_unit | ns | ns |
| using_scheduler | False | False |
| verify | False | False |
| warmup_duration | 5000000000 | 5000000000 |
+-------------------------+------------------------------------------------+------------------------------------------------+ +---------++----------+---------+--------++----------+----------+--------+----------------------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+---------++----------+---------+--------++----------+----------+--------+----------------------+
| 01 || 212.2 | 216.1 | +2%˄ || 4.71 | 4.63 | -2%˄ | 0.0000 |
+| 03 || 80.0 | 70.0 | -13%˄ || 12.49 | 14.29 | +14%˄ | 0.0000 |
+| 06 || 167.1 | 151.3 | -9%˄ || 5.99 | 6.61 | +10%˄ | 0.0000 |
| 07 || 293.7 | 283.4 | -3%˄ || 3.41 | 3.53 | +4%˄ | 0.0000 |
-| 09 || 625.0 | 667.1 | +7% || 1.60 | 1.50 | -6% | 0.0000 |
+| 10 || 156.0 | 140.1 | -10%˄ || 6.41 | 7.14 | +11%˄ | 0.0000 |
| 13 || 417.6 | 417.2 | -0%˄ || 2.39 | 2.40 | +0%˄ | 0.7408 |
| 15 || 111.8 | 114.3 | +2%˄ || 8.95 | 8.75 | -2%˄ | 0.0000 |
| 16 || nan | 163.9 | || 0.00 | 6.10 | +nan%˄ | (run time too short) |
| 17 || 393.9 | 381.9 | -3%˄ || 2.54 | 2.62 | +3%˄ | 0.0000 |
+| 19 || 115.0 | 103.2 | -10%˄ || 8.69 | 9.69 | +11%˄ | 0.0000 |
| 25 || 181.0 | 175.0 | -3%˄ || 5.52 | 5.71 | +3%˄ | 0.0000 |
-| 26 || 127.6 | 136.5 | +7%˄ || 7.84 | 7.33 | -7%˄ | 0.0000 |
| 28 || 2768.3 | 2801.6 | +1% || 0.36 | 0.36 | -1% | 0.0000 |
+| 29 || 487.1 | 464.5 | -5%˄ || 2.05 | 2.15 | +5%˄ | 0.0000 |
+| 31 || 1319.4 | 1249.0 | -5% || 0.76 | 0.80 | +6% | 0.0000 |
| 32 || nan | 48.1 | || 0.00 | 20.81 | +nan%˄ | (run time too short) |
+| 34 || 161.8 | 153.9 | -5%˄ || 6.18 | 6.50 | +5%˄ | 0.0000 |
| 35 || 616.7 | 612.3 | -1% || 1.62 | 1.63 | +1% | 0.0000 |
| 37 || nan | 322.4 | || 0.00 | 3.10 | +nan%˄ | (run time too short) |
| 39a || 1805.7 | 1828.0 | +1% || 0.55 | 0.55 | -1% | 0.0106 |
| 39b || 1806.5 | 1807.6 | +0% || 0.55 | 0.55 | -0% | 0.9134 |
+| 41 || 307.1 | 290.8 | -5%˄ || 3.26 | 3.44 | +6%˄ | 0.0000 |
+| 42 || 98.1 | 81.6 | -17%˄ || 10.19 | 12.25 | +20%˄ | 0.0000 |
| 43 || 906.3 | 890.7 | -2% || 1.10 | 1.12 | +2% | 0.0000 |
| 45 || 119.8 | 115.8 | -3%˄ || 8.35 | 8.63 | +3%˄ | 0.0000 |
| 48 || 1077.0 | 1072.3 | -0% || 0.93 | 0.93 | +0% | 0.2279 |
+| 50 || 128.8 | 109.9 | -15%˄ || 7.77 | 9.10 | +17%˄ | 0.0000 |
+| 52 || 96.6 | 78.9 | -18%˄ || 10.35 | 12.68 | +23%˄ | 0.0000 |
+| 55 || 93.5 | 75.6 | -19%˄ || 10.70 | 13.23 | +24%˄ | 0.0000 |
| 62 || 544.1 | 539.7 | -1%˄ || 1.84 | 1.85 | +1%˄ | 0.0000 |
| 65 || 1739.5 | 1746.9 | +0% || 0.57 | 0.57 | -0% | 0.4397 |
+| 69 || 147.9 | 126.6 | -14%˄ || 6.76 | 7.90 | +17%˄ | 0.0000 |
+| 73 || 93.3 | 78.2 | -16%˄ || 10.71 | 12.79 | +19%˄ | 0.0000 |
| 79 || 476.8 | 467.7 | -2%˄ || 2.10 | 2.14 | +2%˄ | 0.0000 |
| 81 || 199.9 | 206.0 | +3%˄ || 5.00 | 4.86 | -3%˄ | 0.0000 |
| 82 || nan | 406.7 | || 0.00 | 2.46 | +nan%˄ | (run time too short) |
+| 83 || 42.6 | 40.0 | -6%˄ || 23.47 | 25.02 | +7%˄ | 0.0000 |
| 85 || 281.5 | 280.7 | -0%˄ || 3.55 | 3.56 | +0%˄ | 0.0000 |
+| 88 || 755.9 | 631.4 | -16% || 1.32 | 1.58 | +20% | 0.0000 |
+| 91 || 19.6 | 17.3 | -12%˄ || 51.11 | 57.76 | +13%˄ | 0.0000 |
| 92 || nan | 35.2 | || 0.00 | 28.42 | +nan%˄ | (run time too short) |
| 93 || 3792.7 | 3835.1 | +1% || 0.26 | 0.26 | -1% | 0.0242 |
| 94 || nan | 98.1 | || 0.00 | 10.19 | +nan%˄ | (run time too short) |
| 95 || nan | 8891.5 | || 0.00 | 0.11 | +nan% | (run time too short) |
+| 96 || 75.3 | 57.9 | -23%˄ || 13.29 | 17.27 | +30%˄ | 0.0000 |
| 97 || 3293.8 | 3232.3 | -2% || 0.30 | 0.31 | +2% | 0.0206 |
| 99 || 1029.2 | 1045.4 | +2% || 0.97 | 0.96 | -2% | 0.0000 |
+---------++----------+---------+--------++----------+----------+--------+----------------------+
-| Sum || 27165.3 | 36759.7 | +35% || | | | |
+| Geomean || | | || | | +6% | |
+---------++----------+---------+--------++----------+----------+--------+----------------------+
| Notes || ˄ Execution stopped due to max runs reached |
+---------++----------+---------+--------++----------+----------+--------+----------------------+ |
New idea (I don't consider it likely but we don't have any other ideas): maybe the difference is introduced by the new queries that influence the cache or something else. Could you keep everything as is in terms of the actual Hyrise code but use the old query blacklist? Thereby, we should execute the exact same set of queries for old and new, right? |
|
I'll give it a try, yet this should only influence queries that are executed after the new ones, right? |
Yep. |
|
Does not really change anything. Configuration Overview - click to expand +Configuration Overview---+------------------------------------------------+------------------------------------------------+
| Parameter | ds-master - Copy.json | ds-head_old_queries.json |
+-------------------------+------------------------------------------------+------------------------------------------------+
| GIT-HASH | a750ae96866018ca705584a8087bb4cbd0fa72da-dirty | 6c142ad931d8ea12af6574476f0a5d7258f4f83e-dirty |
| benchmark_mode | Ordered | Ordered |
| build_type | release | release |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| compiler | clang 11.0.0 | clang 11.0.0 |
| cores | 0 | 0 |
| data_preparation_cores | 0 | 0 |
| date | 2021-10-06 08:58:30 | 2021-10-06 12:16:36 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| indexes | False | False |
| max_duration | 60000000000 | 60000000000 |
| max_runs | 100 | 100 |
| time_unit | ns | ns |
| using_scheduler | False | False |
| verify | False | False |
| warmup_duration | 5000000000 | 5000000000 |
+-------------------------+------------------------------------------------+------------------------------------------------+ +---------++----------+---------+--------++----------+----------+--------+---------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+---------++----------+---------+--------++----------+----------+--------+---------+
| 01 || 212.2 | 212.2 | -0%˄ || 4.71 | 4.71 | +0%˄ | 0.9955 |
+| 03 || 80.0 | 69.2 | -13%˄ || 12.49 | 14.44 | +16%˄ | 0.0000 |
+| 06 || 167.1 | 148.9 | -11%˄ || 5.99 | 6.72 | +12%˄ | 0.0000 |
| 07 || 293.7 | 288.4 | -2%˄ || 3.41 | 3.47 | +2%˄ | 0.0111 |
| 09 || 625.0 | 651.1 | +4% || 1.60 | 1.54 | -4% | 0.0000 |
+| 10 || 156.0 | 139.5 | -11%˄ || 6.41 | 7.17 | +12%˄ | 0.0000 |
| 13 || 417.6 | 415.6 | -0%˄ || 2.39 | 2.41 | +0%˄ | 0.0742 |
| 15 || 111.8 | 113.4 | +1%˄ || 8.95 | 8.82 | -1%˄ | 0.0000 |
+| 17 || 393.9 | 374.0 | -5%˄ || 2.54 | 2.67 | +5%˄ | 0.0000 |
+| 19 || 115.0 | 99.4 | -14%˄ || 8.69 | 10.06 | +16%˄ | 0.0000 |
+| 25 || 181.0 | 167.2 | -8%˄ || 5.52 | 5.98 | +8%˄ | 0.0000 |
| 26 || 127.6 | 130.9 | +3%˄ || 7.84 | 7.64 | -3%˄ | 0.0000 |
| 28 || 2768.3 | 2783.3 | +1% || 0.36 | 0.36 | -1% | 0.0063 |
+| 29 || 487.1 | 465.7 | -4%˄ || 2.05 | 2.15 | +5%˄ | 0.0000 |
+| 31 || 1319.4 | 1241.1 | -6% || 0.76 | 0.81 | +6% | 0.0000 |
| 34 || 161.8 | 157.1 | -3%˄ || 6.18 | 6.37 | +3%˄ | 0.0000 |
| 35 || 616.7 | 616.0 | -0% || 1.62 | 1.62 | +0% | 0.4734 |
| 39a || 1805.7 | 1808.9 | +0% || 0.55 | 0.55 | -0% | 0.6651 |
| 39b || 1806.5 | 1782.9 | -1% || 0.55 | 0.56 | +1% | 0.0351 |
+| 41 || 307.1 | 288.4 | -6%˄ || 3.26 | 3.47 | +6%˄ | 0.0000 |
+| 42 || 98.1 | 82.3 | -16%˄ || 10.19 | 12.14 | +19%˄ | 0.0000 |
| 43 || 906.3 | 902.2 | -0% || 1.10 | 1.11 | +0% | 0.1513 |
+| 45 || 119.8 | 113.8 | -5%˄ || 8.35 | 8.78 | +5%˄ | 0.0000 |
| 48 || 1077.0 | 1090.6 | +1% || 0.93 | 0.92 | -1% | 0.0053 |
+| 50 || 128.8 | 110.7 | -14%˄ || 7.77 | 9.03 | +16%˄ | 0.0000 |
+| 52 || 96.6 | 80.8 | -16%˄ || 10.35 | 12.37 | +20%˄ | 0.0000 |
+| 55 || 93.5 | 78.1 | -16%˄ || 10.70 | 12.80 | +20%˄ | 0.0000 |
| 62 || 544.1 | 540.6 | -1%˄ || 1.84 | 1.85 | +1%˄ | 0.0000 |
| 65 || 1739.5 | 1712.7 | -2% || 0.57 | 0.58 | +2% | 0.0028 |
+| 69 || 147.9 | 125.4 | -15%˄ || 6.76 | 7.97 | +18%˄ | 0.0000 |
+| 73 || 93.3 | 77.4 | -17%˄ || 10.71 | 12.92 | +21%˄ | 0.0000 |
| 79 || 476.8 | 464.8 | -3%˄ || 2.10 | 2.15 | +3%˄ | 0.0000 |
| 81 || 199.9 | 199.8 | -0%˄ || 5.00 | 5.01 | +0%˄ | 0.8946 |
+| 83 || 42.6 | 38.5 | -10%˄ || 23.47 | 25.97 | +11%˄ | 0.0000 |
| 85 || 281.5 | 276.3 | -2%˄ || 3.55 | 3.62 | +2%˄ | 0.0000 |
+| 88 || 755.9 | 626.8 | -17% || 1.32 | 1.60 | +21% | 0.0000 |
+| 91 || 19.6 | 17.0 | -13%˄ || 51.11 | 58.83 | +15%˄ | 0.0000 |
| 93 || 3792.7 | 3789.3 | -0% || 0.26 | 0.26 | +0% | 0.8235 |
+| 96 || 75.3 | 56.9 | -24%˄ || 13.29 | 17.58 | +32%˄ | 0.0000 |
| 97 || 3293.8 | 3253.8 | -1% || 0.30 | 0.31 | +1% | 0.0410 |
| 99 || 1029.2 | 1031.4 | +0% || 0.97 | 0.97 | -0% | 0.2415 |
+---------++----------+---------+--------++----------+----------+--------+---------+
| Sum || 27165.3 | 26622.6 | -2% || | | | |
+| Geomean || | | || | | +7% | |
+---------++----------+---------+--------++----------+----------+--------+---------+
| Notes || ˄ Execution stopped due to max runs reached |
+---------++----------+---------+--------++----------+----------+--------+---------+ |
|
I looked at the query plans of Q96 and Q88 (not running the full DS benchmark): I cannot reproduce the differences on my local machine. Usually the master's version is even a tiny bit faster. Let's proceed with the other open review aspects of this PR. |
|
Hm, maybe this is a hint for the paging theory?
|
Yes, that could be. |
Bensk1
requested changes
Oct 14, 2021
There was a problem hiding this comment.
I answered/resolved some of the open comments. Apart from that it looks good to me and I only have some minor requests.
I also registered that we want to discuss the following aspects in a larger audience:
- How we want to fail if CASTs fail (two places in this PR)
- Compliance with SQL standard or SQLite in the expression evaluator test
- The discussion around line 1775 of the sql_translator.cpp
|
Decisions from today's meeting with @Bensk1 and @mweisgut:
SQL standard, we fail with the error mentioned above.
We only try to cast value expressions as date, other expressions fail. This PR does not add full support for a In addition, the parser does not parse the following statement: |
|
Great, thanks for the summary!
We should also document that somewhere. |
I added that this is no proper intoduction of a Date type in the PR description; before |
|
Yep. |
Bensk1
approved these changes
Oct 25, 2021
dey4ss
commented
Oct 25, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
With this PR, Hyrise is able to resolve SQL's
CASTandINTERVALstatements. Intervals are implemented only for dates. By applying this, Hyrise is able to execute seven additional TPC-DS queries.This is achieved by a new
IntervalExpression. For its evaluation, date utilities are added. Intervals are evaluated directly by theSQLTranslatoriff they are added to or subsracted from aValueExpressionthat holds a valid date representation. E.g., such an expression can stem from parsing a String or Date literal.Anyway, this PR does not privode a full support of a Date type.
Furthermore, the existing implementation of casts was revisited to match the SQL standard.