-
Notifications
You must be signed in to change notification settings - Fork 28k
/
DatasetBenchmark-results.txt
46 lines (39 loc) · 3.84 KB
/
DatasetBenchmark-results.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
================================================================================================
Dataset Benchmark
================================================================================================
OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1031-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
back-to-back map long: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
RDD 11103 11472 522 9.0 111.0 1.0X
DataFrame 1610 2302 979 62.1 16.1 6.9X
Dataset 2493 2569 108 40.1 24.9 4.5X
OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1031-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
back-to-back map: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
RDD 13568 13584 22 7.4 135.7 1.0X
DataFrame 7016 7019 3 14.3 70.2 1.9X
Dataset 18500 20172 2365 5.4 185.0 0.7X
OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1031-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
back-to-back filter Long: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
RDD 2589 2638 69 38.6 25.9 1.0X
DataFrame 871 919 53 114.8 8.7 3.0X
Dataset 2642 2798 220 37.8 26.4 1.0X
OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1031-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
back-to-back filter: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
RDD 4590 5007 589 21.8 45.9 1.0X
DataFrame 153 188 24 653.0 1.5 30.0X
Dataset 8744 8814 99 11.4 87.4 0.5X
OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1031-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
aggregate: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
RDD sum 4108 4214 150 24.3 41.1 1.0X
DataFrame sum 67 82 10 1487.0 0.7 61.1X
Dataset sum using Aggregator 7972 8557 828 12.5 79.7 0.5X
Dataset complex Aggregator 11833 12021 266 8.5 118.3 0.3X