-
Notifications
You must be signed in to change notification settings - Fork 28k
/
CSVBenchmark-results.txt
67 lines (59 loc) · 6.1 KB
/
CSVBenchmark-results.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
================================================================================================
Benchmark to measure CSV read/write performance
================================================================================================
OpenJDK 64-Bit Server VM 1.8.0_312-b07 on Linux 5.11.0-1022-azure
Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
Parsing quoted values: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
One quoted string 51747 52432 599 0.0 1034931.8 1.0X
OpenJDK 64-Bit Server VM 1.8.0_312-b07 on Linux 5.11.0-1022-azure
Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
Wide rows with 1000 columns: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Select 1000 columns 118598 119530 1380 0.0 118598.1 1.0X
Select 100 columns 41421 41669 221 0.0 41420.9 2.9X
Select one column 34304 34521 325 0.0 34304.3 3.5X
count() 10934 11080 128 0.1 10934.5 10.8X
Select 100 columns, one bad input field 58925 60124 1170 0.0 58925.0 2.0X
Select 100 columns, corrupt record field 72072 72334 252 0.0 72071.7 1.6X
OpenJDK 64-Bit Server VM 1.8.0_312-b07 on Linux 5.11.0-1022-azure
Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
Count a dataset with 10 columns: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Select 10 columns + count() 15067 15151 79 0.7 1506.7 1.0X
Select 1 column + count() 9479 9729 287 1.1 947.9 1.6X
count() 3445 3568 119 2.9 344.5 4.4X
OpenJDK 64-Bit Server VM 1.8.0_312-b07 on Linux 5.11.0-1022-azure
Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
Write dates and timestamps: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Create a dataset of timestamps 1512 1517 5 6.6 151.2 1.0X
to_csv(timestamp) 12346 12393 45 0.8 1234.6 0.1X
write timestamps to files 10548 10694 240 0.9 1054.8 0.1X
Create a dataset of dates 1677 1704 24 6.0 167.7 0.9X
to_csv(date) 7870 7949 136 1.3 787.0 0.2X
write dates to files 5983 6038 66 1.7 598.3 0.3X
OpenJDK 64-Bit Server VM 1.8.0_312-b07 on Linux 5.11.0-1022-azure
Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
Read dates and timestamps: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
read timestamp text from files 2067 2077 10 4.8 206.7 1.0X
read timestamps from files 27461 27701 218 0.4 2746.1 0.1X
infer timestamps from files 56941 57246 352 0.2 5694.1 0.0X
read date text from files 1879 1907 27 5.3 187.9 1.1X
read date from files 10322 10555 309 1.0 1032.2 0.2X
infer date from files 22640 22971 475 0.4 2264.0 0.1X
timestamp strings 3078 3140 62 3.2 307.8 0.7X
parse timestamps from Dataset[String] 29809 30225 449 0.3 2980.9 0.1X
infer timestamps from Dataset[String] 60556 60743 162 0.2 6055.6 0.0X
date strings 3222 3346 114 3.1 322.2 0.6X
parse dates from Dataset[String] 12721 12788 91 0.8 1272.1 0.2X
from_csv(timestamp) 28171 28242 62 0.4 2817.1 0.1X
from_csv(date) 10887 11094 287 0.9 1088.7 0.2X
OpenJDK 64-Bit Server VM 1.8.0_312-b07 on Linux 5.11.0-1022-azure
Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
Filters pushdown: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
w/o filters 19355 19895 474 0.0 193548.5 1.0X
pushdown disabled 19258 19465 343 0.0 192582.3 1.0X
w/ filters 1458 1479 26 0.1 14583.2 13.3X