-
Notifications
You must be signed in to change notification settings - Fork 28k
/
DataSourceReadBenchmark-results.txt
431 lines (385 loc) · 42.4 KB
/
DataSourceReadBenchmark-results.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
================================================================================================
SQL Single Numeric Column Scan
================================================================================================
OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1031-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
SQL Single BOOLEAN Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
SQL CSV 13143 13363 311 1.2 835.6 1.0X
SQL Json 7721 7787 93 2.0 490.9 1.7X
SQL Parquet Vectorized: DataPageV1 110 128 18 143.6 7.0 120.0X
SQL Parquet Vectorized: DataPageV2 90 103 14 175.1 5.7 146.3X
SQL Parquet MR: DataPageV1 1785 1810 35 8.8 113.5 7.4X
SQL Parquet MR: DataPageV2 1554 1557 5 10.1 98.8 8.5X
SQL ORC Vectorized 175 180 4 89.9 11.1 75.2X
SQL ORC MR 1585 1604 27 9.9 100.8 8.3X
OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1031-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
Parquet Reader Single BOOLEAN Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
---------------------------------------------------------------------------------------------------------------------------
ParquetReader Vectorized: DataPageV1 116 117 2 136.1 7.3 1.0X
ParquetReader Vectorized: DataPageV2 110 112 3 142.9 7.0 1.0X
ParquetReader Vectorized -> Row: DataPageV1 45 46 1 347.0 2.9 2.5X
ParquetReader Vectorized -> Row: DataPageV2 40 40 1 394.4 2.5 2.9X
OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1031-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
SQL Single TINYINT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
SQL CSV 15823 15829 8 1.0 1006.0 1.0X
SQL Json 8823 8824 1 1.8 560.9 1.8X
SQL Parquet Vectorized: DataPageV1 142 149 7 110.9 9.0 111.5X
SQL Parquet Vectorized: DataPageV2 140 145 11 112.1 8.9 112.8X
SQL Parquet MR: DataPageV1 1965 1979 20 8.0 124.9 8.1X
SQL Parquet MR: DataPageV2 1833 1837 7 8.6 116.5 8.6X
SQL ORC Vectorized 147 153 7 106.8 9.4 107.4X
SQL ORC MR 1437 1438 2 10.9 91.3 11.0X
OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1031-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
Parquet Reader Single TINYINT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
---------------------------------------------------------------------------------------------------------------------------
ParquetReader Vectorized: DataPageV1 194 197 6 81.0 12.3 1.0X
ParquetReader Vectorized: DataPageV2 194 196 3 80.9 12.4 1.0X
ParquetReader Vectorized -> Row: DataPageV1 183 183 0 86.2 11.6 1.1X
ParquetReader Vectorized -> Row: DataPageV2 182 183 0 86.5 11.6 1.1X
OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1031-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
SQL Single SMALLINT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
SQL CSV 16581 16592 16 0.9 1054.2 1.0X
SQL Json 9305 9308 5 1.7 591.6 1.8X
SQL Parquet Vectorized: DataPageV1 200 227 68 78.8 12.7 83.1X
SQL Parquet Vectorized: DataPageV2 179 187 11 87.7 11.4 92.5X
SQL Parquet MR: DataPageV1 2270 2282 18 6.9 144.3 7.3X
SQL Parquet MR: DataPageV2 1945 1947 3 8.1 123.7 8.5X
SQL ORC Vectorized 176 180 3 89.2 11.2 94.0X
SQL ORC MR 1647 1649 3 9.6 104.7 10.1X
OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1031-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
Parquet Reader Single SMALLINT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
---------------------------------------------------------------------------------------------------------------------------
ParquetReader Vectorized: DataPageV1 244 247 5 64.5 15.5 1.0X
ParquetReader Vectorized: DataPageV2 266 268 4 59.1 16.9 0.9X
ParquetReader Vectorized -> Row: DataPageV1 229 231 5 68.8 14.5 1.1X
ParquetReader Vectorized -> Row: DataPageV2 250 251 1 62.9 15.9 1.0X
OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1031-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
SQL Single INT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
SQL CSV 18059 18090 44 0.9 1148.2 1.0X
SQL Json 9790 9791 1 1.6 622.5 1.8X
SQL Parquet Vectorized: DataPageV1 144 150 7 109.2 9.2 125.4X
SQL Parquet Vectorized: DataPageV2 260 266 13 60.6 16.5 69.6X
SQL Parquet MR: DataPageV1 2241 2263 31 7.0 142.5 8.1X
SQL Parquet MR: DataPageV2 1984 1991 10 7.9 126.2 9.1X
SQL ORC Vectorized 242 249 7 64.9 15.4 74.6X
SQL ORC MR 1693 1700 9 9.3 107.7 10.7X
OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1031-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
Parquet Reader Single INT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
---------------------------------------------------------------------------------------------------------------------------
ParquetReader Vectorized: DataPageV1 247 253 9 63.7 15.7 1.0X
ParquetReader Vectorized: DataPageV2 360 365 6 43.7 22.9 0.7X
ParquetReader Vectorized -> Row: DataPageV1 212 220 9 74.1 13.5 1.2X
ParquetReader Vectorized -> Row: DataPageV2 327 329 3 48.0 20.8 0.8X
OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1031-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
SQL Single BIGINT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
SQL CSV 23621 23622 2 0.7 1501.8 1.0X
SQL Json 12398 12402 5 1.3 788.3 1.9X
SQL Parquet Vectorized: DataPageV1 219 226 10 71.8 13.9 107.9X
SQL Parquet Vectorized: DataPageV2 379 385 9 41.5 24.1 62.4X
SQL Parquet MR: DataPageV1 2319 2338 27 6.8 147.5 10.2X
SQL Parquet MR: DataPageV2 2066 2079 19 7.6 131.4 11.4X
SQL ORC Vectorized 298 341 93 52.8 19.0 79.2X
SQL ORC MR 1844 1844 0 8.5 117.2 12.8X
OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1031-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
Parquet Reader Single BIGINT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
---------------------------------------------------------------------------------------------------------------------------
ParquetReader Vectorized: DataPageV1 294 327 77 53.4 18.7 1.0X
ParquetReader Vectorized: DataPageV2 471 479 15 33.4 30.0 0.6X
ParquetReader Vectorized -> Row: DataPageV1 276 278 4 57.0 17.5 1.1X
ParquetReader Vectorized -> Row: DataPageV2 454 460 11 34.6 28.9 0.6X
OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1031-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
SQL Single FLOAT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
SQL CSV 19058 19073 21 0.8 1211.7 1.0X
SQL Json 12557 12578 29 1.3 798.4 1.5X
SQL Parquet Vectorized: DataPageV1 145 150 6 108.7 9.2 131.8X
SQL Parquet Vectorized: DataPageV2 145 151 9 108.7 9.2 131.7X
SQL Parquet MR: DataPageV1 2197 2199 3 7.2 139.7 8.7X
SQL Parquet MR: DataPageV2 2051 2060 13 7.7 130.4 9.3X
SQL ORC Vectorized 314 318 3 50.0 20.0 60.6X
SQL ORC MR 1737 1742 6 9.1 110.5 11.0X
OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1031-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
Parquet Reader Single FLOAT Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
---------------------------------------------------------------------------------------------------------------------------
ParquetReader Vectorized: DataPageV1 241 244 6 65.4 15.3 1.0X
ParquetReader Vectorized: DataPageV2 240 243 5 65.4 15.3 1.0X
ParquetReader Vectorized -> Row: DataPageV1 213 214 3 74.0 13.5 1.1X
ParquetReader Vectorized -> Row: DataPageV2 212 217 8 74.1 13.5 1.1X
OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1031-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
SQL Single DOUBLE Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
SQL CSV 24573 24625 74 0.6 1562.3 1.0X
SQL Json 16677 16680 5 0.9 1060.3 1.5X
SQL Parquet Vectorized: DataPageV1 209 216 10 75.3 13.3 117.6X
SQL Parquet Vectorized: DataPageV2 208 217 9 75.4 13.3 117.9X
SQL Parquet MR: DataPageV1 2287 2303 23 6.9 145.4 10.7X
SQL Parquet MR: DataPageV2 2153 2182 42 7.3 136.9 11.4X
SQL ORC Vectorized 397 401 4 39.6 25.2 61.9X
SQL ORC MR 1857 1875 25 8.5 118.1 13.2X
OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1031-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
Parquet Reader Single DOUBLE Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
---------------------------------------------------------------------------------------------------------------------------
ParquetReader Vectorized: DataPageV1 291 299 12 54.0 18.5 1.0X
ParquetReader Vectorized: DataPageV2 291 301 13 54.0 18.5 1.0X
ParquetReader Vectorized -> Row: DataPageV1 274 278 5 57.3 17.4 1.1X
ParquetReader Vectorized -> Row: DataPageV2 274 275 4 57.5 17.4 1.1X
================================================================================================
SQL Single Numeric Column Scan in Struct
================================================================================================
OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1031-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
SQL Single TINYINT Column Scan in Struct: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------------------------
SQL ORC MR 2294 2370 108 6.9 145.8 1.0X
SQL ORC Vectorized (Nested Column Disabled) 2228 2236 10 7.1 141.7 1.0X
SQL ORC Vectorized (Nested Column Enabled) 287 289 1 54.7 18.3 8.0X
SQL Parquet MR: DataPageV1 2342 2352 14 6.7 148.9 1.0X
SQL Parquet Vectorized: DataPageV1 (Nested Column Disabled) 2753 2758 7 5.7 175.0 0.8X
SQL Parquet Vectorized: DataPageV1 (Nested Column Enabled) 304 309 8 51.7 19.3 7.5X
SQL Parquet MR: DataPageV2 2216 2220 6 7.1 140.9 1.0X
SQL Parquet Vectorized: DataPageV2 (Nested Column Disabled) 2625 2625 1 6.0 166.9 0.9X
SQL Parquet Vectorized: DataPageV2 (Nested Column Enabled) 305 312 13 51.6 19.4 7.5X
OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1031-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
SQL Single SMALLINT Column Scan in Struct: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------------------------
SQL ORC MR 2292 2304 17 6.9 145.7 1.0X
SQL ORC Vectorized (Nested Column Disabled) 2295 2306 16 6.9 145.9 1.0X
SQL ORC Vectorized (Nested Column Enabled) 324 329 7 48.6 20.6 7.1X
SQL Parquet MR: DataPageV1 2541 2547 9 6.2 161.5 0.9X
SQL Parquet Vectorized: DataPageV1 (Nested Column Disabled) 3086 3088 2 5.1 196.2 0.7X
SQL Parquet Vectorized: DataPageV1 (Nested Column Enabled) 298 305 9 52.8 18.9 7.7X
SQL Parquet MR: DataPageV2 2334 2339 8 6.7 148.4 1.0X
SQL Parquet Vectorized: DataPageV2 (Nested Column Disabled) 2800 2803 4 5.6 178.0 0.8X
SQL Parquet Vectorized: DataPageV2 (Nested Column Enabled) 416 419 3 37.8 26.5 5.5X
OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1031-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
SQL Single INT Column Scan in Struct: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------------------------
SQL ORC MR 2351 2364 19 6.7 149.5 1.0X
SQL ORC Vectorized (Nested Column Disabled) 2344 2358 20 6.7 149.0 1.0X
SQL ORC Vectorized (Nested Column Enabled) 402 406 4 39.1 25.6 5.8X
SQL Parquet MR: DataPageV1 2572 2574 3 6.1 163.5 0.9X
SQL Parquet Vectorized: DataPageV1 (Nested Column Disabled) 3087 3088 2 5.1 196.3 0.8X
SQL Parquet Vectorized: DataPageV1 (Nested Column Enabled) 282 292 15 55.7 17.9 8.3X
SQL Parquet MR: DataPageV2 2390 2418 40 6.6 152.0 1.0X
SQL Parquet Vectorized: DataPageV2 (Nested Column Disabled) 2846 2870 35 5.5 180.9 0.8X
SQL Parquet Vectorized: DataPageV2 (Nested Column Enabled) 454 461 9 34.6 28.9 5.2X
OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1031-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
SQL Single BIGINT Column Scan in Struct: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------------------------
SQL ORC MR 2470 2472 2 6.4 157.1 1.0X
SQL ORC Vectorized (Nested Column Disabled) 2454 2462 12 6.4 156.0 1.0X
SQL ORC Vectorized (Nested Column Enabled) 446 452 10 35.3 28.4 5.5X
SQL Parquet MR: DataPageV1 2668 2679 15 5.9 169.7 0.9X
SQL Parquet Vectorized: DataPageV1 (Nested Column Disabled) 3169 3171 3 5.0 201.5 0.8X
SQL Parquet Vectorized: DataPageV1 (Nested Column Enabled) 353 378 14 44.6 22.4 7.0X
SQL Parquet MR: DataPageV2 2466 2474 11 6.4 156.8 1.0X
SQL Parquet Vectorized: DataPageV2 (Nested Column Disabled) 2898 2898 1 5.4 184.2 0.9X
SQL Parquet Vectorized: DataPageV2 (Nested Column Enabled) 565 570 6 27.8 36.0 4.4X
OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1031-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
SQL Single FLOAT Column Scan in Struct: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------------------------
SQL ORC MR 2435 2449 20 6.5 154.8 1.0X
SQL ORC Vectorized (Nested Column Disabled) 2458 2467 13 6.4 156.3 1.0X
SQL ORC Vectorized (Nested Column Enabled) 444 458 8 35.4 28.2 5.5X
SQL Parquet MR: DataPageV1 2548 2640 130 6.2 162.0 1.0X
SQL Parquet Vectorized: DataPageV1 (Nested Column Disabled) 2968 2971 5 5.3 188.7 0.8X
SQL Parquet Vectorized: DataPageV1 (Nested Column Enabled) 276 295 28 57.0 17.6 8.8X
SQL Parquet MR: DataPageV2 2402 2406 5 6.5 152.7 1.0X
SQL Parquet Vectorized: DataPageV2 (Nested Column Disabled) 2811 2828 24 5.6 178.7 0.9X
SQL Parquet Vectorized: DataPageV2 (Nested Column Enabled) 278 285 7 56.6 17.7 8.8X
OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1031-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
SQL Single DOUBLE Column Scan in Struct: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------------------------
SQL ORC MR 2540 2542 3 6.2 161.5 1.0X
SQL ORC Vectorized (Nested Column Disabled) 2531 2541 14 6.2 160.9 1.0X
SQL ORC Vectorized (Nested Column Enabled) 542 546 5 29.0 34.4 4.7X
SQL Parquet MR: DataPageV1 2643 2674 44 6.0 168.0 1.0X
SQL Parquet Vectorized: DataPageV1 (Nested Column Disabled) 3178 3195 23 4.9 202.1 0.8X
SQL Parquet Vectorized: DataPageV1 (Nested Column Enabled) 348 353 7 45.1 22.1 7.3X
SQL Parquet MR: DataPageV2 2525 2546 30 6.2 160.5 1.0X
SQL Parquet Vectorized: DataPageV2 (Nested Column Disabled) 3005 3009 5 5.2 191.0 0.8X
SQL Parquet Vectorized: DataPageV2 (Nested Column Enabled) 344 356 14 45.8 21.9 7.4X
================================================================================================
SQL Nested Column Scan
================================================================================================
OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1031-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
SQL Nested Column Scan: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------------------------
SQL ORC MR 23536 23843 176 0.0 22445.8 1.0X
SQL ORC Vectorized (Nested Column Disabled) 23036 23334 127 0.0 21969.1 1.0X
SQL ORC Vectorized (Nested Column Enabled) 8504 8623 122 0.1 8110.1 2.8X
SQL Parquet MR: DataPageV1 13540 13645 122 0.1 12913.0 1.7X
SQL Parquet Vectorized: DataPageV1 (Nested Column Disabled) 14310 14430 123 0.1 13647.3 1.6X
SQL Parquet Vectorized: DataPageV1 (Nested Column Enabled) 9081 9458 677 0.1 8660.8 2.6X
SQL Parquet MR: DataPageV2 16024 16350 380 0.1 15281.4 1.5X
SQL Parquet Vectorized: DataPageV2 (Nested Column Disabled) 16714 16847 111 0.1 15939.8 1.4X
SQL Parquet Vectorized: DataPageV2 (Nested Column Enabled) 7080 7782 744 0.1 6752.4 3.3X
================================================================================================
Int and String Scan
================================================================================================
OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1031-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
Int and String Scan: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
SQL CSV 17078 17133 79 0.6 1628.7 1.0X
SQL Json 11708 11723 21 0.9 1116.5 1.5X
SQL Parquet Vectorized: DataPageV1 2002 2002 1 5.2 190.9 8.5X
SQL Parquet Vectorized: DataPageV2 2313 2325 16 4.5 220.6 7.4X
SQL Parquet MR: DataPageV1 4157 4170 18 2.5 396.5 4.1X
SQL Parquet MR: DataPageV2 4052 4067 22 2.6 386.4 4.2X
SQL ORC Vectorized 1971 1989 25 5.3 188.0 8.7X
SQL ORC MR 3646 3648 3 2.9 347.7 4.7X
================================================================================================
Repeated String Scan
================================================================================================
OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1031-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
Repeated String: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
SQL CSV 9526 9547 30 1.1 908.5 1.0X
SQL Json 6867 6883 23 1.5 654.9 1.4X
SQL Parquet Vectorized: DataPageV1 728 738 15 14.4 69.4 13.1X
SQL Parquet Vectorized: DataPageV2 702 714 12 14.9 67.0 13.6X
SQL Parquet MR: DataPageV1 1877 1887 14 5.6 179.1 5.1X
SQL Parquet MR: DataPageV2 1821 1827 8 5.8 173.7 5.2X
SQL ORC Vectorized 422 426 4 24.9 40.2 22.6X
SQL ORC MR 1838 1849 15 5.7 175.3 5.2X
================================================================================================
Partitioned Table Scan
================================================================================================
OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1031-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
Partitioned Table: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
---------------------------------------------------------------------------------------------------------------------------------
Data column - CSV 23701 23707 10 0.7 1506.9 1.0X
Data column - Json 12457 12521 90 1.3 792.0 1.9X
Data column - Parquet Vectorized: DataPageV1 209 219 11 75.3 13.3 113.5X
Data column - Parquet Vectorized: DataPageV2 424 431 7 37.1 27.0 55.9X
Data column - Parquet MR: DataPageV1 2711 2715 6 5.8 172.4 8.7X
Data column - Parquet MR: DataPageV2 2467 2471 6 6.4 156.8 9.6X
Data column - ORC Vectorized 299 306 8 52.6 19.0 79.3X
Data column - ORC MR 2139 2146 9 7.4 136.0 11.1X
Partition column - CSV 6516 6656 198 2.4 414.3 3.6X
Partition column - Json 9845 9849 5 1.6 625.9 2.4X
Partition column - Parquet Vectorized: DataPageV1 43 49 8 361.9 2.8 545.3X
Partition column - Parquet Vectorized: DataPageV2 43 49 9 367.2 2.7 553.3X
Partition column - Parquet MR: DataPageV1 1380 1389 14 11.4 87.7 17.2X
Partition column - Parquet MR: DataPageV2 1374 1381 11 11.5 87.3 17.3X
Partition column - ORC Vectorized 46 52 11 344.4 2.9 519.0X
Partition column - ORC MR 1378 1378 0 11.4 87.6 17.2X
Both columns - CSV 23758 23771 17 0.7 1510.5 1.0X
Both columns - Json 13246 13293 67 1.2 842.1 1.8X
Both columns - Parquet Vectorized: DataPageV1 248 261 16 63.3 15.8 95.4X
Both columns - Parquet Vectorized: DataPageV2 469 480 12 33.5 29.8 50.5X
Both columns - Parquet MR: DataPageV1 2779 2786 10 5.7 176.7 8.5X
Both columns - Parquet MR: DataPageV2 2533 2548 21 6.2 161.1 9.4X
Both columns - ORC Vectorized 338 340 3 46.5 21.5 70.1X
Both columns - ORC MR 2210 2210 0 7.1 140.5 10.7X
================================================================================================
String with Nulls Scan
================================================================================================
OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1031-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
String with Nulls Scan (0.0%): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
SQL CSV 11462 11576 162 0.9 1093.1 1.0X
SQL Json 10487 10489 3 1.0 1000.1 1.1X
SQL Parquet Vectorized: DataPageV1 1321 1327 9 7.9 126.0 8.7X
SQL Parquet Vectorized: DataPageV2 1689 1691 3 6.2 161.1 6.8X
SQL Parquet MR: DataPageV1 3489 3505 22 3.0 332.8 3.3X
SQL Parquet MR: DataPageV2 4243 4246 4 2.5 404.6 2.7X
ParquetReader Vectorized: DataPageV1 959 964 6 10.9 91.5 11.9X
ParquetReader Vectorized: DataPageV2 1341 1345 5 7.8 127.9 8.5X
SQL ORC Vectorized 962 979 15 10.9 91.8 11.9X
SQL ORC MR 3227 3241 20 3.2 307.7 3.6X
OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1031-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
String with Nulls Scan (50.0%): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
SQL CSV 8004 8008 6 1.3 763.3 1.0X
SQL Json 7827 7843 22 1.3 746.5 1.0X
SQL Parquet Vectorized: DataPageV1 1026 1038 17 10.2 97.8 7.8X
SQL Parquet Vectorized: DataPageV2 1265 1276 15 8.3 120.7 6.3X
SQL Parquet MR: DataPageV1 2738 2749 16 3.8 261.1 2.9X
SQL Parquet MR: DataPageV2 3219 3227 12 3.3 306.9 2.5X
ParquetReader Vectorized: DataPageV1 934 938 5 11.2 89.0 8.6X
ParquetReader Vectorized: DataPageV2 1192 1196 6 8.8 113.7 6.7X
SQL ORC Vectorized 1207 1207 1 8.7 115.1 6.6X
SQL ORC MR 3020 3021 1 3.5 288.0 2.7X
OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1031-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
String with Nulls Scan (95.0%): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
SQL CSV 5075 5082 10 2.1 484.0 1.0X
SQL Json 4602 4604 3 2.3 438.9 1.1X
SQL Parquet Vectorized: DataPageV1 228 232 8 46.0 21.8 22.3X
SQL Parquet Vectorized: DataPageV2 281 287 9 37.3 26.8 18.1X
SQL Parquet MR: DataPageV1 1868 1875 10 5.6 178.1 2.7X
SQL Parquet MR: DataPageV2 1798 1803 8 5.8 171.4 2.8X
ParquetReader Vectorized: DataPageV1 241 242 2 43.6 22.9 21.1X
ParquetReader Vectorized: DataPageV2 290 291 3 36.2 27.6 17.5X
SQL ORC Vectorized 404 411 5 25.9 38.5 12.6X
SQL ORC MR 1584 1585 2 6.6 151.1 3.2X
================================================================================================
Single Column Scan From Wide Columns
================================================================================================
OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1031-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
Single Column Scan from 10 columns: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
SQL CSV 2596 2603 10 0.4 2475.4 1.0X
SQL Json 2935 2961 36 0.4 2799.5 0.9X
SQL Parquet Vectorized: DataPageV1 45 49 7 23.5 42.5 58.3X
SQL Parquet Vectorized: DataPageV2 60 65 7 17.5 57.2 43.3X
SQL Parquet MR: DataPageV1 200 207 8 5.3 190.4 13.0X
SQL Parquet MR: DataPageV2 184 190 5 5.7 175.5 14.1X
SQL ORC Vectorized 52 58 7 20.3 49.2 50.3X
SQL ORC MR 155 159 4 6.8 147.7 16.8X
OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1031-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
Single Column Scan from 50 columns: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
SQL CSV 7659 7670 15 0.1 7304.2 1.0X
SQL Json 11990 12203 300 0.1 11434.9 0.6X
SQL Parquet Vectorized: DataPageV1 63 67 6 16.7 59.8 122.1X
SQL Parquet Vectorized: DataPageV2 75 80 8 13.9 71.9 101.6X
SQL Parquet MR: DataPageV1 218 223 8 4.8 208.1 35.1X
SQL Parquet MR: DataPageV2 205 211 9 5.1 195.2 37.4X
SQL ORC Vectorized 67 73 12 15.7 63.8 114.5X
SQL ORC MR 175 179 3 6.0 167.3 43.7X
OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1031-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
Single Column Scan from 100 columns: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
SQL CSV 13640 13681 58 0.1 13008.1 1.0X
SQL Json 22078 22212 189 0.0 21055.5 0.6X
SQL Parquet Vectorized: DataPageV1 94 101 10 11.2 89.3 145.6X
SQL Parquet Vectorized: DataPageV2 109 119 15 9.6 104.2 124.8X
SQL Parquet MR: DataPageV1 255 266 15 4.1 242.9 53.6X
SQL Parquet MR: DataPageV2 237 242 7 4.4 226.1 57.5X
SQL ORC Vectorized 85 93 12 12.3 81.1 160.5X
SQL ORC MR 198 204 7 5.3 188.8 68.9X