/
CoalescedRDDBenchmark-results.txt
40 lines (37 loc) · 4.23 KB
/
CoalescedRDDBenchmark-results.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
================================================================================================
Coalesced RDD , large scale
================================================================================================
OpenJDK 64-Bit Server VM 1.8.0_312-b07 on Linux 5.11.0-1022-azure
Intel(R) Xeon(R) CPU E5-2673 v4 @ 2.30GHz
Coalesced RDD: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
----------------------------------------------------------------------------------------------------------------------------
Coalesce Num Partitions: 100 Num Hosts: 1 409 479 115 0.2 4091.1 1.0X
Coalesce Num Partitions: 100 Num Hosts: 5 295 312 28 0.3 2945.5 1.4X
Coalesce Num Partitions: 100 Num Hosts: 10 293 321 26 0.3 2932.2 1.4X
Coalesce Num Partitions: 100 Num Hosts: 20 284 298 25 0.4 2839.6 1.4X
Coalesce Num Partitions: 100 Num Hosts: 40 275 289 20 0.4 2745.5 1.5X
Coalesce Num Partitions: 100 Num Hosts: 80 257 272 21 0.4 2574.1 1.6X
Coalesce Num Partitions: 500 Num Hosts: 1 897 937 57 0.1 8965.2 0.5X
Coalesce Num Partitions: 500 Num Hosts: 5 386 405 17 0.3 3863.1 1.1X
Coalesce Num Partitions: 500 Num Hosts: 10 329 354 26 0.3 3288.6 1.2X
Coalesce Num Partitions: 500 Num Hosts: 20 323 334 16 0.3 3230.1 1.3X
Coalesce Num Partitions: 500 Num Hosts: 40 312 315 3 0.3 3117.4 1.3X
Coalesce Num Partitions: 500 Num Hosts: 80 306 325 17 0.3 3059.2 1.3X
Coalesce Num Partitions: 1000 Num Hosts: 1 1505 1510 6 0.1 15047.4 0.3X
Coalesce Num Partitions: 1000 Num Hosts: 5 498 512 14 0.2 4984.4 0.8X
Coalesce Num Partitions: 1000 Num Hosts: 10 385 407 28 0.3 3850.7 1.1X
Coalesce Num Partitions: 1000 Num Hosts: 20 333 355 30 0.3 3332.9 1.2X
Coalesce Num Partitions: 1000 Num Hosts: 40 305 326 31 0.3 3052.4 1.3X
Coalesce Num Partitions: 1000 Num Hosts: 80 302 312 16 0.3 3024.8 1.4X
Coalesce Num Partitions: 5000 Num Hosts: 1 6444 6484 55 0.0 64443.7 0.1X
Coalesce Num Partitions: 5000 Num Hosts: 5 2014 2112 85 0.0 20141.6 0.2X
Coalesce Num Partitions: 5000 Num Hosts: 10 1181 1198 20 0.1 11805.9 0.3X
Coalesce Num Partitions: 5000 Num Hosts: 20 747 778 29 0.1 7471.8 0.5X
Coalesce Num Partitions: 5000 Num Hosts: 40 506 509 5 0.2 5058.4 0.8X
Coalesce Num Partitions: 5000 Num Hosts: 80 408 415 11 0.2 4082.7 1.0X
Coalesce Num Partitions: 10000 Num Hosts: 1 11825 11980 263 0.0 118254.8 0.0X
Coalesce Num Partitions: 10000 Num Hosts: 5 3905 3979 116 0.0 39050.3 0.1X
Coalesce Num Partitions: 10000 Num Hosts: 10 2116 2159 44 0.0 21159.9 0.2X
Coalesce Num Partitions: 10000 Num Hosts: 20 1199 1221 19 0.1 11992.1 0.3X
Coalesce Num Partitions: 10000 Num Hosts: 40 745 762 15 0.1 7447.8 0.5X
Coalesce Num Partitions: 10000 Num Hosts: 80 522 537 18 0.2 5218.0 0.8X