Fix API usage

[hyc]

Your read tests didn't use read-only TXNs, so they showed no scaling for parallel reads. With these fixed, on my AMD quadcore laptop I get:

Wrote 10000 items in 0.012149 sec, 823113 ops/s
Write Parallel [2] 10000 items in 0.011838 sec, 844737 ops/s
Write Parallel [4] 10000 items in 0.021464 sec, 465896 ops/s
Write Parallel [8] 10000 items in 0.017587 sec, 568601 ops/s
Write Parallel [16] 10000 items in 0.015424 sec, 648340 ops/s
Read 10000 items in 0.004716 sec, 2120441 ops/s
Read Parallel [2] 20000 items in 0.012886 sec, 1552072 ops/s
Read Parallel [4] 40000 items in 0.014793 sec, 2703981 ops/s
Read Parallel [8] 80000 items in 0.016813 sec, 4758222 ops/s
Read Parallel [16] 160000 items in 0.030883 sec, 5180843 ops/s
Wrote 10000 items in 0.016397 sec, 609867 ops/s
Read 10000 items in 0.004939 sec, 2024701 ops/s
Wrote 10000 items in 0.347584 sec, 28770 ops/s
Read 10000 items in 0.003645 sec, 2743484 ops/s
Wrote 10000 items in 0.356129 sec, 28079 ops/s
Read 10000 items in 0.003839 sec, 2604845 ops/s

[hyc]

Original results on my Intel quadcore laptop Q9300 2.53GHz CPU:

Wrote 10000 items in 0.015838 sec, 631392 ops/s
Write Parallel [2] 10000 items in 0.015643 sec, 639263 ops/s
Write Parallel [4] 10000 items in 0.015856 sec, 630676 ops/s
Write Parallel [8] 10000 items in 0.016044 sec, 623285 ops/s
Write Parallel [16] 10000 items in 0.017219 sec, 580753 ops/s
Read 10000 items in 0.004208 sec, 2376425 ops/s
Read Parallel [2] 20000 items in 0.011748 sec, 1702417 ops/s
Read Parallel [4] 40000 items in 0.02576 sec, 1552795 ops/s
Read Parallel [8] 80000 items in 0.044986 sec, 1778331 ops/s
Read Parallel [16] 160000 items in 0.097731 sec, 1637146 ops/s
Wrote 10000 items in 0.01756 sec, 569476 ops/s
Read 10000 items in 0.004266 sec, 2344116 ops/s
Wrote 10000 items in 0.416034 sec, 24036 ops/s
Read 10000 items in 0.003571 sec, 2800336 ops/s
Wrote 10000 items in 0.419614 sec, 23831 ops/s
Read 10000 items in 0.003636 sec, 2750275 ops/s

New results:

Wrote 10000 items in 0.004874 sec, 2051702 ops/s
Write Parallel [2] 10000 items in 0.005062 sec, 1975503 ops/s
Write Parallel [4] 10000 items in 0.00499 sec, 2004008 ops/s
Write Parallel [8] 10000 items in 0.004967 sec, 2013287 ops/s
Write Parallel [16] 10000 items in 0.003112 sec, 3213367 ops/s
Read 10000 items in 0.00307 sec, 3257328 ops/s
Read Parallel [2] 20000 items in 0.005373 sec, 3722315 ops/s
Read Parallel [4] 40000 items in 0.004905 sec, 8154943 ops/s
Read Parallel [8] 80000 items in 0.010379 sec, 7707871 ops/s
Read Parallel [16] 160000 items in 0.01687 sec, 9484291 ops/s
Wrote 10000 items in 0.01197 sec, 835421 ops/s
Read 10000 items in 0.003727 sec, 2683123 ops/s
Wrote 10000 items in 0.408144 sec, 24501 ops/s
Read 10000 items in 0.002571 sec, 3889537 ops/s
Wrote 10000 items in 0.44082 sec, 22684 ops/s
Read 10000 items in 0.003021 sec, 3310162 ops/s

[hyc]

Updated results on my AMD laptop with A10-4600M APU
Old:

Wrote 10000 items in 0.011976 sec, 835003 ops/s
Write Parallel [2] 10000 items in 0.017238 sec, 580113 ops/s
Write Parallel [4] 10000 items in 0.012007 sec, 832847 ops/s
Write Parallel [8] 10000 items in 0.015422 sec, 648424 ops/s
Write Parallel [16] 10000 items in 0.014966 sec, 668181 ops/s
Read 10000 items in 0.004375 sec, 2285714 ops/s
Read Parallel [2] 20000 items in 0.013256 sec, 1508750 ops/s
Read Parallel [4] 40000 items in 0.027275 sec, 1466544 ops/s
Read Parallel [8] 80000 items in 0.059413 sec, 1346506 ops/s
Read Parallel [16] 160000 items in 0.123994 sec, 1290385 ops/s
Wrote 10000 items in 0.011965 sec, 835770 ops/s
Read 10000 items in 0.005158 sec, 1938735 ops/s
Wrote 10000 items in 0.345668 sec, 28929 ops/s
Read 10000 items in 0.003629 sec, 2755580 ops/s
Wrote 10000 items in 0.343342 sec, 29125 ops/s
Read 10000 items in 0.003956 sec, 2527805 ops/s

New:

Wrote 10000 items in 0.002411 sec, 4147656 ops/s
Write Parallel [2] 10000 items in 0.002378 sec, 4205214 ops/s
Write Parallel [4] 10000 items in 0.002313 sec, 4323389 ops/s
Write Parallel [8] 10000 items in 0.002424 sec, 4125412 ops/s
Write Parallel [16] 10000 items in 0.002387 sec, 4189359 ops/s
Read 10000 items in 0.002968 sec, 3369272 ops/s
Read Parallel [2] 20000 items in 0.011021 sec, 1814717 ops/s
Read Parallel [4] 40000 items in 0.00949 sec, 4214963 ops/s
Read Parallel [8] 80000 items in 0.013322 sec, 6005104 ops/s
Read Parallel [16] 160000 items in 0.022812 sec, 7013852 ops/s
Wrote 10000 items in 0.014251 sec, 701705 ops/s
Read 10000 items in 0.00438 sec, 2283105 ops/s
Wrote 10000 items in 0.343272 sec, 29131 ops/s
Read 10000 items in 0.002631 sec, 3800836 ops/s
Wrote 10000 items in 0.352845 sec, 28341 ops/s
Read 10000 items in 0.003508 sec, 2850627 ops/s

The parallel write results are as expected - write speed is constant regardless of number of threads, since LMDB writes are fully serialized. Read speed is a bit weird here, since it's only a quadcore CPU it shouldn't get any faster past 4 threads. But I have a feeling that dynamic CPU frequency control is showing its effect here; I didn't lock the CPU down to a fixed speed when running the test.

[ayende]

Thanks for that, pulled.

Oren Eini
CEO
Hibernating Rhinos
Cellular: +972-52-548-6969
Office: +972-4-674-7811
Fax: +972-153-4622-7811

On Tue, Dec 3, 2013 at 10:19 PM, hyc notifications@github.com wrote:

Your read tests didn't use read-only TXNs, so they showed no scaling for
parallel reads. With these fixed, on my AMD quadcore laptop I get:

Wrote 10000 items in 0.012149 sec, 823113 ops/s
Write Parallel [2] 10000 items in 0.011838 sec, 844737 ops/s
Write Parallel [4] 10000 items in 0.021464 sec, 465896 ops/s
Write Parallel [8] 10000 items in 0.017587 sec, 568601 ops/s
Write Parallel [16] 10000 items in 0.015424 sec, 648340 ops/s
Read 10000 items in 0.004716 sec, 2120441 ops/s
Read Parallel [2] 20000 items in 0.012886 sec, 1552072 ops/s
Read Parallel [4] 40000 items in 0.014793 sec, 2703981 ops/s
Read Parallel [8] 80000 items in 0.016813 sec, 4758222 ops/s
Read Parallel [16] 160000 items in 0.030883 sec, 5180843 ops/s
Wrote 10000 items in 0.016397 sec, 609867 ops/s
Read 10000 items in 0.004939 sec, 2024701 ops/s
Wrote 10000 items in 0.347584 sec, 28770 ops/s
Read 10000 items in 0.003645 sec, 2743484 ops/s
Wrote 10000 items in 0.356129 sec, 28079 ops/s

Read 10000 items in 0.003839 sec, 2604845 ops/s

You can merge this Pull Request by running

git pull https://github.com/hyc/raven.voron voron

Or view, comment on, or merge it at:

#9
Commit Summary

• Fix API usage

File Changes

• M Performance.Comparison/Other/LMDB/lmdb_perf_test.cpphttps://github.com/Fix API usage #9/files#diff-0(41)

Patch Links:

• https://github.com/ayende/raven.voron/pull/9.patch
• https://github.com/ayende/raven.voron/pull/9.diff

[hyc]

(just for reference, I ran the LevelDB test using HyperLevelDB on my AMD laptop)

Write 100000 items in 0.371018 sec, 269528 ops/s
Write Parallel [2] 100000 items in 0.377052 sec, 265215 ops/s
Write Parallel [4] 100000 items in 0.382794 sec, 261237 ops/s
Write Parallel [8] 100000 items in 0.376744 sec, 265432 ops/s
Write Parallel [16] 100000 items in 0.380318 sec, 262937 ops/s
Read 100000 items in 0.431378 sec, 231815 ops/s
Read Parallel [2] 200000 items in 0.724038 sec, 276228 ops/s
Read Parallel [4] 400000 items in 1.12518 sec, 355499 ops/s
Read Parallel [8] 800000 items in 2.20392 sec, 362988 ops/s
Read Parallel [16] 1600000 items in 4.42898 sec, 361257 ops/s
Write 100000 items in 0.441429 sec, 226536 ops/s
Read 100000 items in 1.08308 sec, 92329 ops/s
Write 100000 items in 31.4589 sec, 3178 ops/s
Read 100000 items in 2.42433 sec, 41248 ops/s
Write 100000 items in 33.0292 sec, 3027 ops/s
Read 100000 items in 24.3814 sec, 4101 ops/s

I see basically no scaling with threads/CPUs for reads or writes.

[ayende]

Yes, that pretty much how we found it to be. Especially on reads.

Oren Eini
CEO
Hibernating Rhinos
Cellular: +972-52-548-6969
Office: +972-4-674-7811
Fax: +972-153-4622-7811

On Wed, Dec 4, 2013 at 1:59 AM, hyc notifications@github.com wrote:

(just for reference, I ran the LevelDB test using HyperLevelDB on my AMD
laptop)

Write 100000 items in 0.371018 sec, 269528 ops/s
Write Parallel [2] 100000 items in 0.377052 sec, 265215 ops/s
Write Parallel [4] 100000 items in 0.382794 sec, 261237 ops/s
Write Parallel [8] 100000 items in 0.376744 sec, 265432 ops/s
Write Parallel [16] 100000 items in 0.380318 sec, 262937 ops/s
Read 100000 items in 0.431378 sec, 231815 ops/s
Read Parallel [2] 200000 items in 0.724038 sec, 276228 ops/s
Read Parallel [4] 400000 items in 1.12518 sec, 355499 ops/s
Read Parallel [8] 800000 items in 2.20392 sec, 362988 ops/s
Read Parallel [16] 1600000 items in 4.42898 sec, 361257 ops/s
Write 100000 items in 0.441429 sec, 226536 ops/s
Read 100000 items in 1.08308 sec, 92329 ops/s
Write 100000 items in 31.4589 sec, 3178 ops/s
Read 100000 items in 2.42433 sec, 41248 ops/s
Write 100000 items in 33.0292 sec, 3027 ops/s
Read 100000 items in 24.3814 sec, 4101 ops/s

I see basically no scaling with threads/CPUs for reads or writes.

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/9#issuecomment-29764803
.

[hyc]

Just for reference, I reran with CPU speed locked to maximum - 2.53GHz for Intel, 2.3GHz for AMD.

Intel:

Wrote 10000 items in 0.003066 sec, 3261578 ops/s
Write Parallel [2] 10000 items in 0.003053 sec, 3275466 ops/s
Write Parallel [4] 10000 items in 0.002991 sec, 3343363 ops/s
Write Parallel [8] 10000 items in 0.003175 sec, 3149606 ops/s
Write Parallel [16] 10000 items in 0.003134 sec, 3190810 ops/s
Read 10000 items in 0.001443 sec, 6930006 ops/s
Read Parallel [2] 20000 items in 0.002323 sec, 8609556 ops/s
Read Parallel [4] 40000 items in 0.00331 sec, 12084592 ops/s
Read Parallel [8] 80000 items in 0.004939 sec, 16197610 ops/s
Read Parallel [16] 160000 items in 0.008534 sec, 18748535 ops/s
Wrote 10000 items in 0.010263 sec, 974373 ops/s
Read 10000 items in 0.003854 sec, 2594706 ops/s
Wrote 10000 items in 0.402111 sec, 24868 ops/s
Read 10000 items in 0.001558 sec, 6418485 ops/s
Wrote 10000 items in 0.417092 sec, 23975 ops/s
Read 10000 items in 0.002995 sec, 3338898 ops/s

AMD:

Wrote 10000 items in 0.002444 sec, 4091653 ops/s
Write Parallel [2] 10000 items in 0.002458 sec, 4068348 ops/s
Write Parallel [4] 10000 items in 0.002396 sec, 4173622 ops/s
Write Parallel [8] 10000 items in 0.002397 sec, 4171881 ops/s
Write Parallel [16] 10000 items in 0.00248 sec, 4032258 ops/s
Read 10000 items in 0.001668 sec, 5995203 ops/s
Read Parallel [2] 20000 items in 0.003321 sec, 6022282 ops/s
Read Parallel [4] 40000 items in 0.003321 sec, 12044564 ops/s
Read Parallel [8] 80000 items in 0.004874 sec, 16413623 ops/s
Read Parallel [16] 160000 items in 0.009871 sec, 16209097 ops/s
Wrote 10000 items in 0.011802 sec, 847314 ops/s
Read 10000 items in 0.004705 sec, 2125398 ops/s
Wrote 10000 items in 0.345979 sec, 28903 ops/s
Read 10000 items in 0.001811 sec, 5521811 ops/s
Wrote 10000 items in 0.352833 sec, 28342 ops/s
Read 10000 items in 0.003541 sec, 2824060 ops/s

Gives a clearer picture of read scaling.

[ayende]

Hm... this is the LevelDB benchmark, right?
I was surprised about the write values, until I realized that the 10,000
writes are probably just all entirely buffered in memory and it is doing
async writes.

Oren Eini
CEO
Hibernating Rhinos
Cellular: +972-52-548-6969
Office: +972-4-674-7811
Fax: +972-153-4622-7811

On Wed, Dec 4, 2013 at 7:19 AM, hyc notifications@github.com wrote:

Just for reference, I reran with CPU speed locked to maximum - 2.53GHz for
Intel, 2.3GHz for AMD.

Intel:

Wrote 10000 items in 0.003066 sec, 3261578 ops/s
Write Parallel [2] 10000 items in 0.003053 sec, 3275466 ops/s
Write Parallel [4] 10000 items in 0.002991 sec, 3343363 ops/s
Write Parallel [8] 10000 items in 0.003175 sec, 3149606 ops/s
Write Parallel [16] 10000 items in 0.003134 sec, 3190810 ops/s
Read 10000 items in 0.001443 sec, 6930006 ops/s
Read Parallel [2] 20000 items in 0.002323 sec, 8609556 ops/s
Read Parallel [4] 40000 items in 0.00331 sec, 12084592 ops/s
Read Parallel [8] 80000 items in 0.004939 sec, 16197610 ops/s
Read Parallel [16] 160000 items in 0.008534 sec, 18748535 ops/s
Wrote 10000 items in 0.010263 sec, 974373 ops/s
Read 10000 items in 0.003854 sec, 2594706 ops/s
Wrote 10000 items in 0.402111 sec, 24868 ops/s
Read 10000 items in 0.001558 sec, 6418485 ops/s
Wrote 10000 items in 0.417092 sec, 23975 ops/s
Read 10000 items in 0.002995 sec, 3338898 ops/s

AMD:

Wrote 10000 items in 0.002444 sec, 4091653 ops/s
Write Parallel [2] 10000 items in 0.002458 sec, 4068348 ops/s
Write Parallel [4] 10000 items in 0.002396 sec, 4173622 ops/s
Write Parallel [8] 10000 items in 0.002397 sec, 4171881 ops/s
Write Parallel [16] 10000 items in 0.00248 sec, 4032258 ops/s
Read 10000 items in 0.001668 sec, 5995203 ops/s
Read Parallel [2] 20000 items in 0.003321 sec, 6022282 ops/s
Read Parallel [4] 40000 items in 0.003321 sec, 12044564 ops/s
Read Parallel [8] 80000 items in 0.004874 sec, 16413623 ops/s
Read Parallel [16] 160000 items in 0.009871 sec, 16209097 ops/s
Wrote 10000 items in 0.011802 sec, 847314 ops/s
Read 10000 items in 0.004705 sec, 2125398 ops/s
Wrote 10000 items in 0.345979 sec, 28903 ops/s
Read 10000 items in 0.001811 sec, 5521811 ops/s
Wrote 10000 items in 0.352833 sec, 28342 ops/s
Read 10000 items in 0.003541 sec, 2824060 ops/s

Gives a clearer picture of read scaling.

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/9#issuecomment-29780359
.

[hyc]

No sorry for the ambiguity, my last update was rerunning the LMDB bench.