benchmark.rst

orphan:
nositemap:

Tarantool - Benchmarks (YCSB)

.. wp_section::
    :class: b-block-lightgray b-benchmark_top b-clearbox
    :title: YCSB - Benchmark results

.. wp_section::
    :class: b-block b-benchmark b-benchmark_page
    :title: Preface

    There are lies, then there is statistics, but the first place in
    misrepresenting the truth is undoubtedly owned by benchmarks.

    Comparing Tarantool with other systems, apples to apples, is not strictly
    correct: the server networking subsystem is fully asynchronous and it's
    possible to proxy all clients via a single socket. In this case, responses
    to queries are sent as soon as they are ready. Most production applications
    use asynchronous and batched I/O with Tarantool.

    As long as the overhead of system calls and context switches is the single
    largest contributor to the cost of serving a single request, use of batched
    and multiplexed I/O produces an order of magnitude better results, when
    compared with traditional multi-threaded workloads. A tool we developed for
    our own use, `nosqlbench <http://github.com/tarantool/nosqlbench>`_,
    utilizes this approach at full.

    However, to compare with the rest of the world, a standardized benchmarking
    kit is more appropriate. This is why `Yahoo! Cloud Serving Benchmark |copy|
    <http://github.com/brianfrankcooper/YCSB>`_ was used to produce the charts
    below. Since YCSB was developed to compare cloud key/value servers, it
    provides a very narrow view at performance of a tested server. For example,
    performance of secondary keys or overhead of locking (which Tarantool
    doesn't have) is not tested at all.

.. wp_section::
    :class: b-block b-benchmark b-benchmark_page
    :title: What is YCSB

    Yahoo! Cloud Serving Benchmark (c) consists of two components:

    - the client, which generates the load according to a workload type and
      analyzes latency and throughput,
    - workload files, which define a single benchmark by describing the size of
      the data set, the total amount of requests, the ratio of read and write
      queries.

    There are 6 major workload types in YCSB:

    - workload **A**, 50/50 update/read ratio, size of the data set is 2 000 000
      key/value pairs,
    - workload **B**, 5/95 update/read ratio, the same size of the data set,
    - workload **C** is 100% read-only,
    - workload **D** 5/95 insert/read ratio, the read load is skewed towards the
      end of the key range,
    - workload **E**, 5/95 ratio of insert/reads over a range of 10 records,
    - workload **F**, 95% read/modify/write, 5% read.

    For additional information on YCSB and workload types, please visit
    `YCSB official page on Github <http://github.com/brianfrankcooper/YCSB>`_.

    All charts below were measured using 5M queries per test, averaged over 8
    consecutive test runs.

    Configuration files for the tested systems can be found `on its Github page
    <https://github.com/bigbes/autoycsb/tree/master/configs>`_.

    .. raw:: html


        <div class="b-block-wrapper_bench">
          <div class="b-benchmark-catalog">
            <ul class="b-switcher">
              <li class="b-switcher-item"><a href="#" class="b-switcher-item-url p-active">A</a></li>
              <li class="b-switcher-item"><a href="#" class="b-switcher-item-url">B</a></li>
              <li class="b-switcher-item"><a href="#" class="b-switcher-item-url">C</a></li>
              <li class="b-switcher-item"><a href="#" class="b-switcher-item-url">D</a></li>
              <li class="b-switcher-item"><a href="#" class="b-switcher-item-url">E</a></li>
              <li class="b-switcher-item"><a href="#" class="b-switcher-item-url">F</a></li>
              <li class="b-switcher-item"><a href="#" class="b-switcher-item-url">MEM</a></li>
            </ul>
          </div>
          <div class="b-benchmark-type">
            <h3 class="b-benchmark-type-title">Workload A</h3>
          </div>
        </div>
        <div class="b-benchmark-grapf"><div id="b-benchmark-grapf-image_1"></div></div>
        <div class="b-block-wrapper_bench">
          <div class="b-benchmark-type">
            <ul class="b-switcher">
              <li class="b-switcher-item">
                <a href="#" class="b-switcher-item-url p-active">READ Latency</a>
              </li>
              <li class="b-switcher-item">
                <a href="#" class="b-switcher-item-url">UPDATE Latency</a>
              </li>
            </ul>
          </div>
        </div>
        <div class="b-benchmark-grapf"><div id="b-benchmark-grapf-image_2"></div></div>