Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create iproto performance test and integrate it into current testing system #5985

Open
EvgenyMekhanik opened this issue Apr 12, 2021 · 0 comments
Labels
feature A new functionality
Milestone

Comments

@EvgenyMekhanik
Copy link
Contributor

EvgenyMekhanik commented Apr 12, 2021

During to work on #5645, I am faced with the necessity to create a performance test for iproto.
Here is a test example:

-- Test starts server with different count of iproto threads, calculates
-- performance and latency of simple iproto calls and saved result in the
-- table format in the file with a name perf.txt.
env = require('test_run')
clock = require('clock')
net_box = require('net.box')
fiber = require('fiber')
test_run = env.new()
test_run:cmd("create server perf with script='perf/iproto.lua'")

test_run:cmd("setopt delimiter ';'")
function latency(bench_count, server_addr, result, threads_count)
     local time_before = clock.monotonic64()
     for _ = 1, bench_count do
        local conn = net_box.new(server_addr)
        pcall(conn.call, conn, 'ping')
     end
     result[threads_count].latency =
         tonumber(clock.monotonic64() - time_before) / (1.0e6 * bench_count)
end;
function benchmark(bench_count, server_addr, result, threads_count)
    local fibers_count = 100
    local fibers = {}
    local time_before = clock.monotonic64()
    for i = 1, fibers_count do
        fibers[i] = fiber.new(function()
            local conn = net_box.new(server_addr)
            for _ = 1, bench_count do
                pcall(conn.call, conn, 'ping')
            end
        end)
        fibers[i]:set_joinable(true)
    end
    for _, f in ipairs(fibers) do
        f:join()
    end
    result[threads_count].benchmark =
        (1.0e9 * fibers_count * bench_count) /
        (1.0e6 * tonumber(clock.monotonic64() - time_before))
end;
function benchmark_save_result(result)
    local one_thread_perf = result[1].benchmark
    local file = io.open("perf.txt", "w+")
    if file == nil then
        return
    end
    io.output(file)
    io.write(" ________________________________________________________________\n")
    io.write("| iproto threads count |     latency, ms     | performance, Mrps |\n")
    io.write("|----------------------------------------------------------------|\n")
    for threads_count, val in pairs(result) do
        io.write(string.format("|          %2d          ", threads_count))
        io.write(string.format("|        %0.3f        ", val.latency))
        io.write(string.format("|      %6.4f       |\n", val.benchmark))
    end
    io.write("|----------------------------------------------------------------|\n")
    io.close(file)
end;
test_run:cmd("setopt delimiter ''");

threads_count = 1
threads_count_max = 16
bench_count = 1000
result = {}
test_run:cmd("setopt delimiter ';'")
while threads_count <= threads_count_max do
    test_run:cmd(string.format('start server perf with args=\"%d\"', threads_count))
    server_addr = test_run:cmd("eval perf 'return box.cfg.listen'")[1]
    result[threads_count] = {}
    latency(bench_count, server_addr, result, threads_count)
    benchmark(bench_count, server_addr, result, threads_count)
    threads_count = threads_count * 2
    test_run:cmd("stop server perf")
end;
test_run:cmd("setopt delimiter ''");

benchmark_save_result(result)

test_run:cmd("cleanup server perf")
test_run:cmd("delete server perf")

but there are several problems with this test.

  1. In case when iproto thread is not a bottleneck, one or more threads have the same performance. We need to create a test, which demonstrates a case when several iproto threads are better than one.
  2. Current test saves the result in a file, we need a test, which runs on CI every time and checks performance.
@EvgenyMekhanik EvgenyMekhanik added feature A new functionality teamC labels Apr 12, 2021
@EvgenyMekhanik EvgenyMekhanik self-assigned this Apr 12, 2021
@kyukhin kyukhin changed the title Create useful iproto performance test and integrate it into current testing system Create iproto performance test and integrate it into current testing system Jul 14, 2021
@kyukhin kyukhin removed the incoming label Jul 29, 2021
@kyukhin kyukhin added this to the wishlist milestone Aug 19, 2021
@TarantoolBot TarantoolBot removed the teamC label Jun 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature A new functionality
Projects
None yet
Development

No branches or pull requests

3 participants