Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Insert performance #127

Open
fireya opened this issue Mar 20, 2018 · 1 comment
Open

Insert performance #127

fireya opened this issue Mar 20, 2018 · 1 comment

Comments

@fireya
Copy link

@fireya fireya commented Mar 20, 2018

Hello!
I have noticed slow insert performance of insert operations in connector. To test this, i have made some benchmarks:
Create space:

box.schema.space.create('pivot')
box.space.pivot:create_index('primary', { type = 'tree', parts = {1, 'num'}, unique = true})
box.space.pivot:create_index('rtree', { type = 'rtree', parts = {2, 'array'}, unique = false})

Test on lua:

box.space.pivot:truncate()
function main_function()
  local t
  for i = 1,1000000,1 do
    t = box.tuple.new({i,{i,i},i})
    box.space.pivot:insert(t)
  end
end
start_time = os.clock()
main_function()
end_time = os.clock()
'insert done in ' .. end_time - start_time .. ' seconds'

Results: 4 seconds, or 250k per second

Test on go:
first, truncate space: box.space.pivot:truncate()
then test:

package main

import (
	"fmt"
	"log"
	"time"

	"github.com/tarantool/go-tarantool"
)

func main() {
	opts := tarantool.Opts{}
	conn, err := tarantool.Connect("10.40.10.143:3301", opts)

	// conn, err := tarantool.Connect("/path/to/tarantool.socket", opts)
	if err != nil {
		fmt.Println("Connection refused: %s", err.Error())
	}
	start := time.Now()
	f := make([]*tarantool.Future, 0)
	for i := 0; i < 1000000; i++ {
		fut := conn.InsertAsync("pivot", []interface{}{i, []int{i, i}, i})
		f = append(f, fut)
	}
	for _, element := range f {
		_, err := element.Get()
		if err != nil {
			fmt.Println("Insert failed: %s", err.Error())
		}
	}
	elapsed := time.Since(start)
	log.Printf("Insert took %s", elapsed)
}

Results: 6.74s or 148k per second

Now .net connector:
truncate: box.space.pivot:truncate()
Test:

using System;
using System.Diagnostics;
using System.Threading.Tasks;
using ProGaudi.Tarantool.Client;
using ProGaudi.Tarantool.Client.Model;

namespace Tarantool.Test
{
    class Program
    {
        static void Main(string[] args)
        {
            using (var box = Box.Connect("10.40.10.143:3301").Result)
            {
                var schema = box.GetSchema();
                var space = schema["pivot"];
                var sw = new Stopwatch();
                var lst = new Task[1000000];
                sw.Start();
                for (int i = 0; i < lst.Length; i++)
                {
                    lst[i] = space.Insert(new TarantoolTuple<int, int[], int>(i, new[] { i, i }, i));
                }

                Task.WaitAll(lst);
                sw.Stop();
                Console.WriteLine(sw.ElapsedMilliseconds);
                Console.ReadKey();
            }
        }
    }
}

Time: 92 sec or 11k per second

@aensidhe
Copy link
Member

@aensidhe aensidhe commented Mar 25, 2018

I have better results (33k per second), but worse than that go results. You can improve by at least 30-50% by batching queries into batches of 1000, right now you're benchmarking .net GC more than actual code of library.

That does not mean, that I will leave it as it is. I'll need to rewrite our or adopt new msgpack library, most problem lie there. Unfortunately, can't provide any ETA as hard promises.

This branch has some drafts of what I want to do with it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.