Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Database insert speed #73

Closed
ToxyMao opened this issue Sep 19, 2022 · 4 comments
Closed

Database insert speed #73

ToxyMao opened this issue Sep 19, 2022 · 4 comments

Comments

@ToxyMao
Copy link

ToxyMao commented Sep 19, 2022

HelloThe database object has 6 indices. The client inserts approximately 15k objects into the base in 5 seconds. This is a very good speed for me. But when the database began to have a 10GB size, the time for the insert of those 15k objects became about 80 seconds. After 10 minutes, the insert speed again became 5 seconds. Does the database sometimes recalculate indexes?
I did tests several times, and every time there was such a situation.
Thank you

@ToxyMao
Copy link
Author

ToxyMao commented Sep 19, 2022

In features I have found: Liana-Trie indexing technology. Database indexes (keys) never need to be defragmented. Speed of insert/update/remove operations doesn't change during the time.
I use speedUpdate=true flag when objects insert

@ToxyMao
Copy link
Author

ToxyMao commented Sep 19, 2022

I have check pidstat on linux, it says that my process in that situation have I/O disk read speed about 5bm/sec, and write speed about 200kb/sec

@hhblaze
Copy link
Owner

hhblaze commented Sep 19, 2022

  • "Does the database sometimes recalculate indexes?"
  • No

If you want go deeper, you have to make more personal "scientific" experiments. They will have to cover performance monitors of database functions, the nature of inserting keys and methods of inserts. When you see bottlenecks, you may refer to DBreeze approaches and your infrastructure.

@hhblaze hhblaze closed this as completed Sep 19, 2022
@hhblaze
Copy link
Owner

hhblaze commented Sep 20, 2022

You could also publish your long-running code, may be it will be also much clear

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants