compact affects performance? #164

Closed
smallnest opened this Issue Jan 10, 2017 · 4 comments

Projects

None yet

2 participants

@smallnest

let me describe my application case:
I have use goleveldb as database to persist realtime data,. I write data into golevedb in 2000 values per second and size of each value is about 6k.

The usage of goleveldb is simple and just like demo in README:
open the file and close until the application exit. Still put values in 2000 qps. All configurations are in default and I have not set any extra options.

A timer deletes expired values per 2 minutes. Expiration time is 30 minutes.

Test after onr day I found disk was occupted and It seems there is no auto compaction for this database.

So I added compaction action into the timer and it compacted the database with deleted range mannually.

Yes, compaction works but it looks compaction affect other operations on this database, for example, get and put. So will it lock the database when compacting?

For my case, is there a good solution for compacting deleted items to avoid disk occupation?

Thanks.

@syndtr
Owner
syndtr commented Jan 11, 2017

Could you post the LOG file?

@smallnest

I will retest and attach the codes and LOG file. Thanks

@smallnest

I test about one hour and get the LOG file and the flame graph.

I can't attach LOG into this issue directly so I uploadec them into my website.

LOG file
LOG

flame grapth
flame graph

@smallnest

I have change logics in my codes and save data concurrently. It reduces time of saving data and it looks all work well.

I close this issue and thanks.

@smallnest smallnest closed this Jan 11, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment