Potential Read Performance Regression

[connorgorman]

What version of Go are you using (go version)?

$ go version
1.13.4

What version of Badger are you using?

master (bce069c)

Does this issue reproduce with the latest master?

yes

What are the hardware specifications of the machine (RAM, OS, Disk)?

8GB, Ubuntu, 100GB HDD

What did you do?

Running 1.6.0 vs master with a mixed workload doing mostly writes and random reads
Value sizes range from 400 to 1000 bytes

What did you expect to see?

I'd expect the performance of master to be better than the performance of 1.6.0 across the board. Instead, it seems that random reads have more variability in speed.

What did you see instead?

I've seen slow downs of up to 200x. Unfortunately, I haven't been able to provide an isolated code sample that I could share, but was curious if anything has shown up in any benchmarks.

[jarifibrahim]

@connorgorman were you running badger with default options? You might want to disable compression, encryption, and cache and then run your benchmarks.

[connorgorman]

@jarifibrahim I was running with the default options, but with NumLevelZeroTables = 2 and NumLevelZeroTablesStall = 5. I believe those other features are off by default (except the cache), but I will set the flags and comment again. Thanks

[connorgorman]

@jarifibrahim So I ran it again with

badger.DefaultOptions(path).
		WithTruncate(true).
		WithNumLevelZeroTables(2).
		WithNumLevelZeroTablesStall(5).
		WithMaxCacheSize(0).
		WithChecksumVerificationMode(options.NoVerification).
		WithCompression(options.None).
		WithVerifyValueChecksum(false)

and I'm getting similar results in terms of performance regressions. In fact, it looks like reads and writes are generally slower (though some cases appear faster).

Are there any internal metrics in Badger I can use? Do all those options make sense?

[jarifibrahim]

@connorgorman The options make sense. Let me try to run some benchmarks and see what results I get. Ideally, v2 should be faster than v1.6.

It would be useful if you can share your benchmarks.

[jarifibrahim]

Hey @connorgorman, I've raised #1256 to fix this.
Major performance improvement comes from #1256 and #1257 also improves the performance.

Please try out #1257 and let me know how it goes.