How did you improve the performance?

[krowling]

I have 2 questions about HSE Performance.

1. MongoDB Run Phase
   MongoDB/HSE improve 8x more throughput than MongoDB/WiredTiger.
   I think it improved performance by reducing read/write I/O.
   Is this right? If this is correct, How can you reduce I/O?

As I know, YCSB does not support multi-segment key.
Therefore, I think the multi-segmnet key did not affect the performance improvement...

2. MongoDB Load Phase
   MongoDB/HSE delivered more than 6x throughput compared to MongoDB/WiredTiger.
   How did you improve performance in load phase?

Please explain compared to wiredtiger.

Thanks,
sohyun

[smoyergh]

Question-1: Many factors contribute to the performance gains. However the primary factors are 1) reduced write/read amplification (as you mention), and 2) a focus on highly concurrent data structures, including those based on RCU where applicable.

Question-2: HSE plugs into the MongoDB Storage Engine API framework, and in that context we do in fact make use of HSE's multi-segment keys. The YCSB application is a MongoDB client, not a direct HSE client.

As to load phase performance, it's the same answer as Question-1.

[krowling]

Thanks :)

How did you reduce write/read amplification compared to MongoDB/Wiredtiger?

[smoyergh]

HSE uses a fundamentally different data layout. WiredTiger uses a btree by default (it has an LSM mode, but I don't know if it is supported). HSE uses a data layout of our own design that combines some characteristics of a trie with some characteristics of an LSM, along with novel compaction methods.

We anticipate publishing a whitepaper on the design when our schedule permits.

[krowling]

Thanks,

Are you using only the capacity class in mongoDB/HSE?
There is only nvme SSDs, but capacity class in not optional.

[configuration]

256GB DRAM / 4x Micron® 9300 NVMe SSDs 3.2TB in an LVM striped logical volume

[smoyergh]

In the MongoDB results on the HSE Wiki we are using only the capacity media class. We configured 4x Micron 9300 SSDs as a single LVM striped logical volume, and used that logical volume as the capacity media class.