Why load_duration take so much time?

[become-nice]

I use vectorDBBench to test standalone instance, test case is 'Search Performance Test(1M Dataset,768Dim), HNSW index with the default config', load_duration cost more than 3000s. All milvus test case is alse cost hundreds or thousands of seconds.I wonder what this step does and why it takes so long.

[liliu-z]

Can I ask the CPU resource for Milvus server? We put some test result data in this repo, and it shows for a 2c8g hnsw Milvus, it will take ~2400s.
This time is for data insertion and index building

[become-nice]

I use 4c8g hnsw milvus, thank you for your answer!

[liliu-z]

@become-nice That's weird, the loading time should be ~1200s this case. Can I ask how you run Milvus (compile&run or docker image) and what version is that?

[become-nice]

I use docker image to deploy. I wonder if you will include load_duration time when calculating qps.

[become-nice]

2.2.10. What is the configuration when you test the hnsw index.I use the default configuration.

[XuanYang-cn]

I use docker image to deploy. I wonder if you will include load_duration time when calculating qps.

load_duration is the sum of time(load the dataset into Milvus) and time(building hnsw index).

QPS won't use load_duration. To get QPS, we will run concurrency search from 1 to 35 concurrencies, each concurrency search runs for 30s, the QPS is calculated by total_query_count/30 for each concurrency, and we'll take the best QPS result as the final results from all concurrency searches.

[XuanYang-cn]

2.2.10. What is the configuration when you test the hnsw index.I use the default configuration.

We use the default config too

[XuanYang-cn]

The load_duration of standard results for milvus-2c8g-hnsw is 2380s.

load_duration consists of insert_time and index_time, since you're using 4c8g, the index time will be shorter. So the problem should be the insert_time.

For Milvus standalone the insert_time is influenced by network latency and log broker.
we use rocksmq(powered by rocksdb) for milvus standalone log broker, which means the insert_time is very sensitive to rocksdb's disk quality.

Are you using ssd for milvus standalone? @become-nice

[become-nice]

How do I adjust the concurrency, with the default parameters, it is impossible to test the real maximum qps.

[become-nice]

The frequency of my cpu is only 2.2GHz.Maybe your cpu is better than ours.

[liliu-z]

The frequency of my cpu is only 2.2GHz.Maybe your cpu is better than ours.

@XuanYang-cn can you help provide the CPU data. But I don't think this can be the major problem. Any chance we can get the insert time form log?

[XuanYang-cn]

The frequency of my cpu is only 2.2GHz.Maybe your cpu is better than ours.

This's what we've used for the benchmark client and milvus standalone

Model name:                      Intel(R) Xeon(R) Platinum 8375C CPU @ 2.90GHz

[become-nice]

Can you tell me how to adjust the concurrency.

[liliu-z]

How do I adjust the concurrency, with the default parameters, it is impossible to test the real maximum qps.

@become-nice To make it more straight forward, our logic is testing concurrency [1, 5,10,15,20,25,30,35] and pick the best QPS. If you want to customize your concurrency, you can modify the related code and build the benchmark following README.