Question about Vector Insertion and Index Building

[Chisdo]

Hi there, firstly, I got a few questions about my experiment with Milvus. Then I do have some questions about vector insertion and when does index build.
--- My experiment setting: ---
[only start one GPU-enabled Milvus on one computer]
a. Milvus image: 1.0.0 GPU-milvus with RTX 2080 ti * 1, CPU has 12 core[server level]
b. metadata: enable with MySQL
c. Wal: Disable
d. cache_size: 180 GB
e. disk is HDD and other settings are default
f. Then, I create 5 collections and build index at the beggining:
collection1: L2, 2048-dimension, 1024-index_file_size, IVF_PQ {nlist=30000, m=32}
collection2~5: L2, 480-dimension, 1024-index_file_size, IVF_PQ {nlist=30000, m=32} * 4
g. I start 5 threads to insert data to each collection at the same time
h. Use different single queries to test the collection 1 several times.

--- my questions here ---
i. Does 1. g. [inserting data to the different collection concurrently] would affect insertion speed?

ii. I got extremely poor testing results from 1. h. : the searching time is extremely high around 700 seconds per query. I feel like Milvus did brute-force during the test. While the result from get_collection_stats function shows all segments have already got PQ index yet. So I'm wondering that does my Milvus did use index for searching or just do the brute-force? Besides, does Milvus really build index during inserting?

iii. I notice that there are two background operations in Milvus: merging segments and build the index. I'm wondering that if both operations are needed, which one would be done first? Also, can I do the merging operation manually?

iv. If I keep inserting continuous data to a partition in a collection with a small auto_flush_interval [1 second]. May the small auto_flush_interval number would increase the number of segments at this partition?

v. In "Performance Tuning" section, there is a sentence says: "In scenario with continuous data insertion, because Milvus does not index segments with a size less than index_file_size, it uses brute-force search as the query method. ". Does it mean that once the segment is inserted over index_file_size, the segment would be built index immediately?

[shengjun1985]

i. Does 1. g. [inserting data to the different collection concurrently] would affect insertion speed?

No, it doesn't.
Disk IO is the main factor that affects the insertion performance. Multi-threading doesn't help it.

ii. So I'm wondering that does my Milvus did use index for searching or just do the brute-force?
Besides, does Milvus really build index during inserting?

Please refer to create_index.
According to your description, you firstly specified the index and then inserted the data. So the data inserted must reach index_ file_size (1024 Mb), the index would be created. How about your row_count of the collection 1?
But if your data is static, you can insert the data before specifying the index.
Furthermore, how about the 'nq', 'topk', 'nprobe' of your query?

iii. I'm wondering that if both operations are needed, which one would be done first?
Also, can I do the merging operation manually?

Please refer to merge-data
These two operations depend on the parameter index_ file_size. So they don't work on one segment at the same time.
For different data segments, one is disk IO and the other is CPU computing, so they can be done at the same time.
However, there is no interface for manually merging at present.

iv. If I keep inserting continuous data to a partition in a collection with a small auto_flush_interval [1 second]. May the small auto_flush_interval number would increase the number of segments at this partition?

Yes, it will in a short time.
And after the data merged, there will be no such problem.

v. Does it mean that once the segment is inserted over index_file_size, the segment would be built index immediately?

Need to wait for merging segments.

[yhmo]

Thx for your prompt reply！But I may still have a few confusions about the index.

1. If I understand correctly, for one partition, [after merge] it should have several index-well-build segments, several segments in the build-index waiting list, and only one segment whose size is less than index_file_size. Am I correct?
2. How can I know the Milvus is building index on the background?
3. After conducting a search, the collections would be loaded into the cache. While if I don't do any operations, how long does the cache would be invalid for the next searching approximately?
4. I also met a problem during rebuilding the index for 2048-dimension vectors. The status error-log shows below. For the 2048 dimension vector, the only 'm' option for it is 64. Hence, may I know how to do the 'useFloat16LokkupTables' trick or how to solve this problem?

1. With continually insert, there could be several index-well-build segments in one partition, and several segments in build index waiting list, but not only one segment whose size is less than index_file_size, because the merge strategy need to consider "write-amplify", please refer this article: https://zhuanlan.zhihu.com/p/148106312 (this article is written in Chinese, you can translate it into English by google translator)

2. Currently we don't have interface to tell client what is going on for the building index on background. For our developers, they can know from log, or meta(MySQL/SQLite).

3. Milvus cache size is defined by server_config.yaml, when you perform a search request, milvus will load required segments from disk into cache(cpu memory). If the cache space is enough, the segments will keep in cache. Once the cache space is fully occupied, milvus will evict some segments that long time not used, according to LRU strategy.

[shengjun1985]

4 I also met a problem during rebuilding the index for 2048-dimension vectors. The status error-log shows below. For the 2048 dimension vector, the only 'm' option for it is 64. Hence, may I know how to do the 'useFloat16LokkupTables' trick or how to solve this problem?

Please refer to: #3254 (comment)
That "(m x 1024) <= MaxSharedMemPerBlock of your graphics card" is a constraint on GPU.
If you want to continue to use GPU, you have to set M smaller.

[Chisdo]

Please refer to: #3254 (comment)
That "(m x 1024) <= MaxSharedMemPerBlock of your graphics card" is a constraint on GPU.
If you want to continue to use GPU, you have to set M smaller.

Since the 'dim/m' at most is 32, for the 2048-dimension vector, the 'm' value at least should be 64 [I cannot decline 'm' value anymore]. On the other hand, the 'm=64' setting is too big for my 2080TI graphic card. So I think the 'useFloat16LokkupTables' trick might be my only solution but I didn't find any interface to configure it.

• Is there any way to change 'useFloat16LokkupTables'?
• Can multiple GPU increase the 'MaxSharedMemPerBlock' of the graphic card?
• I think there might be a typo here:
  IVF_PQ
  

[shengjun1985]

Is there any way to change 'useFloat16LokkupTables'?
Can multiple GPU increase the 'MaxSharedMemPerBlock' of the graphic card?

No, it cannot.
This is related to the GPU model but not GPU number.

I think there might be a typo here.

Thank you very much.
We will fix it soon.

[Chisdo]

My pleasure pal. Also appreciate all your answers, which help me a lot!
Btw, I strongly wish to have more parameters to tune for using GPU to build index. It's ok for losing some precision or efficiency. But it's really frustrating for crushing during index building.

[yhmo]

Merge operation only happens on segments which size less than index_file_size.
Auto-build-index operation only happens on segment which size larger/equal than index_file_size.

Two ways to create index:

1. create an empty collection, then specify an index type, then insert data until segments merged to exceed index_file_size.
   steps:
   create_collection()
   create_index()
   insert()

2. create an empty collection, insert data until size larger than index_file_size, then specify an index type
   steps:
   create_collection()
   insert()
   create_index()

The way_1, milvus automatically build index for segments which size larger than index_file_size
The way_2, milvus will build index for all segments except the tiny segment which row count is less than 4096

For your question ii, I guess you are using the way_1. Since the index type has been specified, you saw all the segments are marked as "PQ", but in fact, their build index tasks maybe are still working, index not finished. So you found the search is brute-force search.

[Chisdo]

Hi Yhmo, I get further questions about index building:
If I create an indexed collection [like SQ8] first, then keep inserting data to the collection [treat it as streaming data]. Is there any possibility that I can drop and rebuild the index of the collection while inserting data?

[yhmo]

Milvus has three queues to process different requests:

• create/insert/delete requests put into "DDL" queue
• search requests put into "DQL" queue
• others requests put into another queue

Consider this case:

• client_1 continually call insert() to insert vectors.
• client_2 call drop_index() and create_index()

When client_2 call create_index(), the "DDL" queue will blocked until all segments finish index. At the same time, the client_1 will paused since the queue is blocked by create_index().
But if the client_2 stopped, by ctrl+c or connection break, etc, the create_index request will be removed from the "DDL" queue, and put to background thread to execute, therefore the client_1 can continue insert.