about k8s + Mishards discussion and a few conceptual questions

[Chisdo]

Dear Contributors and fellows, recently I'm studying Mishards on k8s solution right now.
[refers to: https://milvus.io/docs/v1.1.0/mishards.md#Deploy-Mishards-cluster-in-Kubernetes]

1. First of all, there might be a typo in [https://milvus.io/docs/v1.1.0/mishards.md#Usage, section Add CHANGLOGS.md and CONTIRBUTING.md #1].
   I think the "replica" in "-- set mishards.replica=2" should be "replicas".
2. I'm wondering why the number of "mishards.replicas" could be more than 1? Is it for high availability [ha]?
3. I'm wondering why the number of "readonly.replicas" could be more than 1? Is it for using more CPU and cache resources to compute or is it still for ha?
4. . Can I or How can I set up 'wr' machine's replicas more than 1?

Best,
Jiaxin

[yhmo]

1. This is a typo of the documentation, should be "mishards.replicas". Thank you for pointing our this typo. Could you help us fix it? On the top-right of the page there is a button "Edit" can fork/submit pr.
2. Mishards contians several kinds of instances: mishards master, write node, read-only node, mysql server. The "mishards.replicas" is for the mishards master node, it is for high availability of the master service. And mishatds master node receive requests from clients, and dispatch requests to "wr" nodes and "ro" nodes. And the master need to reduce search results from multiple "ro" nodes. Set "mishards.replicas" > 1 can get good qps for certain conditions.
3. "readonly.replicas" means how many read-only node in the cluster, the more "ro" nodes, the better search performance. But it is not for HA.
4. Mishards only allow one "wr" node all the time. Because mishars is a “one write multiple read system".

[Chisdo]

1. Thx for the reply. I have already submitted the change.
2. Besides the topic, I'm wondering that is there a function that can search across multiple collections?
3. My current solution of searching in multiple collections is using a single thread to search multiple collections and merge results. Can I get better performance if I use multi-threading or other parallel computing methods to search?

[yhmo]

1. Thank you. But I didn't found your pr in https://github.com/milvus-io/docs/pulls, could you give me the pr link?
2. Milvus doesn't support search across multiple collections, because there is no clearly requirement for this.
3. Your collections have difference dimension? Or your search parameters is different for each collection? If your collections have same dimension and search parameters are same, I think you can use partition to do this work:
   1. create a collection by create_collection()
   2. create some partitions under the collection, by create_partition()
   3. insert vectors to each partition
   4. search to the collection, milvus will search all the partitions and merge result for you

If your collections have different dimension and search parameters are different, You have multiple collections, milvus will execute search task one by one for each collection. Use multi-threads to send search requests can improve performance a bit better, but not obviously.

[Chisdo]

1. Oh... I forget to create a pr after committing the change. Now I just build a new pr at: Update mishards.md - k8s+milvus typo web-content#590
2. Appreciate for the concept of Partition, while, my main problem is reducing rebuild-index time. Although my vectors are saved in the same dimension and parameters, I still need to split them into several different collections for speeding up rebuild-index time. So can I rebuild the index [drop & re-create] based on Partition granularity?

[yhmo]

Ok, we see the pr.
Milvus doesn't support build index for a single partition.So, split vectors into multiple collections is make sense.