Support multiple indexes

[lming]

It seems the documents are all in a single index. Do you plan to support multiple indexes that user can add documents into and search from separate index?

Thanks for building this great piece of software!

[mosuka]

Hi @lming ,
Sorry for the late reply.

Solr and Elasticsearch can serve multiple indices in a single process, but I don't plan to support that at the moment. Bayard keeps the design simple, serving one index per process.
This is my opinion, as container orchestration systems such as Kubernetes allow more flexibility in resource allocation, there is no longer a need to serve multiple indices on a single node (process) as in the past.

However, while I may support it in the future, it's not in our plans right now.

[lming]

@mosuka thanks for the reply!

I respect your choice but it's arguable that container orchestration of multiple bayard instances is a good replacement of one cluster serving multiple indexes in terms of dev/ops effort - especially when the number of indexes is big and dynamic. (e.g. one index for each customer's data).

[mosuka]

@lming
That makes sense.
I'll try to support multiple indexes when I have time.
Thanks!

[lming]

Great to see that bayard now supports multiple indexes, thanks a lot @mosuka for implementing this!

A question - is there a limit of how many indexes we can operate in a bayard cluster?

[mosuka]

Hi @lming ,
No limit is set. If you want to run many indexes, you will need to increase the number of Bayard nodes as well.
Please note that I have not yet implemented shard rebalancing when nodes are dynamically added to the cluster...