Faster Imports and better import DX: Async vector index building

[etiennedi]

At the moment, a batch request is fully synchronous. This means it will only complete once everything is fully indexed.

Instead, it should be possible to import async where only the object storage (and possibly the inverted index) is written synchronously. Vector indexing happens async in more organized batches.

This should lead to:

• Better CPU utilization by making sure the import queue does not run dry
• Easier to avoid lock contention as some processes can happen in larger batches
• Avoid import vs delete races and contention, i.e. the whole delete re-built logic can be rewritten

To avoid the negative user impact of the index not being available immediately, we need to make sure that:

• If feasible, a search while importing makes use of the partially built index and in addition brute-force objects which are not yet indexed
• It is easily visible to the user whether an index is not fully built yet and when it turns ready. Clients should offer methods to wait until all is ready.

[etiennedi]

Delivered in 1.23