[Enhancement]: query optimization: reduce io when retrieve output fields

[longjiquan]

Is there an existing issue for this?

• I have searched the existing issues

What would you like to be added?

Above figure illustrates how milvus handles a query request:

• 1. Every segments execute the filtering and return all the required output fields;
• 2. Shard delegators reduce all the segment results and then return to the Proxy;
• 3. Proxy reduce all the delegator results and then return to the users;

We already optimize step 1 & 2 by pushing down the limit operator to the segment, which reduce the sort operation of step 2.
However, step 1 may be also very inefficient. With mmap enabled, retrieving the output fields may cost big io compared to retrieving raw data from memory. However, in the reduce phase, in fact many candidates in the segment results will be discarded, so the prefetched io was wasted. So does the proxy's reduce.

So I want to optimize the query workflow by changing the way of retrieving the required output fields:

• 1. Retrieving the output fields after local reduce is done, whether the field enables mmap or not. More cgo calls will be introduced, but much less data will be retrieved. We can consider this optimization will always work since the cost of more cgo calls can be assumed small.
• 2. If the limit is very large and any output fields enable the mmap, just same with change 1, the proxy should remove the mmapped fields when forward the query request to the querynode. Retrieval of the mmapped fields should be done after the reduce. Compared to current logic, more rpcs will be introduced and rpc sometimes is also very time-consuming. So trade-off between the rpc-cost and io-cost should be considered. For now, I would suggest that proxy should do the change only when below conditions are all satisfied:
  • Limit is very large, of course this can be configured dynamically, 1000 by default;
  • Output fields contain any mmapped fields;

Why is this needed?

Reduce the io when retrieving output fields and thus increase the qps.

Anything else?

No response

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Rotten issues close after 30d of inactivity. Reopen the issue with /reopen.