Summary:
This change addresses a limitation in the current vector index search functionality, where results are valid only for a specific tablet. In cases where a table spans multiple tablets, results must be merged to compute the actual top K. Previously, the absence of a merging algorithm restricted testing to a single tablet, severely limiting datasets that vector index could be tested with.
**Key Changes:**
• Implements a simple merge algorithm on the PgClient side to combine top K results from multiple tablets.
• Enables testing of vector indexes on larger datasets that span multiple tablets.
**Temporary Implementation Details:**
• Assumes each request contains only one vector index read operation.
• Requests are forwarded to all tablets, and their results are merged into a single top K result.
**Current Limitations:**
1. Filter Pushdown: If extra filters for non-vector columns are pushed down, results may be incorrect because filter pushdown is not yet implemented.
2. Pagination: Pagination is unsupported. Requests requiring multiple read operations will not function properly.
This is an interim solution designed to facilitate testing while supporting larger datasets, with further improvements planned for future iterations.
**Upgrade/Rollback safety:**
Changes format for not yet released feature.
Jira: DB-14400
Test Plan: Jenkins
Reviewers: tnayak, arybochkin, aleksandr.ponomarenko
Reviewed By: tnayak, arybochkin
Subscribers: ybase, yql
Tags: #jenkins-ready
Differential Revision: https://phorge.dev.yugabyte.com/D40389