You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
We introduced the GetRangeDescriptors API in #92603 as an alternative to scanning meta ranges to fetch RangeDescriptors. The intention here was to allow secondary tenants to fetch range descriptors without directly interacting with the meta ranges.
We set up this API to be able to stream range descriptors back to the client. However, we don't meaningfully make use of this just yet. Instead, we bulk load the requested range descriptors in memory and send them as a single response. This is likely okay for the initial implementation, given we don't expect RangeDescriptors to be particularly large (assuming reasonable keys, ofcourse).
We could explore adding some sort of batching here in the future. Concretely, we'd need to ship a timestamp from the client and use it as the timestamp to perform a paginated scan of meta2 at. We could then stream results back to the client as we paginate through the descriptors. One thing to note is the scan happens in a retryable, and as such, can be restarted multiple times -- we'd need to communicate this back to the client so that it can throw away the accumulated range descriptors so far.
cc @ajwerner following our conversation about this earlier today. @ecwall you might find this interesting as well.
Is your feature request related to a problem? Please describe.
We introduced the
GetRangeDescriptors
API in #92603 as an alternative to scanning meta ranges to fetchRangeDescriptors
. The intention here was to allow secondary tenants to fetch range descriptors without directly interacting with the meta ranges.We set up this API to be able to stream range descriptors back to the client. However, we don't meaningfully make use of this just yet. Instead, we bulk load the requested range descriptors in memory and send them as a single response. This is likely okay for the initial implementation, given we don't expect
RangeDescriptors
to be particularly large (assuming reasonable keys, ofcourse).We could explore adding some sort of batching here in the future. Concretely, we'd need to ship a timestamp from the client and use it as the timestamp to perform a paginated scan of meta2 at. We could then stream results back to the client as we paginate through the descriptors. One thing to note is the scan happens in a retryable, and as such, can be restarted multiple times -- we'd need to communicate this back to the client so that it can throw away the accumulated range descriptors so far.
cc @ajwerner following our conversation about this earlier today. @ecwall you might find this interesting as well.
Jira issue: CRDB-21962
Epic: CRDB-26091
The text was updated successfully, but these errors were encountered: