Historical cache should operate in bulk

[gianm]

Currently, the broker-side cache operates in bulk: it uses the getBulk API, and timeouts apply to the entire bulk get. But the historical-side cache operates on one segment at a time: it uses the get API, and timeouts apply to each cache fetch.

This is fine for local caches, but it's bad for remote caches (like memcached) because it means the historical is not efficient at pulling from them (due to the lack of bulk get) and can grind to a halt when memcached is unreachable (due to the timeout being applied to each segment, and a query potentially hitting many segments, in which case the timeout stacks up).

For many users, switching to broker-side caching is not a good option, since it forces the broker to retrieve and merge results from every individual segment, and doesn't scale well. So ideally we should improve the historical-side caching.

[drcrallen]

@gianm if I recall the broker only pulls per-segment results if it is asked to populate cache results. If you set the broker to use cache but not populate, it should try to pull what it can, but then push down the un-matched stuff to the historicals.

[gianm]

Related: #4947

[github-actions]

This issue has been marked as stale due to 280 days of inactivity.
It will be closed in 4 weeks if no further activity occurs. If this issue is still
relevant, please simply write any comment. Even if closed, you can still revive the
issue at any time or discuss it on the dev@druid.apache.org list.
Thank you for your contributions.

[github-actions]

This issue has been closed due to lack of activity. If you think that
is incorrect, or the issue requires additional review, you can revive the issue at
any time.