High latency on high QPS use case

[jackjlli]

We noticed the latency from offline server becomes high when qps is high (e.g. more than 1000 qps per server host).
The query pattern is:

SELECT sum(count) FROM table1 WHERE column1 IN ("aa", "ab", "ad", ..., "zz")

This page is used to keep track of the methods we've tried to optimize the performance.

[jackjlli]

One of the potential problems we suspect is that using on heap dictionary is not scalable, which keeps all the values of column1 in the memory. If the cardinality of the column is high (e.g. 10 million unique strings in each segment), and there're 10 segments in one offline server host, then retrieving string from hashmap would be a performance concern. Thus, we tried several trie based dictionary strategies to tackle this problem.

[mcvsubbu]

Did you try using (a variant of) StringOffHeapMutableDictionary? If binary search on the dictionary is the issue, then this format will alleviate it while using off-heap memory for hash

[jackjlli]

OnHeapDictionary doesn't use binary search. It just simply uses a hashmap. Binary search isn't the bottleneck though.

[sunithabeeram]

Have you profiled this query to see where the bottleneck is? How big does the IN clause get?

Note that with the latest roaring bit map we have seen significant improvement in the performance of filtering.

[jackjlli]

@sunithabeeram There could be 256 values in the IN clause.

[snleee]

@sunithabeeram We don't use inverted index for this particular query. We sort on column1.

[sunithabeeram]

@snleee I see. Makes sense.
Would be good to profile to figure out the hotspot.