Optimization for selection only queries: Allow early termination

[xiangfu0]

In Pinot CombineOpearator, queries are scheduled with at most 10 threads. For tables with many segments(say 10k), it means each thread will process 1k segments.
For selection only queries (e.g. select * from myTable limit 10), each thread may collect enough results to return after scan a few segments. There is no means to wait and scan all the segments.
This is extremely useful for people want to randomly browse a big table from query console clicks.

[codecov-io]

Codecov Report

Merging #5163 into master will decrease coverage by 0.05%.
The diff coverage is 94.00%.

@@             Coverage Diff              @@
##             master    #5163      +/-   ##
============================================
- Coverage     66.02%   65.97%   -0.06%     
  Complexity       12       12              
============================================
  Files          1050     1050              
  Lines         54117    54112       -5     
  Branches       8071     8067       -4     
============================================
- Hits          35729    35698      -31     
- Misses        15734    15765      +31     
+ Partials       2654     2649       -5     

Impacted Files	Coverage Δ	Complexity Δ
...rg/apache/pinot/core/operator/CombineOperator.java	65.62% <72.72%> (-0.67%)	0.00 <0.00> (ø)
...he/pinot/core/operator/CombineGroupByOperator.java	84.44% <100.00%> (+0.74%)	0.00 <0.00> (ø)
...t/core/operator/CombineGroupByOrderByOperator.java	85.32% <100.00%> (+0.63%)	0.00 <0.00> (ø)
...g/apache/pinot/core/operator/DocIdSetOperator.java	93.54% <100.00%> (+0.44%)	0.00 <0.00> (ø)
...ore/operator/query/AggregationGroupByOperator.java	95.23% <100.00%> (-0.42%)	0.00 <0.00> (ø)
...rator/query/AggregationGroupByOrderByOperator.java	91.66% <100.00%> (-0.65%)	0.00 <0.00> (ø)
...pinot/core/operator/query/AggregationOperator.java	94.44% <100.00%> (-0.56%)	0.00 <0.00> (ø)
...ator/query/DictionaryBasedAggregationOperator.java	88.88% <100.00%> (-0.40%)	0.00 <0.00> (ø)
...erator/query/MetadataBasedAggregationOperator.java	87.50% <100.00%> (ø)	0.00 <0.00> (ø)
...not/core/operator/query/SelectionOnlyOperator.java	97.29% <100.00%> (-0.08%)	0.00 <0.00> (ø)
... and 27 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bbaa1d9...3f568b5. Read the comment docs.

[Jackie-Jiang]

Very good optimization, thanks

[Jackie-Jiang]

@fx19880617 I pushed a commit to address the metadata issue with the early termination. The fixed metadata are: numSegmentsProcessed which is the number of segments after the segment pruning; numTotalDocs which is the total number of documents for the table. Please take a look

[xiangfu0]

@fx19880617 I pushed a commit to address the metadata issue with the early termination. The fixed metadata are: numSegmentsProcessed which is the number of segments after the segment pruning; numTotalDocs which is the total number of documents for the table. Please take a look

lgtm! 👍