Replace BrokerRequestOptimizer with QueryOptimizer to also optimize the PinotQuery

[Jackie-Jiang]

Description

Currently broker only optimizes the filter in BrokerRequest, but leaves the filter in PinotQuery as is. Because of this, we can only use the filter from the BrokerRequest, which is inefficient because there is no native Expression support. It also blocks the PQL deprecation.

In this PR:

• Attach the time boundary to PinotQuery.
• Replace the BrokerRequestOptimizer with QueryOptimizer to also optimize the filter in PinotQuery.
• in BrokerRequestToQueryContextConverter, use PinotQuery to generate QueryContext for SQL queries.

For backward-compatibility, Broker will optimize the filter in both BrokerRequest and PinotQuery.

Upgrade Notes

Please upgrade Brokers before Servers

[codecov-io]

Codecov Report

Merging #6423 (042bae1) into master (1beaab5) will decrease coverage by 21.79%.
The diff coverage is 40.24%.

@@             Coverage Diff             @@
##           master    #6423       +/-   ##
===========================================
- Coverage   66.44%   44.65%   -21.80%     
===========================================
  Files        1075     1320      +245     
  Lines       54773    64481     +9708     
  Branches     8168     9403     +1235     
===========================================
- Hits        36396    28792     -7604     
- Misses      15700    33333    +17633     
+ Partials     2677     2356      -321     

Flag	Coverage Δ
integration	44.65% <40.24%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...ot/broker/broker/AllowAllAccessControlFactory.java	100.00% <ø> (ø)
.../helix/BrokerUserDefinedMessageHandlerFactory.java	52.83% <0.00%> (-13.84%)	⬇️
...org/apache/pinot/broker/queryquota/HitCounter.java	0.00% <0.00%> (-100.00%)	⬇️
...che/pinot/broker/queryquota/MaxHitRateTracker.java	0.00% <0.00%> (ø)
...ache/pinot/broker/queryquota/QueryQuotaEntity.java	0.00% <0.00%> (-50.00%)	⬇️
...ker/routing/instanceselector/InstanceSelector.java	100.00% <ø> (ø)
...ceselector/StrictReplicaGroupInstanceSelector.java	0.00% <0.00%> (ø)
...roker/routing/segmentpruner/TimeSegmentPruner.java	0.00% <0.00%> (ø)
...roker/routing/segmentpruner/interval/Interval.java	0.00% <0.00%> (ø)
...r/routing/segmentpruner/interval/IntervalTree.java	0.00% <0.00%> (ø)
... and 1304 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3d24302...042bae1. Read the comment docs.

[xiangfu0]

lgtm