Support SegmentSelector API to select segment names based on expression

[xiangfu0]

Since Pinot supports batch segments replacement, there is a need to find out what are segments to replace in an existing table based on segment metadata.

E.g. When we want to backfill data on a particular date, we want to find out all the segments for that date.

Sample expression:

1. Find segments for a given time range:
   segment.start.time >= 1614000000 and segment.end.time <1614150000
2. Find segments for a partition:
   segment.partition.id==1

Ideally, if we can have a system logical table like __SEGMENTS_METADATA then we can direct send a query like:

SELECT segment_name FROM __SEGMENTS_METADATA WHERE SEGMENT_START_TIME >= 1614000000 and SEGMENT_END_TIME <1614150000

to query segment metadata.

[Jackie-Jiang]

Is this API mainly for debugging purpose? Should we consider using EXPLAIN for this?

[xiangfu0]

the immediate use case is for data backfill, which we can programmatically replace x segments given some criteria then replace with some new segments.

[Jackie-Jiang]

I think that should be done by enforcing the segment name convention and time alignment. In order to replace segments, all the records must match the filter.
For replacing with time range, @snleee 's design doc covers the algorithm if segments are not strictly time aligned.

[xiangfu0]

This is just one example of usage. Segment name convention is used for single segment replacement, for batch replace, the segment name should be different though.

Another example could be replacing segments with given partitions for a time range.
For debugging purposes. we can send a query to find out how many segments per day for a pinot table, which will be helpful to debug potential data missing for large data bootstrap.

I think that should be done by enforcing the segment name convention and time alignment. In order to replace segments, all the records must match the filter.
For replacing with time range, @snleee 's design doc covers the algorithm if segments are not strictly time aligned.