[INLONG-8125][Sort] Optimizing the speed of transitioning from snapshot to binlog

[e-mhui]

Prepare a Pull Request

[INLONG-8125][Sort] Optimizing the speed of transitioning from snapshot to binlog

• Fixes [Improve][Sort] Optimizing the speed of transitioning from snapshot to binlog #8125

Motivation

When transitioning from the snapshot phase to the binlog phase, the shouldEmit() method is called. Its function is to traverse all splits for each Binlog data and check whether the record is located after the end of the snapshot phase. However, the time complexity of this method is O(n). Since finishedSplitsInfo is locally ordered, we can use binary search to find the split where the current binlog is located, reducing the time complexity from O(n) to O(logn).

  for (FinishedSnapshotSplitInfo splitInfo : finishedSplitsInfo.get(tableId)) {
      if (RecordUtils.splitKeyRangeContains(
              key, splitInfo.getSplitStart(), splitInfo.getSplitEnd())
              && position.isAfter(splitInfo.getHighWatermark())) {
          return true;
      }

Modifications

1. When the MySqlSourceReader receives all the FinishedSnapshotSplitInfos, it sorts the FinishedSnapshotSplitInfos based on the chunkId.
2. Use binary search to find the split which contain the specific key.