Pipe: order historical TsFiles by query priority

[Caideyipi]

Description

This PR adds a historical Pipe source option:

• extractor.history.tsfile.order-by-query-priority
• source.history.tsfile.order-by-query-priority
• default: true

When enabled for historical TsFile-only extraction, selected TsFiles are transferred from lower query/compaction priority to higher priority. This matches the real point coverage semantics used by query and compaction: unsequence files cover sequence files, and within the same seq/unseq category, higher file version wins before higher file-name timestamp.

The receiver loads incoming TsFiles into the unsequence list with monotonically increasing versions, so covered files must be loaded first and covering files later. This preserves duplicated-timestamp overwrite semantics after transfer.

This PR also makes reordered historical progress reporting restart-safe by introducing a time-partition-scoped progress index. Instead of reporting a globally larger ProgressIndex too early, the source can report TimePartitionProgressIndex(timePartitionId, progressIndex) after a TsFile only when that progress cannot cover any remaining resource in the same time partition. Recovery uses this partition progress only for resources from the same timePartitionId, so an out-of-global-order progress in one partition cannot skip an untransferred TsFile in another partition. After all selected reordered historical TsFiles are supplied, the source still emits the global max historical progress to simplify future recovery.

Semantic Changes

For historical TsFile extraction where insertions are captured and deletions are not captured:

1. Historical working TsFile processors are closed synchronously before extraction, so the source has a stable set of TsFiles to order.
2. Selected TsFiles are sorted by MergeReaderPriority in ascending order, then by compaction version and file path as deterministic tie-breakers.
3. TsFiles filtered out by progress, time, path, delete status, pipe-generated status, or pin failure are removed before computing reordered historical progress.
4. Reordered historical TsFile events call skipReportOnCommitAndGeneratedEvents(), so neither the TsFile event nor generated tablet events report the original per-file progress directly.
5. The source scans the sorted resources from tail to head and maintains suffix-minimal progress indexes per TsFileResource.getTimePartition().
6. A resource is allowed to report incremental historical progress only if its progress will not cover any later resource in the same time partition.
7. The incremental report is persisted as TimePartitionProgressIndex, and restart filtering first checks normal global progress coverage, then checks partition-local coverage.
8. After all selected reordered historical TsFiles are supplied, the source emits one ProgressReportEvent with the max progressIndex of the selected resources.

When historical deletions are captured together with insertions, the source keeps the previous progressIndex ordering. Deletion resources only carry progressIndex ordering information, so this avoids changing insertion/deletion ordering semantics.

The option can be set to false to keep the previous progressIndex-based ordering.

Tests / Coverage

Added or updated unit coverage for:

• the new option defaulting to true,
• both source.* and extractor.* disable keys,
• ordering by real query coverage priority, including version before file-name timestamp and unseq after seq,
• falling back to progressIndex order when deletions are extracted together with insertions,
• excluding filtered-out TsFiles from reordered historical progress calculation,
• reporting final global max historical progress after reordered historical resources are consumed,
• computing safe incremental progress per time partition,
• reporting TimePartitionProgressIndex for out-of-global-order cross-partition TsFiles,
• restart filtering with partition-local progress coverage,
• TimePartitionProgressIndex merge, coverage, hybrid blending, and serialization,
• preserving the no-progress-report flag when a TsFile event is shallow-copied,
• making generated tablet events inherit the source TsFile event's no-progress-report behavior.

I did not add an integration test because ordinary end-to-end SQL write scenarios do not deterministically create the exact file-version/file-timestamp/seq-unseq priority conflicts needed for this regression. The deterministic behavior that matters here is the historical source's resource ordering and progress reporting contract, which is covered by unit tests with explicit TsFileResource setup.

Local verification:

• mvn spotless:apply -pl iotdb-core/node-commons,iotdb-core/datanode
• git diff --check
• mvn -pl iotdb-core/node-commons -Dtest=PipeMetaDeSerTest test
• mvn -pl iotdb-core/node-commons -DskipTests install
• mvn -pl iotdb-core/datanode "-Dtest=TsFileResourceProgressIndexTest,PipeHistoricalDataRegionTsFileAndDeletionSourceTest" test
• mvn -pl iotdb-core/datanode "-Dtest=PipeHistoricalDataRegionTsFileAndDeletionSourceTest" -DforkCount=1 test

[codecov]

Codecov Report

❌ Patch coverage is 63.47305% with 122 lines in your changes missing coverage. Please review.
✅ Project coverage is 41.89%. Comparing base (220e7a3) to head (dd886fa).
⚠️ Report is 30 commits behind head on master.

Files with missing lines	Patch %	Lines
...peHistoricalDataRegionTsFileAndDeletionSource.java	63.44%	68 Missing ⚠️
...nsensus/index/impl/TimePartitionProgressIndex.java	58.67%	50 Missing ⚠️
.../event/common/tsfile/PipeTsFileInsertionEvent.java	71.42%	2 Missing ⚠️
...tdb/commons/consensus/index/ProgressIndexType.java	50.00%	2 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #18088      +/-   ##
============================================
+ Coverage     41.65%   41.89%   +0.24%     
  Complexity      318      318              
============================================
  Files          5296     5298       +2     
  Lines        371663   374390    +2727     
  Branches      48088    48345     +257     
============================================
+ Hits         154819   156859    +2040     
- Misses       216844   217531     +687     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[sonarqubecloud]

Quality Gate passed

Issues
16 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud