Fixed concurrency issues caused by write and flush sorting during query execution

[shuwenwei]

Description

When writing and flushing data, the internal TVList sorting mechanism could conflict with active queries.
✅ What’s Changed
✔️ Concurrency control enhancements
• The write and flush sort logic has been revised to safely handle concurrent queries by introducing synchronization and controlled cloning of TVLists under contention.
• The implementation ensures that sorting operations do not corrupt or disrupt query access on the same data structures.

✔️ Memtable & TVList adjustments
• Modified internal MemChunk and TVList management to better isolate sort state changes from active queries.
• New clone-on-flush behaviour ensures that a working TVList can be safely sorted even when there are concurrent readers.

[codecov]

Codecov Report

❌ Patch coverage is 57.89474% with 48 lines in your changes missing coverage. Please review.
✅ Project coverage is 39.51%. Comparing base (7ee7a83) to head (192d2b8).
⚠️ Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
...e/dataregion/memtable/AlignedWritableMemChunk.java	63.82%	17 Missing ⚠️
...che/iotdb/db/utils/datastructure/BinaryTVList.java	0.00%	5 Missing ⚠️
...he/iotdb/db/utils/datastructure/BooleanTVList.java	0.00%	5 Missing ⚠️
...che/iotdb/db/utils/datastructure/DoubleTVList.java	0.00%	5 Missing ⚠️
...ache/iotdb/db/utils/datastructure/FloatTVList.java	0.00%	5 Missing ⚠️
...apache/iotdb/db/utils/datastructure/IntTVList.java	0.00%	5 Missing ⚠️
...pache/iotdb/db/utils/datastructure/LongTVList.java	0.00%	5 Missing ⚠️
...geengine/dataregion/memtable/WritableMemChunk.java	93.33%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #17193      +/-   ##
============================================
- Coverage     39.51%   39.51%   -0.01%     
  Complexity      282      282              
============================================
  Files          5101     5101              
  Lines        341807   341876      +69     
  Branches      43518    43524       +6     
============================================
+ Hits         135062   135081      +19     
- Misses       206745   206795      +50     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Copilot]

Pull request overview

This PR addresses a concurrency hazard between query execution and flush-time sorting of in-memory TVLists by introducing a dedicated “flush-sort clone” path and routing flush encoding through a stable workingListForFlush.

Changes:

• Added cloneForFlushSort() to TVList and implemented it across TVList variants to clone sort-sensitive structures (timestamps/indices) for flush sorting under contention.
• Centralized flush sorting in AbstractWritableMemChunk.sortTvListForFlush() and updated memchunk encoding to read from workingListForFlush rather than the mutable working list.
• Added a regression test covering the interaction between query sorting and flush sorting.

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
iotdb-core/datanode/src/test/java/org/apache/iotdb/db/storageengine/dataregion/memtable/PrimitiveMemTableTest.java	Adds a regression test for query execution interacting with flush-sort behavior.
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/utils/datastructure/TVList.java	Introduces cloneForFlushSort() as a new abstraction for flush-safe sorting.
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/utils/datastructure/LongTVList.java	Implements flush-sort cloning for long TVLists.
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/utils/datastructure/IntTVList.java	Implements flush-sort cloning for int/date TVLists.
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/utils/datastructure/FloatTVList.java	Implements flush-sort cloning for float TVLists.
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/utils/datastructure/DoubleTVList.java	Implements flush-sort cloning for double TVLists.
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/utils/datastructure/BooleanTVList.java	Implements flush-sort cloning for boolean TVLists.
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/utils/datastructure/BinaryTVList.java	Implements flush-sort cloning for binary/text TVLists.
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/utils/datastructure/AlignedTVList.java	Implements flush-sort cloning for aligned TVLists.
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/memtable/WritableMemChunk.java	Switches flush encoding to use workingListForFlush rather than the mutable list.
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/memtable/IWritableMemChunk.java	Adds releaseTemporaryTvListForFlush() to allow releasing the flush-time clone reference.
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/memtable/AlignedWritableMemChunk.java	Switches aligned flush encoding to use workingListForFlush.
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/memtable/AbstractWritableMemChunk.java	Centralizes flush sort logic and introduces workingListForFlush lifecycle.
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/flush/MemTableFlushTask.java	Releases temporary flush TVList reference after encoding.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[sonarqubecloud]

Quality Gate passed

Issues
11 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud