[To dev/1.3] Optimize partition cache getRegionReplicaSet interface performance by batching (#15396)#16030
Closed
Caideyipi wants to merge 760 commits intoapache:masterfrom
Closed
[To dev/1.3] Optimize partition cache getRegionReplicaSet interface performance by batching (#15396)#16030Caideyipi wants to merge 760 commits intoapache:masterfrom
Caideyipi wants to merge 760 commits intoapache:masterfrom
Conversation
…nc manager is closed (apache#14929) (apache#14947) (cherry picked from commit bb8a5f7)
… number of DRs & Modify the implementation of the poll method in PipeRealtimePriorityBlockingQueue to reduce commit queue backlog & Adjust the default thread count related to Pipe for better performance & Significantly reduce pipeMemoryAllocateRetryIntervalMs & Provide a switch for memory control of ConnectorReadFileBuffer (apache#14917) (apache#14946) (cherry picked from commit b498285) Co-authored-by: Steve Yurong Su <rong@apache.org>
) (cherry picked from commit 9733c75)
…hecking of StateMachineProcedure apache#14959 (apache#14961) (cherry picked from commit fb2cb7a)
* Finished * optimize autoCleanPartitionTable * Update SeriesPartitionTable.java --------- Co-authored-by: Potato <tanxinyu@apache.org>
… core / 2 & IoTConsensusV2: Flush old leader when leader transfer & Delay the execution of invalidateSchemaCache after leader change (apache#14963) * Adjust default data region num per node from 5 to to cpu core / 2 (apache#14348) * finish Signed-off-by: OneSizeFitQuorum <tanxinyu@apache.org> * fix it & ut Signed-off-by: OneSizeFitQuorum <tanxinyu@apache.org> --------- Signed-off-by: OneSizeFitQuorum <tanxinyu@apache.org> (cherry picked from commit f1714d0) * merge * merge * spotless --------- Co-authored-by: Potato <tanxinyu@apache.org>
…ment (apache#14965) (apache#14969) * add todo * get memory size (cherry picked from commit e1840d0)
Signed-off-by: OneSizeFitQuorum <tanxinyu@apache.org>
…ection fails (apache#14949) (apache#14973) (cherry picked from commit b638a70)
…of OPC UA Sink (apache#14964) (apache#14978) Co-authored-by: Steve Yurong Su <rong@apache.org>
(cherry picked from commit aad2caa)
Co-authored-by: 2b3c511 <rong.li@timecho.com>
… force close situation & fix reference count management for iteration snapshot (apache#14966) (apache#14984)
…e by default apache#14983 (apache#14986) Signed-off-by: OneSizeFitQuorum <tanxinyu@apache.org>
…pache#14987) * add phi accrual for node failure detection * fix unit error * address review issues * address review issues * address review issues * address review issues * address review issues
…ion of Alter Pipe Procedure. (apache#14982) (apache#15000) (cherry picked from commit 662de31) Co-authored-by: Zhenyu Luo <luoluoyuyu2001@gmail.com>
…ent & Subscription: add parameter control tablet event batch estimate & close memory block in tablet event for iteration snapshot (apache#14991) (apache#15001) (cherry picked from commit c5a2e43) Co-authored-by: VGalaxies <vgalaxies@apache.org>
* Fix frozen IoTConsensus sync lag * spotless (cherry picked from commit f9f0574)
…che#15029) (cherry picked from commit e8d07c1)
* perf: improve seq inserting * rename MergeSortTvListIterator to MergeSortTVListIterator * fix: concurrent indices modification during query sort and flush sort * revert: iterate rows in aligned memChunk * fix: testMemAlignedChunkLoader test * fix: no need to synchronize list since sort is already an synchronized method * fix: optimize mininum time update in TVList * perf: special case for merge sort iterator when no handover occurs
…5922) (apache#15932) Bumps org.apache.commons:commons-lang3 from 3.13.0 to 3.18.0. --- updated-dependencies: - dependency-name: org.apache.commons:commons-lang3 dependency-version: 3.18.0 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…dByPipe mark in tsfile resources (apache#15934)
…pped (2 situations) (apache#15910) (apache#15929) (cherry picked from commit bf8329b)
…add capture history file log (apache#15952) (apache#15951) * Pipe:Fix DataNodeShutdownHook waiting report logic and add capture history file log * update log
… resetting script (apache#15957) * Pipe: Added progressIndex reset function to the resource resetting script * bat-fix
…toricalDataRegionTsFileExtractor (apache#15963)
* Update PipeTsFileResourceManager.java * Update PipeDataRegionAssigner.java * Update PipeTsFileResourceManager.java * Update PipeTsFileResourceManager.java
… to MQTT custom plugins" and "Disable the parameter 'ALLOW_ZERO-BYTE_CITE_SIDED_PROPERTY1 NAME' and fix null pointer exception" (apache#15985) * cp-Enable the kernel to support passing topics to MQTT custom plugins (apache#15523) * cp-Disable the parameter 'ALLOW_ZERO-BYTE_CITE_SIDED_PROPERTY1 NAME' and fix null pointer exception (apache#15866) Co-authored-by: shiwenyan <wenyan.shi@timecho.com> --------- Co-authored-by: shiwenyan <wenyan.shi@timecho.com>
…ving end causes all synchronization to be suspended (apache#15962) (apache#15991) * Pipe: Fix the problem that a cluster crash in the multi-cluster receiving end causes all synchronization to be suspended * add IsEndPointAlive Function * add IsEndPointAlive Function * add IsEndPointAlive Function * add pipeCheckSyncAllClientLiveTimeIntervalMs config * fix * fix * fix * fix (cherry picked from commit d34054d)
…y DN to CN (apache#15983) (apache#15997) (cherry picked from commit 0340423)
…n realtime-only mode (apache#15999) * Pipe: Do not transfer historical tsFiles when restarts in realtime-only mode (apache#15996) * cp
…pache#15998) (apache#16004) Adds logic to check if a hardlink already exists when creating a hardlink. If the existing file matches the source (by MD5), a warning is logged and creation is skipped. If not, the existing file is deleted and creation is retried, with error handling and logging for failures. (cherry picked from commit de05c67)
…vents in realtime extractors (apache#15975) (apache#16003) * Pipe: Report progress index for filter-outed insertion events in realtime extractors (apache#15975) * fix * fix (cherry picked from commit 7214e1f) * Update PipeDataRegionAssigner.java
…apache#16010) * remove-useless * fix
…T & Copied some historical filter logic from dev/1.3 (apache#16019) (apache#16020) * logger * fix-ut * Update PipeHistoricalDataRegionTsFileAndDeletionExtractor.java
…me metric which have not be sent (apache#16015) (apache#16016)
…che#16006) (apache#16012) * Update IoTDBPipePermissionIT.java * refacotr * revert=pom * fix-lock
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
As the title said.
This PR has:
for an unfamiliar reader.
for code coverage.
Key changed/added classes (or packages if there are too many classes) in this PR