Skip to content

Fix ConfignNode LoadManager NPE when removing datanodes#14016

Merged
OneSizeFitsQuorum merged 1 commit intomasterfrom
jira1088
Nov 8, 2024
Merged

Fix ConfignNode LoadManager NPE when removing datanodes#14016
OneSizeFitsQuorum merged 1 commit intomasterfrom
jira1088

Conversation

@OneSizeFitsQuorum
Copy link
Copy Markdown
Contributor

When Datanodes are currently removed, the deletion of some maps of cn loadmanager and the callback of asynchronous rpc may have some concurrency conflicts. In this case, some robustness needs to be improved

Signed-off-by: OneSizeFitQuorum <tanxinyu@apache.org>
@sonarqubecloud
Copy link
Copy Markdown

sonarqubecloud bot commented Nov 7, 2024

Copy link
Copy Markdown
Collaborator

@HxpSerein HxpSerein left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM~

@codecov
Copy link
Copy Markdown

codecov bot commented Nov 7, 2024

Codecov Report

Attention: Patch coverage is 52.94118% with 8 lines in your changes missing coverage. Please review.

Project coverage is 39.87%. Comparing base (bc4dd72) to head (75f86a6).
Report is 12 commits behind head on master.

Files with missing lines Patch % Lines
...iotdb/confignode/manager/load/cache/LoadCache.java 52.94% 8 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #14016      +/-   ##
============================================
- Coverage     39.95%   39.87%   -0.09%     
  Complexity       71       71              
============================================
  Files          4191     4201      +10     
  Lines        265837   266569     +732     
  Branches      32266    32323      +57     
============================================
+ Hits         106222   106283      +61     
- Misses       159615   160286     +671     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Copy Markdown
Collaborator

@liyuheng55555 liyuheng55555 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent use of Optional !

@OneSizeFitsQuorum OneSizeFitsQuorum merged commit b39c325 into master Nov 8, 2024
@OneSizeFitsQuorum OneSizeFitsQuorum deleted the jira1088 branch November 8, 2024 03:41
HxpSerein pushed a commit to HxpSerein/iotdb that referenced this pull request Dec 3, 2024
Signed-off-by: OneSizeFitQuorum <tanxinyu@apache.org>
(cherry picked from commit b39c325)
OneSizeFitsQuorum added a commit that referenced this pull request Dec 6, 2024
…14301)

* Use CountDownLatch to replace Semaphore in IoTConsensus log dispatcher closing #13517

(cherry picked from commit 5599859)

* Remove datanode optimization (#13559)

(cherry picked from commit cc73946)

* Split CnToDnRequestType to sync and async & Add check for adding new request type (#13660)

(cherry picked from commit 8187498)

* Enhance Remove DataNode Test (#13809)

(cherry picked from commit 063fddd)

* [remove datanode] Identify and display invalid nodes before removing #13987

(cherry picked from commit 387b4bf)

* [remove datanode] Fixed jvm heap memory in remove datanode script (#13983)

(cherry picked from commit 78e6f3a)

* [remove datanode] Enhance remove message on environment with only ConfigNode #14007

(cherry picked from commit db7c522)

* Remove useless log "datanodeId -1" #14035

(cherry picked from commit 8a92aaf)

* [remove datanode] Log the node information when submitting RegionMigrateProcedure (#14051)

(cherry picked from commit e437ef3)

* [remove datanode] Enhance remove message on environment with only ConfigNode #14123

(cherry picked from commit f3afc20)

* [remove datanode] Refuse to remove when there are any other unknown or readonly DataNodes in the consensus group (#14145)

* add uknown check

* remove useless check

(cherry picked from commit adcedc0)

* Fix stopping cn leader stuck when region region migration #14175

Signed-off-by: OneSizeFitQuorum <tanxinyu@apache.org>
(cherry picked from commit 9b22f7c)

* [remove datanode] Do not allow regions to inherit the Removing state from datanode (#14185)

(cherry picked from commit 8a6405c)

* [remove datanode] Do not disable the entire region group for one removing region (#14241)

* fix disabled

* fix disabled

(cherry picked from commit 041d292)

* [remove datanode] Not re-submit region migration procedure when leader change or reboot #14277

(cherry picked from commit d101d76)

* Fix ConfignNode LoadManager NPE when removing datanodes #14016

Signed-off-by: OneSizeFitQuorum <tanxinyu@apache.org>
(cherry picked from commit b39c325)

* Fix ConfigNode Partition Metric NPE bug #14144

Signed-off-by: OneSizeFitQuorum <tanxinyu@apache.org>
(cherry picked from commit 4a76dfb)

* Enhance procedure recover policy

(cherry picked from commit 496c62e)

* remove table

* fix IT

---------

Co-authored-by: Li Yu Heng <liyuheng55555@126.com>
Co-authored-by: Potato <tanxinyu@apache.org>
Caideyipi pushed a commit to Caideyipi/iotdb that referenced this pull request Mar 25, 2026
…pache#14301)

* Use CountDownLatch to replace Semaphore in IoTConsensus log dispatcher closing apache#13517

(cherry picked from commit 5599859)

* Remove datanode optimization (apache#13559)

(cherry picked from commit cc73946)

* Split CnToDnRequestType to sync and async & Add check for adding new request type (apache#13660)

(cherry picked from commit 8187498)

* Enhance Remove DataNode Test (apache#13809)

(cherry picked from commit 063fddd)

* [remove datanode] Identify and display invalid nodes before removing apache#13987

(cherry picked from commit 387b4bf)

* [remove datanode] Fixed jvm heap memory in remove datanode script (apache#13983)

(cherry picked from commit 78e6f3a)

* [remove datanode] Enhance remove message on environment with only ConfigNode apache#14007

(cherry picked from commit db7c522)

* Remove useless log "datanodeId -1" apache#14035

(cherry picked from commit 8a92aaf)

* [remove datanode] Log the node information when submitting RegionMigrateProcedure (apache#14051)

(cherry picked from commit e437ef3)

* [remove datanode] Enhance remove message on environment with only ConfigNode apache#14123

(cherry picked from commit f3afc20)

* [remove datanode] Refuse to remove when there are any other unknown or readonly DataNodes in the consensus group (apache#14145)

* add uknown check

* remove useless check

(cherry picked from commit adcedc0)

* Fix stopping cn leader stuck when region region migration apache#14175

Signed-off-by: OneSizeFitQuorum <tanxinyu@apache.org>
(cherry picked from commit 9b22f7c)

* [remove datanode] Do not allow regions to inherit the Removing state from datanode (apache#14185)

(cherry picked from commit 8a6405c)

* [remove datanode] Do not disable the entire region group for one removing region (apache#14241)

* fix disabled

* fix disabled

(cherry picked from commit 041d292)

* [remove datanode] Not re-submit region migration procedure when leader change or reboot apache#14277

(cherry picked from commit d101d76)

* Fix ConfignNode LoadManager NPE when removing datanodes apache#14016

Signed-off-by: OneSizeFitQuorum <tanxinyu@apache.org>
(cherry picked from commit b39c325)

* Fix ConfigNode Partition Metric NPE bug apache#14144

Signed-off-by: OneSizeFitQuorum <tanxinyu@apache.org>
(cherry picked from commit 4a76dfb)

* Enhance procedure recover policy

(cherry picked from commit 496c62e)

* remove table

* fix IT

---------

Co-authored-by: Li Yu Heng <liyuheng55555@126.com>
Co-authored-by: Potato <tanxinyu@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants