Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(port): support multiple datanode/metanode instances using dif… #3016

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

zhang111feng
Copy link

…ferent port on a single node

What this PR does / why we need it:
Support multiple datanode/metanode instances using different port on a single node.In the previous version, every datanode or metanode in the same raft group is required to be configured with the same raftHeartBeatPort and replicaPort because each node splices ports of peers through its own port.
Which issue this PR fixes:

fixes #3004

Special notes for your reviewer:

Release note:

Copy link

netlify bot commented Jan 18, 2024

Deploy Preview for cubefs-check ready!

Name Link
🔨 Latest commit ce18fef
🔍 Latest deploy log https://app.netlify.com/sites/cubefs-check/deploys/65d85733798af30008dbac6d
😎 Deploy Preview https://deploy-preview-3016--cubefs-check.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

Copy link

codecov bot commented Jan 18, 2024

Codecov Report

Attention: Patch coverage is 21.39175% with 305 lines in your changes are missing coverage. Please review.

Project coverage is 41.99%. Comparing base (4c1ca65) to head (ce18fef).
Report is 1 commits behind head on master.

Files Patch % Lines
datanode/partition_raft.go 0.00% 93 Missing ⚠️
master/cluster.go 10.60% 59 Missing ⚠️
datanode/wrap_operator.go 0.00% 53 Missing ⚠️
master/data_partition.go 24.48% 35 Missing and 2 partials ⚠️
raftstore/resolver.go 0.00% 13 Missing ⚠️
metanode/partition.go 0.00% 11 Missing ⚠️
metanode/raft_server.go 0.00% 8 Missing ⚠️
datanode/partition_raftfsm.go 0.00% 7 Missing ⚠️
metanode/partition_fsmop.go 0.00% 7 Missing ⚠️
master/operate_util.go 0.00% 6 Missing ⚠️
... and 5 more
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #3016      +/-   ##
==========================================
+ Coverage   41.95%   41.99%   +0.03%     
==========================================
  Files         577      577              
  Lines      119055   119381     +326     
==========================================
+ Hits        49946    50129     +183     
- Misses      64831    64974     +143     
  Partials     4278     4278              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@leonrayang leonrayang added the first good pr first good pr label Jan 23, 2024
@Victor1319
Copy link
Member

It's a really good and big job, and maybe you should consider the process of upgrading an old cluster because the default values for raftport and replica port are 0 when the older nodes were registered. This could potentially cause issues.

@zhang111feng
Copy link
Author

zhang111feng commented Feb 1, 2024

It's a really good and big job, and maybe you should consider the process of upgrading an old cluster because the default values for raftport and replica port are 0 when the older nodes were registered. This could potentially cause issues.

I have made some attempts to resolve the compatibility issues when upgrading the old version.I have finished the dataNode part, but I think it has something wrong and there is still a state inconsistency problem when synchronizing raft information.The scenario I tested was to restart all the Masters first, and then restart the dataNodes one by one.
I think I need some help and suggestions for improvement, and I will finish the metaNode part after that.

@zhang111feng zhang111feng changed the title [Enhancement]: support multiple datanode/metanode instances using dif… fix(port): support multiple datanode/metanode instances using dif… Feb 1, 2024
@zhang111feng zhang111feng force-pushed the multiport branch 2 times, most recently from 5b9ddab to 19ad51b Compare February 1, 2024 03:13
@zhang111feng zhang111feng changed the title fix(port): support multiple datanode/metanode instances using dif… feat(port): support multiple datanode/metanode instances using dif… Feb 1, 2024
@zhang111feng
Copy link
Author

The problem I encounter now is that after dataNodes are restarted one by one, although the META information in the locally stored dp file can be updated to the new version with port information, the META information may not be correctly changed when other member changes such as del-replica are performed.

@zhang111feng
Copy link
Author

I found that in the branch <develop-hybridcloud-lifecycle>, the branch also changes the structure of dataNode with adding mediaType when it adds the storage tiering function. How does that solve the problem in the process of upgrading an old cluster?

…ent port on a single node

Signed-off-by: CNyuchenZhang <yuchen.zhang6688@gmail.com>
@zhang111feng
Copy link
Author

I think the problem mentioned doesn't exist anymore, but I'm not sure if there are other problems because I haven't fully understood the mechanism here.

Signed-off-by: CNyuchenZhang <yuchen.zhang6688@gmail.com>
@true1064
Copy link
Contributor

I found that in the branch , the branch also changes the structure of dataNode with adding mediaType when it adds the storage tiering function. How does that solve the problem in the process of upgrading an old cluster?
in branch , filed mediaType is only on the master, there no filed added on datanode

@leonrayang leonrayang reopened this Feb 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
first good pr first good pr
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Enhancement]: support multiple datanode/metanode instances using different port on a single node
4 participants