Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature]: curvefs metaserver support asynchronous snapshot #2691

Merged
merged 1 commit into from
Oct 9, 2023
Merged

[Feature]: curvefs metaserver support asynchronous snapshot #2691

merged 1 commit into from
Oct 9, 2023

Conversation

NaturalSelect
Copy link
Contributor

@NaturalSelect NaturalSelect commented Aug 13, 2023 •
edited

What problem does this PR solve?

Issue Number: #1617

Problem Summary:

  • on_snapshot_save() will block the apply thread, it should to be asynchronous.

What is changed and how it works?

What's Changed:

  • Add a logIndex argument to each meta operations.
  • Filter the logs that already applied.

How it Works:

  • Due to the non-idempotency of the log application, we store some sentinel values on KVStorage, there are:
    • inode applied index - record the application progress of the InodeStorage .
    • dentry applied index - record the application progress of the DentryStorage.
    • handle tx applied index - record the application progress of the CommitTx/RollbackTx.
  • Move pengdTx to KVStorage from MetaFStream.
  • Move dentry count and inode count to KVStorage from MetaFStream.
  • Set allow_write_stall to false.
  • Invoke KVStorage::Checkpoint() in other threads.

See also:

Side effects(Breaking backward compatibility? Performance regression?):

  • The older version of metaserver cannot use newer version of metaserver snapshot.
  • If there are Inode and Dentry in MetaFStream, there cannot be loaded by current version.

Check List

  • Relevant documentation/comments is changed or added
  • I acknowledge that all my contributions will be made under the project's license

@NaturalSelect NaturalSelect force-pushed the snapshot-dev branch 11 times, most recently from 8694473 to c114ca8 Compare August 17, 2023 08:54
@NaturalSelect
Copy link
Contributor Author

@wu-hanqing PTAL.

@NaturalSelect NaturalSelect force-pushed the snapshot-dev branch 2 times, most recently from ecd3a5a to 3e32a51 Compare August 18, 2023 06:57
@NaturalSelect NaturalSelect changed the title [WIP][Feature]: curvefs metaserver support asynchronous snapshot [Feature]: curvefs metaserver support asynchronous snapshot Aug 18, 2023
@NaturalSelect NaturalSelect marked this pull request as ready for review August 18, 2023 09:24
wu-hanqing
wu-hanqing reviewed Aug 19, 2023
View reviewed changes
curvefs/src/metaserver/copyset/meta_operator.h Outdated Show resolved Hide resolved
curvefs/src/metaserver/copyset/copyset_node.cpp Outdated Show resolved Hide resolved
curvefs/src/metaserver/copyset/copyset_node.cpp Outdated Show resolved Hide resolved
curvefs/src/metaserver/metastore_fstream.cpp Outdated Show resolved Hide resolved
curvefs/src/metaserver/metastore_fstream.cpp Outdated Show resolved Hide resolved
curvefs/src/metaserver/transaction.cpp Outdated Show resolved Hide resolved
@NaturalSelect NaturalSelect force-pushed the snapshot-dev branch 2 times, most recently from 41a2dc3 to f64b231 Compare August 19, 2023 16:20
@NaturalSelect NaturalSelect force-pushed the snapshot-dev branch 4 times, most recently from 255b660 to 1b42470 Compare August 20, 2023 07:32
wu-hanqing
wu-hanqing reviewed Aug 31, 2023
View reviewed changes
.bazelrc Outdated Show resolved Hide resolved
curvefs/src/metaserver/storage/memory_storage.cpp Outdated Show resolved Hide resolved
curvefs/src/metaserver/metastore.h Outdated Show resolved Hide resolved
curvefs/src/metaserver/metastore.cpp Outdated Show resolved Hide resolved
curvefs/src/metaserver/metastore_fstream.cpp Outdated Show resolved Hide resolved
curvefs/src/metaserver/transaction.h Outdated Show resolved Hide resolved
curvefs/src/metaserver/transaction.h Outdated Show resolved Hide resolved
curvefs/src/metaserver/dentry_storage.cpp Outdated Show resolved Hide resolved
curvefs/src/metaserver/dentry_storage.cpp Show resolved Hide resolved
curvefs/src/metaserver/dentry_storage.cpp Outdated Show resolved Hide resolved
@wu-hanqing
Copy link
Contributor

@Wine93 PTAL~

@wu-hanqing wu-hanqing requested a review from Wine93 August 31, 2023 09:36
@NaturalSelect
Copy link
Contributor Author

I will make a force push later.

wu-hanqing
wu-hanqing reviewed Sep 5, 2023
View reviewed changes
curvefs/src/metaserver/inode_storage.cpp Show resolved Hide resolved
curvefs/src/metaserver/inode_storage.cpp Outdated Show resolved Hide resolved
curvefs/src/metaserver/inode_storage.cpp Outdated Show resolved Hide resolved
curvefs/src/metaserver/inode_storage.cpp Show resolved Hide resolved
curvefs/src/metaserver/inode_storage.cpp Outdated Show resolved Hide resolved
curvefs/src/metaserver/dentry_storage.cpp Outdated Show resolved Hide resolved
curvefs/src/metaserver/dentry_storage.cpp Outdated Show resolved Hide resolved
curvefs/src/metaserver/dentry_storage.cpp Outdated Show resolved Hide resolved
@wu-hanqing
Copy link
Contributor

cicheck

@NaturalSelect NaturalSelect force-pushed the snapshot-dev branch 2 times, most recently from 802a34d to 71bb1e7 Compare September 8, 2023 10:11
@NaturalSelect
Copy link
Contributor Author

@Wine93 PTAL

@NaturalSelect NaturalSelect force-pushed the snapshot-dev branch 4 times, most recently from 2b51efb to 6d3cf1d Compare September 15, 2023 08:00
@NaturalSelect
Copy link
Contributor Author

@Wine93 PTAL~

@Wine93
Copy link
Contributor

Wine93 commented Sep 19, 2023

@Wine93 PTAL~

Reviewing.

@NaturalSelect
Copy link
Contributor Author

@Wine93 PTAL~

Reviewing.

Are there any problems in the pull request?

@Wine93
Copy link
Contributor

Wine93 commented Sep 26, 2023

Hi @NaturalSelect, have you test the case that upgrade metaserver to this version?

@NaturalSelect
Copy link
Contributor Author

Hi @NaturalSelect, have you test the case that upgrade metaserver to this version?

Hello @Wine93.

Do you mean start a metaserver from a old version snapshot? I don't know how to build a test case like that.

@Wine93
Copy link
Contributor

Wine93 commented Sep 26, 2023

Hi @NaturalSelect, have you test the case that upgrade metaserver to this version?

Hello @Wine93.

Do you mean start a metaserver from a old version snapshot? I don't know how to build a test case like that.

yep, you can deploy a CurveFS cluster with old version first and then use curveadm upgrade to upgrade metaserver.

@NaturalSelect
Copy link
Contributor Author

Hi @NaturalSelect, have you test the case that upgrade metaserver to this version?

Hello @Wine93.
Do you mean start a metaserver from a old version snapshot? I don't know how to build a test case like that.

yep, you can deploy a CurveFS cluster with old version first and then use curveadm upgrade to upgrade metaserver.

Thank you for your help.

@NaturalSelect
Copy link
Contributor Author

Hi @NaturalSelect, have you test the case that upgrade metaserver to this version?

Hello @Wine93.
Do you mean start a metaserver from a old version snapshot? I don't know how to build a test case like that.

yep, you can deploy a CurveFS cluster with old version first and then use curveadm upgrade to upgrade metaserver.

I cannot deploy client on WSL2, because kernel module FUSE is statically compiled into the WSL2 kernel.

It makes the curveadm cannot find FUSE module.

Can you help me test it?

See also:WSL Issue

@Wine93
Copy link
Contributor

Wine93 commented Sep 27, 2023 •
edited

Hi @NaturalSelect, have you test the case that upgrade metaserver to this version?

Hello @Wine93.
Do you mean start a metaserver from a old version snapshot? I don't know how to build a test case like that.

yep, you can deploy a CurveFS cluster with old version first and then use curveadm upgrade to upgrade metaserver.

I cannot deploy client on WSL2, because kernel module FUSE is statically compiled into the WSL2 kernel.

It makes the curveadm cannot find FUSE module.

Can you help me test it?

See also:WSL Issue

You can mount filesystem by insecure mode, try it:

curveadm mount myfs /path/to/mount -k

@NaturalSelect
feat(fs): metaserver support asynchronous snapshot
bba13b6 
close: opencurve#1617

old metaserver snapshot that contain `inode` and `dentry` incompatible with new version

Signed-off-by: NaturalSelect <2145973003@qq.com>
@wu-hanqing wu-hanqing merged commit 9f60a9a into opencurve:master Oct 9, 2023
4 checks passed
@NaturalSelect NaturalSelect deleted the snapshot-dev branch October 30, 2023 07:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Wine93 Wine93 Wine93 approved these changes

@wu-hanqing wu-hanqing wu-hanqing approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@NaturalSelect @wu-hanqing @Wine93