Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

curvefs/metaserver: Can metaserver's raft snapshot implementation be asynchronous #1617

Closed
ilixiaocui opened this issue Jun 22, 2022 · 12 comments
Assignees
Labels

Comments

@ilixiaocui
Copy link
Contributor

Is your feature request related to a problem? (你需要的功能是否与某个问题有关?)
Now the raft snapshot on the metaserver side blocks normal io,we need to evaluate whether snapshots can be implemented in an asynchronous manner.

Describe the solution you'd like (描述你期望的解决方法)

Describe alternatives you've considered (描述你想到的折衷方案)

Additional context/screenshots (更多上下文/截图)

@ilixiaocui ilixiaocui added the enhancement improve feature label Jun 22, 2022
@ilixiaocui ilixiaocui changed the title Can metaserver's raft snapshot implementation be asynchronous [CurveFS] Can metaserver's raft snapshot implementation be asynchronous Jun 22, 2022
@ilixiaocui ilixiaocui changed the title [CurveFS] Can metaserver's raft snapshot implementation be asynchronous curvefs/metaserver: Can metaserver's raft snapshot implementation be asynchronous Jun 22, 2022
@wuhongsong
Copy link
Contributor

@ilixiaocui

@ilixiaocui ilixiaocui added the good first issue Good for newcomers label Feb 15, 2023
@ilixiaocui
Copy link
Contributor Author

@ilixiaocui

still have this problem, tag the issue as good first issue.

@Ziy1-Tan
Copy link
Contributor

take

@Ziy1-Tan
Copy link
Contributor

Ziy1-Tan commented Apr 6, 2023

I have read https://github.com/baidu/braft/blob/master/docs/cn/server.md, I think on_snapshot_load and on_snapshot_save block io, point out my misunderstanding if any plz.

@wu-hanqing
Copy link
Contributor

on_snapshot_load doesn't matters because when a replica is loading from snapshot, it should not be a leader of this group and doesn't serve the end user's request.

Currently, we implement on_snapshot_save synchronous because we need to flush rocksdb's memtable to disk, so in the progress other requests are blocked.

But this can be implemented using other methods, for example, we can combine each operation with raft apply index, and when a memtable is flushed to disk then we can callback raft to truncate WAL with the lastest apply index in current memtable.

@Ziy1-Tan
Copy link
Contributor

Ziy1-Tan commented Apr 6, 2023

Let me take a look, thx!:)

@zhanghuidinah
Copy link
Member

@pengpengSir
Copy link
Contributor

can i take this issue?

@wu-hanqing
Copy link
Contributor

can i take this issue?

hi @pengpengSir, since @Ziy1-Tan has already startted, you can find other interesting issues in #2334

@NaturalSelect
Copy link
Contributor

Can I apply for it?

@wuhongsong
Copy link
Contributor

Can I apply for it?

of course,You are welcome to participate, please add ”OpenCurve_bot“ on WeChat.

@1797818494
Copy link

Can I apply for it?

NaturalSelect added a commit to NaturalSelect/curve that referenced this issue Aug 18, 2023
close: opencurve#1617

Signed-off-by: NaturalSelect <2145973003@qq.com>
NaturalSelect added a commit to NaturalSelect/curve that referenced this issue Aug 19, 2023
close: opencurve#1617

Signed-off-by: NaturalSelect <2145973003@qq.com>
NaturalSelect added a commit to NaturalSelect/curve that referenced this issue Aug 19, 2023
close: opencurve#1617

Signed-off-by: NaturalSelect <2145973003@qq.com>
NaturalSelect added a commit to NaturalSelect/curve that referenced this issue Aug 19, 2023
close: opencurve#1617

Signed-off-by: NaturalSelect <2145973003@qq.com>
NaturalSelect added a commit to NaturalSelect/curve that referenced this issue Aug 19, 2023
close: opencurve#1617

Signed-off-by: NaturalSelect <2145973003@qq.com>
NaturalSelect added a commit to NaturalSelect/curve that referenced this issue Aug 20, 2023
close: opencurve#1617

Signed-off-by: NaturalSelect <2145973003@qq.com>
NaturalSelect added a commit to NaturalSelect/curve that referenced this issue Aug 20, 2023
close: opencurve#1617

Signed-off-by: NaturalSelect <2145973003@qq.com>
NaturalSelect added a commit to NaturalSelect/curve that referenced this issue Aug 31, 2023
close: opencurve#1617

Signed-off-by: NaturalSelect <2145973003@qq.com>
NaturalSelect added a commit to NaturalSelect/curve that referenced this issue Aug 31, 2023
close: opencurve#1617

Signed-off-by: NaturalSelect <2145973003@qq.com>
NaturalSelect added a commit to NaturalSelect/curve that referenced this issue Sep 5, 2023
close: opencurve#1617

Signed-off-by: NaturalSelect <2145973003@qq.com>
NaturalSelect added a commit to NaturalSelect/curve that referenced this issue Sep 5, 2023
close: opencurve#1617

Signed-off-by: NaturalSelect <2145973003@qq.com>
NaturalSelect added a commit to NaturalSelect/curve that referenced this issue Sep 8, 2023
close: opencurve#1617

Signed-off-by: NaturalSelect <2145973003@qq.com>
NaturalSelect added a commit to NaturalSelect/curve that referenced this issue Sep 8, 2023
close: opencurve#1617

Signed-off-by: NaturalSelect <2145973003@qq.com>
NaturalSelect added a commit to NaturalSelect/curve that referenced this issue Sep 15, 2023
close: opencurve#1617

Signed-off-by: NaturalSelect <2145973003@qq.com>
NaturalSelect added a commit to NaturalSelect/curve that referenced this issue Sep 15, 2023
close: opencurve#1617

Signed-off-by: NaturalSelect <2145973003@qq.com>
NaturalSelect added a commit to NaturalSelect/curve that referenced this issue Oct 8, 2023
close: opencurve#1617

old metaserver snapshot that contain `inode` and `dentry` incompatible with new version

Signed-off-by: NaturalSelect <2145973003@qq.com>
NaturalSelect added a commit to NaturalSelect/curve that referenced this issue Oct 8, 2023
close: opencurve#1617

old metaserver snapshot that contain `inode` and `dentry` incompatible with new version

Signed-off-by: NaturalSelect <2145973003@qq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

8 participants