Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(store): warm file after create #452

Merged
merged 5 commits into from
Feb 28, 2023

Conversation

ifplusor
Copy link
Collaborator

What problem does this PR solve?

Improve file IO performance.

Issue Number: None

Problem Summary

What is changed and how does it work?

Warm file after create it.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Signed-off-by: James Yin <ywhjames@hotmail.com>
Signed-off-by: James Yin <ywhjames@hotmail.com>
@ifplusor ifplusor added the co/store relating to store label Feb 23, 2023
@ifplusor ifplusor force-pushed the perf-append branch 2 times, most recently from a785df3 to 52eae4c Compare February 23, 2023 09:31
Signed-off-by: James Yin <ywhjames@hotmail.com>
Signed-off-by: James Yin <ywhjames@hotmail.com>
Signed-off-by: James Yin <ywhjames@hotmail.com>
@codecov
Copy link

codecov bot commented Feb 27, 2023

Codecov Report

Merging #452 (83d3217) into main (2a78cb5) will decrease coverage by 0.64%.
The diff coverage is 33.53%.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #452      +/-   ##
==========================================
- Coverage   57.71%   57.07%   -0.64%     
==========================================
  Files         180      184       +4     
  Lines       14614    15001     +387     
==========================================
+ Hits         8434     8562     +128     
- Misses       5552     5807     +255     
- Partials      628      632       +4     
Impacted Files Coverage Δ
...ernal/controller/trigger/validation/subscripton.go 69.29% <ø> (+0.85%) ⬆️
internal/gateway/proxy/deadletter.go 0.00% <0.00%> (ø)
internal/gateway/proxy/direct.go 73.33% <0.00%> (-5.24%) ⬇️
internal/store/io/file_linux.go 0.00% <0.00%> (ø)
internal/store/io/zone/segmentedfile/recovery.go 18.98% <0.00%> (ø)
internal/store/raft/block/executor.go 0.00% <0.00%> (ø)
internal/store/segment/server.go 14.10% <0.00%> (-0.03%) ⬇️
internal/store/vsb/engine_open.go 2.98% <0.00%> (+0.12%) ⬆️
internal/store/wal/recovery.go 57.14% <0.00%> (-2.51%) ⬇️
internal/trigger/client/gcloud_functions.go 0.00% <0.00%> (ø)
... and 41 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e8da787...83d3217. Read the comment docs.

go.mod Show resolved Hide resolved
Copy link
Contributor

@wenfengwang wenfengwang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@wenfengwang wenfengwang merged commit d752543 into vanus-labs:main Feb 28, 2023
@ifplusor ifplusor deleted the perf-append branch February 28, 2023 08:17
wenfengwang pushed a commit that referenced this pull request Mar 23, 2023
* fix(store): open wal without recovery callback

Signed-off-by: James Yin <ywhjames@hotmail.com>

* fix(store): io uring

Signed-off-by: James Yin <ywhjames@hotmail.com>

* feat(store): warm file after create

Signed-off-by: James Yin <ywhjames@hotmail.com>

* feat: support handoff on blocking queue

Signed-off-by: James Yin <ywhjames@hotmail.com>

* feat(store): use blocking queue in wal appending

Signed-off-by: James Yin <ywhjames@hotmail.com>

---------

Signed-off-by: James Yin <ywhjames@hotmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
co/store relating to store high-priority P0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants