Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: large files broke prefetch #1348

Merged
merged 1 commit into from
Jul 11, 2023

Conversation

ccx1024cc
Copy link
Contributor

Details

Files larger than 4G leads to prefetch panic, because the max blob io range is smaller than 4G. This pr changes blob io max size from u32 to u64.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation Update (if none of the other choices apply)

Checklist

  • I have updated the documentation accordingly.
  • I have added tests to cover my changes.

@ccx1024cc ccx1024cc requested a review from a team as a code owner July 6, 2023 01:07
@ccx1024cc ccx1024cc requested review from liubin, luodw, adamqqqplay and bergwolf and removed request for a team July 6, 2023 01:07
@anolis-bot
Copy link
Collaborator

@ccx1024cc , a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/83048

@codecov
Copy link

codecov bot commented Jul 6, 2023

Codecov Report

Merging #1348 (c71e6a3) into master (98834dd) will increase coverage by 0.19%.
The diff coverage is 78.18%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1348      +/-   ##
==========================================
+ Coverage   45.81%   46.00%   +0.19%     
==========================================
  Files         122      122              
  Lines       37733    37806      +73     
  Branches    37733    37806      +73     
==========================================
+ Hits        17286    17394     +108     
+ Misses      19537    19501      -36     
- Partials      910      911       +1     
Impacted Files Coverage Δ
rafs/src/fs.rs 18.67% <0.00%> (ø)
storage/src/cache/worker.rs 62.25% <0.00%> (ø)
storage/src/device.rs 61.03% <100.00%> (+1.82%) ⬆️

... and 6 files with indirect coverage changes

@ccx1024cc ccx1024cc force-pushed the morgan/fix-large-file-panic branch from b5f54b2 to 56cb7e6 Compare July 6, 2023 03:39
@anolis-bot
Copy link
Collaborator

@ccx1024cc , the code has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/83118

@anolis-bot
Copy link
Collaborator

@ccx1024cc , The CI test is completed, please check result:

Test CaseTest Result
nydus_ci❌ FAIL

Sorry, your test job failed. Please get the details in the link.

@anolis-bot
Copy link
Collaborator

@ccx1024cc , The CI test is completed, please check result:

Test CaseTest Result
nydus_ci❌ FAIL

Sorry, your test job failed. Please get the details in the link.

smoke/tests/tool/layer.go Outdated Show resolved Hide resolved
rafs/src/fs.rs Outdated Show resolved Hide resolved
@ccx1024cc ccx1024cc force-pushed the morgan/fix-large-file-panic branch 2 times, most recently from 17c1b9a to d5e1976 Compare July 6, 2023 08:29
@anolis-bot
Copy link
Collaborator

@ccx1024cc , the code has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/83177

@anolis-bot
Copy link
Collaborator

@ccx1024cc , the code has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/83178

@ccx1024cc ccx1024cc requested a review from jiangliu July 6, 2023 08:32
@ccx1024cc ccx1024cc force-pushed the morgan/fix-large-file-panic branch from d5e1976 to 2ec07f4 Compare July 6, 2023 09:08
@anolis-bot
Copy link
Collaborator

@ccx1024cc , the code has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/83189

@ccx1024cc ccx1024cc force-pushed the morgan/fix-large-file-panic branch from 2ec07f4 to 0e7c4cc Compare July 6, 2023 09:14
@anolis-bot
Copy link
Collaborator

@ccx1024cc , the code has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/83192

@anolis-bot
Copy link
Collaborator

@ccx1024cc , The CI test is completed, please check result:

Test CaseTest Result
build rust golang image✅ SUCCESS
compile nydusd✅ SUCCESS
compile ctr remote✅ SUCCESS
compile nydus snapshotter✅ SUCCESS
run container with rafs✅ SUCCESS
run container with zran✅ SUCCESS
run container with rafs and compile linux✅ SUCCESS

Congratulations, your test job passed!

@anolis-bot
Copy link
Collaborator

@ccx1024cc , The CI test is completed, please check result:

Test CaseTest Result
build rust golang image✅ SUCCESS
compile nydusd✅ SUCCESS
compile ctr remote✅ SUCCESS
compile nydus snapshotter✅ SUCCESS
run container with rafs✅ SUCCESS
run container with zran✅ SUCCESS
run container with rafs and compile linux✅ SUCCESS

Congratulations, your test job passed!

rafs/src/fs.rs Outdated Show resolved Hide resolved
@anolis-bot
Copy link
Collaborator

@ccx1024cc , the code has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/84046

@anolis-bot
Copy link
Collaborator

@ccx1024cc , The CI test is completed, please check result:

Test CaseTest Result
build rust golang image✅ SUCCESS
compile nydusd✅ SUCCESS
compile ctr remote✅ SUCCESS
compile nydus snapshotter✅ SUCCESS
run container with rafs✅ SUCCESS
run container with zran✅ SUCCESS
run container with rafs and compile linux❌ FAIL

Sorry, your test job failed. Please get the details in the link.

@anolis-bot
Copy link
Collaborator

@ccx1024cc , the code has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/84077

@anolis-bot
Copy link
Collaborator

@ccx1024cc , The CI test is completed, please check result:

Test CaseTest Result
build rust golang image✅ SUCCESS
compile nydusd✅ SUCCESS
compile ctr remote✅ SUCCESS
compile nydus snapshotter✅ SUCCESS
run container with rafs✅ SUCCESS
run container with zran✅ SUCCESS
run container with rafs and compile linux✅ SUCCESS

Congratulations, your test job passed!

Files larger than 4G leads to prefetch panic, because the max blob io
range is smaller than 4G. This pr changes blob io max size from u32 to
u64.

Signed-off-by: 泰友 <cuichengxu.ccx@antgroup.com>
@anolis-bot
Copy link
Collaborator

@ccx1024cc , the code has been updated, so a new test job has been submitted. Please wait in patience. The test job url: https://tone.openanolis.cn/ws/nrh4nnio/test_result/84081

@anolis-bot
Copy link
Collaborator

@ccx1024cc , The CI test is completed, please check result:

Test CaseTest Result
build rust golang image✅ SUCCESS
compile nydusd✅ SUCCESS
compile ctr remote✅ SUCCESS
compile nydus snapshotter✅ SUCCESS
run container with rafs✅ SUCCESS
run container with zran✅ SUCCESS
run container with rafs and compile linux✅ SUCCESS

Congratulations, your test job passed!

@imeoer imeoer merged commit 31f2170 into dragonflyoss:master Jul 11, 2023
22 checks passed
@bergwolf bergwolf mentioned this pull request Jul 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants