Skip to content
This repository has been archived by the owner on Feb 27, 2023. It is now read-only.

feature: support source cdn pattern #1213

Merged
merged 1 commit into from
Mar 24, 2020
Merged

Conversation

starnop
Copy link
Contributor

@starnop starnop commented Feb 11, 2020

Signed-off-by: Starnop starnopg@gmail.com

Ⅰ. Describe what this PR did

As present, supernode will trigger CDN module to download the file from the source if not exist. However, that will cost some time. It's now allowed when someone wants to use the ability of P2P distribution without caching.

Restrictions

  1. The source URL support for getting ContentLength.
  2. The source URL support for range requests.
  3. MD5 validation is not supported.

Ⅱ. Does this pull request fix one issue?

NONE.

Ⅲ. Why don't you add test cases (unit test/integration test)? (你真的觉得不需要加测试吗?)

NONE.

Ⅳ. Describe how to verify it

Ⅴ. Special notes for reviews

@starnop starnop requested a review from lowzj February 11, 2020 03:43
@starnop starnop changed the title feature: support source cdn pattern [WIP]feature: support source cdn pattern Feb 11, 2020
@starnop starnop changed the title [WIP]feature: support source cdn pattern feature: support source cdn pattern Feb 11, 2020
@starnop starnop force-pushed the source-cdn branch 3 times, most recently from 4ddc723 to 8a786a1 Compare February 11, 2020 06:12
@codecov-io
Copy link

codecov-io commented Feb 11, 2020

Codecov Report

Merging #1213 into master will decrease coverage by 0.06%.
The diff coverage is 44.23%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1213      +/-   ##
==========================================
- Coverage   48.51%   48.45%   -0.07%     
==========================================
  Files         121      121              
  Lines        7682     7746      +64     
==========================================
+ Hits         3727     3753      +26     
- Misses       3649     3682      +33     
- Partials      306      311       +5
Impacted Files Coverage Δ
supernode/config/config.go 72.91% <ø> (ø) ⬆️
dfget/types/register_response.go 0% <ø> (ø) ⬆️
supernode/daemon/mgr/task/manager.go 28.57% <0%> (-0.89%) ⬇️
supernode/server/0.3_bridge.go 0% <0%> (ø) ⬆️
...et/core/downloader/p2p_downloader/target_writer.go 0% <0%> (ø) ⬆️
dfget/core/api/download_api.go 0% <0%> (ø) ⬆️
supernode/daemon/mgr/task/manager_util.go 22.82% <10%> (-0.61%) ⬇️
cmd/supernode/app/root.go 69.74% <100%> (+0.79%) ⬆️
supernode/server/server.go 42.62% <100%> (ø) ⬆️
.../downloader/p2p_downloader/client_stream_writer.go 38.66% <100%> (ø) ⬆️
... and 7 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bae7528...26d9d97. Read the comment docs.

Copy link
Member

@lowzj lowzj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should not delete 5 bytes(piece head and tail) when dfget download a piece and get the data content by piece.RawData() in cdn source pattern.

@starnop starnop force-pushed the source-cdn branch 5 times, most recently from 68e87db to 9a3ced9 Compare March 23, 2020 02:54
Copy link
Member

@lowzj lowzj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The function dfget/core/uploader/peer_server#amendRange should not compute the piece meta size to support cdn-source pattern.

if maxLength < 0 {
return pieceRange
}
indexs := strings.Split(pieceRange, separator)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/indexs/indexes

Signed-off-by: Starnop <starnopg@gmail.com>
@starnop
Copy link
Contributor Author

starnop commented Mar 23, 2020

The function dfget/core/uploader/peer_server#amendRange should not compute the piece meta size to support cdn-source pattern.

DONE. PTAL again.

Copy link
Member

@lowzj lowzj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

This pull request doesn't support the task which downloads a part of source file. I will fix it later.

@lowzj lowzj merged commit d7d4ff4 into dragonflyoss:master Mar 24, 2020
sungjunyoung pushed a commit to sungjunyoung/Dragonfly that referenced this pull request May 8, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants