store: do not resume a download when we already have the whole thing #3640

Merged
merged 2 commits into from Aug 3, 2017

Conversation

Projects
None yet
4 participants
Member

chipaca commented Aug 2, 2017

Without this a .partial file that has been completely downloaded still
triggers a request to the server (which will then come back with a
416).

This lets you drop a .snap in /var/lib/snapd/snaps as a .partial,
which could be used to speed up tests. It'll still be checksummed
(twice), but it won't hit the network for the download unless the
store says it's got a delta.

Doing the same thing for delta downloads is left as an exercise to the
reader.

zyga approved these changes Aug 2, 2017

Looks ok, LGTM

Commit messages though ;-)

store: do not resume a download when we already have the whole thing
Without this a .partial file that has been completely downloaded still
triggers a request to the server (which will then come back with a
416).

This lets you drop a .snap in /var/lib/snapd/snaps as a .partial,
which could be used to speed up tests. It'll still be checksummed
(twice), but it won't hit the network for the download unless the
store says it's got a delta.

Doing the same thing for delta downloads is left as an exercise to the
reader.
Member

chipaca commented Aug 2, 2017

@zyga better? :-)

codecov-io commented Aug 2, 2017

Codecov Report

Merging #3640 into master will decrease coverage by 0.01%.
The diff coverage is 60%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #3640      +/-   ##
==========================================
- Coverage    75.2%   75.19%   -0.02%     
==========================================
  Files         387      388       +1     
  Lines       33452    33491      +39     
==========================================
+ Hits        25159    25183      +24     
- Misses       6481     6494      +13     
- Partials     1812     1814       +2
Impacted Files Coverage Δ
store/store.go 79.7% <60%> (-0.28%) ⬇️
interfaces/builtin/kvm.go 61.53% <0%> (ø)
cmd/snap/cmd_aliases.go 95% <0%> (+1.66%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e037060...fbc1b58. Read the comment docs.

@zyga zyga requested a review from pedronis Aug 3, 2017

Contributor

zyga commented Aug 3, 2017

Much better, thank you!

some wonderings

@@ -1359,7 +1359,22 @@ func (s *Store) Download(ctx context.Context, name string, targetPath string, do
url = downloadInfo.DownloadURL
}
- err = download(ctx, name, downloadInfo.Sha3_384, url, user, s, w, resume, pbar)
+ if downloadInfo.Size == 0 || resume < downloadInfo.Size {
@pedronis

pedronis Aug 3, 2017

Contributor

are we sure resume cannot be > ?

@chipaca

chipaca Aug 3, 2017

Member

I know it can, but it should then fail the checksum

store/store.go
- err = download(ctx, name, downloadInfo.Sha3_384, url, user, s, w, resume, pbar)
+ if downloadInfo.Size == 0 || resume < downloadInfo.Size {
+ err = download(ctx, name, downloadInfo.Sha3_384, url, user, s, w, resume, pbar)
+ } else if downloadInfo.Sha3_384 != "" {
@pedronis

pedronis Aug 3, 2017

Contributor

this is weird, shouldn't this always be the case ? and if not, shouldn't we error earlier?

@chipaca

chipaca Aug 3, 2017

Member

there are tests that don't provide the checksum

@pedronis

pedronis Aug 3, 2017

Contributor

yes, maybe, but you changed real runtime code, the old code was always downloading, something seems off now

@pedronis

pedronis Aug 3, 2017

Contributor

what I mean, if you think we cannot error, it seems more conservative to behave as before in the case == "", not do a random thing as with this code

@chipaca

chipaca Aug 3, 2017

Member

gah, turns out the only test that didn't set it that mattered was my own. fixed.

thank you

@chipaca chipaca merged commit 81dc18a into snapcore:master Aug 3, 2017

7 checks passed

artful-amd64 autopkgtest finished (success)
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
xenial-amd64 autopkgtest finished (success)
Details
xenial-i386 autopkgtest finished (success)
Details
xenial-ppc64el autopkgtest finished (success)
Details
yakkety-amd64 autopkgtest finished (success)
Details
zesty-amd64 autopkgtest finished (success)
Details

@chipaca chipaca deleted the chipaca:do-not-resume-completed-downloads branch Aug 3, 2017

mvo5 added a commit to mvo5/snappy that referenced this pull request Aug 14, 2017

store: do not resume a download when we already have the whole thing (#…
…3640)

* store: do not resume a download when we already have the whole thing

Without this a .partial file that has been completely downloaded still
triggers a request to the server (which will then come back with a
416).

This lets you drop a .snap in /var/lib/snapd/snaps as a .partial,
which could be used to speed up tests. It'll still be checksummed
(twice), but it won't hit the network for the download unless the
store says it's got a delta.

Doing the same thing for delta downloads is left as an exercise to the
reader.

* store: address review feedback (thanks pedronis)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment