Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Image hashes may not match when a new cloud-image is uploaded #825

Closed
townsend2010 opened this issue Jun 6, 2019 · 1 comment · Fixed by #834
Closed

Image hashes may not match when a new cloud-image is uploaded #825

townsend2010 opened this issue Jun 6, 2019 · 1 comment · Fixed by #834
Assignees
@townsend2010
Labels
bug
Projects
19.10 cycle v0.8.0

Comments

@townsend2010
Copy link
Collaborator

We ran into a situation where new minimal cloud images were uploaded, but we were getting image hash mismatch errors.

Turns out that the SHA256SUM file is cached by Qt and it was using an older cached version of that file instead or retrieving the new one.

We need to force Qt to always hit the actual URL when retrieving the hash file.
@townsend2010 townsend2010 self-assigned this Jun 6, 2019
@townsend2010 townsend2010 added this to To do in 19.10 cycle via automation Jun 6, 2019
@townsend2010 townsend2010 added this to To do in v0.8.0 via automation Jun 6, 2019
@townsend2010
Copy link
Collaborator Author

townsend2010 commented Jun 10, 2019 •
edited

After some investigation about what Qt is doing, it uses RFC 2616 sections 13.2.3 and 13.2.4 on determining when something in the cache is expired. It does not actual see if there is a newer file on the server first when determining this.

After discovering this and applying it to how we treat files, I think it's best just to bypass using Qt caching at all. The files we retrieve that are cached are relatively small, so I don't see much of an impact on just downloading the file every time we need it to ensure it is up to date.

In the future, if this is a problem, we should develop our own caching and use the LastModified header to determine if there is a newer file.

townsend2010 added a commit that referenced this issue Jun 10, 2019
@townsend2010
url_downlaoder: Remove QNetworkDiskCache caching
8013d77 
The Qt network caching doesn't work for our needs since it doesn't necessarily
get teh latest file from the server nad using "AlwaysNetwork" defeats the purpose
of caching.

In the future, custom caching will be introduced that fits Multipass's needs.

Fixes #825
@townsend2010 townsend2010 mentioned this issue Jun 10, 2019
Closed
townsend2010 added a commit that referenced this issue Jun 10, 2019
@townsend2010
url_downloader: Remove QNetworkDiskCache caching
05f3207 
The Qt network caching doesn't work for our needs since it doesn't necessarily
get the latest file from the server and using "AlwaysNetwork" defeats the purpose
of caching.

In the future, custom caching will be introduced that fits Multipass's needs.

Fixes #825
townsend2010 added a commit that referenced this issue Jun 10, 2019
@townsend2010
url_downloader: Remove QNetworkDiskCache caching
8eaccf3 
The Qt network caching doesn't work for our needs since it doesn't necessarily
get the latest file from the server and using "AlwaysNetwork" defeats the purpose
of caching.

In the future, custom caching will be introduced that fits Multipass's needs.

Fixes #825
townsend2010 added a commit that referenced this issue Jun 10, 2019
@townsend2010
url_downloader: Remove QNetworkDiskCache caching
0d6d112 
The Qt network caching doesn't work for our needs since it doesn't necessarily
get the latest file from the server and using "AlwaysNetwork" defeats the purpose
of caching.

In the future, custom caching will be introduced that fits Multipass's needs.

Fixes #825
townsend2010 added a commit that referenced this issue Jun 10, 2019
@townsend2010
url_downloader: Remove QNetworkDiskCache caching
e7e2686 
The Qt network caching doesn't work for our needs since it doesn't necessarily
get the latest file from the server and using "AlwaysNetwork" defeats the purpose
of caching.

In the future, custom caching will be introduced that fits Multipass's needs.

Fixes #825
townsend2010 added a commit that referenced this issue Jun 11, 2019
@townsend2010
url_downloader: Directly use network cache when downloading data
37d5434 
Due to how QNetwork handles caching via QNetworkAccessManager and issues it
causes, instead directly access the network cache. This includes comparing the
Last-Modified header of the URL to what is cached and also use cached data (if
availble) when there is an issue downloading the data from the URL.

Fixes #825, fixes #829
@townsend2010 townsend2010 mentioned this issue Jun 11, 2019
Merged
@townsend2010 townsend2010 moved this from To do to In progress in 19.10 cycle Jun 13, 2019
bors bot added a commit that referenced this issue Jun 14, 2019
@bors @townsend2010
Merge #834
af773b8 
834: url_downloader: Directly use network cache when downloading data r=gerboland a=townsend2010

Due to how QNetwork handles caching via QNetworkAccessManager and issues it
causes, instead directly access the network cache. This includes comparing the
Last-Modified header of the URL to what is cached and also use cached data (if
availble) when there is an issue downloading the data from the URL.

Fixes #825, fixes #829

Co-authored-by: Chris Townsend <christopher.townsend@canonical.com>
@bors bors bot closed this as completed in #834 Jun 14, 2019
19.10 cycle automation moved this from In progress to Done Jun 14, 2019
v0.8.0 automation moved this from To do to Done Jun 14, 2019
Saviq pushed a commit that referenced this issue Jun 17, 2019
@bors @townsend2010 @Saviq
Merge #834
a5a157b 
834: url_downloader: Directly use network cache when downloading data r=gerboland a=townsend2010

Due to how QNetwork handles caching via QNetworkAccessManager and issues it
causes, instead directly access the network cache. This includes comparing the
Last-Modified header of the URL to what is cached and also use cached data (if
availble) when there is an issue downloading the data from the URL.

Fixes #825, fixes #829

Co-authored-by: Chris Townsend <christopher.townsend@canonical.com>
@Saviq Saviq mentioned this issue Jun 17, 2019
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@townsend2010 townsend2010

Labels
bug
Projects
No open projects
19.10 cycle
  
Done
v0.8.0
Done
Milestone
No milestone
1 participant
@townsend2010