Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conan client should implement HTTP/1.1 resumable downloads if the server supports it. #3610

Closed
ftrofin opened this issue Sep 25, 2018 · 2 comments · Fixed by #6791
Closed

Comments

@ftrofin
Copy link

ftrofin commented Sep 25, 2018

Conan version 1.7.3
OS: Mac OS 10.13.6
Artifactory Enterprise 6.1.0 rev 60100900

SUMMARY: Many developers work remotely and need to sync large binaries (for example test files). If the connection times out or drops and the conan client times out, when the conan install command is re-issued, the download starts again from the beginning. Instead it should resume from where it was interrupted.

DESCRIPTION: Since HTTP/1.1 resumable downloads can be implemented by using ranges in the HTTP header (if the server supports it). Artifactory has been supporting this feature since version 3.7.0: https://stackoverflow.com/questions/51027395/does-jfrogs-artifactory-support-client-side-download-resume. Since Artifactory is the most used Conan repository server, the client code should be able to handle this as well.
This is not a hypothetical scenario: in my company we have large binaries (multiple GB) which we need to sync from time to time and this feature is a must have if we want to adopt Conan on a large scale.

STEPS TO REPRO:

  1. Create a large binary Conan package and upload to a remote.
  2. Remove it from the local repo.
  3. Install it again using conan install. During download (say about half way through) remove the network cable and wait for the conan client to time out (it will print an error and return).
  4. Re-connect the network and re-issue the conan install command. Watch the progress bar carefully. It will start downloading again from zero. I expect it should resume from where it was interrupted (approximately, depending on the chunk size).
@lasote
Copy link
Contributor

lasote commented Sep 25, 2018

Hi, thanks for raising the issue.
Although Artifactory supports it, it has to be also implemented in the Conan specific protocol and of course in the client side. It would be an important amount of time to implement both sides. Currently, we are working on the definition of an API v2 also in Artifactory, so we will take it into account.

@memsharded
Copy link
Member

#6791 has implemented resumable downloads, so this should fix this issue. It will be released next 1.25.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Performance
  
Closed
Development

Successfully merging a pull request may close this issue.

4 participants