Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

curl error #123

Open
weir12 opened this issue Apr 5, 2020 · 3 comments
Open

curl error #123

weir12 opened this issue Apr 5, 2020 · 3 comments

Comments

@weir12
Copy link

weir12 commented Apr 5, 2020

Hi:
I encountered this error when trying to download a large folder

Requesting folder 'Vero-Infected' from OSF
Downloading Vero-Infected.zip
Downloading: 30 GB     Error in curl::curl_fetch_disk(x$url$url, x$disk, handle = x$url$handle) : 
  transfer closed with outstanding read data remaining
Calls: osf_download ... verb_func -> <Anonymous> -> crul_fetch -> <Anonymous>
Execution halted

How to solve this error ,thanks!
btw,If there is another way to download OSF data,I would be happy to try it:)

@aaronwolen
Copy link
Member

Can you verify the error is consistently reproducible (seems like it might be connectivity related). If it is, please post your code so I can attempt to reproduce.

@weir12
Copy link
Author

weir12 commented Apr 6, 2020

The error does not seem to consistently repeated.
Sometimes it's an error with a timeout aroused by curl
here are my code

library(osfr)
anes_project <- osf_retrieve_node("https://osf.io/8f6n9")
anes_files <- osf_ls_files(anes_project)
anes_files=osf_ls_files(anes_files[1,])
osf_download(
  anes_files,
  recurse = TRUE,
  conflicts = "overwriter",
  verbose = F,
  progress = T

Can osfr continue to download partially completed files?(like wget -c)
Thank you for your support!

@weir12
Copy link
Author

weir12 commented Apr 6, 2020

I ran the command several more times.Each time it's blocked at random,Maybe it's stuck at 30GB or 40GB.For a large folder(500GB), this error can be fatal.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants