Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cannot download certain files #72

Closed
equal-l2 opened this issue May 26, 2017 · 19 comments
Closed

Cannot download certain files #72

equal-l2 opened this issue May 26, 2017 · 19 comments

Comments

@equal-l2
Copy link

equal-l2 commented May 26, 2017 •
edited

Snatch built from devel HEAD (fac87df) cannot download these files even with -t1.

Output:

# Waiting a response from the remote server... OK !
# Remote content length: 384.57 KB
Downloading 1 chunks:
Chunk 0 ▎ |##############################################################################################################################################| 100.00 % 352.65 KB/s
thread '<unnamed>' panicked at 'Cannot download the chunk 0, due to error Invalid Status provided', src/download.rs:167
note: Run with `RUST_BACKTRACE=1` for a backtrace.
Done! Your download is available in 0.1.1.zip
@Jean-Serge
Copy link
Collaborator

Hello 😄

Which version of rust are you using ?

There is no issue from my side to download those files with rust 1.19.

@equal-l2
Copy link
Author

I've used stable (1.17) for build.

I'll try beta and nightly now.

@equal-l2
Copy link
Author

equal-l2 commented May 26, 2017 •
edited

The error still occurs with beta(0.18-beta.3) and nightly(1.19.0-nightly 5f3966864).

I'm running this on Mac.
Can it be a problem?

@Jean-Serge
Copy link
Collaborator

No I've just tried on a Mac with nightly without any problem. 🤔 Very strange

@k0pernicus
Copy link
Contributor

Got the same problem of @equal-l2

This is my stack trace:

wpp00031:snatch antonin$ snatch https://github.com/derniercri/snatch/archive/0.1.1.zip -t 1
# Waiting a response from the remote server... OK !
# Remote content length: 384.57 KB
Downloading 1 chunks:
Chunk 0 ▎ |############################################################################################################| 100.00 % 472.07 KB/s
thread '<unnamed>' panicked at 'Cannot download the chunk 0, due to error Invalid Status provided', src/download.rs:167
note: Run with `RUST_BACKTRACE=1` for a backtrace.
Done! Your download is available in 0.1.1.zip

Also, the output file has to be deleted if the download was not ok.

@Jean-Serge
Copy link
Collaborator

Could you both try by putting the -t 1 option before URL ?

@equal-l2
Copy link
Author

The problem will occur without -t 1, at least for me.

@Jean-Serge
Copy link
Collaborator

I am only able to reproduce this bug when putting the -t 1 at the end of the command. This bug become stranger...

@k0pernicus
Copy link
Contributor

Same thing with @equal-l2

@Jean-Serge
Copy link
Collaborator

@equal-l2 Can you tell me :

  • do you have the same issue using -t 2 ?
  • how many thread does it use without this option ?

@equal-l2
Copy link
Author

equal-l2 commented May 26, 2017 •
edited

@Jean-Serge

do you have the same issue using -t 2 ?

Yes.

how many thread does it use without this option ?

4 threads by default.

@Jean-Serge
Copy link
Collaborator

Can you send me the trace when using the -t 2 option ?

@equal-l2
Copy link
Author

equal-l2 commented May 26, 2017 •
edited

Here.

# Waiting a response from the remote server... OK !
[WARNING] Cannot get the remote content length, using an HEADER request.
[WARNING] Trying to send an HTTP request, to get the remote content length...
Response { status: Ok, headers: Headers { Date: Fri, 26 May 2017 14:05:23 GMT
, Content-Type: application/zip
, Content-Length: 393798
, Access-Control-Allow-Origin: https://render.githubusercontent.com
, Content-Security-Policy: default-src 'none'; style-src 'unsafe-inline'
, Strict-Transport-Security: max-age=31536000
, Vary: Authorization,Accept-Encoding
Vary: Accept-Encoding
, X-Content-Type-Options: nosniff
, X-Frame-Options: deny
, X-XSS-Protection: 1; mode=block
, ETag: "c77b663f454ad7d29fd74de937d48c11ce794846"
, Content-Disposition: attachment; filename=snatch-0.1.1.zip
, X-Geo-Block-List:
, X-GitHub-Request-Id: DB47:24692:4A2E:84D2:59283623
, }, version: Http11, url: "https://codeload.github.com/derniercri/snatch/zip/0.1.1", status_raw: RawStatus(200, "OK"), message: Http11Message { is_proxied: false, method: None, stream: Wrapper { obj: Some(Reading(SizedReader(remaining=393798))) } } }
# Remote content length: 384.57 KB
Downloading 2 chunks:
0 / 196899 [---------------------------------------------------] 0.00 % 0.00/s
Chunk 1 ▎ |###########################################| 100.00 % 125.31 KB/s
thread '<unnamed>' panicked at 'Cannot download the chunk 1, due to error Invalid Status provided', src/download.rs:167
stack backtrace:
   0:        0x10ecd9b73 - std::sys::imp::backtrace::tracing::imp::unwind_backtrace::h884a721e113c3303
   1:        0x10ecdbec8 - std::panicking::default_hook::{{closure}}::h7a7d734b2824d103
   2:        0x10ecdbb0c - std::panicking::default_hook::h3eb11bd6cbfdc331
   3:        0x10ecde317 - std::panicking::rust_panic_with_hook::h8b9b25777425677b
   4:        0x10ecde1b4 - std::panicking::begin_panic::h69fe06a9d67b6165
   5:        0x10ecde122 - std::panicking::begin_panic_fmt::haf8695bfb6c52436
   6:        0x10e9a5835 - libsnatch::download::download_chunks::{{closure}}::hba634e8d9a54e7f0
   7:        0x10e96e84a - std::sys_common::backtrace::__rust_begin_short_backtrace::h9aa379327fb9c20e
   8:        0x10e9724d3 - std::thread::Builder::spawn::{{closure}}::{{closure}}::h26e5f94d0d3f8281
   9:        0x10e9a122a - <std::panic::AssertUnwindSafe<F> as core::ops::FnOnce<()>>::call_once::hb26a4adcb8ed6a76
  10:        0x10e9728a9 - std::panicking::try::do_call::h995fb58d60f03586
  11:        0x10ecdf44a - __rust_maybe_catch_panic
  12:        0x10e9726dc - std::panicking::try::h9d5ab9499fa7bb66
  13:        0x10e971795 - std::panic::catch_unwind::h8a7792b5472127dd
  14:        0x10e9722db - std::thread::Builder::spawn::{{closure}}::hdbd2a9b8fdfc6dc3
  15:        0x10e98c4e3 - <F as alloc::boxed::FnBox<A>>::call_box::h711bf8f596d161c0
  16:        0x10ecdb4f5 - std::sys::imp::thread::Thread::new::thread_start::h993b7e5cb7871f66
  17:     0x7fffab43f93a - _pthread_body
Downloading 2 chunks: 86 - _pthread_start
Chunk 0 ▎ |###########################################| 100.00 % 113.60 KB/s  liChunk 1 ▎ |###########################################| 100.00 % 125.31 KB/s
stack backtrace:
   0:        0x10ecd9b73 - std::sys::imp::backtrace::tracing::imp::unwind_backtrace::h884a721e113c3303
   1:        0x10ecdbec8 - std::panicking::default_hook::{{closure}}::h7a7d734b2824d103
   2:        0x10ecdbb0c - std::panicking::default_hook::h3eb11bd6cbfdc331
   3:        0x10ecde317 - std::panicking::rust_panic_with_hook::h8b9b25777425677b
   4:        0x10ecde1b4 - std::panicking::begin_panic::h69fe06a9d67b6165
   5:        0x10ecde122 - std::panicking::begin_panic_fmt::haf8695bfb6c52436
   6:        0x10e9a5835 - libsnatch::download::download_chunks::{{closure}}::hba634e8d9a54e7f0
   7:        0x10e96e84a - std::sys_common::backtrace::__rust_begin_short_backtrace::h9aa379327fb9c20e
   8:        0x10e9724d3 - std::thread::Builder::spawn::{{closure}}::{{closure}}::h26e5f94d0d3f8281
   9:        0x10e9a122a - <std::panic::AssertUnwindSafe<F> as core::ops::FnOnce<()>>::call_once::hb26a4adcb8ed6a76
  10:        0x10e9728a9 - std::panicking::try::do_call::h995fb58d60f03586
  11:        0x10ecdf44a - __rust_maybe_catch_panic
  12:        0x10e9726dc - std::panicking::try::h9d5ab9499fa7bb66
  13:        0x10e971795 - std::panic::catch_unwind::h8a7792b5472127dd
  14:        0x10e9722db - std::thread::Builder::spawn::{{closure}}::hdbd2a9b8fdfc6dc3
  15:        0x10e98c4e3 - <F as alloc::boxed::FnBox<A>>::call_box::h711bf8f596d161c0
  16:        0x10ecdb4f5 - std::sys::imp::thread::Thread::new::thread_start::h993b7e5cb7871f66
  17:     0x7fffab43f93a - _pthread_body
  18:     0x7fffab43f886 - _pthread_start
Done! Your download is available in 0.1.1.zip

@equal-l2 equal-l2 changed the title Cannot download some files Cannot download certain files May 31, 2017
@equal-l2
Copy link
Author

equal-l2 commented Jun 1, 2017

I've finally noticed these URLs don't support split download.

@k0pernicus
Copy link
Contributor

Hello @equal-l2, I solved this issue few days ago actually - I forgot to describe the issue, sorry...
I fixed the issue on my personal branch - a few refactoring and I will push a solution to download a file using only one thread, and a better support for servers that don't support the split behaviour.

@equal-l2
Copy link
Author

equal-l2 commented Jun 1, 2017

@k0pernicus
I'm glad to hear that, thank you!

@k0pernicus
Copy link
Contributor

k0pernicus commented Jun 1, 2017 •
edited

No problem, I will push the code this week-end ;-)

@k0pernicus
Copy link
Contributor

k0pernicus commented Jun 3, 2017 •
edited

Hello @equal-l2,
the solution has been merged (commit #73) ;-)
You can now update your local project to download your files - a changelog has been added in the README.
Please to close the issue if the solution is OK for you!

@equal-l2
Copy link
Author

equal-l2 commented Jun 3, 2017

It works perfectly now.
Thank you again, @k0pernicus !

@equal-l2 equal-l2 closed this as completed Jun 3, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@k0pernicus @Jean-Serge @equal-l2