Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

-O option broken with HTTP redirects #349

Open
tylerknott opened this issue Jan 18, 2017 · 0 comments
Open

-O option broken with HTTP redirects #349

tylerknott opened this issue Jan 18, 2017 · 0 comments
Labels

Comments

@tylerknott
Copy link

tylerknott commented Jan 18, 2017

For example, wpull -O x http://www.ncbi.nlm.nih.gov/pubmed/16713885 (which redirects to https://www.ncbi.nlm.nih.gov/pubmed/16713885) outputs the following:

INFO Fetching ‘http://www.ncbi.nlm.nih.gov/pubmed/16713885’.
100.0% [=========================] 252.0 B 0:00:00 1.0 KiB/s
INFO Fetched ‘http://www.ncbi.nlm.nih.gov/pubmed/16713885’: 301 Moved Permanently. Length: 252 [text/html; charset=iso-8859-1].
INFO Fetching ‘https://www.ncbi.nlm.nih.gov/pubmed/16713885’.
/usr/local/lib/python3.4/dist-packages/wpull/protocol/http/client.py:185: UserWarning: HTTP session did not complete.
warnings.warn(_('HTTP session did not complete.'))
ERROR Fatal exception.
Traceback (most recent call last):
File "/usr/local/lib/python3.4/dist-packages/wpull/application/app.py", line 152, in run
yield from pipeline.process()
File "/usr/local/lib/python3.4/dist-packages/wpull/pipeline/pipeline.py", line 194, in process
yield from self._process_one_worker()
File "/usr/local/lib/python3.4/dist-packages/wpull/pipeline/pipeline.py", line 215, in _process_one_worker
task.result()
File "/usr/lib/python3.4/asyncio/futures.py", line 277, in result
raise self._exception
File "/usr/lib/python3.4/asyncio/tasks.py", line 235, in _step
result = coro.send(value)
File "/usr/local/lib/python3.4/dist-packages/wpull/pipeline/pipeline.py", line 119, in process
item = yield from self.process_one(_worker_id=worker_id)
File "/usr/local/lib/python3.4/dist-packages/wpull/pipeline/pipeline.py", line 103, in process_one
yield from task.process(item)
File "/usr/local/lib/python3.4/dist-packages/wpull/application/tasks/download.py", line 492, in process
yield from session.app_session.factory['Processor'].process(session)
File "/usr/local/lib/python3.4/dist-packages/wpull/processor/delegate.py", line 29, in process
return (yield from processor.process(item_session))
File "/usr/local/lib/python3.4/dist-packages/wpull/processor/web.py", line 92, in process
return (yield from session.process())
File "/usr/local/lib/python3.4/dist-packages/wpull/processor/web.py", line 186, in process
yield from self._process_loop()
File "/usr/local/lib/python3.4/dist-packages/wpull/processor/web.py", line 245, in _process_loop
exit_early, wait_time = yield from self._fetch_one(cast(Request, self._item_session.request))
File "/usr/local/lib/python3.4/dist-packages/wpull/processor/web.py", line 276, in _fetch_one
self._file_writer_session.process_response(response)
File "/usr/local/lib/python3.4/dist-packages/wpull/writer.py", line 511, in process_response
if not self._stream.readable():
ValueError: I/O operation on closed file
CRITICAL Sorry, Wpull unexpectedly crashed.
CRITICAL Please report this problem to the authors at Wpull's issue tracker so it may be fixed. If you know how to program, maybe help us fix it? Thank you for helping us help you help us all.
INFO Exiting with status 1.

Downloading the URL without -O or downloading the https version of the URL both work correctly. This is with version 2.0.1 running on Python 3.4.2 on 32-bit Debian Jessie.

@chfoo chfoo added the bug label Feb 20, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants