Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

unhandled exception during asyncio.run() shutdown #5

Closed
BianTan opened this issue Apr 29, 2022 · 10 comments
Closed

unhandled exception during asyncio.run() shutdown #5

BianTan opened this issue Apr 29, 2022 · 10 comments

Comments

@BianTan
Copy link

BianTan commented Apr 29, 2022

下载链接:https://www.bilibili.com/video/BV1JP4y1K774

unhandled exception during asyncio.run() shutdown
task: <Task finished name='Task-1813' coro=<Downloader._get_media() done, defined at
/Users/biandan/Desktop/Lighting-bilibili-download-0.3/lighting_downloader.py:336>
exception=LocalProtocolError('Invalid input ConnectionInputs.RECV_DATA in state ConnectionState.CLOSED')>
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/h2/connection.py", line 224,
in process_input
func, target_state = self.transitions[(self.state, input)]
KeyError: (<ConnectionState.CLOSED: 3>, <ConnectionInputs.RECV_DATA: 11>)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpcore/_async/http2.py",
line 111, in handle_async_request
status, headers = await self._receive_response(
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpcore/_async/http2.py",
line 228, in _receive_response
event = await self._receive_stream_event(request, stream_id)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpcore/_async/http2.py",
line 259, in _receive_stream_event
await self._receive_events(request, stream_id)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpcore/_async/http2.py",
line 278, in _receive_events
events = await self._read_incoming_data(request)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpcore/_async/http2.py",
line 340, in _read_incoming_data
events = self._h2_state.receive_data(data)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/h2/connection.py", line 1463,
in receive_data
events.extend(self._receive_frame(frame))
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/h2/connection.py", line 1487,
in _receive_frame
frames, events = self._frame_dispatch_tableframe.class
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/h2/connection.py", line 1673,
in _receive_data_frame
events = self.state_machine.process_input(
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/h2/connection.py", line 228,
in process_input
raise ProtocolError(
h2.exceptions.ProtocolError: Invalid input ConnectionInputs.RECV_DATA in state ConnectionState.CLOSED

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpx/_transports/default.py",
line 60, in map_httpcore_exceptions
yield
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpx/_transports/default.py",
line 353, in handle_async_request
resp = await self._pool.handle_async_request(req)
File
"/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpcore/_async/connection_pool.py",
line 253, in handle_async_request
raise exc
File
"/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpcore/_async/connection_pool.py",
line 237, in handle_async_request
response = await connection.handle_async_request(request)
File
"/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpcore/_async/http_proxy.py", line
310, in handle_async_request
return await self._connection.handle_async_request(request)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpcore/_async/http2.py",
line 141, in handle_async_request
raise LocalProtocolError(exc) # pragma: nocover
httpcore.LocalProtocolError: Invalid input ConnectionInputs.RECV_DATA in state ConnectionState.CLOSED

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/Users/biandan/Desktop/Lighting-bilibili-download-0.3/lighting_downloader.py", line 349, in _get_media
await asyncio.gather(*cors)
File "/Users/biandan/Desktop/Lighting-bilibili-download-0.3/lighting_downloader.py", line 366, in _get_media_part
async with self.client.stream("GET", random.choice(media_urls),
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/contextlib.py", line 199, in aenter
return await anext(self.gen)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpx/_client.py", line 1549,
in stream
response = await self.send(
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpx/_client.py", line 1593,
in send
response = await self._send_handling_auth(
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpx/_client.py", line 1621,
in _send_handling_auth
response = await self._send_handling_redirects(
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpx/_client.py", line 1658,
in _send_handling_redirects
response = await self._send_single_request(request)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpx/_client.py", line 1695,
in _send_single_request
response = await transport.handle_async_request(request)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpx/_transports/default.py",
line 352, in handle_async_request
with map_httpcore_exceptions():
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/contextlib.py", line 153, in exit
self.gen.throw(typ, value, traceback)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpx/_transports/default.py",
line 77, in map_httpcore_exceptions
raise mapped_exc(message) from exc
httpx.LocalProtocolError: Invalid input ConnectionInputs.RECV_DATA in state ConnectionState.CLOSED
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/h2/connection.py", line 224,
in process_input
func, target_state = self.transitions[(self.state, input)]
KeyError: (<ConnectionState.CLOSED: 3>, <ConnectionInputs.RECV_DATA: 11>)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpcore/_async/http2.py",
line 111, in handle_async_request
status, headers = await self._receive_response(
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpcore/_async/http2.py",
line 228, in _receive_response
event = await self._receive_stream_event(request, stream_id)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpcore/_async/http2.py",
line 259, in _receive_stream_event
await self._receive_events(request, stream_id)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpcore/_async/http2.py",
line 278, in _receive_events
events = await self._read_incoming_data(request)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpcore/_async/http2.py",
line 340, in _read_incoming_data
events = self._h2_state.receive_data(data)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/h2/connection.py", line 1463,
in receive_data
events.extend(self._receive_frame(frame))
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/h2/connection.py", line 1487,
in _receive_frame
frames, events = self._frame_dispatch_tableframe.class
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/h2/connection.py", line 1673,
in _receive_data_frame
events = self.state_machine.process_input(
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/h2/connection.py", line 228,
in process_input
raise ProtocolError(
h2.exceptions.ProtocolError: Invalid input ConnectionInputs.RECV_DATA in state ConnectionState.CLOSED

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpx/_transports/default.py",
line 60, in map_httpcore_exceptions
yield
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpx/_transports/default.py",
line 353, in handle_async_request
resp = await self._pool.handle_async_request(req)
File
"/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpcore/_async/connection_pool.py",
line 253, in handle_async_request
raise exc
File
"/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpcore/_async/connection_pool.py",
line 237, in handle_async_request
response = await connection.handle_async_request(request)
File
"/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpcore/_async/http_proxy.py", line
310, in handle_async_request
return await self._connection.handle_async_request(request)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpcore/_async/http2.py",
line 141, in handle_async_request
raise LocalProtocolError(exc) # pragma: nocover
httpcore.LocalProtocolError: Invalid input ConnectionInputs.RECV_DATA in state ConnectionState.CLOSED

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/Users/biandan/Desktop/Lighting-bilibili-download-0.3/bili_cmd.py", line 56, in
asyncio.run(main(parser.parse_args()))
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/asyncio/base_events.py", line 646, in
run_until_complete
return future.result()
File "/Users/biandan/Desktop/Lighting-bilibili-download-0.3/bili_cmd.py", line 9, in main
await d.get_series(args.key, quality=args.q, image=args.image)
File "/Users/biandan/Desktop/Lighting-bilibili-download-0.3/lighting_downloader.py", line 255, in get_series
await asyncio.gather(*cors)
File "/Users/biandan/Desktop/Lighting-bilibili-download-0.3/lighting_downloader.py", line 309, in get_video
await asyncio.gather(
File "/Users/biandan/Desktop/Lighting-bilibili-download-0.3/lighting_downloader.py", line 349, in _get_media
await asyncio.gather(*cors)
File "/Users/biandan/Desktop/Lighting-bilibili-download-0.3/lighting_downloader.py", line 366, in _get_media_part
async with self.client.stream("GET", random.choice(media_urls),
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/contextlib.py", line 199, in aenter
return await anext(self.gen)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpx/_client.py", line 1549,
in stream
response = await self.send(
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpx/_client.py", line 1593,
in send
response = await self._send_handling_auth(
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpx/_client.py", line 1621,
in _send_handling_auth
response = await self._send_handling_redirects(
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpx/_client.py", line 1658,
in _send_handling_redirects
response = await self._send_single_request(request)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpx/_client.py", line 1695,
in _send_single_request
response = await transport.handle_async_request(request)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpx/_transports/default.py",
line 352, in handle_async_request
with map_httpcore_exceptions():
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/contextlib.py", line 153, in exit
self.gen.throw(typ, value, traceback)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/httpx/_transports/default.py",
line 77, in map_httpcore_exceptions
raise mapped_exc(message) from exc
httpx.LocalProtocolError: Invalid input ConnectionInputs.RECV_DATA in state ConnectionState.CLOSED

@BianTan
Copy link
Author

BianTan commented Apr 29, 2022

还有希望可以加上字幕下载功能

@HFrost0
Copy link
Owner

HFrost0 commented Apr 30, 2022

@BianTan 👋

下载链接:https://www.bilibili.com/video/BV1JP4y1K774 报错 httpx.LocalProtocolError

我尝试下载了这个链接,在本地python3.8和3.9以及3.10没能复现错误,或许你可以尝试重新下载,看看这个错误是否是因为网络条件不佳导致。作者对目前已知的因为网络原因出现的错误进行了try catch,包括httpx.RemoteProtocolError以及httpx.ReadTimeout。如果证实httpx.LocalProtocolError是因为网络不佳的原因引发,在后续更新中会对其进行catch,并做重试处理。

还有希望可以加上字幕下载功能

谢谢你的建议,在后续更新中会添加此功能

ps:另外我发现这个视频标题中的&没有被正常的解析,这个在后续更新中可能会去修复🥲
类似

@BianTan
Copy link
Author

BianTan commented Apr 30, 2022

或许你可以尝试重新下载,看看这个错误是否是因为网络条件不佳导致。

我使用python3.9和3.10重新测试下载这个链接的视频,每次都是到了
【Udemy付费课程】 Git & GitHub - The Practical Guide -->Git&GitHub 使用指南(中英文字幕)-P51-054 Applying
the Fast-Forward Merge 完成
后面就报错
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/h2/connection.py", line 224, in
process_input
func, target_state = self.transitions[(self.state, input)]
KeyError: (<ConnectionState.CLOSED: 3>, <ConnectionInputs.SEND_HEADERS: 0>)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/httpcore/_async/http2.py", line
105, in handle_async_request
await self._send_request_headers(request=request, stream_id=stream_id)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/httpcore/_async/http2.py", line
202, in _send_request_headers
self._h2_state.send_headers(stream_id, headers, end_stream=end_stream)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/h2/connection.py", line 766, in
send_headers
self.state_machine.process_input(ConnectionInputs.SEND_HEADERS)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/h2/connection.py", line 228, in
process_input
raise ProtocolError(
h2.exceptions.ProtocolError: Invalid input ConnectionInputs.SEND_HEADERS in state ConnectionState.CLOSED

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/httpx/_transports/default.py",
line 60, in map_httpcore_exceptions
yield
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/httpx/_transports/default.py",
line 353, in handle_async_request
resp = await self._pool.handle_async_request(req)
File
"/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/httpcore/_async/connection_pool.py",
line 253, in handle_async_request
raise exc
File
"/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/httpcore/_async/connection_pool.py",
line 237, in handle_async_request
response = await connection.handle_async_request(request)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/httpcore/_async/http_proxy.py",
line 310, in handle_async_request
return await self._connection.handle_async_request(request)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/httpcore/_async/http2.py", line
141, in handle_async_request
raise LocalProtocolError(exc) # pragma: nocover
httpcore.LocalProtocolError: Invalid input ConnectionInputs.SEND_HEADERS in state ConnectionState.CLOSED

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/Users/biandan/Desktop/Lighting-bilibili-download-0.3/bili_cmd.py", line 56, in
asyncio.run(main(parser.parse_args()))
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/base_events.py", line 647, in
run_until_complete
return future.result()
File "/Users/biandan/Desktop/Lighting-bilibili-download-0.3/bili_cmd.py", line 9, in main
await d.get_series(args.key, quality=args.q, image=args.image)
File "/Users/biandan/Desktop/Lighting-bilibili-download-0.3/lighting_downloader.py", line 255, in get_series
await asyncio.gather(*cors)
File "/Users/biandan/Desktop/Lighting-bilibili-download-0.3/lighting_downloader.py", line 309, in get_video
await asyncio.gather(
File "/Users/biandan/Desktop/Lighting-bilibili-download-0.3/lighting_downloader.py", line 349, in _get_media
await asyncio.gather(*cors)
File "/Users/biandan/Desktop/Lighting-bilibili-download-0.3/lighting_downloader.py", line 366, in _get_media_part
async with self.client.stream("GET", random.choice(media_urls),
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/contextlib.py", line 181, in aenter
return await self.gen.anext()
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/httpx/_client.py", line 1549, in
stream
response = await self.send(
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/httpx/_client.py", line 1593, in
send
response = await self._send_handling_auth(
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/httpx/_client.py", line 1621, in
_send_handling_auth
response = await self._send_handling_redirects(
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/httpx/_client.py", line 1658, in
_send_handling_redirects
response = await self._send_single_request(request)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/httpx/_client.py", line 1695, in
_send_single_request
response = await transport.handle_async_request(request)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/httpx/_transports/default.py",
line 353, in handle_async_request
resp = await self._pool.handle_async_request(req)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/contextlib.py", line 137, in exit
self.gen.throw(typ, value, traceback)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/httpx/_transports/default.py",
line 77, in map_httpcore_exceptions
raise mapped_exc(message) from exc
httpx.LocalProtocolError: Invalid input ConnectionInputs.SEND_HEADERS in state ConnectionState.CLOSED

谢谢你的建议,在后续更新中会添加此功能

期待😚 希望可以配置单独下载字幕

@BianTan
Copy link
Author

BianTan commented Apr 30, 2022

设备是 M1 的 Mac mini

@BianTan
Copy link
Author

BianTan commented Apr 30, 2022

啊,原来报错了不删除文件重新执行命令可以继续下载

@HFrost0
Copy link
Owner

HFrost0 commented Apr 30, 2022

@BianTan

啊,原来报错了不删除文件重新执行命令可以继续下载

是的,我在特性里面提了一下,看来需要放在更显眼的位置🤣

httpx.LocalProtocolError

我又试了几次,M1芯片的Mac在转译和原生的情况下都能正常运行,我猜测可能是由于我们的地区不同,所请求的b站服务器不一致导致的。错误可能是由于请求的服务器对于http2协议支持出了问题,为了测试这个想法,可以暂时将lighting_downloader.py中的第29行

self.client = httpx.AsyncClient(headers=headers, cookies=cookies, http2=True)

改为

self.client = httpx.AsyncClient(headers=headers, cookies=cookies, http2=False)

使用http1协议访问

@BianTan
Copy link
Author

BianTan commented May 3, 2022

为了测试这个想法,可以暂时将lighting_downloader.py中的第29行

改了还是一样捏,不过我不使用 -num 20 就稳一点了

@HFrost0
Copy link
Owner

HFrost0 commented May 3, 2022

改了还是一样捏,不过我不使用 -num 20 就稳一点了

你指的是-max_con吗,最好别设太大,会有很多问题,感觉5以内已经足够了

@HFrost0
Copy link
Owner

HFrost0 commented May 3, 2022

你可能需要的下载字幕功能已经有了,最新版本中使用--subtitle就可以下载rst格式的字幕了

python bili_cmd.py get_series 'url' --subtitle

字幕会下到视频目录下面的extra文件夹里面,我用的IINA播放器是可以自动导入的

@HFrost0
Copy link
Owner

HFrost0 commented May 12, 2022

请问最新版本还存在问题吗?

@HFrost0 HFrost0 closed this as completed May 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants