Errors while using latest version of websocket

[kapkirl]

Hello.
I was playing with sample code like this:

from pyppeteer import launch
import logging
logging.basicConfig(level=logging.DEBUG)

async def main():
    browser = launch(options={
        'headless': True,
        'timeout': 10000,  # Maximum time in milliseconds to wait for the browser instance to start
    })
    url = 'http://httpbin.org/anything'
    page = await browser.newPage()

    response = await page.goto(url, options={
        'timeout': 3000,
        'waitUntil': 'load'})
    print('response status: {}'.format(response.status))
    await browser.close()

loop = asyncio.get_event_loop()
loop.set_debug(enabled=True)
loop.run_until_complete(main())

Using python 3.6 and websockets==3.3 it works without any troubles.
But when I upgraded websocket to 4.0.1 - it starts falling with such message:

response status: 200
Task exception was never retrieved
future: <Task finished coro=<Connection._recv_loop() done, defined at /usr/local/lib/python3.6/site-packages/pyppeteer/connection.py:47> exception=InvalidState('Cannot write to a WebSocket in the CLOSING state',)>
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/pyppeteer/connection.py", line 53, in _recv_loop
resp = await self.connection.recv()
File "/usr/local/lib/python3.6/site-packages/websockets/protocol.py", line 309, in recv
loop=self.loop, return_when=asyncio.FIRST_COMPLETED)
File "/usr/local/lib/python3.6/asyncio/tasks.py", line 307, in wait
return (yield from _wait(fs, timeout, return_when, loop))
File "/usr/local/lib/python3.6/asyncio/tasks.py", line 390, in _wait
yield from waiter
concurrent.futures._base.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/pyppeteer/connection.py", line 58, in _recv_loop
break
File "/usr/local/lib/python3.6/site-packages/websockets/client.py", line 390, in aexit
yield from self.ws_client.close()
File "/usr/local/lib/python3.6/site-packages/websockets/protocol.py", line 370, in close
self.timeout, loop=self.loop)
File "/usr/local/lib/python3.6/asyncio/tasks.py", line 352, in wait_for
return fut.result()
File "/usr/local/lib/python3.6/site-packages/websockets/protocol.py", line 642, in write_frame
"in the {} state".format(self.state.name))
websockets.exceptions.InvalidState: Cannot write to a WebSocket in the CLOSING state

Any suggestions, how to handle this?

[kennethreitz]

i'm running into this also

[cnicodeme]

Yes, I confirm this too.

On my side, this occurs when I call

    browser.close()
    try:
        loop.run_until_complete(asyncio.gather(*asyncio.Task.all_tasks()))
    except Exception as e:
        # The above mentionned exception is caught here
        pass

[miyakogi]

Thank you for reporting.
I reproduced this.
I will try to fix.

[miyakogi]

Fixed by 6822b1e

[cnicodeme]

Great, Thanks!

Just out of curiosity, how this will be released as a new package?

[jhadenfeldt]

Looks like the new version will be coming this weekend: #41 (comment)

[scythargon]

I still encounter this in 0.0.14! T_T

[scythargon]

Though a bit different:

future: <Task finished coro=<Connection._async_send() done, defined at /usr/local/lib/python3.6/site-packages/pyppeteer/connection.py:61> exception=ConnectionClosed('WebSocket connection is closed: code = 1006 (connection closed abnormally [internal]), no reason',)>
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/pyppeteer/connection.py", line 64, in _async_send
    await self.connection.send(msg)
  File "/usr/local/lib/python3.6/site-packages/websockets/protocol.py", line 334, in send
    yield from self.ensure_open()
  File "/usr/local/lib/python3.6/site-packages/websockets/protocol.py", line 470, in ensure_open
    raise ConnectionClosed(self.close_code, self.close_reason)
websockets.exceptions.ConnectionClosed: WebSocket connection is closed: code = 1006 (connection closed abnormally [internal]), no reason

[miyakogi]

Latest version is 0.0.17.
Please update pyppeteer.

[scythargon]

Thank! And thank you for your work on this library - it's very good mostly.
Updated, still the same issue.

Task exception was never retrieved
future: <Task finished coro=<Connection._async_send() done, defined at /usr/local/lib/python3.6/site-packages/pyppeteer/connection.py:61> exception=ConnectionClosed('WebSocket connection is closed: code = 1006 (connection closed abnormally [internal]), no reason',)>
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/pyppeteer/connection.py", line 64, in _async_send
    await self.connection.send(msg)
  File "/usr/local/lib/python3.6/site-packages/websockets/protocol.py", line 334, in send
    yield from self.ensure_open()
  File "/usr/local/lib/python3.6/site-packages/websockets/protocol.py", line 470, in ensure_open
    raise ConnectionClosed(self.close_code, self.close_reason)
websockets.exceptions.ConnectionClosed: WebSocket connection is closed: code = 1006 (connection closed abnormally [internal]), no reason

And the most sad thing is I don't see how do I intercept and handle this error, do you know how?
Because it freezes the whole python process. BTW noticed that child chrome process at this point becomes a zombie-process.

[miyakogi]

Could you show the sample code to reproduce this error?

[scythargon]

I'll try to prepare one later, because it requires quite a few libraries to be incorporated together
https://stackoverflow.com/questions/49743638/python-script-produces-zombie-processes-only-in-docker