HTTP/2 usage and Windows support?

[McSpidey]

Hi

I just found your excelent tuturial on how to use Quart with gunicorn, which I know isn't supported on Windows 10 so decided to attempt to use hypercorn instead. Unfortunately it's crashing giving this error.

---

hypercorn --keyfile key.pem --certfile cert.pem --ciphers ECDHE+AESGCM --bind localhost:5000 http2test:app
Running on https://localhost:5000 (CTRL + C to quit)
Traceback (most recent call last):
File "c:\users\x\appdata\local\programs\python\python37-32\Lib\runpy.py", line 193, in run_module_as_main
"main", mod_spec)
File "c:\users\x\appdata\local\programs\python\python37-32\Lib\runpy.py", line 85, in run_code
exec(code, run_globals)
File "C:\Users\x.virtualenvs\QuartTest-s-3dKnLB\Scripts\hypercorn.exe_main.py", line 9, in
File "c:\users\x.virtualenvs\quarttest-s-3dknlb\lib\site-packages\hypercorn_main.py", line 159, in main
run_multiple(config)
File "c:\users\x.virtualenvs\quarttest-s-3dknlb\lib\site-packages\hypercorn\run.py", line 234, in run_multiple
process.start()
File "c:\users\x\appdata\local\programs\python\python37-32\Lib\multiprocessing\process.py", line 112, in start
self._popen = self._Popen(self)
File "c:\users\x\appdata\local\programs\python\python37-32\Lib\multiprocessing\context.py", line 223, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "c:\users\x\appdata\local\programs\python\python37-32\Lib\multiprocessing\context.py", line 322, in _Popen
return Popen(process_obj)
File "c:\users\x\appdata\local\programs\python\python37-32\Lib\multiprocessing\popen_spawn_win32.py", line 65, in init
reduction.dump(process_obj, to_child)
File "c:\users\x\appdata\local\programs\python\python37-32\Lib\multiprocessing\reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
TypeError: can't pickle SSLContext objects

Traceback (most recent call last):
File "", line 1, in
File "c:\users\x\appdata\local\programs\python\python37-32\Lib\multiprocessing\spawn.py", line 99, in spawn_main
new_handle = reduction.steal_handle(parent_pid, pipe_handle)
File "c:\users\x\appdata\local\programs\python\python37-32\Lib\multiprocessing\reduction.py", line 87, in steal_handle
_winapi.DUPLICATE_SAME_ACCESS | _winapi.DUPLICATE_CLOSE_SOURCE)
PermissionError: [WinError 5] Access is denied

Is there a working usage example? It's working fine with http1.

Thanks and keep up the great work!

[pgjones]

This is interesting, I think if you rollback to version 0.2.4 and confine yourself to a single worker (the default) it will work. I've some issues to fix with Windows and multiple workers (especially with the 0.3.x releases).

[McSpidey]

Thanks, I can confirm that works.

Command line from the gunicorn tutorial, slightly modified.

hypercorn --workers 1 --keyfile key.pem --certfile cert.pem --ciphers ECDHE+AESGCM --bind localhost:5000 http2test:app

[pgjones]

I think I've fixed this on the master branch, would you be able to try it and confirm?

[McSpidey]

I can confirm HTTP2 on Windows is working in '0.3.2' when restricted to 1 worker.

[pgjones]

I think this is good with 0.4.1, please reopen if not.

[McSpidey]

Just confirming 1 worker is still fine, but 2 or more crashes.

[pgjones]

I can't reproduce the crash, do you have a stack trace?

[McSpidey]

Here's my start command
hypercorn --workers 2 --keyfile key.pem --certfile cert.pem --ciphers ECDHE+AESGCM --bind localhost:5000 http2test:app

And the stack trace```

hypercorn --workers 2 --keyfile key.pem --certfile cert.pem --ciphers ECDHE+AESGCM --bind localhost:5000 http2test:app
Running on https://localhost:5000 (CTRL + C to quit)
Process Process-1:
Traceback (most recent call last):
File "c:\users\username\appdata\local\programs\python\python37-32\Lib\multiprocessing\process.py", line 297, in _bootstrap
self.run()
File "c:\users\username\appdata\local\programs\python\python37-32\Lib\multiprocessing\process.py", line 99, in run
self._target(*self._args, **self._kwargs)
File "c:\users\username.virtualenvs\quarttest-s-3dknlb\lib\site-packages\hypercorn\asyncio\run.py", line 193, in asyncio_worker
run_single(app, config, loop=loop, sock=sock, shutdown_event=shutdown_event)
File "c:\users\username.virtualenvs\quarttest-s-3dknlb\lib\site-packages\hypercorn\asyncio\run.py", line 139, in run_single
server = loop.run_until_complete(create_server)
File "c:\users\username\appdata\local\programs\python\python37-32\Lib\asyncio\base_events.py", line 568, in run_until_complete
return future.result()
File "c:\users\username\appdata\local\programs\python\python37-32\Lib\asyncio\base_events.py", line 1380, in create_server
server._start_serving()
File "c:\users\username\appdata\local\programs\python\python37-32\Lib\asyncio\base_events.py", line 268, in _start_serving
sock.listen(self._backlog)
OSError: [WinError 10022] An invalid argument was supplied
Task was destroyed but it is pending!
task: <Task pending coro=<Lifespan.handle_lifespan() done, defined at c:\users\username.virtualenvs\quarttest-s-3dknlb\lib\site-packages\hypercorn\asyncio\lifespan.py:21> wait_for=>

[pgjones]

Hmm, this is a very difficult bug, I can only reproduce it intermittently - I think there is a race issue when creating servers (sockets) at the same time on windows. I think this commit, 693093f, works around this. Could you try it?

[McSpidey]

Just tested this and while at first it didn't help, I increased the delay however to time.sleep(1.0 * random.random()) and then it started working intermitently. Increasing the worker count appears to make it fail more often still even with the higher delay.

It's somewhat functional now though.

[pgjones]

Could you try with the experimental branch, specifically this commit?
Given it does seem to be a race condition, a Lock seems a much better solution than the random delay.

[McSpidey]

I just downloaded those three changed files manually and applied them, the behaviour was different but it crashes.

Behaviour
Start App
New additional delay
Looks like it's starting up (prints the localhost:500 to the screen)
New additional delay
Error

Here's the error.
hypercorn --workers 10 --keyfile key.pem --certfile cert.pem --ciphers ECDHE+AESGCM --bind localhost:5000 http2test:app
Running on https://localhost:5000 (CTRL + C to quit)
Process Process-7:
Traceback (most recent call last):
File "C:\Users\username\AppData\Local\Programs\Python\Python37-32\Lib\multiprocessing\process.py", line 297, in _bootstrap
self.run()
File "C:\Users\username\AppData\Local\Programs\Python\Python37-32\Lib\multiprocessing\process.py", line 99, in run
self._target(*self._args, **self._kwargs)
File "c:\users\username.virtualenvs\quarttest-s-3dknlb\lib\site-packages\hypercorn\asyncio\run.py", line 207, in asyncio_worker
startup_lock=startup_lock,
File "c:\users\username.virtualenvs\quarttest-s-3dknlb\lib\site-packages\hypercorn\asyncio\run.py", line 142, in run_single
server = loop.run_until_complete(create_server)
File "C:\Users\username\AppData\Local\Programs\Python\Python37-32\Lib\asyncio\base_events.py", line 584, in run_until_complete
return future.result()
File "C:\Users\username\AppData\Local\Programs\Python\Python37-32\Lib\asyncio\base_events.py", line 1397, in create_server
server._start_serving()
File "C:\Users\username\AppData\Local\Programs\Python\Python37-32\Lib\asyncio\base_events.py", line 287, in _start_serving
sock.listen(self._backlog)
OSError: [WinError 10022] An invalid argument was supplied
Task was destroyed but it is pending!
task: <Task pending coro=<Lifespan.handle_lifespan() done, defined at c:\users\username.virtualenvs\quarttest-s-3dknlb\lib\site-packages\hypercorn\asyncio\lifespan.py:21> wait_for=>

[pgjones]

Interesting, looks like the same issue but after a new delay. This is maybe not a race condition then, although the intermittent nature suggests so...
I'm a bit stuck on this, I'll have to think/google more.