WIP: Implement h2 protocol

[Vibhu-Agarwal]

Closes #47

HTTP Parsing

• Using h2

PR #214

It was great! I've picked a lot of stuff from there.
I've mostly worked on extending the PR, removing bugs and modernizing the code.
I've opted to omit a few implementations though, on which I'd like to have some discussion.

Motivation from Hypercorn's code

I also read the hypercorn's code, which has the implementation for HTTP/2 already. I got a pretty good idea of http2's working from there.

priority

It uses priority to build a priority tree and manage the streams.
I haven't yet implemented this in this PR (as it wasn't, in #214). Should we use it?
If we use it, it will also help in implementing code for handling h2 events like WindowUpdated, PriorityUpdated and RemoteSettingsChanged (which are currently ignored).

stream_buffers

Hypercorn also uses StreamBuffers to handle the data of different streams per connection. This may be due to the fact that h2 doesn't buffer data internally for sending. I'm not sure though, as hypercorn checks the chunk size (before sending) anyway.
I haven't yet implemented this in this PR (as it wasn't, in #214). Should we use it?

Server Push

I wasn't sure if I should go ahead with HTTP Server-Push's implementation or not, as its status is not clear.
Should we support it?

In this PR

• Some h2-specific changes in SSL-Context

  • uvicorn/uvicorn/config.py

    Line 215 in a8dcf14

    self.h2_ssl_ciphers = "ECDHE+AESGCM"

  • uvicorn/uvicorn/config.py

    Lines 116 to 125 in a8dcf14

    	if enable_h2:
    	ctx.options |= (
    	ssl.OP_NO_SSLv2
    	| ssl.OP_NO_SSLv3
    	| ssl.OP_NO_TLSv1
    	| ssl.OP_NO_TLSv1_1
    	| ssl.OP_NO_COMPRESSION
    	| ssl.OP_CIPHER_SERVER_PREFERENCE
    	)
    	ctx.set_alpn_protocols(["h2", "http/1.1"])

• No websocket support, yet.

  • Unlike h11, a Connection: Upgrade (to webocket) request raises h2.exceptions.ProtocolError here:
    

    uvicorn/uvicorn/protocols/http/h2_impl.py

    Lines 219 to 221 in b683626

    	try:
    	events = self.conn.receive_data(data)
    	except h2.exceptions.ProtocolError:

  • This can be resolved through the usage of Extended CONNECT Method.
    

    uvicorn/uvicorn/protocols/http/h2_impl.py

    Line 111 in b683626

    h2.settings.SettingCodes.ENABLE_CONNECT_PROTOCOL: 1,

    
    That will have to be handled here:
    

    uvicorn/uvicorn/protocols/http/h2_impl.py

    Lines 284 to 285 in b683626

    	if self.scope["method"] == "CONNECT" and websocket_protocol:
    	# TODO: Websocket Extended CONNECT Implementation

Tests

• For tests for config.py,
  • It didn't have much to cover up the coverage, but SSL-things had to be tested. So, one for that.
• For tests for h11_impl.py,
  • These two upgrades need tests:
    1. h2c-upgrade
    2. H2-Prior-Knowledge Connection-Preface
  • Work: I added one for both of them (got the coverage; but checks in response still remain):
    1. h2c-upgrade: I added one for a simple GET request.
       For some reason, test_h2c_upgrade_request is causing failure of some un-related tests in Python3.8 and Python3.9 (runs fine in Python <=3.7). Commit 81f21be
       Manual-Testing: I'm not sure how to manually test this one.
    2. H2-Prior-Knowledge Connection-Preface: The test is really a minimal one, not even hitting any endpoint. So I'm not even sure if this is sufficient or not.
       Manual-Testing: I tried this with a POST request using curl. It works. I'm not sure how to replicate it in Python or even get raw HTTP request out of it.
  • Basically, I'm out of ideas to implement tests for these.
• For tests for h2_impl.py,
  • Just like HTTPToolsProtocol and H11Protocol, H2Protocol needs testing for a lot of stuff.
  • I saw two options of implementation methods:
    1. test_http.py way (using mocks). I tried this out (had to modify get_connected_protocol() a bit) but got messed up with valid raw HTTP/2 requests and SSL-Context passing. Couldn't debug and get it to work.
    2. test_ssl.py way (using httpx). I installed Starlette, added a (very) small app, and tried out a test with a simple POST request. It kinda worked out. So I've added this one in the PR.
  • Manual-Testing: I tried with my (Firefox) browser using a sane app on simple GET requests. It seems to work fine.
    I haven't tried broken apps and others with exceptions.

WIP

• I've marked a lot of places in the code with # TODO: Task and added some comments for decision-related discussion or missing implementation.
• I've also tried to keep the code of h2_impl.py similar to httptools_impl.py and h11_impl.py.

Help Needed

I'm just giving it a go. Hope to complete this soon 🤞
Will need reviews, inputs, tests, ideas and a lot of help from everyone 🤗

[Vibhu-Agarwal]

^ So that some discussion could be started.
The Work is still In-Progress though.

[Kludex]

Do we need starlette? What's the idea here?

[Kludex]

We want to do the same as httpx: https://github.com/encode/httpx/blob/ab64f7c41fc0fbe638dd586fecf0689c847109bb/setup.py#L66

[Kludex]

We should, yep.

[tomchristie]

So, I'm neither necessarily for or against this, but I do have an alternate perspective that I think is worth sharing/considering.

See the discussion in... #1105

[tomchristie]

It may well make sense for us to deliberately limit the scope of Uvicorn to being an HTTP/1.1 server.
The tighter the scope, the more focused our resources can be on making sure we do a fantastic job in a particular space.

[Kludex]

@Vibhu-Agarwal Thanks for the effort here, but I think by #1105 we'll highlight hypercorn as the recommendation for HTTP/2.

Sorry for taking so long to act here.