aiohttp support for http/2 protocol

[catroot]

Is there any activity to implement http/2 capability to aiohttp or any reason to not implement it?

[samuelcolvin]

aiohttp already works fine with nginx-1.10.0 over http2. That should cover much of http2. I guess aiohttp will take some modification to support "Server push", but even nginx doesn't support that yet.

[catroot]

I'm glad, that nginx supports partial http2 features. They takes their own way.
The point is to have some info about native support of http2 for aiohttp.
It will be nice if @asvetlov takes care of them as a developer of http stack for asyncio.
Because aiohttp makes a good pretension to be a favorite python http server for a long time (imho).
Without support for http2 the solution doesn't seems to be completed.
I can't get the point why http2 stays out of focus of developing http stack for asyncio, eventough all browsers already have support for them at their current versions.

[ludovic-gasc]

I'm not an aiohttp core dev, this post only represent my personal opinion:

I already spent a lot of my (personal) time only to understand correctly HTTP/1.1 and have enough understanding in aiohttp to put asyncio daemons on production.
I can't imagine the time necessary to implement aiohttp, not only to be an advanced user.

Now, I can deliver and debug daemons quickly with this stack, and for now, it's enough efficient for our needs.
What's the incentive to break everything, relearn at least HTTP/2 debugging, if with HTTP/1.1 + WebSockets already cover the needs ?
Yes, for a company like Google, HTTP/2 is clearly a very good investment, but for a simple human like me who wants feed his family, I'm pretty happy with the features of aiohttp, and it's also the case for my clients ;-)

However, don't misinterpret me: If you want HTTP/2, just code it. As a pull request of aiohttp or a new project, just code it, I'll pretty happy to test.
But, to my understanding, it isn't a small time investment, you need to reimplement a lot of things.
And in case you are interested in to give money for an implementation, certainly people could be interested in to help you.

[fafhrd91]

from my production experience i found that frame based protocols are very slow if parser is written in python. so to make aiohttp work well with http/2 we really need c-module for that.

[Lukasa]

@fafhrd91 That's not really true.

Frame based protocols have the potential to be substantially faster than line-based protocols written in Python if care is taken with the way the library is designed.

For example, consider aiohttp's current parser/buffer interaction. This implements a coroutine that repeatedly searches a data buffer for a newline character. These loops are written in C, which can be relatively fast, but nonetheless require that every single byte in the buffer be touched. This is not true of HTTP/2, which has length-prefixed frames and length-prefixed header fields. This makes parsing comparatively trivial: splitting data into logical chunks doesn't require searching for delimiters, as everything is length-prefixed.

Regardless, in any moderately complex web application written on top of aiohttp the vast majority of the time will be spent in the application code itself. Performance improvements in the parser are useful (giving more CPU time to the application), but the gains obtained from moving to a C-based parser will be relatively minimal.

[fafhrd91]

we (at KeepSafe) have one service in production that uses frame-based binary protocol. it is written on top of aiohttp. and i think this was of the architectural mistakes that i made when developing it. while with python you can write very nice high level apis, and abstract-away all framing and binary low-level functionality into easy to use, high level api, python is so slow and unpredictable under high load that it kills all good things. now we are migrating from that implementation to plain stateless http. so i am not very excited about http2.

[catroot]

it is written on top of aiohttp

@fafhrd91, it isn`t shared anywhere? The protocol is the other kind than http2, right? Anyway, it will be nice to see what behind the hood in your solution. Sad to hear about unpredictable behaviour caused by python itself.

[mickeyl]

Well, I don't think it's a question of whether HTTP/2 is a good idea or not. It has been accepted as a standard and will gain wide-spread adoption. HTTP/2-only services will come, sooner or later. If we can't talk to HTTP/2 services with aiohttp, we will lose flexibility.

(FWIW, my #1 for loving to see the HTTP/2 client side implemented in aiohttp is in order to communicate with Apple's new PUSH service which is a HTTP/2 server).

[Lukasa]

Perhaps more importantly, if aiohttp doesn't grow HTTP/2 support it will be at risk of being overtaken by a project that is willing to do HTTP/2. Both Twisted and Tornado are working on their HTTP/2 support, with Twisted likely to ship a HTTP/2 server sometime this year (I know that because I'm building it).

If aiohttp is uninterested in HTTP/2 (which it's allowed to be, you can run the project however you like!) I wouldn't be surprised to see someone pick up h11 and hyper-h2 and use those as the basis for an alternative asyncio-based HTTP implementation. Having those two projects around greatly lowers the barrier to entry for someone else wanting to build alternative to aiohttp.

[asvetlov]

See also #320

[fafhrd91]

if anyone wants to work on http2 please open new issue

[lock]

This thread has been automatically locked since there has not been
any recent activity after it was closed. Please open a new issue for
related bugs.

If you feel like there's important points made in this discussion,
please include those exceprts into that new issue.