Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support HTTP/2.0 #10981

Open
inotnako opened this issue May 5, 2015 · 51 comments
Open

support HTTP/2.0 #10981

inotnako opened this issue May 5, 2015 · 51 comments
Labels
:Distributed/Network Http and internode communication implementations >feature high hanging fruit Team:Distributed Meta label for distributed team

Comments

@inotnako
Copy link

inotnako commented May 5, 2015

Is ElasticSearch planning to add support (or maybe plugin for) protocol HTTP/2.0?
Like in https://github.com/grpc/grpc(used HTTP/2.0) or may be with grpc

@clintongormley clintongormley added high hanging fruit :Distributed/Network Http and internode communication implementations labels Jan 18, 2016
@dakrone
Copy link
Member

dakrone commented Sep 27, 2016

This is now possible since we have moved to Netty 4, since Netty supports HTTPP/2.0 in 4.1+ /cc @jasontedor

@jasontedor
Copy link
Member

To be clear, Netty 4 supports HTTP/2 but there is effort required to make Elasticsearch use it, and ensure that it works properly.

@KodrAus
Copy link

KodrAus commented Sep 11, 2017

Is there any new thinking on supporting h2? It seems like a fruit hanging so high it'll probably take a magic beanstalk to reach, but I'm interested in any thoughts you might have on the cost-benefit equation. I imagine there are bigger wins in first-class client support and alternative APIs that take advantage of more stream-oriented connections in supporting h2 within Elasticsearch itself over just exposing through a h2-enabled proxy.

@mantasindrasius
Copy link

Any updates on this? With the deprecated TransportClient, there should for sure be HTTP/2 or/and gRPC support.

@jasontedor
Copy link
Member

At this time there is no update on this other than that we are not currently working on it; this not to say that we will not, only that we feel there are higher priority projects. I do not agree that the deprecation of the transport client implies that there should be HTTP/2 support. Can share why you think that the two are tied together?

@mantasindrasius
Copy link

Thanks for a quick answer.

HTTP/1.1 is limited in the sense that any concurrent operations cannot happen on the same connection and creating a new connection is a huge performance trade-off.

HTTP/2 or gRPC which is based on the former is multiplexing by nature, so is closer to the non-blocking TransportClient.

If we want to keep the transport overhead low, we can't go with REST+HTTP/1.1.

@jasontedor
Copy link
Member

@mantasindrasius Thanks for the feedback. I agree with you on the benefits of HTTP/2.

I do think it's worth noting that with HTTP/1.1 keep alive behavior and a connection pool that you can mitigate some of the disadvantages of HTTP/1.1 versus HTTP/2.0. The best that HTTP/1.1 can offer related to multiplexing is pipelining. Yes, this suffers from head-of-line blocking but it does allow some connection reuse too.

That said, even weighing these advantages and disadvantages, I would prefer that we continue to not tie removal of the transport client to the introduction of HTTP/2. Removing the transport client will bring us major advantages in continuing to develop the server side of Elasticsearch without maintaining a client that is so intimately tied to the internal protocol. These advantages outweigh the loss of benefits of the transport client.

Rest assured that HTTP/2 is on our roadmap, just not at the top of the list at the moment.

@mantasindrasius
Copy link

Thanks for that. Do you also think there could be gRPC support for elastic or it's not on the table?

@jasontedor
Copy link
Member

jasontedor commented Apr 4, 2019

@mantasindrasius gRPC is not on the table at the moment.

@Asgoret
Copy link

Asgoret commented Jun 14, 2019

@jasongoodwin ELK don't work in OKD 3.11 (Openshift) correctly because there is no support of H2\grpc protocols.
See: elastic/kibana#7104 and openshift/origin-aggregated-logging#1465

UPD#1: Yeah...i disable h2 and kibana work perfectly. Enable and stop working) Is there ant information when kibana will support h2?

@ayushsangani
Copy link
Contributor

Any updates on the support of Http 2.0 in ES?

@rjernst rjernst added the Team:Distributed Meta label for distributed team label May 4, 2020
@cangiremir
Copy link

cangiremir commented Jun 16, 2020

Any actions to support gRPC or HTTP/2?

@weitzjdevk
Copy link

weitzjdevk commented Jun 18, 2020

Are there any news here? AWS NLB has finally ALPN support for the TLS Listener, which would help for the gRPC handshake/http2 connection

https://www.reddit.com/r/aws/comments/gsdo3w/nlb_now_supports_alpn_on_tls_listeners/

@bachden
Copy link

bachden commented Aug 13, 2020

Waiting for http2 (gRPC maybe) supporting.

@bradquarry
Copy link

A team at Verizon is asking for gRPC support. +1

@YohanSciubukgian
Copy link

Do you have any update to share about HTTP/2 support please ?

@gokcesurenkok
Copy link

Do you have any plan to do that in near future ? i would like to see it if it is in your roadmap.
Thanks

@weitzj
Copy link

weitzj commented Nov 13, 2020

Any updates here?

AWS ALB has finally gRPC support built in
GCP had it all the time

@cyberfifi
Copy link

+1 from Compass. We need this feature.

@emcpadden
Copy link

This is definitely needed!

@Farenheith
Copy link

I endorse http2 too!

One solution for me at this moment, if anyone is interested, is to avoid elasticsearch oficcial client (in a nodejs service that only uses _search), and to use superagent + agentkeepalive, with, in my case of use, maxSockets uninformed, assuming the default value of Infinity.
This way I have the connection pool working under the hood, mitigating the http1 limitation as @jasontedor well said.

@gguillen
Copy link

+1 Upvote

@JhennerTigreros
Copy link

A team at Platzi is asking for gRPC support. +1

@Dyqer
Copy link

Dyqer commented Nov 11, 2021

+1 Upvote

@blackheaven
Copy link

+1 Upvote

@JoeGilkey
Copy link

+1 as a nice to have.

@Jaimenms
Copy link

Jaimenms commented Feb 9, 2022

+1 Upvote

@cjmcgraw
Copy link

+1 this would be very helpful to my company

@kkdeok
Copy link

kkdeok commented Jun 9, 2022

+1 please

@marinrusu1997
Copy link

+1 Upvote

1 similar comment
@KennyLindahl
Copy link

+1 Upvote

@BiditPakrashi
Copy link

Any update on GRPC support ?

@mhlee-lee
Copy link

+1 please

1 similar comment
@fsedano
Copy link

fsedano commented Aug 30, 2022

+1 please

@nakulpathak3
Copy link

+1 Faire would like this too

@raff-s
Copy link

raff-s commented Nov 4, 2022

+1 Upvote

1 similar comment
@DaemonSnake
Copy link

+1 Upvote

@nealcola
Copy link

nealcola commented Dec 2, 2022

+1 please

@vineelyalamarthy
Copy link

@clintongormley are there discussions happening some where like in wiki pages/ discord etc on this topic roadmap/implementation. Are open source contributors welcome to participate in those? I would love to contribute to this feature.

@lukasojd
Copy link

+1

@henningandersen
Copy link
Contributor

This is not a high priority to us internally at the moment. There is no to little actual benefits from http/2 over what we have today. We would welcome a contribution, but would like to remind of our contributions guidelines, highlighting that this change is likely to require considerable test additions.

@otrosien
Copy link

@henningandersen thanks for the feedback. Out of curiosity: "There is no to little actual benefits from http/2 over what we have today" - how was that evaluated?

@loeffel-io
Copy link

+1

@magicleo
Copy link

Any update on GRPC support ?

@patpatpat123
Copy link

Upvoting this many year old idea

@firebook
Copy link

Any update? We really need it!

@DaveCTurner
Copy link
Contributor

Unfortunately there's nothing in the discussion above to say why this is so important. HTTP/2.0 (and now HTTP/3.0) offer significant advantages for certain applications, particularly those involving web browsers or web-browser-like interaction patterns, but we cannot see the benefits over HTTP/1.1 in the specific context of an application like ES whose clients are expected to use a pool of long-lived connections (and possibly pipelining) to achieve the same results. We just don't encounter cases where the use of HTTP/1.1 is the limiting factor for either performance or expressivity in a meaningful way right now. I'm sorry if we're missing something obvious, but without some more concrete motivation for addressing this request it's unlikely we'll take on the significant amount of work involved in doing so in the foreseeable future.

@trycoon
Copy link

trycoon commented Feb 14, 2024

@DaveCTurner HTTP/2 is of little importance to me… But I was hoping that gRPC could lower the transmission latency that JSON and REST adds to the querying. We use Elastic as a smarty cache to filter information and answer user requests, several thousend per second. If we could shave of some additional 10ms for each 100KB JSON response by using gRPC, that would be a nice improvement.
And to be able to use gRPC we need HTTP/2… 🤗

@DaveCTurner
Copy link
Contributor

If we could shave off some additional 10ms for each 100KB JSON response by using gRPC, that would be a nice improvement.

That would indeed be very nice, but I don’t see how it could happen. The current wire protocol doesn’t add anything like that much latency.

@mselmansezgin
Copy link

@DaveCTurner the wire protocol doesn't add anything for additional latency but it also detracts from the ability to use protobuf instead of JSON. This eliminates our chance to reduce the overall size of the payload.

@DaveCTurner
Copy link
Contributor

Adding HTTP/2.0 support wouldn't itself yield any meaningful savings in this area because today ES needs the data it processes to use a self-describing format such as JSON (although if you care about payload size it'd be better to use one of the other supported formats instead). Protobuf isn't naturally self-describing in the way ES needs today, so we'd either end up adding loads of metadata or else just making a thin protobuf wrapper around one of the existing formats, neither of which would be meaningfully smaller than today's wire protocol.

We could probably teach ES to speak protobuf more naturally too, separating schema from content better, but that'd be a separate enhancement. If we did that it'd be able to speak protobuf over the existing HTTP/1.1 interface as well. In other words this point is pretty much orthogonal to the conversation about HTTP/2.0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Distributed/Network Http and internode communication implementations >feature high hanging fruit Team:Distributed Meta label for distributed team
Projects
None yet
Development

No branches or pull requests