HTTP/2 protocol in Julia

[mkschulze]

GOAL
In order to be able to eventually have a grpc library available in Julia, either via writing a new one or revamping gRPC.jl or GRPC.jl, there would need to be a way to make use of the HTTP/2 protocol.

Current situation and possible solutions
There are several solutions that have been started, but are either not completed or not maintained anymore.

Solution 1:
HTTP2.jl
This library is not maintained anymore.

It would require the HPack.jl 0.2.0 (Julia v1.0 compatibility), which itself is not maintained anymore. It looks like it already has been done maybe here in this tree, but has never been released.

The author @samoconnor is not active on GitHub anymore, his last contribution is from March 2019.
Solution: HTTP2.jl could be forked and setup to be working again.

Solution 2:
There is HTTP.jl
@samoconnor started to integrate the HPack.jl into HTTP.jl here

And there is an open PR here: #JuliaWeb/HTTP.jl#331
I asked @quinnj on Slack about the state and he wants to look into it and estimate the required workload and if it would need sponsoring or not.
Solution: We could wait for his response and decide, wether it makes sense to finish that PR or not or look for sponsors or smth.

Solution 3:
Calling into C
Somebody mentioned this MIT licensed pack to be a solution in this issue: #JuliaWeb/HTTP.jl#448
It is about this technology: litespeed-quic-http3-open-source

This is the pack, not sure if HTTP/3 or QUIC actually offer the HTTP/2 parts
https://github.com/litespeedtech/lsquic

But there is the C compression library here
Found on their official site here

Solution: This could be called within Julia.

Solution 4:
Calling grpcio from Python via PyCall and don't care for a Julia implementation for now.
Grakn themselves use the grpcio for their Python Client.

[grlap]

Calling into nghttp2 lib (nghttp2_jll), that seems to be working.

[mkschulze]

Calling into nghttp2 lib (nghttp2_jll), that seems to be working.

oh nice! thx for the hint.

[FrankUrbach]

Imho for now we can close this issue because the implementation we have is for now sufficient.

[quinnj]

It'd be nice to probably have a NGHTTP2.jl package that wrapped the ccalls in a more Julia API. That would facilitate incorporating support into HTTP.jl.

[grlap]

I will share what I got so far.

[mkschulze]

@ScottPJones managed to get these packages to work again.

HTTP/2.jl
HPack.jl

That was required for gRPC.jl to work again.
It works now, but we don't know in which state it is yet.

@markdroth from gRPC mentioned they would help facilitate the work, clarify requirements, do API reviews, set up interop tests, but that hasn't started yet.

Also, there is HTTP/2 support now in LibCURL.jl, but that isn't compatible to gRPC.

[quinnj]

Can you explain why libcurl isn't compatible with grpc?

[mkschulze]

Not in detail, I just asked that in the gRPC Gitter and got this reply:

Can libcurl be used as a http/2 layer to build a gRPC lib based on shared c?

Eric Anderson
@ejona86
Feb. 08 19:03
No, there's no support for that.

M. Schulze
@mkschulze
Feb. 09 21:50
okay, and is that planned maybe or would it be much of a problem? I ask because starting from Julia 1.6, there will be libcurl in the standard library, which supports HTTP/2, so that could ease development of a gRPC library in Julia a lot potentially I think.

Eric Anderson
@ejona86
Feb. 09 23:53

I don't think that really changes difficulty of using C core from Julia; the API would stay the same. Best case the binaries are a little bit smaller, but I wouldn't assume that'd be the case.

On the other side, someone could use it to build a native Julia implementation.

But no, I'm not aware of any planned support. You're the first to mention libcurl+grpc to my knowledge.

Then he mentioned this as a starting point to wrap their C core library: grpc/grpc#15460

So, maybe it could work, it just isn't supported right now.