http/2 initiative

[mikegwhit]

working to gather all related info in this ticket, and will track an effort to bring http/2 into either PillarJS (for routing compatibility only) or Express 5.x.

@dougwilson notes here (in 2015!) #2761 (comment) that PillarJS can be used alongside native module.. so we need to assess if this is going to be something that uses express.js or just PillarJS with HTTP/2

#3730 -- most recent work that picks up on #3390

#2781 notes request for push-promises

#3778 (comment) mentions HTTP2

#2237 mentions HTTP/2 support in 2015 for Express5.x

#3390 is a proposal PR for HTTP/2 via #3388

#3390 (comment) is probably the most comprehensive TODO List i have seen

notably he modifies also:
superagent
supertest
cookies
cookie-session
vhost

also since the start of these efforts we now have chatgpt to help us. after some trial and error i got ChatGPT to provide us a useful TODO list:

Integrate HTTP/2 Core Module:

Replace usage of the HTTP/1 module with Node.js's http2 for server creation, ensuring dual support for HTTP/1 and HTTP/2.
TLS Configuration for HTTP/2:

Implement mandatory TLS setup for HTTP/2, providing configuration options for SSL certificates.
Re-architecture Request/Response Objects:

Adapt Express.js request and response objects to handle HTTP/2's stream-based communication, ensuring API compatibility.
Support for HTTP/2 Streams:

Enable handling of HTTP/2 streams in middleware and routes, allowing for efficient data handling (e.g., file uploads/downloads).
Implement Header Compression:

Utilize HPACK compression for headers in both requests and responses to maximize efficiency as per HTTP/2 specification.
HTTP/2 Server Push:

Develop API methods for HTTP/2 server push, enabling servers to preemptively send resources to the client.
Graceful Fallback to HTTP/1.x:

Ensure the server can gracefully downgrade to HTTP/1.x for clients that do not support HTTP/2.
Optimize for HTTP/2 Features:

Leverage HTTP/2 features like multiplexing and prioritization to optimize request and response management.
Update Middleware for HTTP/2 Compatibility:

Audit and update existing Express middleware to be compatible with HTTP/2, focusing on stream and headers handling.
Comprehensive Test Suite:

Develop a test suite covering HTTP/2 functionality, including stream management, server push, and header compression.
Documentation & Examples:

Provide detailed documentation and examples on setting up and using HTTP/2 with Express.js, including TLS configuration.
Benchmarking & Performance Analysis:

Benchmark HTTP/2 performance against HTTP/1.x, addressing any bottlenecks identified.

[mikegwhit]

CC @QuantGeekDev

I pulled in changes from #3390 today and you have to run his test suite via HTTP2_TEST npm test which fails. it might be useful if we start with the other separate modules that are mentioned above. i didn't see pillarjs in the list also but assume it is relevant.

[QuantGeekDev]

CC @QuantGeekDev

I pulled in changes from #3390 today and you have to run his test suite via HTTP2_TEST npm test which fails. it might be useful if we start with the other separate modules that are mentioned above. i didn't see pillarjs in the list also but assume it is relevant.

Noted. Going to browse through this

[cScarlson]

I would love to see this feature come to fruition. Can we maybe rank-order all the tasks we know of from, say, lowest-hanging fruit to highest so that different people can jump into smaller tasks to learn the codebase more effectively? I believe this would be a big win, itself, so that entire subfeatures can get completed before breaking larger tasks down into subtasks. Someone with more knowledge of the existing ExpressJS codebase needs to provide their expertise here, if possible. Otherwise, the rest of the comminity & users should have a good stab at the rank-ordered list.

[wesleytodd]

Hey, this is a great job combining a bunch of the historical context around HTTP2 and Express. Thanks for putting that together. Just wanted to drop a note that we are in the process of building a plan for the project's next few years and if you are interested in being involved in pushing http2 support forward please stick around. We will need help with this but we need to get v5 out the door before we can start it. So you might not get a lot of engagement from collaborators just yet, but please don't take that as ignoring this, just an ordering and prioritization thing.

[mikegwhit]

understandable @wesleytodd i would have assumed as much..
FWIW we discussed yesterday (people in this issue ticket) "why http2":

in general this is the summary:

• HTTP2 reports benchmark gains around 14% in nginx / apache where it has been implemented
• HTTP2 is slower than HTTP without SSL which may or may not be true, see: [1] [2]
• SPDY was the go-to solution for Express but SPDY broke as of Node 16, perhaps solving those issues is more relevant
• we concluded it's better to just fix express than to tell people to migrate to fastify
• i experienced a 25% speed gain using SPDY with Node 14 vs. without
• industry happy medium is to use a CDN for frontend production, HTTP for local development (see important assumption above)
• HTTP2 benefits API's less because it is frontends that are typically requesting 150 paths per page load, and frontends are incentivized to use CDN's
• while HTTP2 only provides modest gains (see important assumption above) for what an overhaul it would require, interesting speed gain opportunities open up by redesigning express (required for HTTP2 likely) to be asynchronous and implementing a worker thread pool
• a gateway like Nginx could just work via HTTP2 or HTTP with SSL and forward requests to HTTP without SSL express servers, nullifying all gains by implementing HTTP2 (see important assumption above)

the net is that while HTTP2 itself might not be so important since there are workarounds with a Nginx gateway pointed at non-SSL HTTP (see important assumption above) and CDN's, there are interesting gains made by rearchitecting Express to handle requests via worker thread pools which could be done alongside HTTP2 work..