try undici http client

[rifler]

What problem are you trying to solve?

speed improvement

Describe the feature

use new http/1.1 client https://github.com/nodejs/undici, which is faster than builtin http

original tweet - https://twitter.com/matteocollina/status/1298148085210775553

Checklist

• I have read the documentation and made sure this feature doesn't already exist.

[ghermeto]

I might be able to help with that. We are discussing how Undici fits in the long term HTTP technical plans in the next web-server-frameworks team meeting. If the interface will be kept I can implement it.

cc/ @mmarchini

[szmarczak]

We can solve this in two ways:

1. wrap http ClientRequest so it has a similar interface to undici (might have a tiny performance impact, but I'd have to benchmark this)

2. write an UndiciRequest class which has the same interface as Request (zero performance impact)

[szmarczak]

To everyone here, how do you use http.Agents? Do you use it to modify requests?

The agent class has a .addRequest(request, options) method. It is undocumented, however used by the native http module and many popular HTTP proxy agents overwrite it to mutate the requests. For example:

https://github.com/TooTallNate/node-http-proxy-agent/blob/ef184d6b7af42e02920b015ecea557a0ac201fdc/src/agent.ts#L110

Currently it's not possible to mutate the request using undici. See nodejs/undici#887 (comment)

I made a PoC wrapper that allows native Agents to be used with undici. The drawback here is that it will error when an agent tries to mutate the request.

A workaround would be to soft-fail but that can lead to unexpected behavior.

[szmarczak]

I agree with the concept that it should not be possible to mutate the request. So eventually we can just soft-fail (nop, do nothing).

If anyone has any arguments for the opposite, please comment.

[ronag]

process.emitWarning?

[titanism]

We used got at first, but switched to undici. Following along this issue, I think the only other performance bottleneck between using http (got) vs net (undici) we had discovered was the setup time with got.extend and iteration through the loop related to mutableDefaults.

Anyways - thanks for making got and undici - we rewrote 🍊 Tangerine (npm install tangerine) to use undici now (instead of got) and released it today at https://github.com/forwardemail/tangerine. Benchmarks including HTTP benchmarks with got vs undici are at the bottom of README.

[titanism]

Oh, also - as soon as we switched from using got (or any other HTTP library) to using undici, our package tangerine became ⚡ faster ⚡ than the Native Node.js dns module!!! 😲

[JaneJeon]

Unfortunately "just use undici directly" isn't an option for me, given 1. community packages (got-scraping is a godsend), and 2. hooks