Support for hybrid (node) module polyfills

[IgorMinar]

Describe the feature

Unenv's presets allow us to mark certain modules as natively supported by a given host environment. For example Cloudflare's workerd natively supports several APIs in node:buffer, and the cloudflare preset specifies this module as natively supported:

unenv/src/presets/cloudflare.ts

Line 8 in c6dca1d

"buffer",

The issue I see is that workerd supports only some node:buffer APIs but not all. Specifically it supports Buffer and SlowBuffer APIs, but not Blob or File, which are part of Node's buffer module.

What I'd really like is if I could modify the Cloudflare preset in a way that would create a hybrid module polyfill for node:buffer which would provide the natively supported APIs of workerd's node:module while, using the unenv mock (or polyfills` for the remaining APIs.

I gave this a shot locally, but I wonder if this is actually possible at all with the current way unenv works. The problem I see is that we'd need a way to preserve access to the "native" node:buffer while also defining an alias for it. And since all aliases are flattened, I don't see a way how we can have both. Am I missing something? Is there some creative solution to this?

Or is the only way to achieve this to modify workerd to alias node:* modules via an alternative module prefix (e.g. node-unenv:*) so that unenv can internally use these modules while overriding node:* specifiers using module aliases?

Additional information

• Would you be willing to help implement this feature?

[pi0]

It seems an interesting idea. If workerd could provide an alternative way to access node polyfills like workerd:node:buffer or node-unenv:buffer as you mentioned it would make it much easier since we could partially import them in our polyfills (even opened a better way to opt-in to things like async context)

An alternative would involved unenv + specific bundler (rollup/vite/nitro) plugin that preserves external imports from node_modules/unenv/cloudflare.mjs. It is certainly doable but also would be not most stable path because we need to also allow node: externals in renderChunk hook and in there we don't have a way to tell if source of node:buffer import was the unenv polyfill parent or another file for example by another lib or plugin.

[IgorMinar]

ok, thanks for the confirmation @pi0. I'll discuss with the team, and get back to you.

[IgorMinar]

We discussed this and it appears that the easiest way for us to support this is via a newly proposed (and almost landed) process.getBuiltinModule API.

It would be fairly easy for us to implement it and return the native module which could then be used by unenv to compose a hybrid polyfill.

If user code uses process.getBuiltinModule to access the native module then they'll see only the workerd version of the API, which is fine as we expect only very specialized code and not general npm packages to use this newly added node API.

[IgorMinar]

One observation that came out of cloudflare/workerd#2129 is that if we want the polyfills to be truly hybrid, we'll need to update internal module and symbol references to be absolute so that we don't end up with multiple implementations of an API due to internal references not respecting the presets overrides.

I don't believe that this is a show stopper, it's just something that will take special consideration.

[pi0]

Thanks for the pointer. I think in platforms such as worked that can actually implement Socket, unenv and nitro can be configured to directly leverage it (via preset + external alias when imported but also the current mocks have their use: The network stack of unenv is actually in use today, it allows direct fetch calls in server-side of Nitro without an actually network round-trip. (if it uses an actual Socket, we make that round-trip)