Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement window.fetch into core #19393

Open
thecodingdude opened this issue Mar 16, 2018 · 107 comments

Comments

@thecodingdude
Copy link

commented Mar 16, 2018

https://github.com/bitinn/node-fetch

It would make sense if window.fetch was implemented into core. it seems to be a stable enough API that would make a good candidate for inclusion. Not sure what the process is from here but thought I'd raise an issue :)

@jasnell

This comment has been minimized.

Copy link
Member

commented Mar 16, 2018

this has come up from time to time but has not had much traction just yet. Let's see what folks think tho :-)

@devsnek

This comment has been minimized.

Copy link
Member

commented Mar 16, 2018

bradley and i were discussing this in a roundabout way on the subject of importing from urls. if that feature was introduced (and i think in general we do want it) we would need to implement this: https://html.spec.whatwg.org/multipage/webappapis.html#fetch-a-single-module-script which uses the fetch spec. as another note if this was added in core i would want to pull in an existing c++ implementation from one of the browsers. however at a bare minimum node will definitely be adding Request and Response objects, it just might not add a function called fetch

@mscdex

This comment has been minimized.

Copy link
Contributor

commented Mar 16, 2018

-1 this kind of higher-level functionality is best left to userland

@guybedford

This comment has been minimized.

Copy link
Contributor

commented Mar 16, 2018

This would definitely be useful in simple cross-platform APIs and I think is what a lot of people use node-fetch for already. Also it would be nice if HTTP/1 v HTTP/2 negotiation can be handled automatically like in browsers as well.

@styfle

This comment has been minimized.

Copy link
Contributor

commented Mar 17, 2018

I would love this! ❤️

Isomorphic JS is one of the big reasons people who start with JS on the front end, eventually pick up Node.js on the backend.

You get to run the exact same function in the browser and the server and the one place of contention I keep finding is window.fetch.

Some of the code that runs on the server and client needs to make HTTP requests to another server (think microservices with server side rendering and client side rendering).

One case for bringing it into core is that making HTTP requests (client) is closely tied to responding to HTTP requests (server).

And we now have isomorphic URL parsing so now all we need is fetch! Let’s make fetch happen!

@mikemaccana

This comment has been minimized.

Copy link
Contributor

commented Mar 17, 2018

Fetch is 'low level' according to it's author and major supporter hence missing a bunch of features like support for content types, JSON not being default, no query string encoding. Apps that need a high level quality HTTP client available in all JavaScript environments can continue using superagent.

@devsnek

This comment has been minimized.

Copy link
Member

commented Mar 17, 2018

@mikemaccana the fetch we are talking about is https://fetch.spec.whatwg.org/ and i don't think its appropriate to be plugging other http libraries

@mikemaccana

This comment has been minimized.

Copy link
Contributor

commented Mar 17, 2018

@devsnek Yes I know, that's the one I was specifically referring to. I don't have any particular enjoyment of superagent asides from it being:

  • a full featured HTTP client
  • available in node and the browser
  • more popular than fetch
  • has JSON as a default
  • encodes query strings
  • uses content types to determine response body
  • uses HTTP verbs as method names, so you can happily .get() and .post() things rather than 'fetching with method POST' which is a somewhat odd mental model

If fetch supported these I'd suggest it be included in node. To repeat: I've asked fetch's author and main proponent, Jake Archibald, why it doesn't support these things and it's stated that fetch is designed to be a low level API and things like sensible defaults can/should be added by higher level APIs. As a result I see no reason to use fetch in node, or in the browser.

@joyeecheung

This comment has been minimized.

Copy link
Member

commented Mar 18, 2018

@mikemaccana

These are actually desirable properties for the argument of including fetch in Node.js core, if we want to keep the core low-level and small. I believe we will need to eventually provide a promisified API for http/http2 anyway, and fetch as an existing spec for a similar set of low-level functionality as our http is something worth considering. In my experience the missing pieces in fetch feels pretty similar to the missing pieces in Node.js's http, the major difference is that you have a spec'ed API to count on.

Also I kind of doubt the "more popular than fetch" part, I don't think there are as many people using super-agent in the browser as in Node, given that fetch simply exists in the browser (modulo situations needing polyfills)?

Although, before we start implementing fetch, I believe we will need to introduce the stream API in the browser? Is introducing yet another stream in Node.js on the table?

The one time I've been forced to resort to XHR in the browser was when I needed progress, although I think with the browser's ReadableStreams it's possible to do the same thing with an API that's a bit awkward (res.body.read().byteLength)- if I'd implement progress in Node.js I think I'll need to use chunk.byteLength from the chunk being emitted in the data event, which is where the difference between the two streams start to matter.

Also, the fetch spec does not seem to include timeout or agents (yet?) at the moment, there might be more functionalities missing compared to our existing http API. For reference, node-fetch seems to implement these non-standard options. Again, not sure if our implementation should implement non-standard functionalities even if they supply the missing pieces compared to the old API.

@joyeecheung

This comment has been minimized.

Copy link
Member

commented Mar 18, 2018

also cc @TimothyGu you might be interested?

@joyeecheung

This comment has been minimized.

Copy link
Member

commented Mar 18, 2018

@thecodingdude

entirely disagree; Node uses v8, and by extension, should implement as many as v8 features as possible that make sense. fetch is one of those where developers wouldn't need to npm install request or node-fetch which are very popular libraries so this functionality warrants being in core.

Technically fetch is not a v8 feature though, it's an API spec'ed by WHATWG, whereas v8 implements ECMA-262, a spec by ECMA TC39 - if v8 implemented fetch then there would not be this feature request, because we basically just expose what v8 exposes.

BTW: I don't think we are talking about pulling npm packages into the core? Rather, we are talking about implementing the spec on our own and pulling in the WPT to test compliance, possibly with a bunch of code written in C++, much like what we did for WHATWG URL.

@mikemaccana

This comment has been minimized.

Copy link
Contributor

commented Mar 18, 2018

@thecodingdude the continued popularity of non-fetch libraries isn't a personal opinion, it is a fact - superagent had 1.6 million downloads this week . Nor is the lack of reasonable high-level defaults in fetch: again (again) that is acknowledged by fetch's author. Please don't reframe verifiable objective technical facts as irrelevant subjective opinions because you do not like them.

please don't plug other libraries here, they are irrelevant to our discussion.

developers wouldn't need to npm install request or node-fetch

😂👍

@bnoordhuis

This comment has been minimized.

Copy link
Member

commented Mar 19, 2018

I'm personally -1 for two reasons:

  1. Having two different APIs (fetch() and http.request()) for doing the same thing is uncohesive.

  2. fetch() is not flexible enough to support things like proxying, custom TLS certificates, etc. It's handled transparently by the browser but that won't work in node.js.

currently, 84 people agree

I wouldn't put too much stock in that. GH polls are easily gamed through Twitter brigading and whatnot. I'd take them more seriously if it cost a dollar to cast an upvote.

@mikemaccana

This comment has been minimized.

Copy link
Contributor

commented Mar 19, 2018

thecodingdude I have commented with a bullet pointed list of technical reasons why fetch is unsuitable as a general purpose HTTP client. I am aware you "do not care how/good bad fetch is". I think this is wrong: node should pick and adopt quality APIs. That you're personally uninterested in fetch's quality does not make it irrelevant to the discussion. The browser vendors were comparing fetch to window.xhr, node is not.

@Jamesernator

This comment has been minimized.

Copy link

commented Mar 19, 2018

@mikemaccana Why does fetch need to have high level features like "HTTP verbs as method names" for it to be included in core? You can trivially write your own methods that do this, in addition to the other high level features you've mentioned.

The nice thing about having standard APIs shared across both the browser and Node is that by writing the high-level API to target the low-level one (in this case fetch) you can get consistent behaviour much more easily than having to make your API work around differences between different APIs in node and the browser.

@devsnek

This comment has been minimized.

Copy link
Member

commented Mar 19, 2018

@thecodingdude any reason this was closed? discussion still seems to be moving

@thecodingdude

This comment has been minimized.

Copy link
Author

commented Mar 19, 2018

@devsnek yeah, I am not satisfied with the way this discussion and issue went, and I realise that github issues are simply not the appropriate forum for discussing whether features should land into core. I much prefer the node-eps since that was a proper document outlining the proposal. My intention was to gather the thoughts of the core team, and whomever else is involved in merging features into core (planning, coding, testing etc). Instead the discussion here so far is basically meaningless. It should be about feasibility, what would be required, and when such a feature could be merged.

Go take a look at this issue and try and follow the avalanche of conversation about websockets in core. There's too much noise and not a straightforward approach to proposing new features. I think PHP's rfc system is a good example of getting new features added. Clear, concise, accountable.

I'm down for discussing an rfc and its merits in a separate issue but it's clear this isn't going anywhere, with little to no interaction from TSC or anyone else that matters so there's no point leaving it up just to spam people's inboxes.

I'll refrain from making such issues in the future, my bad.
cc @jasnell

@mikemaccana

This comment has been minimized.

Copy link
Contributor

commented Mar 19, 2018

@Jamesernator Why does fetch need to have high level features like "HTTP verbs as method names" for it to be included in core? You can trivially write your own methods that do this, in addition to the other high level features you've mentioned

Sure. I, and everyone else that needs to use query strings, can write a method to encode query strings. Better yet: let's include a good quality, batteries included HTTP client in node where everyone doesn't need to fix it.

@thecodingdude looks like you deleted your comment, but to answer anyway: HTTP2 is necessary because it is required to speak the current version of HTTP. fetch is not necessary, because it is but one of many HTTP clients, and not a very good one, for the reasons discussed.

@Jamesernator

This comment has been minimized.

Copy link

commented Mar 19, 2018

@mikemaccana But the reason for including fetch in Node core isn't about batteries included, it's so that you can write code that targets both the browser and Node at once. That's why Node also supports the URL constructor (and it's even now global like in the browser in node 10) and the docs even refer to the old url.Url objects as the legacy url API.

This is a good thing, as it means some module that uses dynamic import for instance they don't need to include any library to be able to load resources relative to it e.g.:

...

export default async function isWordForLanguage(language="english") {
    const wordDataUrl = new URL(`./resources/${ language }.csv`, import.meta.url)
    const words = await loadSomehow(wordDataUrl)
    return makeIsWord(words)
}

Now that code has no platform specific behavior (assuming a loadSomehow method which could potentially be fetch). The benefits to this should be fairly obvious:

  • Using APIs that are standardised and included in both the browser and Node means you'll get support in both contexts if there's any issues with implementations (and hence no chance of dead or deprecated libraries)
  • Smaller libraries for high-level features as they don't need to smooth over platform differences, they just use the universal API (URL or fetch or whatever) and build directly on top of that
  • Ditto for your own libraries that need high-level behaviour

I think it's a pointless endeavour to create another http library in Node if it isn't for interoperability. If Node creates another one that isn't in the browser then you simply get the same situation you have now, if you want browser inter-operable code you another library built on top of it just to get interoperability with browsers.

If you think it's important for browsers to support high-level APIs out of the box then it's probably worth supporting the efforts to make high level APIs part of the browser and make clear your goals what you want to see out of high-level APIs. I'm sure many developers would appreciate getting high-level API specifications.

Even better would be if there was a way to get Node and Browsers to collaborate on high-level features, I'm not sure how this might work but if it could be done then it'd be a lot easier to propose high-level APIs that target all environments instead of being silo-ed into Node or the browsers.

@mikemaccana

This comment has been minimized.

Copy link
Contributor

commented Mar 20, 2018

Thanks @Jamesernator - it's nice to know there's an effort to resolve this on a larger scale. Thanks also for being polite and actually reading my comments before responding.

@guybedford

This comment has been minimized.

Copy link
Contributor

commented Mar 25, 2018

I tend to think that were NodeJS to have been developed today, there's no doubt that fetch would have been implemented in the name of aligning with browser APIs. Yes there are some really interesting edge cases, but @TimothyGu has handled these really well in the node-fetch integration.

@thecodingdude do you mind if I reopen? We potentially could still have things line up here - I don't think the detailed discussion has run its course fully quite yet and I think there's still fruitful discussion to be had personally.

@bnoordhuis node-fetch handles the differences here by supporting the NodeJS-specific agent option that delegates features like proxy https://github.com/bitinn/node-fetch#options. It breaks the universal API, but could also be done conditionally quite easily, so I tend to think it seems a fairly good compromise with this stuff. It could also be decided that such a divergence is a bad idea as well though certainly.

The key thing though is that this fetch API is purposely restricted - HTTP and HTTP/2 APIs still remain fundamental, this could just provide the easy universal experience. There's nothing wrong with having two APIs to do the same thing, when one builds on top of the other to provide a compelling user experience.

@thecodingdude

This comment has been minimized.

Copy link
Author

commented Mar 25, 2018

@guybedford feel free to make a new issue discussing this further if that's the direction that is most suitable. I'd much prefer technical discussions about the implementation itself, and not what fetch does/doesn't do vs alternatives which is why I closed the issue in the first place.

Quite honestly, I don't see why this discussion needs to happen on github - at the least is should be locked to collaborators who are going to be working on the code and discussions focused entirely on implementing fetch, identical to the browser where possible, and nothing more and nothing less.

@guybedford

This comment has been minimized.

Copy link
Contributor

commented Mar 26, 2018

@thecodingdude I'm still not quite sure I follow why you need a new issue here. Discussing the problem boundaries should surely be the first step for any new feature, and allowance should be made for a wider discussion at the start as well. 16 comments hardly seems off the rails as well.

Certainly the technicalities of whether it should be JS or C++, or where to draw the line on pooling and proxies are important, but things have to run their own course.

@bnoordhuis

This comment has been minimized.

Copy link
Member

commented Mar 26, 2018

There's nothing wrong with having two APIs to do the same thing, when one builds on top of the other to provide a compelling user experience.

But it's not 'on top of' - they do the same thing, just with different interfaces.

Arguments in favor of fetch() boil down to 'convenience' - not a good reason for inclusion in core.

@guybedford

This comment has been minimized.

Copy link
Contributor

commented Mar 26, 2018

Thanks @bnoordhuis these are important points as to where the line is in Node, and I must admit I'm not familiar with these sorts of discussions historically, but so long as Node strives to provide core utilities, this seems like an important one to include to me.

But it's not 'on top of' - they do the same thing, just with different interfaces.

Currently node-fetch is built on top of the http module, I'd personally prefer such a JS-based implementation here, working from exactly what @TimothyGu has already built as one of the most popular libraries. The pattern of having high and low level APIs for the same thing is an established one.

Arguments in favor of fetch() boil down to 'convenience' - not a good reason for inclusion in core.

Convenience is a hugely important argument, the question is by how much to justify the maintenance costs. Node doesn't just support modules and native bindings - it provides server utilities and universal APIs where appropriate. As I've said, fetch seems to me a critical part of the universal toolbox these days.

I'd even push for fetch to transparently integrate HTTP/2 in future, allowing such a feature to not only build on top of the existing builtins, but also to provide a unification that we don't currently have. I know there are complexities to be tackled here, and it's not a given, but it feels like it would be huge win for Node and its users.

@jakearchibald

This comment has been minimized.

Copy link

commented Apr 26, 2019

@mcollina Node's WHATWG URL implementation isn't on the global either, so I don't think that's worth worrying about too much.

@mcollina

This comment has been minimized.

Copy link
Member

commented Apr 26, 2019

I think the best approach would be to have a http.createFetch(opts) API that wraps the agent model, http/https/http2/http3 in a way that is a) coherent with Node.js API b) compatible with the most common usecases for the browser spec c) easy to intercept (our current API is extremely hard to wrap, see http://npm.im/nock).

I think this could give us enough flexibility to solve most of the use cases. There will definitely be some key differences at the semantics/behavior level (caching, connection pooling and security) that makes me wonder if we should call this “fetch” to begin with - however fetch is extremely popular, and the community is happy with node-fetch and other polyfills, so I’m not so much concerned about having something significantly different. We definitely need a new HTTP client API in core, as our current model is aging badly.

(Note that there are significant difference in the WHATWG URL implementation as well, and we are not doing a good job in telling these).

@philsturgeon

This comment has been minimized.

Copy link

commented Apr 26, 2019

@jakearchibald

This comment has been minimized.

Copy link

commented Apr 26, 2019

@philsturgeon

The WHATWG fetch currently ignores caching

That's… incorrect. Large parts of the spec and api are devoted to negotiating caches.

What made you think fetch ignores caches?

@philsturgeon

This comment has been minimized.

Copy link

commented Apr 26, 2019

@devsnek

This comment has been minimized.

Copy link
Member

commented Apr 26, 2019

+1 to a new impl, maybe we can even use napi to integrate it into core

@addaleax

This comment has been minimized.

Copy link
Member

commented Apr 26, 2019

I’m all for vendoring in an npm module that we then expose as fetch, regardless of whether it’s a new implementation or not.

@benjamingr

This comment has been minimized.

Copy link
Member

commented Apr 26, 2019

@mcollina I don't think anyone is concerned with the full fetch specification in terms of caching. Node.js is not a browser but more and more code is shared between client and server and while we don't have the same guarantees as our API surface gets modernised (like fs.promises) I think fetch is not a bad fit. It does streams pretty cleanly, backpressure, cache handling and other things.

There are parts of it that deal with the cross-origin policy which we don't need/want but other than that I think there is merit in a single modern low-level primitive for making http requests in core (fetch being a low level primitive for requests).

What about something like (and I'm bikeshedding):

  • Make an EventTarget subclass of EventEmitter in core.
  • Make a WhatWG stream subclass (or close to) of ReadableStream.
  • Explicitly don't support blobs (but maybe add it later?).
  • Add fetch as experimental on top of http.
@mcollina

This comment has been minimized.

Copy link
Member

commented Apr 26, 2019

I’m pretty firm in not having a single global connection pool, security settings, caching, etc. I’m good with your plan as long as we can create all of that and tear it down if we want to. Node.js become extraordinarily better as it got rid of the default global agent, and I truly do not want that situation to happen again.

(minus some details on streams interop, but those are API details).

@bitinn

This comment has been minimized.

Copy link

commented Apr 26, 2019

On one hand, I would love to see Fetch being in the nodejs core.
On the other, I think people should temper their expectations, as I suspect the end result is going to disappoint many users.

Reasons:

  • People stumping for native Fetch support want to use it in an isomorphic environment: ie. they want to write the same code and run on browsers, nodejs and even react-native.
  • Even if we skip topics like cookie, cache, protocol-relative url and error handling, there is still the problem with WHATWG Stream.
  • My strong preference is for nodejs to implement Fetch with WHATWG Stream, but that means the API won't play with most existing library (sure, you can convert them into nodejs stream, but the interop step itself isn't isomorphic).
  • On the flip side, with multiple implementation of Fetch using nodejs stream, I don't see the reason for yet another Fetch that's inherently non-isomorphic. (We can't even agree on simple things like highWaterMark, which cause code to run fine in browsers but face backpressure on nodejs.)
  • I urge people read limitation of existing Fetch implementations, as well as what we had to do to make Fetch more palatable on server-side, then consider if this is the Fetch you wish for.

https://github.com/bitinn/node-fetch/blob/master/LIMITS.md

@benjamingr

This comment has been minimized.

Copy link
Member

commented Apr 26, 2019

@bitinn what about a node stream subclass that implements the whatwg stream interface?

How important do you think cookie/cache/protocol relative URLs are here? I think error handling and streams are solvable and Node streams are asynciterable anyway already :)

@bitinn

This comment has been minimized.

Copy link

commented Apr 27, 2019

@benjamingr That could be a good compromise.

Pros:

  • People who just want a unified Fetch that abstracts away H1/H2 difference will be happy, they are not necessarily using Fetch in an isomorphic sense, but like the fact that API works similarly, so less things to learn and master.
  • Having Fetch API in node, even a limited one, will move the surrounding infrastructure, so HTTP-related code will become more isomorphic over time.

Cons:

  • People who use Fetch API in an isomorphic sense will demand more than just fetch(): to start there are new Response(), new Request(), new Headers(), Blob, FormData, AbortSignal, then inherent lack of cache and cookie will make people write non-isomorphic code.
  • People who want more control than standard Fetch will demand more than just fetch(): to start they will need to use custom agent, then there will be things like controlling download size, dealing with infinite response, socket support etc.

Fetch API is designed to do a few things very well, but not others. Moving spec in those directions will be difficult, so you almost always have to make judgement calls that alienate people.

See our tests (which is 2700 lines, when actual code is only 500 lines):

https://github.com/bitinn/node-fetch/blob/master/test/test.js

See also how error handling is non-trivial given existing Fetch spec:

bitinn/node-fetch#549

@benjamingr

This comment has been minimized.

Copy link
Member

commented Apr 27, 2019

@bitinn are you attending the summit (https://github.com/nodejs/summit/issues) by any chance? I think it would be great to talk this over with other interested parties (like @mcollina and maybe @jakearchibald if he's attending) and enumerate the challenges in adding fetch to core.

cc @nodejs/open-standards

@benjamingr

This comment has been minimized.

Copy link
Member

commented Apr 27, 2019

I'd also like to add that I've found contributing to the fetch spec in the past a pleasant experience and the people working on fetch to be very welcoming and friendly - so I definitely think we should engage those people when doing this.

@Jamesernator

This comment has been minimized.

Copy link

commented Apr 29, 2019

I think it would be important for Node to ensure that it doesn't behave differently when non-default options are used with fetch.

For example take credentials (cookies), supposing Node didn't ship with credentials support then my opinion would be that it would be better for Node to throw an error on fetch(someUrl, { credentials: 'include' }) instead of silently ignoring the credentials option. Node would instead introduce its own new option e.g. 'bypass' and use fetch(someUrl, { credentials: 'bypass' }) (bikeshed name).

Now obviously it would be a huge pain to have to do fetch(someUrl, { credentials: isNode ? 'bypass' : 'omit' }), instead one could just say that any options that are not provided will just use the host's default which in Node's case would be 'bypass'. Even better yet would be if the fetch spec itself could reflect that non-browser fetch implementations may use different defaults for various options.

Similarly I think it's very important that if Node is to diverge with the spec in a non-compatible way (e.g. to expose extra data or such) it should do so using new properties rather than behaving in a different way to the existing properties.


In my personal opinion it might even be worth considering features like cookies/CORS/caches/etc within Node (whether a global fetch is provided with these features or not I don't personally care).

For example Node could consider a factory with things like cookie jars and such similar to other http libraries:

import { makeFetch, createPersistentCookieJar, createInMemoryCache } from 'fetch'

const cache = createInMemoryCache({ limit: os.totalmem() / 4 })
const cookieJar = createPersistentCookieJar('./cookies.cookiejar')

const fetch = makeFetch({ cache, cookieJar, origin: 'https://mycoolwebsite.com' })

// Use fetch like a browser fetch

Having said that I still think it would be a lot better for Node to have a good subset of fetch than to not implement it all or be delayed for ages working on a complete implementation.

@jakearchibald

This comment has been minimized.

Copy link

commented Apr 29, 2019

For example take credentials (cookies), supposing Node didn't ship with credentials support then my opinion would be that it would be better for Node to throw an error on fetch(someUrl, { credentials: 'include' }) instead of silently ignoring the credentials option.

I'm not so sure. In the browser, if you call fetch(someUrl, { credentials: 'include' }) when there are no no credentials (new site visit, incognito mode, just cleared cookies) it does not reject.

It feels like Node should behave like a browser with no credentials.

Node would instead introduce its own new option e.g. 'bypass' and use fetch(someUrl, { credentials: 'bypass' }) (bikeshed name).

How would this differ from 'omit'? I don't think Node should add new options without adding them to the spec.

@Jamesernator

This comment has been minimized.

Copy link

commented Apr 29, 2019

@jakearchibald I think I was getting confused with CORS when trying to make an example, my example should've been fetch(someUrl, { mode: 'bypass-cors' }) where we completely ignore CORS but can still read the response as this is the typical behaviour most people want in Node.

Obviously there's alternatives:

  • Simply allow reading normally opaque responses in Node (treating file:/// as the origin)
  • Add another mechanism just for Node for reading opaque responses (treating file:/// as the origin)
  • Simply ignore all cors-related flags and read any response

I'd prefer to lean on the side of caution and go with either of the first three ideas and not just ignore existing flags (instead throwing on use of things node doesn't want to support like no-cors or actually supporting them)

@bitinn

This comment has been minimized.

Copy link

commented Apr 29, 2019

To @Jamesernator and others reading this thread.

The tough part about getting Fetch as close to spec as possible is the lack of context, in browsers you take that for granted: cookies, cache, sessions can all be considered a part of it.

This is probably why fetch-h2 invented its context object, because it's more natural for HTTP/2.

This is also why node-fetch didn't, because we like the fact nodejs HTTP/1 is context-less, and you control connections through custom agent.

My take: Things will be easier if nodejs can provide a new HTTP abstraction layer where you don't think about HTTP/1 or HTTP/2; yes, it's as hard as writing a spec-compliant Fetch, but you don't have to be spec-compliant when you invent a new API, you can effectively delay the compliant step to when you wrap this API into Fetch.


The second issue is simply API, you are allowed do things like new Response(new FormData()).Blob() and per spec it should just work.

My take: Things will be easier if nodejs can provide things like Blob, FormData and WHATWG Stream in the first place. Without them, you rely on userland to implement a compliant solution, but neither Blob nor FormData expose a way to reliably "detect" themselves, so you rely on name guessing and API duck-typing. (This is what node-fetch does.)


The third issue is spec change. There were instances where Fetch spec change breaks existing API. For example, reading Set-Cookie headers are still a hack in node-fetch because Headers API expect get(name) to be a single string, when you can have multiple Set-Cookie headers.

Browsers doesn't have to worry about this due to their context (there used to be getAll() but this API was dropped), the spec never needed to move and add an extra API.

My take: if nodejs officially show intent to implement Fetch, then I have a feeling Fetch spec team could be convinced to add special API for things like Set-Cookie (barring the security issue). This potentially extend to other spec-related dilemma.

Thx for reading.


@benjamingr Unfortunately I am not going to the summit (work mostly in C# nowadays), but I sure hope nodejs team get to discuss this issue with fetch spec team and browser vendors.

@sheerun

This comment has been minimized.

Copy link

commented Apr 29, 2019

Maybe it would be better to turn the tables and instead of trying to 100% match official fetch spec, create node's own fetch spec that is simpler in scope than original fetch and is also dead-simple to polyfill with original fetch (ideally 1:1 API). With this change you don't need to expose global fetch or Request or Response because no longer you're trying to implement original fetch.

An API could look something like this:

const { fetch, Response, Request } = require('http')

And on browser-side it would be transformed to something like:

const { fetch, Response, Request } = window
@benjamingr

This comment has been minimized.

Copy link
Member

commented Apr 29, 2019

I talked with @jakearchibald in private (asking about the summit) and since a lot of involved parties won't attend - I was thinking about maybe spinning this into a small team of interested parties.

@jakearchibald @bitinn and any other party maybe interested: I think this is worthwhile and I think we should discuss this. I think the start of June (after JSConf and the summit) would be a good time frame to discuss this.

@nodejs/open-standards wdyt?

@yordis

This comment has been minimized.

Copy link

commented Apr 29, 2019

@sheerun about your comment,

I would prefer that we put those things in the global object so we do not need to create special cases unless there is a technical reason for it (I don't mind re-exporting things).

const { fetch, Response, Request } = globalThis; // don't forget that we have this now
@guybedford

This comment has been minimized.

Copy link
Contributor

commented Apr 29, 2019

If there are going to be implementation differences, a module approach makes more sense like @sheerun describes, as a browser shim can be designed for the Node case, just like with workers.

@sheerun

This comment has been minimized.

Copy link

commented Apr 29, 2019

@yordis It's just example. I'm pretty sure the shim woudn't be as simple and rather something like:

const { fetch, Response, Request } = __makeNodeFetch();

where __makeNodeFetch is injected by polyfilling bundler / library

EDIT: I would also suggest official shim be shipped by node team instead of created by 3rd party

@benjamingr

This comment has been minimized.

Copy link
Member

commented Jun 2, 2019

Hey, just an update that we had a meeting about this in the summit and here are the notes https://docs.google.com/document/d/1tn_-0S_FG_sla81wFohi8Sc8YI5PJiTopEzSA7UaLKM/edit there is also the PR linked to above that is in a very early stage.

We actively need people to step up and help with the above issues.

@yordis

This comment has been minimized.

Copy link

commented Jul 6, 2019

@benjamingr question out of ignorant.

How hard would be to adapt Deno implementation of fetch into NodeJS? (please don't focus on Deno vs NodeJS 🙏)

Looking at the code it seems that could be portable but I am not sure of the walls I will face doing so.

Did anyone try that?

Should we try that?

@benjamingr

This comment has been minimized.

Copy link
Member

commented Jul 6, 2019

How hard would be to adapt Deno implementation of fetch into NodeJS?

Very, Deno's implementation is very incomplete and lacks a lot of features. Starting off from node-fetch like this PR is a lot more feature complete already. Moreover, Deno's implementation works by hooking into tokio with flatbuffers (we use neither) and really wouldn't make much sense for node.

(There is no enmity between Deno and Node.js, it's really not a competition :) )

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.