Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Alternative libraries to request #3143

Open
reconbot opened this issue Apr 1, 2019 · 56 comments
Open

Alternative libraries to request #3143

reconbot opened this issue Apr 1, 2019 · 56 comments
Labels

Comments

@reconbot
Copy link
Contributor

@reconbot reconbot commented Apr 1, 2019

Since the announcement of request going into "maintenance mode" (full details in #3142) I'd like to collect a list of alternative libraries to use. Please comment below and I'll update this table. When we have a list of good alternatives we should add this to the readme.

In no particular order and dreadfully incomplete;

Package Name Bundle Size API Style Summary
node-fetch 0.4kb promise / stream A light-weight module that brings window.fetch to Node.js
bent 1kb fp / promise / stream Functional HTTP client w/ async/await
got 48.4kb promise / stream Simplified HTTP requests
make-fetch-happen 442kb promise / stream make-fetch-happen is a Node.js library that wraps node-fetch-npm with additional features node-fetch doesn't intend to include, including HTTP Cache support, request pooling, proxies, retries, and more!
axios 11.9kb promise / stream Promise based HTTP client for the browser and node.js
unfetch 1kb promise / stream Tiny 500b fetch "barely-polyfill"
superagent 18kb chaining / promise Small progressive client-side HTTP request library, and Node.js module with the same API, sporting many high-level HTTP client features
tiny-json-http 22kb promise Minimalist HTTP client for GET and POSTing JSON payloads
needle 164kb chaining / promise The leanest and most handsome HTTP client in the Nodelands
urllib 816kb callback / promise Help in opening URLs (mostly HTTP) in a complex world — basic and digest authentication, redirections, cookies and more.
@reconbot reconbot added the neverstale label Apr 1, 2019
@jeffscottward

This comment has been minimized.

Copy link

@jeffscottward jeffscottward commented Apr 1, 2019

As a frontend focused guy who also does node.js from time to time, axios has been my go to.
Easy API, from Facebook, works on browsers and node? Done

@bnb

This comment has been minimized.

Copy link

@bnb bnb commented Apr 1, 2019

Per a recent discussion with @mikeal, I have Bent a try. As a Node.js developer whose been using request for a while now, bent was definitely an easy transition - highly recommended 💖

@paambaati

This comment has been minimized.

Copy link
Contributor

@paambaati paambaati commented Apr 1, 2019

@reconbot You might want to add got, needle and urllib.

@simov

This comment has been minimized.

Copy link
Member

@simov simov commented Apr 1, 2019

Well, it feels kind of wrong to promote my own little library here, but since that's the goal of the issue, here it is: request-compose is a functional, 0 deps HTTP client with support for promises, streams, and a bunch of other useful options, most of which are very close to the ones found in request.

I also wrote an article about it. The general idea is that everyone is encouraged to compose their own HTTP clients, specifically tailored to their own needs.

As for the bundle size, I've no idea, but it should be pretty small, though this client was never designed to be used in the browser.

@csantanapr

This comment has been minimized.

Copy link

@csantanapr csantanapr commented Apr 1, 2019

It might be good to add the following columns to the table:

  • Number of stars in Github (yes I already know this is not the only factor when choosing a lib)
  • Number of npm downloads (maybe weekly, same stat as npm website, and yes I already know this is not the only factor when choosing a lib)

When putting side by side these numbers some libs have thousands of stars and million of downloads weekly, vs others in the hundreds.

My 2 cents, OK to ignore and move on, no need to correct or dispute the comment.

@reconbot

This comment has been minimized.

Copy link
Contributor Author

@reconbot reconbot commented Apr 1, 2019

@csantanapr I agree, it might be worth comparing feature sets too. Proxy support, cache support, auth features etc. If you use a specific feature of request and need to find it elsewhere, this would be a good time to talk about it.

@kreig303

This comment has been minimized.

Copy link

@kreig303 kreig303 commented Apr 1, 2019

axios gets my vote, especially as a front-ender.

@JamesMGreene

This comment has been minimized.

Copy link
Contributor

@JamesMGreene JamesMGreene commented Apr 1, 2019

Worth a look: ky (frontend) and ky-universal (isomorphic)

@rmngrc

This comment has been minimized.

Copy link

@rmngrc rmngrc commented Apr 2, 2019

Axios user here. That way, all our teams can use the same library regardless the environment: browser or nodejs (running in server or serverless). Very well maintained, and all our people love it.

@sindresorhus

This comment has been minimized.

Copy link

@sindresorhus sindresorhus commented Apr 2, 2019

We have a good comparison between got, request, node-fetch, axios, and superagent in the got readme: https://github.com/sindresorhus/got#comparison
(PR welcome if you see any inaccuracies. We've tried to keep it as neutral as possible)

Got also has a migration guide for moving from request: https://github.com/sindresorhus/got/blob/master/migration-guides.md

@bajtos bajtos mentioned this issue Apr 2, 2019
0 of 1 task complete
@tracker1

This comment has been minimized.

Copy link

@tracker1 tracker1 commented Apr 2, 2019

For me, I tend to do wrappers around fetch api, so node-fetch is my goto. Despite the negative aspects, I usually load it onto global.fetch in node, so I can rely on it always being available, much like in the browser (via polyfill for older browsers). Can also use isomorphic-fetch which is pretty much a wrapper around node-fetch for node, and the fetch polyfill (or already available fetch) in browser. Since I don't have to support legacy browsers, I just use the global, and establish the global for use in node.

@vdeturckheim

This comment has been minimized.

Copy link

@vdeturckheim vdeturckheim commented Apr 4, 2019

Hey, Wreck (https://www.npmjs.com/package/wreck) is what I use

@Velveeta

This comment has been minimized.

Copy link

@Velveeta Velveeta commented Apr 4, 2019

I would prefer something that mimics the fetch api on the client. Libs like axios, superagent, etc are higher level abstractions on top of a standard request library. As a replacement for the low-level request library, I'd like to see something that mirrors the low-level equivalent on the client for the purposes of universal js development. Libs like axios and superagent can then just reimplement themselves on top of that, and its users can continue using them, but those shouldn't be considered foundational for this purpose.

@kreig303

This comment has been minimized.

Copy link

@kreig303 kreig303 commented Apr 4, 2019

@Velveeta I went and looked at the axios codebase and see no evidence that it is based on a "lower-level standard request library". Please tell me how you came to this conclusion?

@reconbot

This comment has been minimized.

Copy link
Contributor Author

@reconbot reconbot commented Apr 4, 2019

@sindresorhus's comparison is by far the better approach than my list above. https://github.com/sindresorhus/got#comparison

node-fetch/isomorphic-fetch is a suitable low level building block for most clients. I'd love to see a fetch based request shim.

@simov

This comment has been minimized.

Copy link
Member

@simov simov commented Apr 4, 2019

I would wrap fetch with nicer API any day. Well, I guess that's just a matter of preference, but implying that the fetch API is great just because it's a defacto standard in the browsers is just wrong. I know it's less noise to have it isomorphic on both sides, but that don't make it any better.

@dar5hak

This comment has been minimized.

Copy link

@dar5hak dar5hak commented Apr 4, 2019

There's r2 by @mikeal himself. It is meant to be a spiritual successor to request. It has a Promise API and is 16kb compressed.

@ofrobots

This comment has been minimized.

Copy link

@ofrobots ofrobots commented Apr 4, 2019

Axios may work okay in the browser, but that hasn't been our experience with it on Node.js. Also, I am not sure if it is actively maintained anymore.

image

@Velveeta

This comment has been minimized.

Copy link

@Velveeta Velveeta commented Apr 4, 2019

@kreig303 I haven't looked into the internals of axios, so I wasn't aware of that. Looks like it's currently based on regular XHR's, which makes sense, since it's a solution for both client and server requests. I simply meant that axios is pretty feature rich, and something a little more bare bones should be considered for a foundational module like a replacement for request, and then let other more feature rich libs build on top of that if they desire. I opted for something that mirrors the fetch API specifically for the purposes of having a consistent API on both client and server (like the XHR's that underly axios), and because it's the logical successor to XHR's. If a nicer API wrapper is desired, there's plenty of opportunity to wrap it and release another library with that optimal API, but I'm all for feature and API parity between client and server wherever it can be done.

@simov

This comment has been minimized.

Copy link
Member

@simov simov commented Apr 4, 2019

Well, one of the issues we have in request is too many features, and too much exposed state, even the one that's considered internal. It's both a curse and a bless to have so many features. It's a bless because that's why it is so popular, and it was first. It's a curse because without a huge amount of constant effort to keep the codebase clean, straightforward, and generally exciting to work with, the project eventually dies. And that's not even a request's problem, it's the user's own perspective of always wanting to put something out of their own layer, and instead put it under the blanket somewhere else.

Well, I guess axios have the same faith ..

So what we can all do instead, is put at least some amount of effort into understanding how the wheel works, and then try to think through each individual task at hand, and see which wheel fits best.

@kreig303

This comment has been minimized.

Copy link

@kreig303 kreig303 commented Apr 4, 2019

@ofrobots that's a bit of a selective screenshot for such a popularly used library. Here's mine:
Screen Shot 2019-04-04 at 1 58 24 PM

FWIW I don't recall if I'd used it as a back-end lib, so I am in no position to verify your claims (unless you had a peculiar use case it didn't cover).

Martii added a commit to Martii/OpenUserJS.org that referenced this issue Feb 13, 2020
* Please read their CHANGELOGs
* Delete op retested
* *request* is deprecated and another suitable alternative needs to be found. See request/request/issues/3143 and request/request/issues/3142. Note we didn't get the deprecation warning in dev but examined with CHANGELOGs/commits on this package.
  * Affected code *(a lot)*:
    * app.js *(TLS certficate ping)*
    * libs/githubClient.js *(github import/browse)* Related to OpenUserJS#1705
    * libs/repoManager *(github import/browse)* Related to OpenUserJS#1705
    * controllers/scriptStorage.js *(webhook hooks)*
Martii added a commit to OpenUserJS/OpenUserJS.org that referenced this issue Feb 13, 2020
* Please read their CHANGELOGs
* Delete op retested
* *request* is deprecated and another suitable alternative needs to be found. See request/request/issues/3143 and request/request/issues/3142. Note we didn't get the deprecation warning in dev but examined with CHANGELOGs/commits on this package.
  * Affected code *(a lot)*:
    * app.js *(TLS certficate ping)*
    * libs/githubClient.js *(github import/browse)* Related to #1705
    * libs/repoManager *(github import/browse)* Related to #1705
    * controllers/scriptStorage.js *(webhook hooks)*

Auto-merge
@franciscop

This comment has been minimized.

Copy link

@franciscop franciscop commented Feb 18, 2020

node-fetch is being reported incorrectly and only the "browser" version is reported (defeating the point of a Node.js list). This is what seems to be wrongly measured:

Instead, either of these should be measured:

They are all around ~40kb

@franciscop

This comment has been minimized.

Copy link

@franciscop franciscop commented Feb 18, 2020

unfetch is also incorrectly reported:

  • The homepage says that "Use in Node.JS is handled by isomorphic-unfetch", so it should be reporting the combination of both.
  • isomorphic-unfetch uses node-fetch (code, docs) for Node.js, so its reported size should be at least that of node-fetch (see my previous comment).
@mikeal

This comment has been minimized.

Copy link
Member

@mikeal mikeal commented Feb 18, 2020

Since it has been brought up so much I should say a bit about my experience with node-fetch.

First of all, it’s quite an achievement. The amount of code and engineering effort that has gone in to it is much greater than what we ever put into request. fetch seems like a small API and I think people assume the effort to provide a compatible API in Node.js is nominal, but it really is not.

As a result, the code base is massive. It’s a sizable dependency in Node.js, which you likely won’t see at all in browser bundles, but it’s not as though dependency size is not an issue in Node.js, particularly in serverless environments.

node-fetch is indispensable when testing because it does all the work of fully emulating the browser’s APIs, but if you’re using it in an application, even one that is being run in Node.js and in the browser, it’s just too much code and too much indirection to be worth it.

IMO, the right approach at this time for a library that wants to be an http client in both Node.js and browsers is to implement a uniform API with a split implementation using fetch in the browser and require(‘http’) in Node.js. Applications, and http clients, should not target fetch or require(‘http’) directly and should not rely on emulating these API’s on either side. This is actually a lot easier than you might think, as you can see in the implementation of bent which is incredibly small https://github.com/mikeal/bent/tree/master/src

@domenic

This comment has been minimized.

Copy link

@domenic domenic commented Feb 18, 2020

@mikeal I'm having a hard time squaring

As a result, the code base is massive. It’s a sizable dependency in Node.js, which you likely won’t see at all in browser bundles, but it’s not as though dependency size is not an issue in Node.js, particularly in serverless environments.

with the 0.4 kB bundle size listed in the OP, which is the smallest of all the alternatives given?

@mikeal

This comment has been minimized.

Copy link
Member

@mikeal mikeal commented Feb 18, 2020

@domenic the complexity of emulating the browser API’s is the main problem, it’s a lot of unnecessary code and indirection when trying to debug. You’ve got the Blob API, you’ve got a lot of marshalling for the body, you’ve got almost 400 lines of header marshalling, and that’s not even looking at the actual API that gets exposed.

Like I said, it’s impressive, but it’s also just a ton of terse, clever, and ultimately unnecessary code if you want to do anything except emulate the fetch API.

@ThisIsMissEm

This comment has been minimized.

Copy link

@ThisIsMissEm ThisIsMissEm commented Feb 18, 2020

@mikeal you didn't even mention that there's a tonne more code required for node-fetch to be 100% compatible with the fetch API: it doesn't support readable and writable streams from what-wg (something that you need when emulating environments like Cloudflare Workers.

@domenic

This comment has been minimized.

Copy link

@domenic domenic commented Feb 18, 2020

Hmm, I still don't quite understand how to square "a ton" of "ultimately unnecessary" code with "0.4 kB, less than every other entry on the table and 0.25x the size of bent" (which is supposedly "the right approach" and "incredibly small").

@mikeal

This comment has been minimized.

Copy link
Member

@mikeal mikeal commented Feb 18, 2020

@domenic are you comparing the browser bundle size? I’m talking about the complexity of debugging these in Node.js. In the browser I would expect most of the node-fetch code to be non-existent, so I don’t really understand what you’re comparing.

@domenic

This comment has been minimized.

Copy link

@domenic domenic commented Feb 18, 2020

I'm comparing the value in the OP; I'm not sure how that is measured. Perhaps it is not measured correctly, which would be good information to update the OP with!

@mikeal

This comment has been minimized.

Copy link
Member

@mikeal mikeal commented Feb 18, 2020

@domenic ah yes, those are all browser bundle sizes, and since the post is pretty old many of them may be out of date although the bent figure is still close enough.

@solderjs

This comment has been minimized.

Copy link

@solderjs solderjs commented Feb 19, 2020

@root/request - an 80/20 drop-in replacement written in 500 LoC, and ZERO dependencies:

Created and tested against the behavior of request.js, on purpose.

https://git.rootprojects.org/root/request.js

@emanuelcasco

This comment has been minimized.

Copy link

@emanuelcasco emanuelcasco commented Feb 20, 2020

Hi everyone! Im doing a little research in order find to find a request's worthy replacement for my projects. For now, this is what i've put together: https://github.com/emanuelcasco/http-packages-benchmark

Recommendations and opinions are welcome of course!

@jslegers

This comment has been minimized.

Copy link

@jslegers jslegers commented Feb 21, 2020

Às request is now officially deprecated, I could not stress more the importance of officially proposing postman-request as a feature-complete drop-in replacement for request, and possibly @root/request for those who just need a limited subset of request and don't care about eg. streams.

This allows any package maintainer to drop request and get rid of the annoying deprecation message without spending more than a few minutes of dev time on this issue, and without having to refactor their entire library or app. It took me quite some time & frustration to figure out even that such drop-in replacements exist.

And yes, I'm aware that just "deprecation" doesn't break anything. Yes, technically everyone can still use request and possibly continue to use it for maybe even decades to come. That's not what deprecation is supposed to be used for, though. Deprecation IS supposed to act as a call to action, as a "grace period" for people to upgrade their code until someone somewhere decides to pull a plug.

I really, really hate it when "deprecation" is just used to mark "end-of-support" or "end-of-maintenance", as appears to be the case here. But I'd be bothered a lot less but this, if there was an officially supported AND actively maintained feature-complete drop-in replacement like postman-request.

In fact, has anyone considered handing over maintenance of this package to the Postman team? Instead of deprecating request, why not propose for them to port postman-request to request and let them become the new official maintainers?

takamin added a commit to takamin/npm-dlc that referenced this issue Feb 21, 2020
The package `request` was fully deprecated at Feb 11th 2020.
As new package, the Axios is adopted which is almost compatible.

* [request - npm](https://www.npmjs.com/package/request)
* [Request’s Past, Present and Future - GitHub #3142](request/request#3142)
* [Alternative libraries to request - GitHub #3143](request/request#3143)
* [axios - npm](https://www.npmjs.com/package/axios)
takamin added a commit to takamin/npm-dlc that referenced this issue Feb 21, 2020
The package `request` was fully deprecated at Feb 11th 2020.
As new package, the Axios is adopted which is almost compatible.

* [request - npm](https://www.npmjs.com/package/request)
* [Request’s Past, Present and Future - GitHub #3142](request/request#3142)
* [Alternative libraries to request - GitHub #3143](request/request#3143)
* [axios - npm](https://www.npmjs.com/package/axios)
@fuxingZhang

This comment has been minimized.

Copy link

@fuxingZhang fuxingZhang commented Feb 22, 2020

sorry to promote my own little library here

Designed for use in nodejs only

const http = require('@zhangfuxing/http');
const assert = require('assert');

(async () => {
  // http
  const httpRes = await http.get('http://baidu.com');
  assert(httpRes.includes('<html>'));

  // https
  const httpsRes = await http.get('https://cnodejs.org/api/v1/topics?limit=1&mdrender=false');
  assert(httpsRes.success === true);

  // download file: use pipe
  const fs = require('fs');
  const res = await http.get('http://localhost:3000', {
    responseType: "stream"
  })
  res.pipe(require('fs').createWriteStream('zfx.txt'))
  // or use pipeline
  const stream = require('stream');
  const util = require('util');
  const pipeline = util.promisify(stream.pipeline);
  const res = await http.get(`${url}/stream`, {
    responseType: "stream"
  });
  await pipeline(res, fs.createWriteStream('zfx.txt'));

  // post Buffer
  const res = await http.post('http://localhost/upload', Buffer.from('abc'));
  assert(res.success === true);

  // post Stream
  const fs = require('fs');
  const readStream = fs.createReadStream('./index.js');
  const res = await http.post('http://localhost/upload', readStream);
  assert(res.success === true);

  // post json
  const data = {
    username: 'zfx',
    password: 'password'
  };
  const res = await http.post('http://localhost/upload', data);
  assert(res.success === true);
  
  // post application/x-www-form-urlencoded
  const data = {
    username: 'zfx',
    password: 'password'
  };
  const options = {
    headers: {
      'Content-Type': 'application/x-www-form-urlencoded'
    }
  };
  const res = await http.post('http://localhost/upload', data, options);
  assert(res.success === true);

  // post FormData
  const FormData = require('form-data');
  const form = new FormData();
  const fs = require('fs');
  const readStream = fs.createReadStream('./index.js');
  form.append('my_field', 'my value');
  form.append('my_buffer', Buffer.from('abc'));
  form.append('my_file', readStream);
  // Set filename by providing a string for options
  form.append('my_file', readStream, '1.js' );
  // provide an object.
  form.append('my_file', readStream, { 
    filename: 'bar.jpg', 
    contentType: 'image/jpeg', 
    knownLength: 19806
  });
  const formHeaders = form.getHeaders();
  const res = await http.post('http://localhost/upload', form, {
    headers: {
      ...formHeaders,
    },
  });
  assert(res.success === true);

  // head
  const res = await http.head(url);
  assert(res.statusCode === 200);
  assert(res.statusMessage === 'OK');
  assert(res.headers && typeof res.headers === 'object');
  assert(res.statusCode === 200);
  assert(res.data === '');

  // options
  const res = await http.options(url);
  assert(res === 'GET,HEAD,POST,PUT,PATCH,DELETE'); 

https://github.com/fuxingZhang/http

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
You can’t perform that action at this time.