Request’s Past, Present and Future

[mikeal]

Before I go into the details and reasoning I’ll get straight to the point. The most valuable thing request can do for the JavaScript ecosystem is to go into maintenance mode and stop considering new features or major releases.

Apologies in advance to the other committers on request that have been doing their best to improve it, but it’s for the best.

2009

The first version of request was one of the first modules ever created for the Node.js ecosystem. The earliest versions were written to APIs that pre-date the standard callback interface, streams, node_modules and npm. For the first few years, request and Node.js evolved together, each learning from the other. As Node.js improved and migrated core interfaces so did request. As request adopted changes to the core http library and streams it also informed improvements like the pipe event (which enabled request’s one line proxy) and one of Core http’s many re-writes (the one I had to write).

npm

request was one of the first modules added to the npm registry. As npm grew so did dependence on request. Even now, when npm is used far more for front-end than back-end work, request remains one of the most depended on modules in the registry. As I write this, 41K modules depend on request and it is downloaded 14 million times a week.

The place request has in the Node.js ecosystem is no longer one of an innovator but of an incumbent. If you Google for how to do something with HTTP in Node.js the examples are likely to show request as the client and express as the server. This has two notably bad effects.

It’s much harder for new libraries accomplishing similar tasks to gain adoption because of the incumbent position request holds over the ecosystem. It’s also very hard to change request in any meaningful way as the change not only may not be adopted by the majority of its dependents but it would put it out of alignment with the thousands of blog posts and stack overflow responses that use request.

Modern JavaScript

The last few years have been dramatic ones in JavaScript. Features people had talked about for years went from ideas, to standards, to features you can reliably depend on in most environments. The speed at which these have been adopted is staggering, mostly thanks to auto-updating browsers and an aggressive Node.js release schedule.

The patterns at the core of request are out of date. A few people might argue with that assessment, and I know who they are so I won’t be surprised, but it’s true. I have often been skeptical of the impact some of these features would have only to find myself adopting them wholesale not long after they are available in only the latest release of Node.js.

There’s a transition happening now in the ecosystem to these patterns. How messy that will be is still up in the air and I’m not going to try and read the tea leafs and figure out what the future looks like in that regard. The question for request is “Do we try to survive through that transition?” A year ago, I thought the answer was obvious and that we would, but now I’m convinced of the opposite.

A version of request written to truly embrace these new language patterns is, effectively, a new module. I’ve explored this space a bit already and have a project I’m quite happy with but it is incompatible with request in every conceivable way. What’s the value in a version of request that is incompatible with the old patterns yet not fully embracing the new ones? What’s the point in being partially compatible when there’s a whole world of new modules, written by new developers, that are re-thinking these problems with these patterns in mind?

The best thing for these new modules is for request to slowly fade away, eventually becoming just another memory of that legacy stack. Taking the position request has now and leveraging it for a bigger share of the next generation of developers would be a disservice to those developers as it would drive them away from better modules that don’t have the burden of request’s history.

Maintenance Mode

Here’s the plan.

• request will stop accepting new features.
• request will stop considering breaking changes.
• The committers that are still active will try to merge fixes in a timely fashion, no promises though.
• Releases will be fully automated, any merge into master will be published. I’ve already built this for some other projects using GitHub Actions.
  • We’re going to have to remove inactive collaborators and enforce 2fa, because commit rights will effectively become npm publish rights.

[reconbot]

I fully support this, I think a warning message and/or deprecating new releases is in order.

As for the change in process and guidelines, it makes my job a lot easier 👌

[simov]

Very well said @mikeal. I'm pinning this issue to gain more visibility.

[reconbot]

Things we might do - please discuss and volunteer!

• update readme with current state of project
• update ci publishing pipeline @mikeal
• provide a doc with some guidance on request alternatives #3143
• add a warning message on install of the package to use another package and reference the doc
• pick a date to stop support (I vote 6 months, but 12 is probably friendlier)
• close all feature requests and feature prs
• review and merge relevant bug fixes
• add github issue and pr templates explaining that features wont be merged
• deprecate the next major version (3.x) so project’s in active maintenance get a warning but older projects continue as usual

[analog-nico]

It makes a lot of sense! I will slowly adopt this policy for the request-promise family as well. Cheers to your important contributions to the node ecosystem!

[zzz6519003]

TLDR:
The patterns at the core of request are out of date
More specific pls?

[jleppert]

There's nothing wrong with the patterns request uses. On the contrary, there is a huge community in the javascript ecosystem that still uses these patterns. In my experience it is far larger a community than the vocal minority (most large companies) who have the resources to constantly tear up perfectly working code bases for nothing more than developer vanity and arrogance.

I'm sorry that you have fallen into this trap, request has been a great service to the community and I sincerely hope you reconsider your decision.

[jlippold]

Yea, I'm saddened that this is gone. Callbacks aren't bad, neither are promises or async await.

[jbg]

I think what you're missing @SimpleSamples is that the rest of the warnings you pasted have nothing to do with the deprecation warning that brought you here. You don't need to do anything about the deprecation but you might want to do something about your missing package.json (or whatever is causing those other warnings).

[jslegers]

So what ARE we supposed to do now with all of those packages that are using request under the hood?

I tried replacing request with @root/request in one such package, assuming it was indeed a drop-in replacement, but I couldn't get it to work.

I also tried replacing request with something like...

const httprequest = require('http').request;
const httpsrequest = require('https').request;

... and...

const request = parsedUrl.protocol === 'http' ? httprequest : httpsrequest`

... but I couldn't get that to work either.

So, now what? In absence of a drop-in replacement that actually does deliver on its promise, are we supposed to all just live with having multiple dependencies in our node_modules that rely on a deprecated package, several of which don't seem to be maintained? And why?

I get that request has become outdated in several aspects, but by deprecating this package without offering a suitable drop-in replacement, 41K modules are now directly depending on a deprecated package. If we consider the packages that use at least one of these 41K modules as a dependency, we may be talking about hundreds of thousands if not millions of packages that are affected.

Sure, I guess for some packages, it's easy to replace request with something like fetch, axios, superagent or Node.js's native http.request & https.request. But eg. in the case where requests are piped to another request (as with html2canvas-proxy), I struggle to figure out what the hell is going on there... and I can't really afford to spend many hours trying to replace just a few lines of deprecated code while I should actually be doing more important stuff.

I've always been weary of relying too much on a multitude of interdependent packages that are loaded in background with a package manager. Yes, I suppose it can offload a lot of the heavy lifting to third parties, but instead you get a whole bunch of other headaches to deal with.

Package managers give us a false sense of security. The whole leftpad debacle 4 years ago seems to have failed to open people's eyes with respect to the risks involved. I'm sure this won't make a difference either. Still, I feel like I must stress that there is something seriously wrong when one deprecated or broken package can impact millions of packages across the entire ecosystem. And this is likely to only get worse, as more an more projects will become abandoned, deprecated or even broken as more time passes, and we'll all be living in dependency hell...

But hey... I guess at least that means there'd always be a demand for JS developers to clean up the $%#@ mess...

[CliffS]

@jslegers

Still, I feel like I must stress that there is something seriously wrong when one deprecated or broken package can impact millions of packages across the entire ecosystem.

The only thing that is wrong is the panic you and others appear to be suffering from. leftpad went away, deleted. That can't happen now. Request has simply been deprecated; it isn't going anywhere. If it works now, it will continue to work in the same way.

There is no impact on millions of packages, unless you count a benign warning.

I also tried replacing request with something like...

Please stop panicking; please stop trying to fix a non-existent problem. Use whatever packages you like: request's deprecation is not going to break them. Gradually their package maintainers may move to another package. Or they may not. It doesn't matter. Nothing has changed, other than the appearance of one little message.

there'd always be a demand for JS developers to clean up the $%#@ mess...

There is no mess. Just progress.

[jslegers]

There is no impact on millions of packages, unless you count a benign warning.

Deprecating eg. a part of your API or a library basically means that you officially designate it as "obsolete" and you actively encourage users to opt for something else.

Deprecation is typically used as an intermediary stage between officially supporting something and officially dropping support for something, to give developers the time to replace whatever the thing is that you've deprecated until you that thing is no longer available or backwards compatible.

Deprecation warnings are supposed to make you nervous. They are intended as a call to action. Basically, the point of deprecation is to offer developers a “grace period”, allowing them update to their code before someone pulls the plug.

And they should not be used for any other purpose. Deprecation isn't supposed to just inform your users that "Our API doesn't follow the latest coding standards" or "I don't have the time to maintain this project anymore"... even though the library is pretty stable & pretty safe to use in +99% of all uses cases and likely to continue working fine for at least the next decade or so. That's not what deprecation means, and using deprecation warnings just to express a message like that sets a very bad precedent IMO.

Also, it's just plain ugly to have your npm install logs full of deprecation warnings. It looks sloppy. It's kind of a red flag and creates a bad first impression to people trying out your library or framework. Especially if people are actually paying you to use your library / framework, you want to give them a nice / clean install process with zero warnings.

Nothing has changed, other than the appearance of one little message.

That one little message looks sloppy and is supposed to have no other purpose but as a call to action... a call to replace an obsolete package with something else.

That may not matter to you, but it definitely matters to me and other people out there.

There is no mess. Just progress.

I guess you're one of those people who can't tell the difference between change and progress.

Either way, I noticed others in the comments suggested using postman-request. Unlike @root/request, that one does seem to work as a drop-in replacement, so I'll be updating all my packages with this one for the time being...

[riclf]

I think what you're missing @SimpleSamples is that the rest of the warnings you pasted have nothing to do with the deprecation warning that brought you here. You don't need to do anything about the deprecation but you might want to do something about your missing package.json (or whatever is causing those other warnings).

Touché !

[SimpleSamples]

The point has been made yet personal attacks continue. You guys are very intelligent and highly capable technically but there is room for improvement in personal expertise.

[jslegers]

The point has been made yet personal attacks continue. You guys are very intelligent and highly capable technically but there is room for improvement in personal expertise.

Being smart unfortunately does not prevent people from letting their emotions cloud their judgements... especially when their stuff happens like things getting deprecated seemingly without a good reason or a general consensus on what's the purpose of deprecation.

Anyway, I think I made my point quite clearly. I'd like to finish by encouraging @mikeal, @reconbot or any other maintainer of this project to officially propose postman-request as a feature-complete drop-in replacement for request, and possibly @root/request for those who just need a limited subset of request and don't care about eg. streams. This allows any package maintainer to drop request and get rid of the annoying deprecation message without spending more than a few minutes of dev time on this issue, and without having to refactor their entire library or app.

[riclf]

@mikeal coming up from the reality of request being deprecated, I'd like to ask you for one moment of reflection that will be helpful to some or maybe many of us. You have 2 later http request modules, subsequent to request: r2 and bent.

May I ask you to give us a short summary of the differences, benefits, and plus or minus's of moving to one of therse request replacements, over the other. I trust your work.

Thank you for this time, and may I be sure to say Thank You for the years of the request module.

-Ric