request is super slow to require

[ttmarek]

Description

const request = require('request')

takes 100 to 150ms to execute (on Linux). Whereas needle, a similar library, only takes 20 to 25ms. I haven't tested needle's require-time on Windows, but request's require-time clocked in at around 800ms on Windows 7, which is pretty ridiculous. The time to require a module may not be an issue for a lot of applications, but it is a concern for electron apps. For example, if I were to build an electron app that called some api, my app's startup time would take an automatic hit of around 100ms just to require in request (and even longer on Windows). Any delay longer than 100ms is felt by a user, so right of the bat, my application loses any chance at being perceived as fast.

I've been working around this by using node's archaic http module, but I'm at a point where its increasingly hard for me to escape request. It is so damn popular. Basically any npm module that talks to a server uses request under the hood and consequently inherits request's slow require-time.

Steps to Reproduce

console.time('time-to-require')
const request = require('request')
console.timeEnd('time-to-require')

Going Forward

Can we bring down request's require time (at least to needle's levels)?

[starzou]

+1

[molszanski]

975ms in my node.js docker container (linux vm)

[mikeal]

It would be nice to see some actual tracing here so that we know for sure what is taking all this time, but my guess would be that it's because of the modularity and the number of dependencies we have. For electron, you could work around this by using browserify --node to build a single bundle (assuming that the problem is in the number of unique files and not in the time to interpret).

Also, I'm curious if you are using npm 2 or 3. If it's npm 3 it may be taking longer because the dependency lookups have to go backwards from the deps back to the top level of the tree.

[molszanski]

My findings:

• Docker drive is slow, without it 150ms
• 30% of the time it is a dependency lookup. I've removed that overhead penalty with require 'cache-require-paths'

Didn't research further

[mikeal]

@molszanski interesting that lookups take that long, I wonder how much of the remaining time is file reading vs interpretation.

[molszanski]

Yeah, more research is required

[matteocontrini]

On Windows here, request takes on average 120ms while needle takes 70ms. I'd like to see this investigated and improved as well.

[neophob]

on a constraint arm device request takes more than 1200ms to load (measured using https://github.com/henvic/require-time).
the problem seems to be the huge dependencies request needs, more than 60 packages, also see nodejs/node#8259 for more details

Dependency Overview over time:

• 2.74 -> 66 packages needed
• 2.64 -> 52 packages needed
• 2.54 -> 49 packages needed
• 2.53 -> 32 packages needed
• 2.44 -> 23 packages needed

I guess I stick with v2.53

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[reconbot]

This is a general dependency issue. We have a lot of them, they are big. It's known.