Standalone option makes browserify 10x slower #633

Closed
mourner opened this Issue Feb 5, 2014 · 13 comments

Comments

Projects
None yet
7 participants
@mourner

mourner commented Feb 5, 2014

$ browserify -v
3.24.11

$ time browserify app.js # without standalone
...
real    0m0.616s
user    0m0.565s
sys 0m0.060s

$ time browserify app.js -s app # with standalone
...
real    0m6.525s
user    0m6.389s
sys 0m0.179s

What could make such a huge difference in performance? In theory it shouldn't affect it much, right? It's just a small chunk of code for UMD added...

@andreypopp

This comment has been minimized.

Show comment
Hide comment
@andreypopp

andreypopp Feb 5, 2014

Contributor

It also does derequire to rename all require calls to some other name (not to conflict with AMD loaders).

Contributor

andreypopp commented Feb 5, 2014

It also does derequire to rename all require calls to some other name (not to conflict with AMD loaders).

@mourner

This comment has been minimized.

Show comment
Hide comment
@mourner

mourner Feb 5, 2014

Weird that derequire takes so much time... Is it so much more complex than running a simple search/replace?

mourner commented Feb 5, 2014

Weird that derequire takes so much time... Is it so much more complex than running a simple search/replace?

@thlorenz

This comment has been minimized.

Show comment
Hide comment
@thlorenz

thlorenz Feb 5, 2014

Collaborator

@mourner it builds an AST to find all requires.
That is done via esprima which is still somewhat slow. So if you do that on a lot of files you'll run into problems.

I actually put parsing ASTs on a separate process for that same reason for findex, which builds ASTs for entire project dependencies.
I'm not sure if that could be applied to browserify, but here it is.
Basically forks another worker to do the JS parsing and gets message back. That way main process isn't blocked.
You could potentially spin up multiple of these workers and parse multiple files in parallel.

On the other hand the better solution may just be to find a faster JS parser.

Collaborator

thlorenz commented Feb 5, 2014

@mourner it builds an AST to find all requires.
That is done via esprima which is still somewhat slow. So if you do that on a lot of files you'll run into problems.

I actually put parsing ASTs on a separate process for that same reason for findex, which builds ASTs for entire project dependencies.
I'm not sure if that could be applied to browserify, but here it is.
Basically forks another worker to do the JS parsing and gets message back. That way main process isn't blocked.
You could potentially spin up multiple of these workers and parse multiple files in parallel.

On the other hand the better solution may just be to find a faster JS parser.

@andreypopp

This comment has been minimized.

Show comment
Hide comment
@andreypopp

andreypopp Feb 5, 2014

Contributor

By using search/replace you can accidentally replace require inside a comment or a string literal.

I think the faster solution could be built on top of https://github.com/creationix/js-linker/blob/master/mine.js (if it tracks the position we can splice bundle several times after finding requires).

//cc @thlorenz @defunctzombie

Contributor

andreypopp commented Feb 5, 2014

By using search/replace you can accidentally replace require inside a comment or a string literal.

I think the faster solution could be built on top of https://github.com/creationix/js-linker/blob/master/mine.js (if it tracks the position we can splice bundle several times after finding requires).

//cc @thlorenz @defunctzombie

@mourner

This comment has been minimized.

Show comment
Hide comment
@mourner

mourner Feb 5, 2014

@andreypopp yeah, mine.js looks very promising for this!

mourner commented Feb 5, 2014

@andreypopp yeah, mine.js looks very promising for this!

@mourner

This comment has been minimized.

Show comment
Hide comment
@mourner

mourner Feb 6, 2014

BTW, meanwhile, could we add an option in browserify to export standalone build without AMD support, just global/module.exports, completely skipping derequire this way?

mourner commented Feb 6, 2014

BTW, meanwhile, could we add an option in browserify to export standalone build without AMD support, just global/module.exports, completely skipping derequire this way?

@andreypopp

This comment has been minimized.

Show comment
Hide comment
@andreypopp

andreypopp Feb 6, 2014

Contributor

@mourner you can create a separate module:

module.exports = window.MyLib = require('./main.js');

and browserify it to get a "standalone" build

Contributor

andreypopp commented Feb 6, 2014

@mourner you can create a separate module:

module.exports = window.MyLib = require('./main.js');

and browserify it to get a "standalone" build

@calvinmetcalf

This comment has been minimized.

Show comment
Hide comment
@calvinmetcalf

calvinmetcalf Feb 12, 2014

Contributor

so for derequire it's currently using espirma but only parsing it once, it finds uses of require as a functional argument and uses splice to replace them, if we can find a faster way to do that I am totally open to pulls/updating it

and @andreypopp you want to use module.exports = global.MyLib = require('./main.js');

Contributor

calvinmetcalf commented Feb 12, 2014

so for derequire it's currently using espirma but only parsing it once, it finds uses of require as a functional argument and uses splice to replace them, if we can find a faster way to do that I am totally open to pulls/updating it

and @andreypopp you want to use module.exports = global.MyLib = require('./main.js');

@defunctzombie

This comment has been minimized.

Show comment
Hide comment
@defunctzombie

defunctzombie Feb 13, 2014

Contributor

I don't fully understand why derequire is needed in browserify. The final bundle is self contained so our requires don't leak. Is it because external require functions leak in?

Contributor

defunctzombie commented Feb 13, 2014

I don't fully understand why derequire is needed in browserify. The final bundle is self contained so our requires don't leak. Is it because external require functions leak in?

@andreypopp

This comment has been minimized.

Show comment
Hide comment
@andreypopp

andreypopp Feb 13, 2014

Contributor

@defunctzombie to make --standalone work as advertised as an AMD module. Because AMD loaders rely on parsing of require(...) calls to infer deps they are left confused.

Contributor

andreypopp commented Feb 13, 2014

@defunctzombie to make --standalone work as advertised as an AMD module. Because AMD loaders rely on parsing of require(...) calls to infer deps they are left confused.

@Rich-Harris Rich-Harris referenced this issue in gingerhendrix/broccoli-browserify May 28, 2014

Open

Use cache for faster bundles? #10

@substack

This comment has been minimized.

Show comment
Hide comment
@substack

substack Jul 24, 2014

Collaborator

derequire is removed in v5 for performance reasons

Collaborator

substack commented Jul 24, 2014

derequire is removed in v5 for performance reasons

@substack substack closed this Jul 24, 2014

@mourner

This comment has been minimized.

Show comment
Hide comment

mourner commented Jul 24, 2014

👍

@zertosh

This comment has been minimized.

Show comment
Hide comment
@zertosh

zertosh Mar 9, 2015

Member

FWIW, derequire@2.0.0 is super fast now (see calvinmetcalf/derequire#26)

Member

zertosh commented Mar 9, 2015

FWIW, derequire@2.0.0 is super fast now (see calvinmetcalf/derequire#26)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment