unload to others for common low level methods #13

Closed
jdalton opened this Issue Mar 3, 2013 · 28 comments

Comments

Projects
None yet
8 participants

jdalton commented Mar 3, 2013

Prime could use a lib like Lo-Dash for its low level utility methods. Lo-Dash covers issues like old IE's [[DontEnum]] bug that prime.each handles, but also fixes Firefox/WebKit/Opera prototype enumeration bugs when iterating functions, shift and splice bugs on array-like objects, function type checking bugs, and others.

Here is a list of API Prime could punt:

  • prime.each => _.forIn
  • prime.has => _.has
  • array.count => _.size
  • array.every => _.every
  • array.filter => _.filter
  • array.forEach => _.forEach
  • array.indexOf => _.indexOf
  • array.isArray => _.isArray
  • array.lastIndexOf => _.lastIndexOf
  • array.map => _.map
  • array.some => _.some
  • number.random => _.random
  • number.times => _.times
  • object.each => _.forIn
  • object.filter => _.pick
  • object.keys => _.keys
  • object.map => _.assign
  • object.values => _.values
  • type => _.isArguments, _.isArray, _.isBoolean, _.isDate, _.isFunction, _.isRegExp

"Collections" methods, including _.every and _.some, will also work with strings, array-like objects and Object objects.

There's also support for intuitive chaining as well and has something similar to Prime's implement through _.mixin.

Prime would also get support for

  • _.bind, _.clone, _.cloneDeep, _.defaults, _.extend, _.isFinite, _.isNaN, _.isObject, _.isPlainObject, _.merge, _.partial, _.partialRight, _.reduce, _.reduceRight, and lots of other methods.

You can leverage custom builds to include the methods and exports needed.

jdalton commented Mar 3, 2013

After some discussion with @kamicane, before Prime can move forward, we'll have to make some adjustments. Lo-Dash will maintain a separate npm package or packages called lodash-node which will basically be lodash modularize exports=node, exploding Lo-Dash into modules by method category, keeping it in sync with the main lodash module. That way Prime and others can cherry pick more easily the modules required w/o managing the built step on their end.

another option is to use http://moutjs.com which is already split into modules and contains fixes for many crossbrowser issues as well.

jdalton commented Mar 5, 2013

@millermedeiros Prime is written with CommonJS exports as its base, not AMD, though.

mout is converted into CJS during npm prepublish using nodefy so no drawbacks.

jdalton commented Mar 5, 2013

mout is converted into CJS during npm prepublish using nodefy

That's crazy hot!

so no drawbacks.

Lo-Dash provides environment fixes, old and new, to ensure consistency across environments. Fixes that Mout doesn't cover. I dig that recent Mout activity was inspired by Lo-Dash, but Mout isn't at the same level yet. There needs to be more than "it's pre-sliced".

@jdalton both libs have strengths and weakness. mout have many features that aren't present on Lo-Dash (will probably only increase) and implements a saner behavior on some methods (like isNaN). We are implementing most features and fixing problems as needed, that's why we did not add support for things like cyclic references - no users stumbled into a problem related to that so far, so we are keeping it simple, that will give us flexibility to move fast without breaking backwards compatibility and might also give us a big performance advantage in the future. Code on master is already considerably faster than v0.4 but performance is surely not our focus right now since most methods are already fast enough - consistency and cohesion have higher priority at the moment. Both projects will evolve over time, specially if they have more users/contributors, the important thing is to define what are the priorities and which lib will help better to reach that goal, my intention was to show other options so they could analyse and decide. Cheers.

jdalton commented Mar 5, 2013

both libs have strengths and weakness. mout have many features that aren't present on Lo-Dash (will probably only increase)

I don't think the gap can be chalked up to "both have strengths and weakness". Lo-Dash is tested in lots of cli and browsers before each release, has some of the most perf conscious devs out there as core contributors, offers cross-browser/environment consistency, and is extremely customizable.

and implements a saner behavior on some methods (like isNaN).

As a side note, is Mout's isNaN method basically a carbon copy of Lo-Dash? It looks similar down to the code comments and the new Number(NaN) guard which I introduced to Mout/Underscore/Lo-Dash.

We are implementing most features and fixing problems as needed, that's why we did not add support for things like cyclic references - no users stumbled into a problem related to that so far, so we are keeping it simple, that will give us flexibility to move fast without breaking backwards compatibility and might also give us a big performance advantage in the future.

Mout's lack of robustness is not a feature. Lo-Dash's features, like supporting circular references, are implemented because devs have run into these issues IRL, not just with Lo-Dash, but with Underscore, and other popular libs. Also, because of Lo-Dash's close relationship with Underscore it has the benefit of being hammered on by Lo-Dash and Underscore users (more eyes, more fixes).

Both projects will evolve over time, specially if they have more users/contributors, the important thing is to define what are the priorities and which lib will help better to reach that goal, my intention was to show other options so they could analyse and decide.

At its core I don't think Mout offers anything over Lo-Dash expect that is pre-sliced into what you think the "right" consumption is. Lo-Dash isn’t prescriptive about how devs build and consume it.

I won't go over each attack here since bashing Lo-Dash was never my intention. Just note that isNaN is not a carbon copy of Lo-Dash:

_.isNaN([]); // false
_.isNaN(''); // false
// mout really checks if value "is NOT a number" without coercing the value
mout.isNaN([]); // true
mout.isNaN(''); // true

jdalton commented Mar 5, 2013

I won't go over each attack here since bashing Lo-Dash was never my intention.

I didn't mean for anything to come off as an attack, I just didn't want to gloss over the differences.

Just note that isNaN is not a carbon copy of Lo-Dash:

I'm sorry I missed the subtle, yet differentiating, implementation detail :)
The merits of isNaN checks have been debated at length and are really a side track to this issue.

@kamicane kamicane closed this Jul 5, 2013

jdalton commented Sep 14, 2013

@kamicane Sorry for the long delay. Just a heads up, Lo-Dash 2.0 now has individual npm packages and lodash-node to allow you to offload to lodash modern or compat modules. See Kit's post for more info.

Owner

kamicane commented Sep 14, 2013

A package for each method is extremely overkill. A module for each method would have worked fine for every situation. Having to add every single method I want to use in package.json is definitely not something I'm looking forward to.

jdalton commented Sep 14, 2013

A package for each method is extremely overkill. A module for each method would have worked fine for every situation.

Then you'd want lodash-node. It's a single package with modules for each method :D

Having to add every single method I want to use in package.json is definitely not something I'm looking forward to.

Also, just so ya know, using the --save option as part of npm install command like npm i --save lodash.clone will automatically write it to your package.json and save you some typing.

Owner

kamicane commented Sep 15, 2013

Sorry,

This is simply bad practice, clutters npm (even more than it already is) and is useful to nobody.
Not sure why you did this one-package-per-method thing, lodash-node is clearly the (only) way to go for npm.
Seriously, stop doing this.

jdalton commented Sep 15, 2013

Sorry,

This is simply bad practice, clutters npm (even more than it already is) and is useful to nobody.
Not sure why you did this one-package-per-method thing, lodash-node is clearly the (only) way to go for npm.
Seriously, stop doing this.

Naw, turns out many devs dig it.

One of the big themes of Lo-Dash is that we aren't prescriptive in how devs ingest it. There was a demand for AMD, Node, & npm packages so we delivered ;D

Owner

kamicane commented Sep 15, 2013

Sorry to insist, but people who dig this clearly have no clear idea on how npm works.

jdalton commented Sep 15, 2013

I think @mikeal might be a better person to explain why individual npm packages are a good thing.

I'm super excited to hear why this is a good thing so that I can tell everyone to do it.

mikeal commented Sep 18, 2013

"cluttering npm" is not a real thing, it's an imaginary thing that you can choose to stop imagining.

most npm packages are very small and expose, in many cases, a single function. several authors won't take large modules as a dependency.

also, if you're using browserify small modules are something of a necessity. atm, since i haven't had the time to move over to lodash's newly published small functions, lodash is about 30% of my entire frontend js package. resolving this by doing custom builds of lodash is insane since browserify already provdies a dynamic build system that would work just fine if i could require only the functions I need, which I can now do.

i certainly know how npm works. @isaacs, who was next to me when i read that @jdalton had done this, certainly knows how npm works and was positive about the development.

Contributor

kentaromiura commented Sep 18, 2013

I think kamicane mean that if, for example, you search for lodash https://npmjs.org/search?q=lodash
you get lot of results, in which there are no really simple way to understand what a single module does, also it seems you also promote internals to npm.

I don't see any advantage to do that over using modules instead.
I'm not really an expert of browserify but it seems like it does the same thing of our wrapup,
in that case wrapup/browserify already takes care of using just the code you need without importing the entire library by statically looking at your requires.
So if you need just a little part of a library you just include that, eg: require('prime/array/filter') and in the final build you get only the code you need (in this case this one https://github.com/mootools/prime/blob/master/array/filter.js) plus a bit of code for the general wrapper (https://github.com/mootools/wrapup/blob/master/includes/browser-wrapper.js) and not the entire prime library for example.

Owner

kamicane commented Sep 18, 2013

Following your logic, I should be able to install, say, lodash.forin and lodash.foreach and have the minimum amount of JavaScript possible?

npm install lodash.foreach lodash.forin
npm ls
├─┬ lodash.foreach@2.0.0
│ ├─┬ lodash._basecreatecallback@2.0.0
│ │ ├─┬ lodash._setbinddata@2.0.0
│ │ │ ├─┬ lodash._getobject@2.0.0
│ │ │ │ └── lodash._objectpool@2.0.0
│ │ │ ├─┬ lodash._releaseobject@2.0.0
│ │ │ │ ├── lodash._maxpoolsize@2.0.0
│ │ │ │ └── lodash._objectpool@2.0.0
│ │ │ └── lodash._renative@2.0.0
│ │ ├─┬ lodash.bind@2.0.0
│ │ │ ├─┬ lodash._createbound@2.0.0
│ │ │ │ ├─┬ lodash._createobject@2.0.0
│ │ │ │ │ └── lodash._noop@2.0.0
│ │ │ │ ├── lodash.isfunction@2.0.0
│ │ │ │ └─┬ lodash.isobject@2.0.0
│ │ │ │   └── lodash._objecttypes@2.0.0
│ │ │ └── lodash._renative@2.0.0
│ │ ├── lodash.identity@2.0.0
│ │ └─┬ lodash.support@2.0.0
│ │   └── lodash._renative@2.0.0
│ └─┬ lodash.forown@2.0.0
│   ├── lodash._objecttypes@2.0.0
│   └─┬ lodash.keys@2.0.0
│     ├── lodash._renative@2.0.0
│     ├── lodash._shimkeys@2.0.0
│     └── lodash.isobject@2.0.0
└─┬ lodash.forin@2.0.0
  ├─┬ lodash._basecreatecallback@2.0.0
  │ ├─┬ lodash._setbinddata@2.0.0
  │ │ ├─┬ lodash._getobject@2.0.0
  │ │ │ └── lodash._objectpool@2.0.0
  │ │ ├─┬ lodash._releaseobject@2.0.0
  │ │ │ ├── lodash._maxpoolsize@2.0.0
  │ │ │ └── lodash._objectpool@2.0.0
  │ │ └── lodash._renative@2.0.0
  │ ├─┬ lodash.bind@2.0.0
  │ │ ├─┬ lodash._createbound@2.0.0
  │ │ │ ├─┬ lodash._createobject@2.0.0
  │ │ │ │ └── lodash._noop@2.0.0
  │ │ │ ├── lodash.isfunction@2.0.0
  │ │ │ └── lodash.isobject@2.0.0
  │ │ └── lodash._renative@2.0.0
  │ ├── lodash.identity@2.0.0
  │ └─┬ lodash.support@2.0.0
  │   └── lodash._renative@2.0.0
  └── lodash._objecttypes@2.0.0

As you can clearly see, there are multiple duplicates of every dependency being used here, and following node.js logic each package will use its own copy if available. building this with commonjs build tools will produce a file full of duplicate code.

Unless, of course, you manually depend on the lodash private packages. Which I hope is something you're not telling your users to do.

Why a single package is a good idea in this case: http://blog.millermedeiros.com/mout-and-modularity/

But I guess this sums it up:

$ npm install mout
npm http GET https://registry.npmjs.org/mout
npm http 200 https://registry.npmjs.org/mout
npm http GET https://registry.npmjs.org/mout/-/mout-0.7.0.tgz
npm http 200 https://registry.npmjs.org/mout/-/mout-0.7.0.tgz
mout@0.7.0 node_modules/mout
$ npm ls
tmp@0.0.0 /Users/millermedeiros/Projects/tmp
└── mout@0.7.0

inside your app:

var camelCase = require('mout/string/camelCase');
console.log( camelCase('mout is awesum') );
// > "moutIsAwesum"

touché! 😉

phated commented Sep 18, 2013

@millermedeiros that is the same thing as npm install lodash-node

phated commented Sep 18, 2013

@kamicane npm dedupe

As you can clearly see, there are multiple duplicates of every dependency being used here, and following node.js logic each package will use its own copy if available. building this with commonjs build tools will produce a file full of duplicate code.

Running npm dedupe after you install your lodash deps will fix that for you.

jdalton commented Sep 19, 2013

Whoa npm ddp is awesome :O

$ npm init
$ npm i --save lodash.foreach lodash.forin
$ npm ddp
$ npm ls
├─┬ lodash._basecreatecallback@2.0.0
│ ├─┬ lodash._setbinddata@2.0.0
│ │ ├── lodash._getobject@2.0.0
│ │ ├── lodash._objectpool@2.0.0
│ │ └─┬ lodash._releaseobject@2.0.0
│ │   └── lodash._maxpoolsize@2.0.0
│ ├─┬ lodash.bind@2.0.0
│ │ └─┬ lodash._createbound@2.0.0
│ │   ├─┬ lodash._createobject@2.0.0
│ │   │ └── lodash._noop@2.0.0
│ │   └── lodash.isfunction@2.0.0
│ ├── lodash.identity@2.0.0
│ └── lodash.support@2.0.0
├── lodash._objecttypes@2.0.0
├── lodash._renative@2.0.0
├─┬ lodash.foreach@2.0.0
│ └─┬ lodash.forown@2.0.0
│   └─┬ lodash.keys@2.0.0
│     └── lodash._shimkeys@2.0.0
├── lodash.forin@2.0.0
└── lodash.isobject@2.0.0

I did do a double take when I read

kamicane: Sorry to insist, but people who dig this clearly have no clear idea on how npm works.

because I knew the devs requesting individual packages were well connected to npm and the Node.js community.
It's pretty obvious that kitchen sink style module packages are not the common case.

Also, as @millermedeiros points out, it obscures the utility of npm ls so you can no longer see your dependencies in such a pretty manner ;)

To @kentaromiura, devs were already breaking Lo-Dash into individual modules on their own, this way we can ensure quality and support. Finding lodash or its packages isn't a problem and can be aided through the use of shared keywords.

It looks like a lot of this comes down to not knowing about some of the cooler things in npm, like --save and ddp.

That said, I think there's a place for kitchen sink style module packages. The individual npm packages for Lo-Dash are of the modern build so if a lib was wanting optional support for legacy enviros they'd use something like lodash-node:

var forOwn = require('lodash-node/compat/objects/forOwn');

Again, a big theme in Lo-Dash is that it's not prescriptive in how devs ingest its utilities, making high quality utilities available to as many devs as possible.

Owner

kamicane commented Sep 19, 2013

The only way to get that output with npm dedupe is to manually install:

lodash.isobject
lodash._renative
lodash._basecreatecallback
lodash._objecttypes

other than simply what you wanted originally which is:

lodash.foreach
lodash.forin

jdalton commented Sep 19, 2013

The only way to get that output with npm dedupe is to manually install

Not sure what you mean because that's not right either.
You should be all set using features, like ddp, provided by npm.
Modules still load properly and duplicates are removed for build optimizers.

mikeal commented Sep 25, 2013

ddp short for dedupe moves each common package up a level provided that all the version requirements of that package are met. you may still have "duplicates" when two packages require differing version of the same package, but in that case they aren't truly duplicates.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment