tremendous dependency size

[mechanoid]

After recommended tap over tape to a colleague, he pointed out the big difference in file size due to dependencies.

As a comparison node-tape with dependencies installed weighs in at about 2.4MB disk usage (which i consider too much for a minimal test framework), but looking at node-tap we end in 54MB disk usage.

After digging deeper it looks like the main disk space is used for 2 things

1.) polyfilling (bluebird) with about 700kb
2.) code-coverage
** coveralls with about 6MB
** and nyc around the 45MB i guess

For 1.) i consider it a bad practice to include polyfils by default so in my opinion it should be a peerDependency, which people may drag in if it is necessary.

For 2.) i personally think that the mix of a testing utility and line-coverage is merely useless, especially when it is just a nice wrapper of starting the test util with another Cli-Interface.
So in my opinion it would be nice to make those dependencies also linkable via peerDependencies,
so that they can be implemented if given and still remove their mandatory nature.

Overall i think tap is the much better test framework, but i can understand people moving back to tape (or other solutions) because of that.

Thanks in advance, Falk

[Tokimon]

yes totally agree.
I love all the great features node-tap provides, but personally I hold back because of the different dependencies that could be avoided.

[ORESoftware]

Yeah I agree, I really don't know why it's not more common to store certain big dependencies (like babel, coveralls, istanbul/nyc, webpack) in a single global location. You'd have to take into account the version of the package and point to that version somehow, but IMO someone needs to work on this problem. Perhaps Yarn, the new package manager will think about doing something like this.

[mechanoid]

Hmm i don't think that this gets the point here. I like the isolation between the application folders, i just don't know, why a library like istanbul with an independent command line interface (nyc) has to be included in a "minimal" testing library.

[ORESoftware]

@mechanoid agree and disagree. a minimal testing library like this one probably has less business integrating with nyc/istanbul. But more powerful testing libraries like AVA, Mocha and my own (Suman) provide users with some real benefit by integrating with NYC/Istanbul/Babel etc. However, including NYC/Babel/Istanbul as dependencies makes the install HUGE. So we are stuck between a rock and hard place. You did not address my proposal - why doesn't NPM allow us to store all packages in one central location on any machine? Like Maven. I have never heard a good reason. All packages and all versions of packages can just go in one place. That will save disk space. I don't get it!

[Tokimon]

@mechanoid I see your point and there is definitely something to be done in that matter, but I guess it is put of scope for this testing tool ;)
For the sake of this matter I also believe that the coverage part should be optional to include and not forced upon you, even if I always (and encourage other to as well) include coverage reporting with my testing. I my case I am not necesarily using the CLI interface of TAP to load the files and then the automated coverage report is not working, thus making it a redundant inclusion and I have to do it manually anyway.

[mechanoid]

@ORESoftware at first i see a difference in libraries like tap/tape and ava, mocha, etc. in that the are absolutely not minimal. And a reason why I prefer libraries like tape is that they follow the unix philosophy of doing one thing well rather than doing many not that good.
To the point on different dependency stores I think that it has for sure it good parts (saves lot of disk space for sure), but also its downsides. A thing that happened to me very often in e.g. ruby is, that you have to dig into your lockfile (or whatever your language hands to you) to have a clear state of what is installed. After found out, which version your app references you have to turn over to your central store and lookup that code when you want to check what a certain version does to hunt bugs down etc. An approach of storing app dependencies in the app folder means, that i have all dependant code in one place and nothing else, so if i want to check what a certain library does, i can head to it and start to manipulate it etc. without effecting other projects.

[mechanoid]

@Tokimon i personally do not really believe in loc coverage, but yes i think if it would be optional it would make much more sense, but digging deeper to the tap code base I don't believe that it is that easy to remove that necessity.

[Tokimon]

@mechanoid I haven't looked deep into the code, but it seems strange that it is closely tied to the coverage report :-/

[isaacs]

This is an interesting question, and hits at the fundamental vision and goals of this tool.

Tap is not a "minimal test framework". I don't know where you would have gotten that idea, frankly! There's is literally nothing "minimal" about node-tap. It has pretty much every feature one might want. It's absolutely a "batteries-included" kind of thing, by explicit design.

The goal of this library is that you do tap test/*.js in your package.json file, and everything Just Works. If you want coverage, you can throw a --cov on there, and get reasonable output. If it's running in an environment with a coveralls repo token, it'll automatically ship the coverage data over there. It detects if it's running in a child process, or a tty, and presents output that's optimized for each situation. Like promises more than callbacks? Tap both consumes promises, and test functions also return a promise. It has aliases for all the assertion functions you could possibly imagine (are you imagining a new one? let's talk about it, I'll probably accept a PR!)

The goal is that it works in every version of node from 0.10 on up (because even though it's not "supported", many people use it in production, including friends and employees of mine). When it's safe to drop 0.10 support (which will be a breaking change), it'll probably be worth dropping Bluebird. But since Promise support isn't native in 0.10, it uses a polyfill. (And, at least in 0.12 and 4.x where I tested it, Bluebird is also faster than native promises, though that's a small factor since tap is typically I/O bound, and the speed of the Promise implementation is almost never relevant.)

Looking forward, though my time to spend on OSS has dwindled significantly in light of work and family concerns, the plan is to add support for buffered test output, and eventually use that to also enable asynchronous/parallel tests. All the features!

As if to make the point even more obvious, try running tap test/*.js -Rnyan. There is a nyan cat reporter; your argument is invalid. :D

So, since I do create many "minimal" libraries for various things, and have at many times advocated very publicly for building small modules and small things, why have I not designed tap to be minimal? Why is it, instead, more accurately described as a maximal test framework?

This comes down to trade-offs, and the role that developer tools like tap play in the workflow experience. (Consider also, one would be hard pressed to call npm a "minimal" package manager!) Dev tools should make the development experience easy and enjoyable.

For things that are shipped with an application into production, like the actual web framework that powers your application, or a reusable library, there's a big benefit to be had for being very minimal. It makes debugging easier (and debugging production is fundamentally much harder than anything running locally!) But for development time, I like to fly first class. I want the bells and whistles, I want it to Just Work. I code on a Macbook Pro for pretty much that exact reason. (Though, that's a whole other conversation, as it's yet to be seen if Apple will keep catering to my use case in the future.)

Modularity is much more subtle than merely "smallness". I recommend reading On the Criteria To Be Used in Decomposing Systems into Modules. Each "module" should encapsulate one decision, such that it's straightforward to replace. Some decisions are indeed "minimal", and some are not. Module implementations should be the minimum size required to take a specific stance on a given decision, but no smaller.

In tap's case, it encapsulates a bunch of opinions regarding the decision of "how to run software tests". The resulting implementation also has, within itself, many other decisions that are themselves individually abstracted out into smaller (or, in nyc's case as mentioned here, not-so-smaller) modules of functionality.

The fact that it's so easy to switch from tap to tape (or to ava, or mocha, or whatever else), some of which are certainly a lot more "minimal", indicates to me that this is an appropriate module boundary, and that minimal-ness is just one of several possible options for how one might decide to make this particular decision. Tape is the easiest to switch to, and has the fewest features of any of them. If that is your preference, use tape with my blessing!

In short, I'm definitely open to replacing Bluebird with native promises once 0.10 is no longer supported, and it's entirely possible that there are some ways to make nyc a bit leaner. But I'm not likely to ever remove nyc as a dependency, or not have built-in coverage reporting in this library.

[ORESoftware]

@isaacs right, but why doesn't NPM just use the NPM cache to source modules instead of re-writing modules locally in every single project directory :) I 100% agree with you that it makes sense to include NYC with Node-Tap as a dependency, but the real concern is size on disk for many people. After 3 years working with Node.js, I still do not understand why NPM doesn't just put all versions of modules into one directory. I am missing something....maybe it has to do with locks on folders, or perhaps to avoid global file corruption (making problems only local to one project), etc. Is there anything you can point me to that serves as a definitive answer to why NPM uses local installation and doesn't point to ~/.npm or the like? People surely have the same versions of Babel and NYC/Istanbul and Express installed on their filesystems in 10 different places, I just don't see the point, ATM. Willing to be enlightened.

[ORESoftware]

In other words this thread isn't really about NYC being bundled with Node-Tap, it's convenient for both the library authors and users, to do so. What this thread is about is having to write NYC to the filesystem in the local project. I ultimate do not understand the need to do that.

[isaacs]

@ORESoftware Check out ied and pnpm. That's a fine discussion to have, but this is not the place for it :)

[ORESoftware]

sure thing, are ied and pnpm repos or discussion channels? never heard of either of 'em, will google