gulp.src and gulp.dest logic should be a separate package

[millermedeiros]

I believe this functionality deserves it's own package, the same way as you guys are reusing orchestrator, event-stream, glob-stream, etc.

Even tho this feature is the core of gulp, it could be used outside the task runner context. It could even become the de-facto standard for file manipulation. (so you would have file stream modifiers coded by more people).

I agree that there are benefits of bundling all the features into the same package (so you can simply do npm install --save-dev gulp) but for me it feels like this tool is doing too much (too many responsibilities).

Cheers.

[millermedeiros]

I'm thinking that gulp.src could be maybe 4 modules:

1. one that converts the globs into a Stream of files that only contains the paths (no content or stats).
2. a filereader-stream that adds the content to the paths info.
3. a filestats-stream that adds the stats to the paths info.
4. one that encapsulates all of them for brevity.

and gulp.dest could be just a filewriter-stream that simply does mkdirp and fs.createWriteStream based on the file.path and file.content.

I've been looking for a good API to read files from multiple directories, and process them before copying to a destination for a long time. This might be the one, since it allows piping multiple transforms in a easy way.

[yocontra]

Rambling a bit here...

@millermedeiros I can merge the file format into the output of glob-stream (the path info, not contents). The filestats streams takes this and attaches the stats object. The filereader stream takes this and based on X args (buffer: true, read: false, etc.) and the file.stats creates file.contents. The fileread needs filestats to be executed on the file object otherwise it won't know how to read directories correctly.

Is this what you're suggesting? Thoughts?

[millermedeiros]

It was just an idea, did not put too much thought into that. In fact not
even sure what would be the best format and if the "plugins" should expose
just an async method...

[phated]

Gulp isn't too big, and the file streams could be used without the task management. I see no reason for the decouple.

[millermedeiros]

@phated the problem is not the "size" of gulp, but that at the moment this tool have 4 different responsibilities which are not necessarily closely related:

1. it's a task runner (delegates work to orchestrator)
2. it's a file watcher (delegates work to gaze)
3. it's an argument parser (delegates work to optimist)
4. it's a file processor based on Streams (currently implemented internally)

since responsibilities 1, 2 and 3 are handled by external tools, I think this should also be part of an external package and gulp would just glue the external tools together and provide an unified API (maybe not even necessary). - I don't need that many unrelated dependencies (arguments parser, file watcher and task runner) just to process files.

this is not a problem of file size, but just that smaller tools with single responsibilities are easier to grok and can evolve independently (size of the README would reduce considerably). I would really like to see a future where we can reuse modules without having to code yet another plugin for a different runner... and Streams are a good abstraction for some file operations because they can be piped and doesn't require whole file to be in memory (even tho most examples I saw doesn't benefit from that).

[phated]

Instead, you are advocating for yet another file streaming style. This
style is unique to gulp and doesn't really matter standalone. Gulp doesn't
require plugins, just JS, which is where most of your confusion seems to be
coming from. The reason to make a plugin is to tell people it expects that
style of file stream.
On Nov 30, 2013 11:40 AM, "Miller Medeiros" notifications@github.com
wrote:

@phated https://github.com/phated the problem is not the "size" of
gulp, but that at the moment this tool have 4 different responsibilities
which are not necessarily closely related:

1. it's a task runner (delegates work to orchestrator)
2. it's a file watcher (delegates work to gaze)
3. it's an argument parser (delegates work to optimist)
4. it's a file processor based on Streams (currently implemented
   internally)

since responsibilities 1, 2 and 3 are handled by external tools, I think
this should also be part of an external package and gulp would just
glue the external tools together and provide an unified API (maybe not
even necessary). - I don't need that many unrelated dependencies (arguments
parser, file watcher and task runner) just to process files.

this is not a problem of file size, but just that smaller tools with
single responsibilities are easier to grok and can evolve independently
(size of the README would reduce considerably). I would really like to see
a future where we can reuse modules without having to code yet another
plugin for a different runner... and Streams are a good abstraction for
some file operations because they can be piped and doesn't require
whole file to be in memory (even tho most examples I saw doesn't benefit
from that).

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/28#issuecomment-29558195
.

[millermedeiros]

@phated I'm advocating a standalone file stream standard (same as what you guys currently have), in the hopes of it becoming widely used in different contexts, and that users understands the benefits of Streams over regular callbacks for some tasks... I'll elaborate better about plugins on a separate issue. - xkcd on standards

[yocontra]

Okay so I broke the file object out into https://github.com/wearefractal/vinyl and I'm breaking the read/write for the FS into vinyl out into https://github.com/wearefractal/vinyl-fs

[yyx990803]

+1. vinyl-fs actually allows most gulp plugins to be used anywhere, e.g. inside grunt tasks.

[yocontra]

Closing this - vinyl-fs is coming in the next release