Skip to content

Move path transformation logic into function similiar to gulp.src #11

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Silic0nS0ldier opened this issue Nov 18, 2018 · 4 comments
Closed
Assignees
Labels
enhancement Issue enhances existing functionality or development processes.
Milestone

Comments

@Silic0nS0ldier
Copy link
Member

The path transformation logic is great for reducing IO, but comes at the expense of increased RAM usage since it introduces a clog in the stream. We can resolve this by creating our own version of gulp.src that performs this work during the read phase, which will allow us to perform processing of bundles in a more stream like fashion, namely;

  • Start processing a bundle the second we have its resources, and emit its result during the same _transform invocation that ultimately triggered it.
  • Push all files straight on through with required files cloned (we currently can't due to overwrite logic) so that later processing can run.

Final result should be less used RAM and less read operations.

@Silic0nS0ldier Silic0nS0ldier added this to the major milestone Nov 18, 2018
@Silic0nS0ldier Silic0nS0ldier added the enhancement Issue enhances existing functionality or development processes. label Nov 18, 2018
@Silic0nS0ldier
Copy link
Member Author

Figured out how gulp determines the base path (which directly impacts where files get delivered). Looking much more reasonable getting this implemented however I think it would be better to release it as its own package. A vinyl source with file overriding logic has uses in more than just this library. It may even already exist (which would be awesome if its stable and supported).

@Silic0nS0ldier
Copy link
Member Author

Like so: https://www.npmjs.com/package/vinyl-fs-merge
Will need to be tested, but this would allow substantial performance improvements.

@Silic0nS0ldier
Copy link
Member Author

To simplify testability, the implementation should be split like;

  • File resolution (overrides, etc), returns a map of virtual paths to actual paths
  • Vinyl source
  • Bundle substreams (similar to current system, minus file resolution and waiting for all files to start bundling)

To keep streams composable and permit custom transformations, the source and bundler should remain independent logical steps.

Given this would be a breaking change, the release this would come out in should include #22

@Silic0nS0ldier
Copy link
Member Author

This has been started with https://github.com/userfrosting/vinyl-fs-vpath

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Issue enhances existing functionality or development processes.
Projects
None yet
Development

No branches or pull requests

1 participant