Skip to content

Move path transformation logic into function similiar to gulp.src #11

Description

@Silic0nS0ldier

The path transformation logic is great for reducing IO, but comes at the expense of increased RAM usage since it introduces a clog in the stream. We can resolve this by creating our own version of gulp.src that performs this work during the read phase, which will allow us to perform processing of bundles in a more stream like fashion, namely;

  • Start processing a bundle the second we have its resources, and emit its result during the same _transform invocation that ultimately triggered it.
  • Push all files straight on through with required files cloned (we currently can't due to overwrite logic) so that later processing can run.

Final result should be less used RAM and less read operations.

Activity

  1. added this to the major milestone on Nov 18, 2018
  2. added
    enhancementIssue enhances existing functionality or development processes.
    on Nov 18, 2018
  3. Silic0nS0ldier commented on Jan 14, 2019

    @Silic0nS0ldier
    MemberAuthor

    Figured out how gulp determines the base path (which directly impacts where files get delivered). Looking much more reasonable getting this implemented however I think it would be better to release it as its own package. A vinyl source with file overriding logic has uses in more than just this library. It may even already exist (which would be awesome if its stable and supported).

  4. Silic0nS0ldier commented on Jan 14, 2019

    @Silic0nS0ldier
    MemberAuthor

    Like so: https://www.npmjs.com/package/vinyl-fs-merge
    Will need to be tested, but this would allow substantial performance improvements.

  5. Silic0nS0ldier commented on Jul 16, 2019

    @Silic0nS0ldier
    MemberAuthor

    To simplify testability, the implementation should be split like;

    • File resolution (overrides, etc), returns a map of virtual paths to actual paths
    • Vinyl source
    • Bundle substreams (similar to current system, minus file resolution and waiting for all files to start bundling)

    To keep streams composable and permit custom transformations, the source and bundler should remain independent logical steps.

    Given this would be a breaking change, the release this would come out in should include #22

  6. Silic0nS0ldier commented on Jul 19, 2019

    @Silic0nS0ldier
    MemberAuthor
  7. modified the milestones: major, V4 Alpha on Jul 19, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

enhancementIssue enhances existing functionality or development processes.

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions