Proposal: ctx.actions.alias / ctx.actions.derive() #30385
thesayyn
started this conversation in
Design proposals
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Been staring at the copy_to_bin cost in rules_js and I'm increasingly convinced the real problem is Bazel not providing , not bytes — we spin up one action per file just to move things into bin, and it adds up fast. Feels like something a lot of rulesets would hit, not just us.
Rough idea: one action that aliases a files into their bin locations, instead of N separate copies. Something like:
ctx.actions.alias(src_foo, output = bin_out)No bytes copied — on RBE it's already just the source's digest showing up at the bin path, and locally/sandbox it'd hardlink the content in. One node instead of thousands.
Before I go write this up properly: is this something Bazel would be open to since it introduces a ctx.actions.symlink() like api that does not create a filesystem indirection but a Skyframe one?
As for how this would be materialized would be up to the Sandbox implementation, whether hardlink, symlink, or custom one that could be provided by --sandbox_backend
Alternatives we considered before could not solve the problem quite;
Tried symlinks, but that breaks the NodeJS tools in weird ways, because they tend to follow symlinks.
Clonefile/reflink alternatives; this is what we use today, but action is still there and still needs to run.
Also submitted: bazelbuild/proposals#419
All reactions