You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
So when unarchiving tar, it stores files first to cache dir. It's treaded as immutable.
Then it's "cloned" to real node_modules dir to project via "reflinks", which works on COW file systems (including APFS)
TLDR idea is it's creating a new reference to existing blocks instead of doing data write.
I've run some experimentes in macos, m1 mac.
5 gb node_modules dir cp -r took
5 minutes
go app which do clonefile syscall took
17s
[Error: ENOSYS: function not implemented, copyfile '
does not improve first unarchiving, only incremental (cache exists in fs). But it improves duplicates (if hoisting not solved it).
Describe alternatives you've considered
Fuse.
Linux support is great, macos making fskit public this in 15.4.
This is alternative track, which dramatically increases speed and improves disk usage.
The text was updated successfully, but these errors were encountered:
Tagging @arcanis because this is design decision. I can write poc linker and run benchmarks.
It could be external plugin, but it will share alot of logic with existing nm linker, which should be abstracted and exported in this case.
Native helper could be used accelerating other things. E.g downloads and tar.gz->zip transforms.
Thanks.
Describe the user story
Slow node_modules linker.
Describe the solution you'd like
So I've found orogene package mananger, which uses interesting technic.
https://github.com/orogene/orogene/blob/2dc8d9e9d32b9dcc8e8a33e8a729c2c08772c33f/crates/nassun/src/tarball.rs#L443
So when unarchiving tar, it stores files first to cache dir. It's treaded as immutable.
Then it's "cloned" to real node_modules dir to project via "reflinks", which works on COW file systems (including APFS)
TLDR idea is it's creating a new reference to existing blocks instead of doing data write.
I've run some experimentes in macos, m1 mac.
5 gb node_modules dir
cp -r
took5 minutes
go app which do
clonefile
syscall took17s
code:
So, benefits compared to existing NM linker:
Describe the drawbacks of your solution
Node exposes copyFile syscall, but it does not work for dirs
[Error: ENOSYS: function not implemented, copyfile '
Describe alternatives you've considered
Fuse.
Linux support is great, macos making fskit public this in 15.4.
This is alternative track, which dramatically increases speed and improves disk usage.
The text was updated successfully, but these errors were encountered: