New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Run time dependencies for modules with bindings are bloated #187
Comments
Basically a Nix package (any Nix package, not just NPM-based packages) could get bloated if the resulting binary still contains store paths to dependencies that are not required at runtime. To reduce the footprint of these packages, these dependencies should be eliminated. Now that I'm thinking about your issue. Perhaps this might be caused by the fact that the strip phase is the standard builder environment is probably unable to strip binaries from NPM packages because they reside in non standard sub paths. I need to do a bit of investigation to see what the implications are and if there is a way how to solve this generically. Probably, I need to finish my |
I would also be interested in a smaller package size. I tried injecting the node-slim package, but the derivation fails down the line because it can't find npm e.g. myNodeApp = import ./myNodeApp/default.nix {
inherit pkgs system;
nodejs = pkgs."nodejs-14_x";
# nodejs = pkgs."nodejs-slim-14_x"; # slim does not include npm, causing the build to fail
}; It looks like the nodejs package also includes python (due to node-gyp), whereas nodejs-slim does not: ❯ nix why-depends nixpkgs.nodejs nixpkgs.python3
/nix/store/y9ay04l5mfm255r296vhcjbxjqkjxp39-nodejs-14.16.1
╚═══lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py: …#!/nix/sto>
=> /nix/store/d44wd6n98f93hjr6q1d1phhh1hw7a17d-python3-3.8.8 ❯ nix why-depends nixpkgs.nodejs-slim nixpkgs.python3
'nixpkgs.nodejs-slim' does not depend on 'nixpkgs.python3' |
@W1M0R : The trick I use is to put nodejs-slim before nodejs everywhere where nodejs is listed as a dependency. npm, node-gyp, .. are still provided during the build time, while the runtime dependency is satisfied by the slim version of the package, hence bloated nodejs is not included in the result then. If you want to see a complete example, here is the commit (cluttered with many other changes, but you get the idea): openebs/mayastor@6b03e10 |
Thanks @jkryl, I see what you did there. That should steer me in the right direction. |
That helped, and reduced my docker image size from 280MB to 200MB. Python is no longer part of my image, however gcc is still there. ❯ nix why-depends nixpkgs.nodejs-slim nixpkgs.gcc
'nixpkgs.nodejs-slim' does not depend on 'nixpkgs.gcc' The gcc dependency is not because of nodejs-slim, so that is probably introduced by something else in my image. I see that nodejs-slim still includes perl via openssl, which adds 50MB to the image. I don't think I'll be able to remove that dependency: ❯ nix why-depends nixpkgs.nodejs-slim nixpkgs.perl
/nix/store/7wpvzz0dzsq938778h4aaqrf219ygh2n-nodejs-slim-14.16.1
╚═══bin/node: … "/nix/store/apxpi7w14qkvbkpiy89lryarxv9s>
=> /nix/store/apxpi7w14qkvbkpiy89lryarxv9svngy-openssl-1.1.1k-dev
╚═══nix-support/propagated-build-inputs: … /nix/store/dzqd6vwynd55fwrsk>
=> /nix/store/dzqd6vwynd55fwrsksilj4p51kwy5xr6-openssl-1.1.1k-bin
╚═══bin/c_rehash: …#!/nix/store/6mnbpv86l5ff2vf765hinq95k12dhj85-pe>
=> /nix/store/6mnbpv86l5ff2vf765hinq95k12dhj85-perl-5.32.1 According to these issues, perl should not be a dependency and has been fixed:
I'll see if a nixpkgs update will resolve this issue for me. |
Could |
Could it be that the main issue here is that None the less, nodejs-slim should probably be the default runtime for packages generated by |
As an example when packaging a nodejs app using node2nix into docker, this is the contents:
About 300 MiB untarred, but 100 MiB as a |
I have a project that uses npm modules with C++ bindings. When using node2nix to create a nix package, it has a dependency on gcc, node-source, and nodejs-X packages. However, those are rather build-time dependencies that are not required at run-time. I was able to work around it by doing a couple of changes:
1: I patched the generated nix files and added nodejs-slim package to buildInputs before nodejs package. When shebang in composition script is run, it finds the nodejs-slim package before nodejs package and replaces
/usr/bin/env node
at the beginning of JS scripts by a path to the slim package.2: in postInstall script I added a command to remove all files from build subdirectory of modules with bindings except the actual shared library with the bindings. makefiles and object files were referencing various include paths thus pulling in nodejs source package and compiler package.
3: in postInstall script I added a command to strip debug info from shared library (bindings) for the same reason as in 2.
I create a docker image out of the node package generated by nix and the amount of space that was saved is impressive. The size decreased from 981 MB to just 271 MB. I'm wondering if I could accomplish the same with less hassle and without having to patch generated nix files.
The text was updated successfully, but these errors were encountered: