Load NODE_OPTIONS from a file

[mistic]

Is your feature request related to a problem? Please describe.
Currently there are useful node options(https://nodejs.org/api/cli.html#cli_node_options_options) I would like to be able to easily specify project wide like --max-old-space-size and --preserve-symlinks but there is not really a way to do it through the repository only through NODE_OPTIONS or by specifying it by hand every time an npm script is defined which is not convenient for large sized monorepos.

Describe the solution you'd like
I would like to be able to specify those options in a file that, if present, node will try to read the options from it. The name could be for example .node_options. Each option will be specified per line and will override every other option specified through NODE_OPTIONS env var or directly passed when invoking the node binary.

Describe alternatives you've considered
Right now that is non easy way to accomplish the use case that I have.

[mistic]

/cc @Trott @benjamingr @juanarbol in order to get the correct labels into this issue

[juanarbol]

Hi @mistic, thank you so much for opening the issue, this is a very interesting request (personally, it totally makes sense).

How about trying something more script-based (if you will), just before consider adding that kind of functionality to the core itself, something like this:

Creating that mentioned: .node_options:

// ./node_options.txt
--preserve-symlinks --experimental-vm-modules

And running node like this:

NODE_OPTIONS=$(cat ./node_options.txt) node

[mistic]

@juanarbol thank you for looking into it! Some months ago I dedicated some time to think about that problem. Wether there are some escape routes (like the one you mentioned) I don't think there is one able to support a considerable sized monorepo who needs a cross platform solution. Even if the above solution worked on Windows it will still require everyone to run node through the npm scripts (which probably will need to replicate this all over) or will require to ask each developer to setup the NODE_OPTIONS env var (which is not a good option for repos with a large contributors base).

The best solution I could get into is implemented at elastic/kibana#69628 but even that one has some cons like the need to run a setup phase for the new alternative node global bin script at the yarn install and then run node through a child process launched by a parent node script. However, now that we are considering to move our build system into Bazel that complex escape route will also be a problem so we cannot use it.

I really believe having such a feature into the Node.js cores makes a lot of sense to unblock all the use cases that can be later applied once we have that, not only on development environments but also on production ones (example: applications who bundle and ship a given Node.js version could easily setup node options to be applied for that application, everything easily done through code and cross platform proof).

[juanarbol]

I used to face a very similar issue, no matter what your approach is, you will need to set the NODE_OPTIONS env variable manually. What do you think of creating a .env file in the root of the monorepo, and use something like dotenv.

// setupEnv.js
function setupEnv () {
  require('dotenv').config()
}
module.exports = setupEnv

And append this into your main file,

// main.js
const setupEnv = require('./setupEnv.js')
setupEnv()
...

And your .env file could look something like this:

NODE_OPTIONS=....

[mistic]

@juanarbol I can be wrong but I think that won't work right? I believe dontenv will only set vars on process.env at run time and options like --max-old-space-size will not work like that as they need to be set before node even starts. Also even if dotenv was trying to do anything else differently it won't work because we cannot set env variables on child processes and populate them into the parent one. I believe that feature is something that can only be addressed at the node core itself.

[juanarbol]

What about this?

https://www.npmjs.com/package/dotenv#preload

[mistic]

@juanarbol I think that won't also work across all options. While it might work for preserve-symlinks as it happens before the resolution mechanism took start, I believe for others like setup the max memory it won't as node as the need to know about those configurations before passing that down into the low level libraries to spawn the processes.

Even if that works I just feels it will be as problematic and work intensive as defining NODE_OPTIONS or pass the flags on each npm script. I feel like we need a way to declare the node configuration in a file on the workspace root of the repo so I can define widely available options for that repo and then I can just run node <script> through the bash without worrying about specify or not such options. I feel like that option makes a lot of sense and that is like something we should have available for Node.js 😃

[juanarbol]

Yeap, I'm out of recommendations, I would like to read more opinions of the community about this.

BTW, thanks again for the issue :)

[mistic]

Thanks for all the recommendations here @juanarbol . On my past explorations about that matter I went probably with all of them and thats why I realise we need such a thing on Node.js, as none of the available options is fully compliant with the requirements ! Hope others find it as much useful as I do and that we can move forward with an implementation for it!

[jasnell]

Sorry for the delay on responding to this. I'm really not convinced that it's worth the additional complexity to add this to core. Yes, I get that it's a bit of a headache for monorepo type projects but I would look to the tooling that is optimized to support those for improvements before bringing this in to core.

[mistic]

@jasnell thanks for your feedback. We are moving our dev/build tooling into bazel in a big monorepo and that problem was one of the first we spot as Bazel builds and puts the results of the build in folders outside of the repo that are later symlinked into it. The most convenient way we could found was to spawn a child process ensuring --preserve-symlinks for every entry point https://github.com/elastic/kibana/blob/master/src/setup_node_env/ensure_node_preserve_symlinks.js#L109 which is not ideal in my point of view. Preserve symlinks would be a better default (in my opinion) for nodejs, but as this was already tried in the past and reverted, I really think we will be better served with a way we can enforce certain configurations, that are only able to be specified when starting up, repository wide.

[dudeful]

Any updates on this?

I'm running into the same issue here. I have a 35Gb virtual machine running on Azure and I'm currently using it only to perform memory-intensive development tests. Having multiple terminals opened, each running a script that requires more than the default 4144Mb, at the same time makes the "export NODE_OPTIONS" approach very inconvenient.

I would appreciate if someone could offer some tips regarding how to make this situation more convenient.

[benjamingr]

@dudeful add the export to your shell init file so that it gets set whenever you open a terminal? (.zshrc / .bashrc)