UV_THREADPOOL_SIZE undefined in Windows

[rhernandog]

• Node.js Version: 8.11.2
• OS: Windows 7 64
• Scope (install, code, runtime, meta, other?): startup
• Module (and version) (if relevant): none

Hi,

I'm not completely sure if this should be here or on the libuv repo, so correct me if I'm in the wrong place asking the wrong question.

I was running some benchmarks with apache benchmark on an express server and I tried to modify the number of threads in the thread pool through my index.js file with no success:
process.env.UV_THREADPOOL_SIZE=1;

I ran it through the command line and it worked, then I wanted to check the amount of threads by doing a console.log(process.env.UV_THREADPOOL_SIZE) call. The thing is that I forgot to remove that line of code and when I restarted my laptop I got undefined from the console call.

So in fact when you start a fresh boot of the machine the UV_THREADPOOL_SIZE property is not set in the process.env object. Why is that?, Why the property is not added with it's default size (4 as I understand from the libuv docs) at startup?.

On Start Up

console.log(process.env.UV_THREADPOOL_SIZE); // -> undefined

[gireeshpunathil]

#cat 1429.js
process.env.UV_THREADPOOL_SIZE=1;
console.log(process.env.UV_THREADPOOL_SIZE)
#node 1429.js
1
#

it does not happen to me. Just to be sure that we are on the same page: index.js is the only code you are having, or do you spawn other processes? I guess the settings in the index.js do not propagate to child processes, can you please check?

[rhernandog]

@gireeshpunathil Thanks for the answer.

Yep just one file. Here's an extremely simplified set up to reproduce the issue:

index.js

console.log( process.env.UV_THREADPOOL_SIZE ); // -> undefined

const express = require("express");

// EXPRESS
const app = express();

app.get("/", (req, res) => {
  res.send("Hi didly ho!!!");
});

app.get("/fast", (req, res) => {
  res.send("FAST FAST FAST!!!!");
});

app.listen(3000, null, () => {
  console.log("express working on port 3000!!!");
})

This in a folder with just express 4.16.3 installed. Run node index and this is the result:

As I mentioned this happens on Windows 7 64b (I believe most windows OS as well, haven't tested in 8 or 10 yet).

Also is worth mentioning that if I set the thread pool size via command line it is defined and set and after that I can change it through the index.js file or any other file. So doing this first:

$ SET UV_THREADPOOL_SIZE=1

Then running the index file gives this result:

$ node index
1
express working on port 3000!!!

[gireeshpunathil]

where are you doing:
process.env.UV_THREADPOOL_SIZE=1; in index.js ? I don't see it in your code (previous comment)

setting the value in the command line is the right behavior, and the right way to use it - so that different runs can use different values, if need be, without changing the source.

[rhernandog]

Yep, in that file is not there, it is just the most reduced code sample to reproduce that.

A simple solution is to create an NPM script:

SET UV_THREADPOOL_SIZE=1 && node index

But I was trying to actually avoid that. At the end I was just curious about why process.env.UV_THREADPOOL_SIZE returns undefined in Windows OS' before setting the value via CLI and not on OSX, because in OSX you can set the number of threads through a file. I just want to know if it's by design or not, if it's because how the specific OS works, etc. Just looking for an answer of why 😄

[gireeshpunathil]

#uname
Darwin
#node
> console.log( process.env.UV_THREADPOOL_SIZE );
undefined
undefined
> 

I don't see anything special with macos. If UV_THREADPOOL_SIZE is not defined in the command line (export UV_THREADPOOL_SIZE ... ) or in the source (process.env.UV_THREADPOOL_SIZE ...) it is always going to be undefined.

I fear there is a disconnect between us. Am I missing something from your explanation?

[rhernandog]

No you're getting everything right, It was just I wanted to know why is undefined in one OS and not in another, just that.

[gireeshpunathil]

ok, let us resolve this confusion.

1. Can you run:
   console.log( process.env.UV_THREADPOOL_SIZE ); in both win and mac to see if it differs?

2. Can you run:

process.env.UV_THREADPOOL_SIZE = 1;
console.log( process.env.UV_THREADPOOL_SIZE ); 

in both and see if it dffers?

3. Can you run:

$ export UV_THREADPOOL_SIZE=1
and
console.log( process.env.UV_THREADPOOL_SIZE );

in UNIX and
> set UV_THREADPOOL_SIZE=1
and
console.log( process.env.UV_THREADPOOL_SIZE );

in windows and see if they differ?

In all cases I would expect they don't.

[gireeshpunathil]

ping @rhernandog

[gireeshpunathil]

inactive, closing; pls reopen if the the issue is still relevant

[Polyhistor]

@gireeshpunathil the issue exists on windows. I'm currently running on windows 10 and Node v12. I've logged process.env, it's got no UV_THREADPOOL_SIZE property on it...

The exact same behaviour exists on Ubuntu 10 and Node V10

[lorand-horvath]

Having the exact same problem on Win 7 and Node 12.22.12. By default, process.env.UV_THREADPOOL_SIZE is undefined, unless I set it from CMD with set UV_THREADPOOL_SIZE=1, for example.

The other very strange issue that I must mention is that setting process.env.UV_THREADPOOL_SIZE = 1; in the js file doesn't have any effect on the actual number of threads. It only does if I set it via the command line, BEFORE running node index.js
A similar report can be found here: https://stackoverflow.com/questions/68691702/process-uv-threadpool-size-size-is-not-working-in-windows

[gireeshpunathil]

re-opening, as there are at least 2 affected parties

@nodejs/libuv - is there anything special with UV_THREADPOOL_SIZE on windows? I couldn't find anything either in the code or in docs; yet the reports say that the value is not honoured.

[saghul]

AFAIS Node calls uv_os_setenv here: https://github.com/nodejs/node/blob/5fad0b93667ffc6e4def52996b9529ac99b26319/src/node_env_var.cc#L149

But it does not check for the return value, and that function might fail: https://github.com/libuv/libuv/blob/97dcdb1926f6aca43171e1614338bcef067abd59/src/win/util.c#L1543

[saghul]

Setting it from within might not work because it's only read when initializing the threadpool, and that might have happened already, it needs to be set as an env var before calling node.

[lorand-horvath]

Setting process.env.UV_THREADPOOL_SIZE = 1; works on MacOS but doesn't on Windows. I'm not sure about Linux though, haven't checked. So which behavior is the correct one? Should we be able to set it only from the command-line (outside the node process) or is it supposed to also be configurable inside the process (i.e. to be able to set it from within our code)?

[saghul]

I don't think it was ever intended to be set within, since any change in internal code that could trigger the threadpool being started would yield it useless, as it seems to be the case.

[lorand-horvath]

@saghul Then why does it work in the MacOS version of Node.js? And how? And why can't it be made to work the same way under Windows? We discuss this for some time now, but we don't seem to have a clue about actually answering these questions. Or do we?

[saghul]

Then why does it work in the MacOS version of Node.js?

By chance.

And how?

Let's dive in. The threadpool is created here: https://github.com/libuv/libuv/blob/97dcdb1926f6aca43171e1614338bcef067abd59/src/threadpool.c#L193 That function is part of an initialization that is made only once : https://github.com/libuv/libuv/blob/97dcdb1926f6aca43171e1614338bcef067abd59/src/threadpool.c#L248

Every time work is queued in the thread pool it will be initialized if it hasn't already: https://github.com/libuv/libuv/blob/97dcdb1926f6aca43171e1614338bcef067abd59/src/threadpool.c#L266

Now, there are a number of cases wheen work is submitted to the thread pool: https://github.com/libuv/libuv/search?q=uv__work_submit namely get{addr,name}info, filesystem ops, random byte generation and arbitrary work through uv_queue_work.

The only way to have the option work on user code is if none of the above has happened before user code is reached. I don't think Node can make that guarantee. It might be the case for macOS, as I said, by chance, but that can be broken inadvertently if any operation that triggers the threadpool initialization becomes part of the Node initialization process, that is, what runs before your code.

And why can't it be made to work the same way under Windows?

Because, as I said above, Node can't make the guarantee that it will never need the APIs that would trigger the threadpool initialization as part of the Node process bootstrap.

We discuss this for some time now, but we don't seem to have a clue about actually answering these questions. Or do we?

I'd say it has always been clear, as in, not guaranteed, but the fact that it worked on some platforms might have thrown people off.

Updating the docs would be a good thing here IMHO.

[preveen-stack]

windows 7 is past end of life. Wondering if we should track this

[lorand-horvath]

windows 7 is past end of life. Wondering if we should track this

This also happens in Windows 10, 11 etc. So yes, it needs to be fixed.