NumExpr detected 48 cores but "NUMEXPR_MAX_THREADS" not set, so enforcing safe limit of 8.

[cgirardot]

Dear author,

the -t option seems to be ignored. I am always getting the warning message NumExpr detected 48 cores but "NUMEXPR_MAX_THREADS" not set, so enforcing safe limit of 8.
I tried setting the NUMEXPR_MAX_THREADS but the same warning is issued.

> chess --version
0.3.2

[nickmachnik]

Hi,
sidenote: the threads are set with the -p flag. But I suppose you have done that, otherwise you should have gotten a 'unrecognized argument' error.
What -p does is to set the number of worker threads that run the comparisons of matrix pairs. If you set e.g. -p 6 and watch your top you should see that after the 'launching workers' log six new processes spawn, irrespective of "NUMEXPR_MAX_THREADS".
That being said, I am not sure where the "NUMEXPR_MAX_THREADS" warning is coming from, and I get it too, so I'll investigate.

[cgirardot]

indeed -p
I am sending the jobs to the cluster and set -p 8 and the SLURM command accordingly. Since jobs don t get killed, I assumed either this is a weird warning or no multithreading occurs. I'll try running local but this is a minor aspect anyway.

[sidiropoulos]

Hi, I am experiencing the same issue as @cgirardot on a PBS cluster. No matter what value I set with -p I will always get the same message.
2020-11-02 10:37:31,485 INFO Note: NumExpr detected 40 cores but "NUMEXPR_MAX_THREADS" not set, so enforcing safe limit of 8. 2020-11-02 10:37:31,485 INFO NumExpr defaulting to 8 threads.

I am also inspecting the run in at interactive node and I notice that there is only one chess process running.

The same message gets printed when I run chess extract. Maybe that could be a hint to the issue?

Thanks,
Nikos

[nickmachnik]

Hi @sidiropoulos, thank you for reporting this. NumExpr is used in PyTables, which is again used in FAN-C (see https://fan-c.readthedocs.io/en/latest/fanc-executable/fanc_basic.html#numexpr-threadpool-configuration). The message refers to multi-threading in NumExpr only, not in chess in general.
As the message is confusing, we remove it with the 0.3.5 patch.

Chess itself uses multi-threading for the actual matrix-comparisons, as I said above. If you set -p > 1 then you should observe in your system monitor process viewer that new chess processes spawn after the "INFO Launching workers" log message.