Skip to content

Commit

Permalink
[imt] Introduce env var ROOT_MAX_THREADS:
Browse files Browse the repository at this point in the history
This overrides the number of threads that ROOT will be using in the thread pool.
This can be set to be higher or lower than the number of physical cores; it can
be specified as decimal, octal, or hex.

Fixes issue #9805.

Mention the env var in `ROOT::EnableImplicitMT()`'s doc.
  • Loading branch information
Axel-Naumann committed Jun 28, 2023
1 parent 187beb6 commit 416f75c
Show file tree
Hide file tree
Showing 2 changed files with 21 additions and 1 deletion.
3 changes: 3 additions & 0 deletions core/base/src/TROOT.cxx
Expand Up @@ -521,6 +521,9 @@ namespace Internal {
/// a hint for ROOT: it will try to satisfy the request if the execution
/// scenario allows it. For example, if ROOT is configured to use an external
/// scheduler, setting a value for 'numthreads' might not have any effect.
/// To avoid this heuristic, the maximum number of threads can be set by the
/// environment variable `ROOT_MAX_THREADS`: `export ROOT_MAX_THREADS=2` will
/// set the maximum number of active threads to 2.
///
/// \note Use `DisableImplicitMT()` to disable multi-threading (some locks will remain in place as
/// described in EnableThreadSafety()). `EnableImplicitMT(1)` creates a thread-pool of size 1.
Expand Down
19 changes: 18 additions & 1 deletion core/imt/src/RTaskArena.cxx
Expand Up @@ -5,9 +5,11 @@
#include "ROpaqueTaskArena.hxx"
#include "TError.h"
#include "TROOT.h"
#include "TSystem.h"
#include "TThread.h"
#include <fstream>
#include <mutex>
#include <string>
#include <thread>
#include "tbb/task_arena.h"
#define TBB_PREVIEW_GLOBAL_CONTROL 1 // required for TBB versions preceding 2019_U4
Expand Down Expand Up @@ -43,9 +45,24 @@
namespace ROOT {
namespace Internal {

// To honor cgroup quotas if set: see https://github.com/oneapi-src/oneTBB/issues/190
// Honor environment variable `ROOT_MAX_THREADS` if set.
// Also honor cgroup quotas if set: see https://github.com/oneapi-src/oneTBB/issues/190
int LogicalCPUBandwidthControl()
{
if (const char *envMaxThreads = gSystem->Getenv("ROOT_MAX_THREADS")) {
char *str_end = nullptr;
long maxThreads = std::strtol(envMaxThreads, &str_end, 0 /*auto-detect base*/);
if (str_end == envMaxThreads && maxThreads == 0) {
Error("ROOT::Internal::LogicalCPUBandwidthControl()",
"cannot parse number in environment variable ROOT_MAX_THREADS; ignoring.");
} else if (maxThreads < 1) {
Error("ROOT::Internal::LogicalCPUBandwidthControl()",
"environment variable ROOT_MAX_THREADS must be >= 1, but set to %ld; ignoring.",
maxThreads);
} else
return maxThreads;
}

#ifdef R__LINUX
// Check for CFS bandwith control
std::ifstream f("/sys/fs/cgroup/cpuacct/cpu.cfs_quota_us"); // quota file
Expand Down

0 comments on commit 416f75c

Please sign in to comment.