Repository navigation
How to control number of CPU cores used? #2807
|
I see it use only half of CPU, it detected 4 cores which is correct but there is 8 threads. In other inferences I use |
Replies: 1 comment 1 reply
|
There's no OMP_NUM_THREADS=7 ./coli chat --model /path/to/modelOn Windows (cmd): The launcher uses Why it uses 4 by default: the default team is the physical core count (or only the P-cores on a hybrid CPU), not the logical threads ( So |
There's no
--threadsflag. The thread count is the standard OpenMP variableOMP_NUM_THREADS, andcolionly picks a default when you haven't set it yourself:On Windows (cmd):
set OMP_NUM_THREADS=7before runningcoli, or in PowerShell$env:OMP_NUM_THREADS=7.The launcher uses
setdefault, so your value wins over the automatic one (c/coli#L578-L582), and a savedcoli tuneprofile won't override it either, because variables already in your environment are treated as explicit. It's also listed in docs/ENVIRONMENT.md.Why it uses 4 by default: the default team is the physical core count (or only the P-cores on a hybrid CPU), not the logical …