Skip to content
Discussion options

You must be logged in to vote

There's no --threads flag. The thread count is the standard OpenMP variable OMP_NUM_THREADS, and coli only picks a default when you haven't set it yourself:

OMP_NUM_THREADS=7 ./coli chat --model /path/to/model

On Windows (cmd): set OMP_NUM_THREADS=7 before running coli, or in PowerShell $env:OMP_NUM_THREADS=7.

The launcher uses setdefault, so your value wins over the automatic one (c/coli#L578-L582), and a saved coli tune profile won't override it either, because variables already in your environment are treated as explicit. It's also listed in docs/ENVIRONMENT.md.

Why it uses 4 by default: the default team is the physical core count (or only the P-cores on a hybrid CPU), not the logical …

Replies: 1 comment 1 reply

Comment options

You must be logged in to vote
1 reply
@A13xund3r
Comment options

Answer selected by A13xund3r
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
Q&A
Labels
None yet
2 participants