even after offloading all layers to gpu, is it normal for CPU to use up all threads at 100% utilization?

[hiqsociety]

even after offloading all layers to gpu, is it normal for CPU to use up all threads at 100% utilization?

i thought i already offloaded to gpu (which i did) but why is CPU still being used?

[KerfuffleV2]

When offloading all layers, you usually want to set threads to 1 or a low value. The way llama.cpp uses threads basically has them spin, eating 100% CPU even if they don't have work to do. I think there was a behavior recently added to set threads to 1 when offloading all layers (not 100% sure it got merged). If so, you could try making sure you're not specifying -t and see how many threads it chooses automatically in that situation.

[Manouchehri]

Isn't spinning CPUs unnecessarily at 100% not great for power efficiency?

[hiqsociety]

@KerfuffleV2 thx for explaining! i didnt specify -t and it's using all cores.
making -t 1 actually makes the whole thing faster!

should really make it automatic t = 1.

[KerfuffleV2]

#2915 is what I was thinking of. I guess it didn't get merged in yet.