Slightly modify the detection of the number of CPU cores

[rschu1ze]

AMD offers 48/67/64 core systems (Milan), and soon CPUs with up to 96 cores (Genoa). Intel also sells Xeon CPUs with >32 cores. If the system has >= 32 logical cores then the current logic is to use only half of them. If SMT (HyperThreading) is on, then ClickHouse will effectively utilize only the physical cores (i.e. "disable" HyperThreading by means of the software).

However, if SMT is disabled (logical == physical core count), which is not uncommon, then ClickHouse uses only half the available physical cores.

With this PR, the 32-core threshold is retained but the number of used cores is only reduced if SMT is on.

Changelog category (leave one):

• Performance Improvement

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Remove the limitation that on systems with >=32 cores and SMT disabled ClickHouse uses only half of the cores

[rschu1ze]

@nickitat Do you know if SMP/hyperthreading generally benefits/deteriorates performance? In case of the latter, it will be better to check programmatically if SMP is on and halve the max used cores only in this case.

[nickitat]

looks like generally (e.g. if we take the whole set of clickbench queries) we should benefit from hyper threading. it was the case when we brought it back for 16 cores instances. it is because normal queries do have stalls on memory access and branch mispredictions. only heavy number crunching queries without lots of memory accesses (like Q29 from CB) could probably slow down, but it is fairly marginal case.
so if we run CB on machines with different number of cores I would bet that we will see better relative time with HT for both intel and amd even for cpu_count >= 32.

[rschu1ze]

Tnx, this makes sense.
Even if HT isn't beneficial, overhead will be low I guess (1-3%?).
(I could actually not find not much material how HT impacts other DBs, so I guess it is not a big point of contention.)

@alexey-milovidov Feel free to merge or advice further.

[nickitat]

imo it worth to do measurements anyway to be confident in what we are doing. e.g. launch all c6a.<n>xlarge, c5.<n>xlarge for n > 4 and run clickbench.

[rschu1ze]

Some funny measurements of ClickBench on a 128-vCore / 64 physical core instance (m6i.32xlarge):

• current HEAD: 64 threads --> user time: 3.463s
• not artificially reducing the thread number: 128 threads --> user time: 3.677s

(averaging 6 measurements each)

So with HyperThreadin,g CPU consumption is surprisingly 6% worse. Of course, this could be because ClickBench is CPU-intensive.

I nevertheless think that the code should be made aware of HT so we don't underutilize >32-core systems with HT disabled. We can keep the existing threshold though.

[alexey-milovidov]

Hyperthreading cores don't increase performance linearly.

If a query has a low number of cache misses and branch mispredictions, hyperthreading makes performance worse.
If a query has a high number of cache misses and branch mispredictions, hyperthreading makes performance better, but usually in the 10..50% range. Therefore, using all HT cores for a single query by default will worsen performance on many concurrent queries.

Cache misses mainly appear while doing large GROUP BY, DISTINCT, IN, and JOIN. But on a very large number of cores, the memory becomes the bottleneck earlier than we start using all hyper-threaded cores.

It will make sense to allow more cores by default, but queries should automatically stop using so many threads when concurrent queries start running. I can be done with #37285, but this setting is not enabled by default.

[alexey-milovidov]

PS. SMT, not SMP.