kernel warning: cacheinfo: Unable to detect cache hierarchy for CPU 0

[debug-richard]

On BCM2837/BCM2711 running in ARMv8 mode linux kernel 5.10 shows the warning:

cacheinfo: Unable to detect cache hierarchy for CPU 0

It looks like the L1/L2/L3 hierarchy is missing which may have a huge impact on performance for some applications.
I think the device tree needs to be updated to fix this warning.

[pelwell]

What do you suggest needs to be added to the dtbs?

[popcornmix]

I did see this on a search: https://patchwork.kernel.org/project/linux-arm-kernel/patch/20180726003532.18751-2-andre.przywara@arm.com/

I doubt it will have any effect on performance (I'm sure L1 and L2 caches are already active).

[debug-richard]

The error message comes from linux/drivers/base/cacheinfo.c.
The file description says:

cacheinfo support - processor cache information via sysfs

Therefore, the "cache" directory in /sys/devices/system/cpu/cpu0/ is missing.
On my amd64 machine this directory contains all the cache information.

"getconf -a | grep -i cache" also returns only the L1 cache size:

LEVEL1_ICACHE_SIZE                 0
LEVEL1_ICACHE_ASSOC                0
LEVEL1_ICACHE_LINESIZE             64
LEVEL1_DCACHE_SIZE                 0
LEVEL1_DCACHE_ASSOC                0
LEVEL1_DCACHE_LINESIZE             64
LEVEL2_CACHE_SIZE                  0
LEVEL2_CACHE_ASSOC                 0
LEVEL2_CACHE_LINESIZE              0
LEVEL3_CACHE_SIZE                  0
LEVEL3_CACHE_ASSOC                 0
LEVEL3_CACHE_LINESIZE              0
LEVEL4_CACHE_SIZE                  0
LEVEL4_CACHE_ASSOC                 0
LEVEL4_CACHE_LINESIZE              0

It looks like the cache information is not available in userspace.
Some (special) software uses this information to optimize performance. (for example DPDK (I played around with this some time ago))

I also found this issue:
https://forums.raspberrypi.com/viewtopic.php?t=272199
Seems like Kubernetes is also using this information.

The idea behind this is to adapt the data to the cache size in order to optimize the performance.
Hardcore optimizations but without the cache information they don't work.

I couldn't find a reliable source for BCM cache sizes and my knowledge of the device tree is limited, so I can't help you much.

[pelwell]

The last line of that forum thread says:

File a bug against kubernetes. They shouldn't be doing that on ARM.

[debug-richard]

The relevant Kubernetes issues show that they suppressed the warning but did not change the cache detection.
k3s-io/k3s#1937
google/cadvisor#2550

There is no point in disabling it as the driving forces in the armv8 server business (Amazon Graviton) seem to provide this information.

[lurch]

I couldn't find a reliable source for BCM cache sizes

There's (some?) cache info at https://www.raspberrypi.com/documentation/computers/processors.html and in the datasheets linked to from there.

EDIT: also https://www.raspberrypi.com/documentation/computers/config_txt.html#disable_l2cache

[popcornmix]

You can query the cache sizes from co processor registers:
https://forums.raspberrypi.com/viewtopic.php?p=1943936#p1943936

[debug-richard]

I checked some device trees for other SoCs and created a fix raspberrypi/linux#4751.
Cache docu for the BCM2837 is a mess and can be found in some old datasheets/magpi.
Luckily, the BCM2711 docu is right on the website.
The cache alignment can be found in the official ARM docu.