"BPF: Invalid name" warnings when booting into Linux 6.1.15

[Ma27]

Describe the bug

I installed Linux kernel at 6.1.15 on my workstation that I previously built on my own. This caused the following log-lines after booting:

bootlog

[    0.526684] Run /init as init process
[    0.526685]   with arguments:
[    0.526686]     /init
[    0.526687]   with environment:
[    0.526687]     HOME=/
[    0.526688]     TERM=linux
[    0.526689]     BOOT_IMAGE=(crypto0)//kernels/v877jhvpwp5bqs236yw0drbny8xqprlx-linux-6.1.15-bzImage
[    0.555216] stage-1-init: [Sat Mar 11 10:29:29 UTC 2023] loading module amdgpu...
[    0.555453] BPF: [93325] ENUM a..percpu 
[    0.555465] BPF: size=4 vlen=2
[    0.555470] BPF:  
[    0.555474] BPF: Invalid name
[    0.555479] BPF: 
[    0.555482] failed to validate module [backlight] BTF: -22
[    0.559458] BPF: [93327] STRUCT a..percpu 
[    0.559465] BPF: size=16 vlen=4
[    0.559471] BPF:  
[    0.559474] BPF: Invalid name
[    0.559479] BPF: 
[    0.559483] failed to validate module [i2c_core] BTF: -22
[    0.586072] BPF: [93327] STRUCT a..percpu 
[    0.586086] BPF: size=104 vlen=9
[    0.586092] BPF:  
[    0.586096] BPF: Invalid name
[    0.586101] BPF: 
[    0.586105] failed to validate module [drm] BTF: -22
[    0.586190] ACPI: bus type drm_connector registered
[    0.586671] BPF: [93326] FUNC a..percpu 
[    0.586678] BPF: type_id=93308
[    0.586683] BPF:  
[    0.586687] BPF: Invalid name
[    0.586692] BPF: 
[    0.586696] failed to validate module [fb_sys_fops] BTF: -22
[    0.586988] BPF: [93325] FUNC a..percpu 
[    0.586995] BPF: type_id=62649
[    0.587000] BPF:  
[    0.587004] BPF: Invalid name
[    0.587009] BPF: 
[    0.587013] failed to validate module [sysimgblt] BTF: -22
[    0.587364] BPF: [93327] FUNC a..percpu 
[    0.587371] BPF: type_id=50694
[    0.587376] BPF:  
[    0.587380] BPF: Invalid name
[    0.587385] BPF: 
[    0.587388] failed to validate module [sysfillrect] BTF: -22
[    0.587870] BPF: [93325] FUNC a..percpu 
[    0.588092] BPF: type_id=72837
[    0.588310] BPF:  
[    0.588519] BPF: Invalid name
[    0.588727] BPF: 
[    0.588932] failed to validate module [syscopyarea] BTF: -22
[    0.597724] BPF: [93325] FUNC a..percpu 
[    0.597940] BPF: type_id=36471
[    0.598147] BPF:  
[    0.598357] BPF: Invalid name
[    0.598565] BPF: 
[    0.598773] failed to validate module [drm_kms_helper] BTF: -22
[    0.606366] BPF: [93325] FUNC a..percpu 
[    0.606591] BPF: type_id=1
[    0.606812] BPF:  
[    0.607034] BPF: Invalid name
[    0.607263] BPF: 
[    0.607487] failed to validate module [drm_display_helper] BTF: -22
[    0.608405] BPF: [93328] STRUCT a..percpu 
[    0.608644] BPF: size=32 vlen=4
[    0.608881] BPF:  
[    0.609118] BPF: Invalid name
[    0.609358] BPF: 
[    0.609595] failed to validate module [drm_buddy] BTF: -22
[    0.610414] BPF: [93327] STRUCT a..percpu 
[    0.610658] BPF: size=16 vlen=4
[    0.610901] BPF:  
[    0.611137] BPF: Invalid name
[    0.611381] BPF: 
[    0.611618] failed to validate module [i2c_algo_bit] BTF: -22
[    0.613862] BPF: [93325] STRUCT a..percpu 
[    0.614114] BPF: size=32 vlen=4
[    0.614355] BPF:  
[    0.614595] BPF: Invalid name
[    0.614835] BPF: 
[    0.615075] failed to validate module [gpu_sched] BTF: -22
[    0.616464] BPF: [93325] STRUCT a..percpu 
[    0.616710] BPF: size=48 vlen=2
[    0.616954] BPF:  
[    0.617196] BPF: Invalid name
[    0.617438] BPF: 
[    0.617679] failed to validate module [wmi] BTF: -22
[    0.620057] BPF: [93325] ENUM a..percpu 
[    0.620313] BPF: size=4 vlen=2
[    0.620559] BPF:  
[    0.620802] BPF: Invalid name
[    0.621047] BPF: 
[    0.621297] failed to validate module [video] BTF: -22
[    0.621621] ACPI: video: Video Device [VGA] (multi-head: yes  rom: no  post: no)
[    0.621675] input: Video Bus as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0A08:00/device:17/LNXVIDEO:00/input/input0
[    0.622583] BPF: [93325] TYPEDEF a..percpu 
[    0.622832] BPF: type_id=93309
[    0.623080] BPF:  
[    0.623326] BPF: Invalid name
[    0.623569] BPF: 
[    0.623812] failed to validate module [iommu_v2] BTF: -22
[    0.624155] AMD-Vi: AMD IOMMUv2 loaded and initialized
[    0.625665] BPF: [93325] ENUM a..percpu 
[    0.625915] BPF: size=4 vlen=2
[    0.626159] BPF:  
[    0.626402] BPF: Invalid name
[    0.626644] BPF: 
[    0.626885] failed to validate module [agpgart] BTF: -22
[    0.627156] Linux agpgart interface v0.103
[    0.630856] BPF: [93325] STRUCT a..percpu 
[    0.631107] BPF: size=32 vlen=4
[    0.631354] BPF:  
[    0.631594] BPF: Invalid name
[    0.631834] BPF: 
[    0.632076] failed to validate module [ttm] BTF: -22
[    0.632941] BPF: [93325] STRUCT a..percpu 
[    0.633188] BPF: size=104 vlen=9
[    0.633436] BPF:  
[    0.633677] BPF: Invalid name
[    0.633920] BPF: 
[    0.634163] failed to validate module [drm_ttm_helper] BTF: -22
[    1.400005] tsc: Refined TSC clocksource calibration: 1696.804 MHz
[    1.400018] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x18755c21f57, max_idle_ns: 440795228299 ns
[    2.213983] clocksource: Switched to clocksource tsc
[    2.220469] BPF: [93326] TYPEDEF a..percpu 
[    2.220796] BPF: type_id=58
[    2.221111] BPF:  
[    2.221421] BPF: Invalid name
[    2.221733] BPF: 
[    2.222050] failed to validate module [amdgpu] BTF: -22
[    2.223070] [drm] amdgpu kernel modesetting enabled.
[    2.234807] amdgpu: Virtual CRAT table created for CPU
[    2.234822] amdgpu: Topology: Add CPU node

Except for these log-lines everything is behaving normal. Also I only managed to trigger this on my laptop, but not on any of my servers.

The following hardware is affected: Lenovo T14s AMD Gen 1. I also use the zfs kernel module and the configurations from nixos-hardware for said model.

Since I couldn't reproduce the problem with 6.1.14, I tried to bisect the problem to a specific commit, without luck though. When I bisected, every single revision I built booted without these warnings. At the end I rebuilt Linux 6.1.15 (with a random env new var in the drv to trigger a rebuild) and it was still gone, so I suspect the problem is some kind of build-system problem.

Since I can't reproduce it anymore I haven't filed an upstream bug so far because I can't rule out an issue in nixpkgs safely.

In fact I found a thread that seemed related[1], but considering that the root-cause was a problem with KUnit on 6.3 it's most likely not.

Neither my time nor my knowledge about BPF internals is sufficient to investigate what's wrong there currently, so I filed an issue to document this and to let others chime in.

Steps To Reproduce

None, unfortunately. I didn't manage to trigger this while rebuilding kernels.

However, a flat-file binary cache containing the build product of the problematic kernel is available at https://wolke.mbosch.me/s/gaZNLMytbd9EbrH.
Revision b65d565 (nix-build -A linuxKernel.kernels.linux_6_1) was used to build the kernel.

Notify maintainers

cc @alyssais @NeQuissimus @TredwellGit (linux kernel team)
cc @K900 @lovesegfault @fabianhjr (for regularly working on kernel-related things)

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

 - system: `"x86_64-linux"`
 - host os: `Linux 6.1.15, NixOS, 22.11 (Raccoon), 22.11.20230304.211b3c0`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.13.3`
 - nixpkgs: `/nix/store/w9fcmpn3vh9pvsjyfnjdc6aq83cxlap3-00qljhzqp0d08v83a96giy3r6kj65g7c-source`

[1] https://lore.kernel.org/all/9f682c4d-e7b7-5e23-84f5-cea4fdac2085@leemhuis.info/

[K900]

This happens when your kernel and your modules come from different instances of a build, e.g. one is substituted and the other isn't. Before #181456 this would die entirely on boot. I still haven't found the time to investigate the cause.

[Ma27]

Btw, I afaiu #222337 fixed the issue, or am I missing something (cc @K900 @alyssais ).

[K900]

No, BTFs are still not reproducible.

[alyssais]

They are now, right?

[K900]

They should be, let's close this and reopen if it ever happens again.