Skip to content

Add check for BTF sysfs #3847

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jun 13, 2025
Merged

Add check for BTF sysfs #3847

merged 5 commits into from
Jun 13, 2025

Conversation

rlmenge
Copy link
Contributor

@rlmenge rlmenge commented Jun 9, 2025

This PR adds a check for the presence of BTF data on the target system by verifying the existence of the /sys/kernel/btf/ directory with both kernel (vmlinux) and loaded module data

This directory indicates that BTF (BPF Type Format) data is available, which is crucial for certain BPF features and tools. BTF data is required for advanced BPF capabilities and observability on modern Linux kernels. Ensuring its presence helps validate system readiness for BPF workloads. Currently this test has been validated for AzureLinux scenarios.

Specifically this PR

  • Updates lsmod to return the loaded modules
  • Implements verify_btf_sysfs_exists to ensure /sys/kernel/btf/vmlinux and /sys/kernel/btf/ are present.

Testing
Confirmed it will not run on a non AzL3 system (CBL-Mariner 2.0)

2025-06-11 16:10:44.366[135876417206080][INFO] lisa.RootRunner                   BpfSuite.verify_btf_sysfs_exists: SKIPPED  before_case skipped: Unsupported system: 'CBL-Mariner/Linux'. BPF support promised on AzureLinux 3.0 and later.
2025-06-11 16:10:44.366[135876417206080][INFO] lisa.RootRunner test result summary
2025-06-11 16:10:44.366[135876417206080][INFO] lisa.RootRunner     TOTAL    : 1
2025-06-11 16:10:44.366[135876417206080][INFO] lisa.RootRunner     QUEUED   : 0
2025-06-11 16:10:44.367[135876417206080][INFO] lisa.RootRunner     ASSIGNED : 0
2025-06-11 16:10:44.367[135876417206080][INFO] lisa.RootRunner     RUNNING  : 0
2025-06-11 16:10:44.367[135876417206080][INFO] lisa.RootRunner     FAILED   : 0
2025-06-11 16:10:44.367[135876417206080][INFO] lisa.RootRunner     PASSED   : 0
2025-06-11 16:10:44.367[135876417206080][INFO] lisa.RootRunner     SKIPPED  : 1

Confirm the check fails gracefully on systems without /sys/kernel/btf/.

2025-06-12 15:06:49.326[126099552990912][ERROR] lisa.case[verify_btf_sysfs_exists] BTF sysfs missing for modules: xt_owner, mlx5_ib, ib_uverbs, ib_core, mlx5_core, mlxfw, psample, tls, xt_tcpudp, xt_conntrack, nft_compat, nft_fib_ipv6, nft_masq, nft_nat, nft_fib_ipv4, nft_fib, nft_chain_nat, nf_tables, xt_LOG, nf_log_syslog, cfg80211, 8021q, garp, mrp, stp, llc, sm4_ce_cipher, sm4, sm3_ce, sm3, sha3_ce, sha512_ce, sha512_arm64, hyperv_fb, sch_fq_codel, fuse, efi_pstore, dmi_sysfs, hid_generic, crct10dif_ce, ghash_ce, hid_hyperv, gf128mul, hid, sha2_ce, sha256_arm64, sha1_ce, serio_raw, ebt_ip, ip6table_nat, ip6table_mangle, ip6table_filter, ip6_tables, iptable_security, iptable_nat, nf_nat, nf_conntrack, nf_defrag_ipv6, nf_defrag_ipv4, iptable_mangle, iptable_filter, ip_tables, x_tables, aes_ce_blk, aes_ce_cipher
2025-06-12 15:06:49.329[126099818538688][ERROR] lisa.case[verify_btf_sysfs_exists] case failed
...
2025-06-12 15:06:52.126[126100310230848][INFO] lisa.RootRunner                   BpfSuite.verify_btf_sysfs_exists: FAILED   failed. AssertionError: [The following modules are missing /sys/kernel/btf entries: xt_owner, mlx5_ib, ib_uverbs, ib_core, mlx5_core, mlxfw, psample, tls, xt_tcpudp, xt_conntrack, nft_compat, nft_fib_ipv6, nft_masq, nft_nat, nft_fib_ipv4, nft_fib, nft_chain_nat, nf_tables, xt_LOG, nf_log_syslog, cfg80211, 8021q, garp, mrp, stp, llc, sm4_ce_cipher, sm4, sm3_ce, sm3, sha3_ce, sha512_ce, sha512_arm64, hyperv_fb, sch_fq_codel, fuse, efi_pstore, dmi_sysfs, hid_generic, crct10dif_ce, ghash_ce, hid_hyperv, gf128mul, hid, sha2_ce, sha256_arm64, sha1_ce, serio_raw, ebt_ip, ip6table_nat, ip6table_mangle, ip6table_filter, ip6_tables, iptable_security, iptable_nat, nf_nat, nf_conntrack, nf_defrag_ipv6, nf_defrag_ipv4, iptable_mangle, iptable_filter, ip_tables, x_tables, aes_ce_blk, aes_ce_cipher] Expected <['xt_owner', 'mlx5_ib', 'ib_uverbs', 'ib_core', 'mlx5_core', 'mlxfw', 'psample', 'tls', 'xt_tcpudp', 'xt_conntrack', 'nft_compat', 'nft_fib_ipv6', 'nft_masq', 'nft_nat', 'nft_fib_ipv4', 'nft_fib', 'nft_chain_nat', 'nf_tables', 'xt_LOG', 'nf_log_syslog', 'cfg80211', '8021q', 'garp', 'mrp', 'stp', 'llc', 'sm4_ce_cipher', 'sm4', 'sm3_ce', 'sm3', 'sha3_ce', 'sha512_ce', 'sha512_arm64', 'hyperv_fb', 'sch_fq_codel', 'fuse', 'efi_pstore', 'dmi_sysfs', 'hid_generic', 'crct10dif_ce', 'ghash_ce', 'hid_hyperv', 'gf128mul', 'hid', 'sha2_ce', 'sha256_arm64', 'sha1_ce', 'serio_raw', 'ebt_ip', 'ip6table_nat', 'ip6table_mangle', 'ip6table_filter', 'ip6_tables', 'iptable_security', 'iptable_nat', 'nf_nat', 'nf_conntrack', 'nf_defrag_ipv6', 'nf_defrag_ipv4', 'iptable_mangle', 'iptable_filter', 'ip_tables', 'x_tables', 'aes_ce_blk', 'aes_ce_cipher']> to be empty, but was not.
2025-06-12 15:06:52.126[126100310230848][INFO] lisa.RootRunner test result summary
2025-06-12 15:06:52.127[126100310230848][INFO] lisa.RootRunner     TOTAL    : 1
2025-06-12 15:06:52.127[126100310230848][INFO] lisa.RootRunner     QUEUED   : 0
2025-06-12 15:06:52.127[126100310230848][INFO] lisa.RootRunner     ASSIGNED : 0
2025-06-12 15:06:52.127[126100310230848][INFO] lisa.RootRunner     RUNNING  : 0
2025-06-12 15:06:52.127[126100310230848][INFO] lisa.RootRunner     FAILED   : 1
2025-06-12 15:06:52.127[126100310230848][INFO] lisa.RootRunner     PASSED   : 0
2025-06-12 15:06:52.128[126100310230848][INFO] lisa.RootRunner     SKIPPED  : 0

Run the new check on a target AZL3 VM system and verify it passes on systems with BTF support.

2025-06-11 15:56:02.480[125591412008640][INFO] lisa.case[verify_btf_sysfs_exists] test case 'BpfSuite.verify_btf_sysfs_exists' is running
2025-06-11 15:56:06.175[125591269398208][INFO] lisa.case[verify_btf_sysfs_exists] BTF sysfs confirmed for /sys/kernel/btf/vmlinux.
2025-06-11 15:56:14.481[125591269398208][INFO] lisa.case[verify_btf_sysfs_exists] BTF sysfs confirmed for all loaded modules.
2025-06-11 15:56:15.572[125591412008640][INFO] lisa.case[verify_btf_sysfs_exists] result: PASSED, elapsed: 13.091 sec
2025-06-11 15:56:16.289[125591412008640][INFO] lisa.[azure].del[generated_0] skipped to delete environment generated_0, as on runbook, keep_environment value is set to always and env status is EnvironmentStatus.Connected
2025-06-11 15:56:16.289[125591412008640][INFO] lisa.[azure].del[generated_0] node ip addresses: [...]
2025-06-11 15:56:16.295[125591981520704][INFO] lisa.RootRunner ________________________________________
2025-06-11 15:56:16.295[125591981520704][INFO] lisa.RootRunner                   BpfSuite.verify_btf_sysfs_exists: PASSED   
2025-06-11 15:56:16.295[125591981520704][INFO] lisa.RootRunner test result summary
2025-06-11 15:56:16.295[125591981520704][INFO] lisa.RootRunner     TOTAL    : 1
2025-06-11 15:56:16.295[125591981520704][INFO] lisa.RootRunner     QUEUED   : 0
2025-06-11 15:56:16.295[125591981520704][INFO] lisa.RootRunner     ASSIGNED : 0
2025-06-11 15:56:16.296[125591981520704][INFO] lisa.RootRunner     RUNNING  : 0
2025-06-11 15:56:16.296[125591981520704][INFO] lisa.RootRunner     FAILED   : 0
2025-06-11 15:56:16.296[125591981520704][INFO] lisa.RootRunner     PASSED   : 1
2025-06-11 15:56:16.296[125591981520704][INFO] lisa.RootRunner     SKIPPED  : 0
2025-06-11 15:56:16.300[125591981520704][INFO] lisa.notifier[Html] report: /home/mariner_user/repos/lisa/runtime/log/20250611/20250611-155106-264/lisa.html

@rlmenge rlmenge marked this pull request as draft June 9, 2025 18:09
@rlmenge rlmenge force-pushed the rlmenge/bpf-check branch from 7c99876 to e639510 Compare June 9, 2025 18:13
rlmenge added 2 commits June 9, 2025 18:34
The BTF sysfs interface is important because it provides a standardized way for
the kernel to expose BPF Type Format (BTF) data to user space, enabling
advanced introspection of kernel and module types. This accessibility is
crucial for BPF tooling, such as bpftool and libbpf, to perform type-aware
operations, debugging, and verification.The BTF sysfs files are at
/sys/kernel/btf/ by standard, where each entry corresponds to BTF data for the
kernel or loaded kernel modules.

Add bpf testsuite which can be expanded with more tests
@rlmenge rlmenge force-pushed the rlmenge/bpf-check branch from e639510 to 545e589 Compare June 9, 2025 18:34
@rlmenge rlmenge marked this pull request as ready for review June 9, 2025 20:06
@rlmenge rlmenge force-pushed the rlmenge/bpf-check branch from 44972ce to 44d83f3 Compare June 10, 2025 19:36
@squirrelsc
Copy link
Member

@LiliDeng LGTM

@LiliDeng
Copy link
Collaborator

@rlmenge is it expected to see case fail against image microsoftcblmariner azure-linux-3 azure-linux-3 latest?

ls: cannot access '/sys/kernel/btf/xt_owner': No such file or directory

@rlmenge
Copy link
Contributor Author

rlmenge commented Jun 11, 2025

@rlmenge is it expected to see case fail against image microsoftcblmariner azure-linux-3 azure-linux-3 latest?

ls: cannot access '/sys/kernel/btf/xt_owner': No such file or directory

Correct. There is a bug where we claim to have full bpf support but our modules are being stripped of btf data and the fix has not yet gone in yet. The fix is here: microsoft/azurelinux#13813. I was asked to merge a test to check for this before merging the fix

@LiliDeng LiliDeng merged commit ecb2138 into microsoft:main Jun 13, 2025
27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants