Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Linux v5.3-rc1+: bpf issue when linking with lld #619

Closed
dileks opened this issue Jul 26, 2019 · 42 comments
Closed

Linux v5.3-rc1+: bpf issue when linking with lld #619

dileks opened this issue Jul 26, 2019 · 42 comments

Comments

@dileks
Copy link

@dileks dileks commented Jul 26, 2019

Hi,

I see a problem in next-20190723 when linking with ld.lld.

After login I land in the maintenance mode as some systemd services failed to load:

  1. systemd-journald
  2. systemd-udevd
[Fri Jul 26 08:08:43 2019] systemd[453]: systemd-udevd.service: Failed to connect stdout to the journal socket, ignoring: Connection refused

This is not seen when I link with ld.bfd from Debian/buster.

In both cases I use clang-9.

$ clang-9 --version
ClangBuiltLinux clang version 9.0.0 (git://github.com/llvm/llvm-project 1e6b12dd3158a554c63d332df6b25f407eed9556) (based on LLVM 9.0.0)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /home/sdi/src/llvm-toolchain/install/bin

Applied patches (just for the records):

drm/i915: Remove redundant user_access_end() from __copy_from_user() error path
objtool: Improve UACCESS coverage
xen/trace: avoid clang warning on function pointers
drm/i915: Fix i915_gemfs_init() NULL dereference

Excerpt from my dmesg-log:

[Fri Jul 26 08:08:42 2019] BUG: unable to handle page fault for address: ffffffff85403370
[Fri Jul 26 08:08:42 2019] #PF: supervisor read access in kernel mode
[Fri Jul 26 08:08:42 2019] #PF: error_code(0x0000) - not-present page
[Fri Jul 26 08:08:42 2019] PGD 7620e067 P4D 7620e067 PUD 7620f063 PMD 44fe85063 PTE 800fffff8a3fc062
[Fri Jul 26 08:08:42 2019] Oops: 0000 [#1] SMP PTI 
[Fri Jul 26 08:08:42 2019] CPU: 2 PID: 417 Comm: (journald) Not tainted 5.3.0-rc1-5-amd64-cbl-asmgoto #5~buster+dileks1
[Fri Jul 26 08:08:42 2019] Hardware name: LENOVO 20HDCTO1WW/20HDCTO1WW, BIOS N1QET83W (1.58 ) 04/18/2019
[Fri Jul 26 08:08:42 2019] RIP: 0010:___bpf_prog_run+0x40/0x14f0
[Fri Jul 26 08:08:42 2019] Code: f3 eb 24 48 83 f8 38 0f 84 a9 0c 00 00 48 83 f8 39 0f 85 8a 14 00 00 0f 1f 00 48 0f bf 43 02 48 8d 1c c3 48 83 c3 08 0f b6 33 <48> 8b 04 f5 10 2e 40 85 48 83 f8 3b 7f 62 48 83 f8 1e 0f 8f c8 00
[Fri Jul 26 08:08:42 2019] RSP: 0018:ffff992ec028fcb8 EFLAGS: 00010246 
[Fri Jul 26 08:08:42 2019] RAX: ffff992ec028fd60 RBX: ffff992ec00e9038 RCX: 0000000000000002
[Fri Jul 26 08:08:42 2019] RDX: ffff992ec028fd40 RSI: 00000000000000ac RDI: ffff992ec028fce0
[Fri Jul 26 08:08:42 2019] RBP: ffff992ec028fcd0 R08: 0000000000000000 R09: ffff992ec028ff58
[Fri Jul 26 08:08:42 2019] R10: 0000000000000000 R11: ffffffff849b8210 R12: 000000007fff0000
[Fri Jul 26 08:08:42 2019] R13: ffff992ec028feb8 R14: 0000000000000000 R15: ffff992ec028fce0
[Fri Jul 26 08:08:42 2019] FS:  00007f5d20f1d940(0000) GS:ffff8ba3d2500000(0000) knlGS:0000000000000000
[Fri Jul 26 08:08:42 2019] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[Fri Jul 26 08:08:42 2019] CR2: ffffffff85403370 CR3: 0000000445b3e001 CR4: 00000000003606e0
[Fri Jul 26 08:08:42 2019] Call Trace:
[Fri Jul 26 08:08:42 2019]  __bpf_prog_run32+0x44/0x70
[Fri Jul 26 08:08:42 2019]  ? flush_tlb_func_common+0xd8/0x230
[Fri Jul 26 08:08:42 2019]  ? mem_cgroup_commit_charge+0x8c/0x120
[Fri Jul 26 08:08:42 2019]  ? wp_page_copy+0x464/0x7a0
[Fri Jul 26 08:08:42 2019]  seccomp_run_filters+0x54/0x110
[Fri Jul 26 08:08:42 2019]  __seccomp_filter+0xf7/0x6e0
[Fri Jul 26 08:08:42 2019]  ? do_wp_page+0x32b/0x5d0
[Fri Jul 26 08:08:42 2019]  ? handle_mm_fault+0x90d/0xbf0
[Fri Jul 26 08:08:42 2019]  syscall_trace_enter+0x182/0x290
[Fri Jul 26 08:08:42 2019]  do_syscall_64+0x30/0x90
[Fri Jul 26 08:08:42 2019]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[Fri Jul 26 08:08:42 2019] RIP: 0033:0x7f5d220d7f59
[Fri Jul 26 08:08:42 2019] Code: 00 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 07 6f 0c 00 f7 d8 64 89 01 48
[Fri Jul 26 08:08:42 2019] RSP: 002b:00007ffd11332b48 EFLAGS: 00000246 ORIG_RAX: 000000000000013d
[Fri Jul 26 08:08:42 2019] RAX: ffffffffffffffda RBX: 000055bf8ab34010 RCX: 00007f5d220d7f59
[Fri Jul 26 08:08:42 2019] RDX: 000055bf8ab34010 RSI: 0000000000000000 RDI: 0000000000000001
[Fri Jul 26 08:08:42 2019] RBP: 000055bf8ab97fb0 R08: 000055bf8abbe180 R09: 00000000c000003e
[Fri Jul 26 08:08:42 2019] R10: 000055bf8abbe1e0 R11: 0000000000000246 R12: 00007ffd11332ba0
[Fri Jul 26 08:08:42 2019] R13: 00007ffd11332b98 R14: 00007f5d21f087f8 R15: 000000000000002c
[Fri Jul 26 08:08:42 2019] Modules linked in: i2c_dev parport_pc sunrpc ppdev lp parport efivarfs ip_tables x_tables autofs4 ext4 crc32c_generic mbcache crc16 jbd2 btrfs zstd_decompress zstd_compress algif_skcipher af_alg sd_mod dm_crypt dm_mod raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 uas raid0 usb_storage multipath linear scsi_mod md_mod hid_cherry hid_generic usbhid hid crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel aesni_intel aes_x86_64 i915 glue_helper crypto_simd nvme i2c_algo_bit cryptd psmouse xhci_pci drm_kms_helper e1000e i2c_i801 xhci_hcd intel_lpss_pci nvme_core intel_lpss drm usbcore thermal wmi video button
[Fri Jul 26 08:08:42 2019] CR2: ffffffff85403370
[Fri Jul 26 08:08:42 2019] ---[ end trace 867b35c7d6c6705a ]---
[Fri Jul 26 08:08:42 2019] RIP: 0010:___bpf_prog_run+0x40/0x14f0
[Fri Jul 26 08:08:42 2019] Code: f3 eb 24 48 83 f8 38 0f 84 a9 0c 00 00 48 83 f8 39 0f 85 8a 14 00 00 0f 1f 00 48 0f bf 43 02 48 8d 1c c3 48 83 c3 08 0f b6 33 <48> 8b 04 f5 10 2e 40 85 48 83 f8 3b 7f 62 48 83 f8 1e 0f 8f c8 00
[Fri Jul 26 08:08:42 2019] RSP: 0018:ffff992ec028fcb8 EFLAGS: 00010246
[Fri Jul 26 08:08:42 2019] RAX: ffff992ec028fd60 RBX: ffff992ec00e9038 RCX: 0000000000000002
[Fri Jul 26 08:08:42 2019] RDX: ffff992ec028fd40 RSI: 00000000000000ac RDI: ffff992ec028fce0
[Fri Jul 26 08:08:42 2019] RBP: ffff992ec028fcd0 R08: 0000000000000000 R09: ffff992ec028ff58
[Fri Jul 26 08:08:42 2019] R10: 0000000000000000 R11: ffffffff849b8210 R12: 000000007fff0000
[Fri Jul 26 08:08:42 2019] R13: ffff992ec028feb8 R14: 0000000000000000 R15: ffff992ec028fce0
[Fri Jul 26 08:08:42 2019] FS:  00007f5d20f1d940(0000) GS:ffff8ba3d2500000(0000) knlGS:0000000000000000
[Fri Jul 26 08:08:42 2019] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[Fri Jul 26 08:08:42 2019] CR2: ffffffff85403370 CR3: 0000000445b3e001 CR4: 00000000003606e0

Inspired by #282 CONFIG_SECCOMP panic with LLD I disabled CONFIG_SECCOMP=n then I see a different call-trace.
You wanna see this one?

What files do you need?

  1. vmlinux?
  2. kernel/seccomp.o?
  3. bpf?

@kees
Can you enlighten the correlations between bpf and seccomp?

I tried some kernel-boot-parameter:

  1. nokaslr
  2. mitigations=off
  3. apparmor=off (default enabled in Debian/buster)
  4. hardened_usercopy=off

Any other parameters I can test?

I will test in QEMU, too.

Attached are my kernel-config and dmesg-log.

config-5.3.0-rc1-5-amd64-cbl-asmgoto.txt
dmesg_5.3.0-rc1-5-amd64-cbl-asmgoto.txt

@dileks dileks changed the title next-20190723: Hi,bpf seccomp issue? next-20190723: bpf seccomp issue? Jul 26, 2019
@dileks dileks changed the title next-20190723: bpf seccomp issue? next-20190723: bpf/seccomp - systemd/journal issue? Jul 26, 2019
@dileks
Copy link
Author

@dileks dileks commented Jul 26, 2019

clang-9 and ld.lld-9 with CONFIG_SECCOMP=n:

[Fri Jul 26 07:41:30 2019] BUG: unable to handle page fault for address: ffffffffbac03370
[Fri Jul 26 07:41:30 2019] RIP: 0010:___bpf_prog_run+0x40/0x14f0
[Fri Jul 26 07:41:30 2019] #PF: supervisor read access in kernel mode
[Fri Jul 26 07:41:30 2019] Code: f3 eb 24 48 83 f8 38 0f 84 a9 0c 00 00 48 83 f8 39 0f 85 8a 14 00 00 0f 1f 00 48 0f bf 43 02 48 8d 1c c3 48 83 c3 08 0f b6 33 <48> 8b 04 f5 10 2e c0 ba 48 83 f8 3b 7f 62 48 83 f8 1e 0f 8f c8 00
[Fri Jul 26 07:41:30 2019] #PF: error_code(0x0000) - not-present page
[Fri Jul 26 07:41:30 2019] RSP: 0018:ffffa8380025fa88 EFLAGS: 00010246
[Fri Jul 26 07:41:30 2019] PGD 21140e067
[Fri Jul 26 07:41:30 2019] P4D 21140e067
[Fri Jul 26 07:41:30 2019] RAX: ffffa8380025fb30 RBX: ffffa838000d1038 RCX: 0000000000000000
[Fri Jul 26 07:41:30 2019] PUD 21140f063
[Fri Jul 26 07:41:30 2019] RDX: ffffa8380025fb10 RSI: 00000000000000ac RDI: ffffa8380025fab0
[Fri Jul 26 07:41:30 2019] PMD 450828063
[Fri Jul 26 07:41:30 2019] RBP: ffffa8380025faa0 R08: ffff8c7ac5888e00 R09: 0000000000000000
[Fri Jul 26 07:41:30 2019] PTE 800ffffdef1fc062
[Fri Jul 26 07:41:30 2019] R10: ffff8c7acfe2e400 R11: ffffffffba1b5de0 R12: 0000000000000000
[Fri Jul 26 07:41:30 2019] R13: ffffa838000d1000 R14: 0000000000000000 R15: ffffa8380025fab0
[Fri Jul 26 07:41:30 2019] Oops: 0000 [#16] SMP PTI
[Fri Jul 26 07:41:30 2019] FS:  00007f80f0e98d40(0000) GS:ffff8c7ad2480000(0000) knlGS:0000000000000000
[Fri Jul 26 07:41:30 2019] CPU: 3 PID: 453 Comm: systemd-udevd Tainted: G      D           5.3.0-rc1-6-amd64-cbl-asmgoto #6~buster+dileks1
[Fri Jul 26 07:41:30 2019] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[Fri Jul 26 07:41:30 2019] Hardware name: LENOVO 20HDCTO1WW/20HDCTO1WW, BIOS N1QET83W (1.58 ) 04/18/2019
[Fri Jul 26 07:41:30 2019] CR2: ffffffffbac03370 CR3: 00000004494ce005 CR4: 00000000003606e0
[Fri Jul 26 07:41:30 2019] RIP: 0010:___bpf_prog_run+0x40/0x14f0
[Fri Jul 26 07:41:30 2019] Code: f3 eb 24 48 83 f8 38 0f 84 a9 0c 00 00 48 83 f8 39 0f 85 8a 14 00 00 0f 1f 00 48 0f bf 43 02 48 8d 1c c3 48 83 c3 08 0f b6 33 <48> 8b 04 f5 10 2e c0 ba 48 83 f8 3b 7f 62 48 83 f8 1e 0f 8f c8 00
[Fri Jul 26 07:41:30 2019] RSP: 0018:ffffa83800423a88 EFLAGS: 00010246
[Fri Jul 26 07:41:30 2019] RAX: ffffa83800423b30 RBX: ffffa838000d1038 RCX: 0000000000000000
[Fri Jul 26 07:41:30 2019] RDX: ffffa83800423b10 RSI: 00000000000000ac RDI: ffffa83800423ab0
[Fri Jul 26 07:41:30 2019] RBP: ffffa83800423aa0 R08: ffff8c7ac496b800 R09: 0000000000000000
[Fri Jul 26 07:41:30 2019] R10: ffff8c7acb3ff500 R11: ffffffffba1b5de0 R12: 0000000000000000
[Fri Jul 26 07:41:30 2019] R13: ffffa838000d1000 R14: 0000000000000000 R15: ffffa83800423ab0
[Fri Jul 26 07:41:30 2019] FS:  00007f80f0e98d40(0000) GS:ffff8c7ad2580000(0000) knlGS:0000000000000000
[Fri Jul 26 07:41:30 2019] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[Fri Jul 26 07:41:30 2019] CR2: ffffffffbac03370 CR3: 000000044673c005 CR4: 00000000003606e0
[Fri Jul 26 07:41:30 2019] Call Trace:
[Fri Jul 26 07:41:30 2019]  __bpf_prog_run32+0x44/0x70
[Fri Jul 26 07:41:30 2019]  ? security_sock_rcv_skb+0x3f/0x60
[Fri Jul 26 07:41:30 2019]  sk_filter_trim_cap+0xe4/0x220
[Fri Jul 26 07:41:30 2019]  ? __skb_clone+0x2e/0x100
[Fri Jul 26 07:41:30 2019]  netlink_broadcast_filtered+0x2df/0x4f0
[Fri Jul 26 07:41:30 2019]  netlink_sendmsg+0x34f/0x3c0
[Fri Jul 26 07:41:30 2019]  ___sys_sendmsg+0x315/0x330
[Fri Jul 26 07:41:30 2019]  ? alloc_set_pte+0x17f/0x650
[Fri Jul 26 07:41:30 2019]  ? filemap_map_pages+0xa2/0x470
[Fri Jul 26 07:41:30 2019]  ? do_read_fault+0x104/0x2a0
[Fri Jul 26 07:41:30 2019]  ? handle_mm_fault+0x768/0xbf0
[Fri Jul 26 07:41:30 2019]  __x64_sys_sendmsg+0x97/0xe0
[Fri Jul 26 07:41:30 2019]  do_syscall_64+0x59/0x90
[Fri Jul 26 07:41:30 2019]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[Fri Jul 26 07:41:30 2019] RIP: 0033:0x7f80f1689914
[Fri Jul 26 07:41:30 2019] Code: 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b5 0f 1f 80 00 00 00 00 48 8d 05 e9 5d 0c 00 8b 00 85 c0 75 13 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 54 c3 0f 1f 00 41 54 41 89 d4 55 48 89 f5 53
[Fri Jul 26 07:41:30 2019] RSP: 002b:00007fffc5743008 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
[Fri Jul 26 07:41:30 2019] RAX: ffffffffffffffda RBX: 000055a1947c0360 RCX: 00007f80f1689914
[Fri Jul 26 07:41:30 2019] RDX: 0000000000000000 RSI: 00007fffc5743030 RDI: 000000000000000e
[Fri Jul 26 07:41:30 2019] RBP: 000055a1947c0820 R08: 000000000000000f R09: 000055a1947758f0
[Fri Jul 26 07:41:30 2019] R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
[Fri Jul 26 07:41:30 2019] R13: 0000000000000000 R14: 000000000000008a R15: 00007fffc5743020
[Fri Jul 26 07:41:30 2019] Modules linked in: i2c_dev parport_pc nfsd ppdev auth_rpcgss nfs_acl lp lockd parport grace sunrpc efivarfs ip_tables x_tables autofs4 ext4 crc32c_generic mbcache crc16 jbd2 btrfs zstd_decompress zstd_compress algif_skcipher af_alg sd_mod dm_crypt dm_mod raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 uas raid0 usb_storage multipath linear scsi_mod md_mod hid_cherry hid_generic usbhid hid crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel xhci_pci nvme i915 aesni_intel xhci_hcd aes_x86_64 glue_helper crypto_simd i2c_algo_bit cryptd e1000e psmouse drm_kms_helper i2c_i801 nvme_core intel_lpss_pci usbcore intel_lpss drm thermal wmi video button
[Fri Jul 26 07:41:30 2019] CR2: ffffffffbac03370
[Fri Jul 26 07:41:30 2019] ---[ end trace 5096168d3266d949 ]---

Attached are my kernel-config and dmesg-log.

config-5.3.0-rc1-6-amd64-cbl-asmgoto.txt
dmesg_5.3.0-rc1-6-amd64-cbl-asmgoto.txt

@dileks dileks changed the title next-20190723: bpf/seccomp - systemd/journal issue? next-20190723: bpf/seccomp - systemd/journald issue? Jul 26, 2019
@dileks
Copy link
Author

@dileks dileks commented Jul 27, 2019

From [1]:

It sounds like clang miscompiles interpreter.
modprobe test_bpf
should be able to point out which part of interpreter is broken.

BROKEN: test_bpf: #294 BPF_MAXINSNS: Jump, gap, jump, ... jited:0

  • Sedat -

Steps to reproduce:

# sysctl -n net.core.bpf_jit_enable
1

# modprobe -v test_bpf

More details see thread on netdev ML [2].

@nickdesaulniers @jpoimboe
Looking at "bpf: Disable GCC -fgcse optimization for ___bpf_prog_run()" [3]

Can we have something similiar for clang?

[1] https://marc.info/?l=linux-netdev&m=156420984607234&w=2
[2] https://marc.info/?t=156412966200001&r=1&w=2
[3] https://git.kernel.org/linus/3193c0836f203a91bef96d88c64cccf0be090d9c

@dileks
Copy link
Author

@dileks dileks commented Jul 27, 2019

@MaskRay

With clang-9 + ld.bfd I am not seeing this issue.

So playing with clang-9 kbuild-flags does not make many sense to me.

You have an idea where to look at?

From [1]:

I tried with hopping to turn off "global common subexpression elimination":

index 383c87300b0d..92f934a1e9ff 100644
--- a/arch/x86/net/Makefile
+++ b/arch/x86/net/Makefile
@@ -3,6 +3,8 @@
 # Arch-specific network modules
 #

+KBUILD_CFLAGS += -O0
+
 ifeq ($(CONFIG_X86_32),y)
         obj-$(CONFIG_BPF_JIT) += bpf_jit_comp32.o
 else

Still see...

BROKEN: test_bpf: #294 BPF_MAXINSNS: Jump, gap, jump, ... jited:0

[1] https://marc.info/?l=linux-netdev&m=156421540608064&w=2

@jpoimboe
Copy link

@jpoimboe jpoimboe commented Jul 28, 2019

Looking at "bpf: Disable GCC -fgcse optimization for ___bpf_prog_run()" [3]

Can we have something similiar for clang?

Not sure if clang has that optimization, but either way it shouldn't cause a panic. I disabled that option in that function in GCC because it was producing code which objtool couldn't understand.

@dileks
Copy link
Author

@dileks dileks commented Jul 29, 2019

@MaskRay @GeorgiiR

I switched to Linux v5.3-rc2 and the issue still remains.

My clang-9 and ld.lld are from llvm-project.git#release/9.x Git branch:

$ clang-9 --version
(ClangBuiltLinux clang version 9.0.0 (git://github.com/llvm/llvm-project 1634b4bc934d67cb5fa356a925ba8efca2259f12) (based on LLVM 9.0.0)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /home/sdi/src/llvm-toolchain/install/bin

When llinking with ld.bfd I do not see the issue and can boot normally on bar metal.

I did a objdump of vmlinux.o and generated a diff like this:

$ objdump -M intel -d linux.clang9-bfd/vmlinux.o > vmlinux_o_clang9-bfd.txt
$ objdump -M intel -d linux.clang9-lld/vmlinux.o > vmlinux_o_clang9-lld.txt
$ diff -uprN vmlinux_o_clang9-bfd.txt vmlinux_o_clang9-lld.txt > vmlinux_o.diff

This is an example:

 0000000000082600 <bpf_int_jit_compile>:
    82600:      e8 00 00 00 00          call   82605 <bpf_int_jit_compile+0x5>
@@ -148773,9 +152339,16 @@ Disassembly of section .text:
    8431b:      be cc 00 00 00          mov    esi,0xcc
    84320:      5d                      pop    rbp
    84321:      e9 00 00 00 00          jmp    84326 <jit_fill_hole+0x16>
-   84326:      66 90                   xchg   ax,ax
-   84328:      0f 1f 84 00 00 00 00    nop    DWORD PTR [rax+rax*1+0x0]
-   8432f:      00 
+   84326:      cc                      int3   
+   84327:      cc                      int3   
+   84328:      cc                      int3   
+   84329:      cc                      int3   
+   8432a:      cc                      int3   
+   8432b:      cc                      int3   
+   8432c:      cc                      int3   
+   8432d:      cc                      int3   
+   8432e:      cc                      int3   
+   8432f:      cc                      int3

I see a lot of nop and xchg (ld.bfd) lines turned into cc int3 (ld.lld).

I looked through recent changes in lld tree and wanted to run some LLD tests, but was not able to include and run the tests.

I open tc-build issue #42.

[1] ClangBuiltLinux/tc-build#42

@nathanchance
Copy link
Member

@nathanchance nathanchance commented Jul 30, 2019

I'm going to try to reproduce this in a VM as I don't have a local x86 machine to test this on.

I am curious though, did this first start on next-20190723 or does it happen on earlier versions?

@MaskRay
Copy link
Member

@MaskRay MaskRay commented Jul 31, 2019

 0000000000082600 <bpf_int_jit_compile>:
    82600:      e8 00 00 00 00          call   82605 <bpf_int_jit_compile+0x5>
@@ -148773,9 +152339,16 @@ Disassembly of section .text:
    8431b:      be cc 00 00 00          mov    esi,0xcc
    84320:      5d                      pop    rbp
    84321:      e9 00 00 00 00          jmp    84326 <jit_fill_hole+0x16>
-   84326:      66 90                   xchg   ax,ax
-   84328:      0f 1f 84 00 00 00 00    nop    DWORD PTR [rax+rax*1+0x0]
-   8432f:      00 
+   84326:      cc                      int3   
+   84327:      cc                      int3   
+   84328:      cc                      int3   
+   84329:      cc                      int3   
+   8432a:      cc                      int3   
+   8432b:      cc                      int3   
+   8432c:      cc                      int3   
+   8432d:      cc                      int3   
+   8432e:      cc                      int3   
+   8432f:      cc                      int3

In a SHF_EXECINSTR output section, if there is a gap between two adjacent input sections. ld.bfd fills in the gap with nops while lld uses trap instructions (0xcc on x86):

// binutils-gdb/bfd/cpu-i386.c`:`bfd_arch_i386_fill`
  /* xchg %ax,%ax */
  static const char nop_2[] = { 0x66, 0x90 };

  /* nopl 0L(%[re]ax,%[re]ax,1) */
  static const char nop_8[] =
    { 0x0f, 0x1f, 0x84, 0x00, 0x00, 0x00, 0x00, 0x00};

Does the bfd code somehow assume the gaps consist of nops?

Can you upload linux.clang9-{bfd,lld}/vmlinux.o somewhere or make it easier to reproduce 🙂?

@dileks
Copy link
Author

@dileks dileks commented Jul 31, 2019

@dileks
Copy link
Author

@dileks dileks commented Jul 31, 2019

@MaskRay @nathanchance

This is with Linux v5.3-rc2 and this two patches from @jpoimboe and @arndb:

drm/i915: Remove redundant user_access_end() from __copy_from_user() error path
xen/trace: avoid clang warning on function pointers

LLVM toolchain versions:

[ clang9-bfd ]

clang-9: 1634b4bc934d67cb5fa356a925ba8efca2259f12 (from llvm-project.git#release/9.x Git branch)
ld.bfd:  2.31.1-16 (from binutils Debian/buster aka version 10 package)
ld.lld:  see clang-9

Looked into the archived build-dirs:

$ cd linux.clang9-bfd
$ readelf --string-dump .comment vmlinux

String dump of section '.comment':
  [     0]  ClangBuiltLinux clang version 9.0.0 (git://github.com/llvm/llvm-project 1634b4bc934d67cb5fa356a925ba8efca2259f12) (based on LLVM 9.0.0)


$ cd linux.clang9-lld
$ readelf --string-dump .comment vmlinux

String dump of section '.comment':
  [     0]  ClangBuiltLinux clang version 9.0.0 (git://github.com/llvm/llvm-project 1634b4bc934d67cb5fa356a925ba8efca2259f12) (based on LLVM 9.0.0)
  [    88]  Linker: LLD 9.0.0 (git://github.com/llvm/llvm-project 1634b4bc934d67cb5fa356a925ba8efca2259f12)

UPDATE-1: Both clang-9 versions are identical.
UPDATE-2: I upgraded my llvm-toolchain to version v9.0.0-rc1 - nope.

[ Attachments ]

MD5SUM.txt
vmlinux_o_clang9-bfd.txt.gz
vmlinux_o_clang9-lld.txt.gz

@dileks dileks changed the title next-20190723: bpf/seccomp - systemd/journald issue? Linux v5.3-rc2 and next-20190723: bpf/seccomp - systemd/journald issue? Jul 31, 2019
@dileks
Copy link
Author

@dileks dileks commented Jul 31, 2019

This is the first time I hoped it does not work :-).

I took the archived linux-5.1.y kernel-config and did:

$ scripts/config -d DRM_AMDGPU

I can boot on bare metal:

$ cat /proc/version 
Linux version 5.1.18-1-amd64-cbl-asmgoto (sedat.dilek@gmail.com@iniza) (ClangBuiltLinux clang version 9.0.0 (git://github.com/llvm/llvm-project 6aa75a25bdeea9cdc4b04cdd91e82e680444bf4b) (based on LLVM 9.0.0)) #1~buster+dileks1 SMP 2019-07-31

$ readelf --string-dump .comment vmlinux

String dump of section '.comment':
  [     0]  ClangBuiltLinux clang version 9.0.0 (git://github.com/llvm/llvm-project 6aa75a25bdeea9cdc4b04cdd91e82e680444bf4b) (based on LLVM 9.0.0)
  [    89]  Linker: LLD 9.0.0 (git://github.com/llvm/llvm-project 6aa75a25bdeea9cdc4b04cdd91e82e680444bf4b)

Loading module test_bpf is OK:

# modprobe -v test_bpf 
insmod /lib/modules/5.1.18-1-amd64-cbl-asmgoto/kernel/lib/test_bpf.ko

I will try latest Linux v5.2.5 and report later.

[ Attachments ]

config-5.1.18-1-amd64-cbl-asmgoto.txt
dmesg_5.1.18-1-amd64-cbl-asmgoto_modprobe-test_bpf.txt

@dileks
Copy link
Author

@dileks dileks commented Aug 1, 2019

All good with Linux v5.2.5.

@dileks
Copy link
Author

@dileks dileks commented Aug 1, 2019

I did a very time intensive git-bisect:

$ git bisect log
git bisect start
# good: [2519374d2a6b8aa5d395393f21e74232409c2e82] Linux 5.2.5
git bisect good 2519374d2a6b8aa5d395393f21e74232409c2e82
# bad: [609488bc979f99f805f34e9a32c1e3b71179d10b] Linux 5.3-rc2
git bisect bad 609488bc979f99f805f34e9a32c1e3b71179d10b
# good: [0ecfebd2b52404ae0c54a878c872bb93363ada36] Linux 5.2
git bisect good 0ecfebd2b52404ae0c54a878c872bb93363ada36
# good: [17a20acaf171124017f43bc70bb4d7ca88070659] Merge tag 'usb-5.3-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/usb
git bisect good 17a20acaf171124017f43bc70bb4d7ca88070659
# good: [8de262531f5fbb7458463224a7587429800c24bf] Merge tag 'mfd-next-5.3' of git://git.kernel.org/pub/scm/linux/kernel/git/lee/mfd
git bisect good 8de262531f5fbb7458463224a7587429800c24bf
# good: [5f4fc6d440d77a2cf74fe4ea56955674ac7e35e7] Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net
git bisect good 5f4fc6d440d77a2cf74fe4ea56955674ac7e35e7
# good: [af6af87d7e4ff67324425daa699b9cda32e3161d] Merge tag 'armsoc-dt' of git://git.kernel.org/pub/scm/linux/kernel/git/soc/soc
git bisect good af6af87d7e4ff67324425daa699b9cda32e3161d
# bad: [44b912cd0b55777796c5ae8ae857bd1d5ff83ed5] Merge tag 'for-linus-20190722' of git://git.kernel.org/pub/scm/linux/kernel/git/brauner/linux
git bisect bad 44b912cd0b55777796c5ae8ae857bd1d5ff83ed5
# bad: [e6023adc5c6af79ac8ac5b17939f58091fa0d870] Merge branch 'core-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
git bisect bad e6023adc5c6af79ac8ac5b17939f58091fa0d870
# good: [168c79971b4a7be7011e73bf488b740a8e1135c8] Merge tag 'kbuild-v5.3-2' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild
git bisect good 168c79971b4a7be7011e73bf488b740a8e1135c8
# good: [07ab9d5bc53d7fe84047be1d403566123ab9cfaa] Merge tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm
git bisect good 07ab9d5bc53d7fe84047be1d403566123ab9cfaa
# bad: [3193c0836f203a91bef96d88c64cccf0be090d9c] bpf: Disable GCC -fgcse optimization for ___bpf_prog_run()
git bisect bad 3193c0836f203a91bef96d88c64cccf0be090d9c
# bad: [d99a6ce70ec6ed990b74bd4e34232fd830d20d27] x86/kvm: Fix fastop function ELF metadata
git bisect bad d99a6ce70ec6ed990b74bd4e34232fd830d20d27
# good: [cac9b9a4b08304f11daace03b8b48659355e44c1] stacktrace: Force USER_DS for stack_trace_save_user()
git bisect good cac9b9a4b08304f11daace03b8b48659355e44c1
# bad: [e55a73251da335873a6e87d68fb17e5aabb8978e] bpf: Fix ORC unwinding in non-JIT BPF code
git bisect bad e55a73251da335873a6e87d68fb17e5aabb8978e
# good: [87b512def792579641499d9bef1d640994ea9c18] objtool: Add support for C jump tables
git bisect good 87b512def792579641499d9bef1d640994ea9c18
# first bad commit: [e55a73251da335873a6e87d68fb17e5aabb8978e] bpf: Fix ORC unwinding in non-JIT BPF code

@jpoimboe

Can you look at this?

e55a73251da335873a6e87d68fb17e5aabb8978e is the first bad commit
commit e55a73251da335873a6e87d68fb17e5aabb8978e
Author: Josh Poimboeuf <jpoimboe@redhat.com>
Date:   Thu Jun 27 20:50:47 2019 -0500

    bpf: Fix ORC unwinding in non-JIT BPF code
    
    Objtool previously ignored ___bpf_prog_run() because it didn't understand
    the jump table.  This resulted in the ORC unwinder not being able to unwind
    through non-JIT BPF code.
    
    Now that objtool knows how to read jump tables, remove the whitelist and
    annotate the jump table so objtool can recognize it.
    
    Also add an additional "const" to the jump table definition to clarify that
    the text pointers are constant.  Otherwise GCC sets the section writable
    flag and the assembler spits out warnings.
    
    Fixes: d15d356887e7 ("perf/x86: Make perf callchains work without CONFIG_FRAME_POINTER")
    Reported-by: Song Liu <songliubraving@fb.com>
    Signed-off-by: Josh Poimboeuf <jpoimboe@redhat.com>
    Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
    Acked-by: Alexei Starovoitov <ast@kernel.org>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Kairui Song <kasong@redhat.com>
    Cc: Steven Rostedt <rostedt@goodmis.org>
    Cc: Borislav Petkov <bp@alien8.de>
    Cc: Daniel Borkmann <daniel@iogearbox.net>
    Link: https://lkml.kernel.org/r/881939122b88f32be4c374d248c09d7527a87e35.1561685471.git.jpoimboe@redhat.com
    Signed-off-by: Ingo Molnar <mingo@kernel.org>

:040000 040000 4735e9d14fa416c1c361ec3923440a3d586a627d 31de80b85c7b0292e47a719ecb6b1a451de2f8ef M      kernel
@dileks
Copy link
Author

@dileks dileks commented Aug 1, 2019

@dileks
Copy link
Author

@dileks dileks commented Aug 1, 2019

@jpoimboe @MaskRay

Attached is the object file with reverted:

commit e55a732 "bpf: Fix ORC unwinding in non-JIT BPF code"

bpf-core_clang9-lld_reverted-e55a73251da3.o.gz

@dileks
Copy link
Author

@dileks dileks commented Aug 1, 2019

After reverting the above culprit commit I can boot into Linux v5.3-rc2 with clang-9and lld with no issues.

@dileks dileks changed the title Linux v5.3-rc2 and next-20190723: bpf/seccomp - systemd/journald issue? Linux v5.3-rc1+: bpf issue when linking with lld? Aug 5, 2019
@dileks dileks changed the title Linux v5.3-rc1+: bpf issue when linking with lld? Linux v5.3-rc1+: bpf issue when linking with lld Aug 5, 2019
@nickdesaulniers
Copy link
Member

@nickdesaulniers nickdesaulniers commented Aug 7, 2019

cc @jpoimboe .

Thanks for reporting and finding the offending commit @dileks .

Otherwise GCC sets the section writable flag and the assembler spits out warnings.

So I'd be curious which section this symbol should be in, then we can check what's the difference between GCC and Clang.

@dileks
Copy link
Author

@dileks dileks commented Aug 9, 2019

@nickdesaulniers @jpoimboe

From whom/where is this quote from?

To clarify:
Identical clang-9 snapshot version.
Changing from BFD to LLD shows this issue.

I am not saying that clang-9 is mis- or not mis-compiling.

The diff from culprit commit e55a732:

--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -1299,7 +1299,7 @@ static u64 ___bpf_prog_run(u64 *regs, const struct bpf_insn *insn, u64 *stack)
 {
 #define BPF_INSN_2_LBL(x, y)    [BPF_##x | BPF_##y] = &&x##_##y
 #define BPF_INSN_3_LBL(x, y, z) [BPF_##x | BPF_##y | BPF_##z] = &&x##_##y##_##z
-       static const void *jumptable[256] = {
+       static const void * const jumptable[256] __annotate_jump_table = {
                [0 ... 255] = &&default_label,
                /* Now overwrite non-defaults ... */
                BPF_INSN_MAP(BPF_INSN_2_LBL, BPF_INSN_3_LBL),
@@ -1558,7 +1558,6 @@ static u64 ___bpf_prog_run(u64 *regs, const struct bpf_insn *insn, u64 *stack)
                BUG_ON(1);
                return 0;
 }
-STACK_FRAME_NON_STANDARD(___bpf_prog_run); /* jump table */
 
 #define PROG_NAME(stack_size) __bpf_prog_run##stack_size
 #define DEFINE_BPF_PROG_RUN(stack_size) \

The last thing I checked was to see if .rodata..c_jump_table section exists when linking with LLD.

Here is __annotate_jump_table defined (I have set CONFIG_STACK_VALIDATION=y) :

[ include/linux/compiler.h ]

/* Unreachable code */
#ifdef CONFIG_STACK_VALIDATION
/*
 * These macros help objtool understand GCC code flow for unreachable code.
 * The __COUNTER__ based labels are a hack to make each instance of the macros
 * unique, to convince GCC not to merge duplicate inline asm statements.
 */
#define annotate_reachable() ({                                         \
        asm volatile("%c0:\n\t"                                         \
                     ".pushsection .discard.reachable\n\t"              \
                     ".long %c0b - .\n\t"                               \
                     ".popsection\n\t" : : "i" (__COUNTER__));          \
})
#define annotate_unreachable() ({                                       \
        asm volatile("%c0:\n\t"                                         \
                     ".pushsection .discard.unreachable\n\t"            \
                     ".long %c0b - .\n\t"                               \
                     ".popsection\n\t" : : "i" (__COUNTER__));          \
})
#define ASM_UNREACHABLE                                                 \
        "999:\n\t"                                                      \
        ".pushsection .discard.unreachable\n\t"                         \
        ".long 999b - .\n\t"                                            \
        ".popsection\n\t"

/* Annotate a C jump table to allow objtool to follow the code flow */
#define __annotate_jump_table __section(".rodata..c_jump_table") <--- XXX: See here

#else
#define annotate_reachable()
#define annotate_unreachable()
#define __annotate_jump_table
#endif

#ifndef ASM_UNREACHABLE
# define ASM_UNREACHABLE
#endif
#ifndef unreachable
# define unreachable() do {             \
        annotate_unreachable();         \
        __builtin_unreachable();        \
} while (0)
#endif <--- XXX: CONFIG_STACK_VALIDATION

Blame blame blame:

$ git blame include/linux/compiler.h | grep annotate_jump_table
87b512def7925 (Josh Poimboeuf          2019-06-27 20:50:46 -0500 121) #define __annotate_jump_table __section(".rodata..c_jump_table")
87b512def7925 (Josh Poimboeuf          2019-06-27 20:50:46 -0500 126) #define __annotate_jump_table

$ git log --oneline 87b512def7925 -1
87b512def792 (refs/bisect/good-87b512def792579641499d9bef1d640994ea9c18) objtool: Add support for C jump tables

Checking for this section in object-files:

$ ll bpf-core_clang9-**_o.txt
-rw-r--r-- 1 sdi sdi 6097712 Aug  7 00:45 bpf-core_clang9-bfd_o.txt
-rw-r--r-- 1 sdi sdi 6098622 Aug  7 00:46 bpf-core_clang9-lld_o.txt
-rw-r--r-- 1 sdi sdi 6067542 Aug  7 00:50 bpf-core_clang9-lld_reverted-e55a73251da3_o.txt

$ grep '.rodata..c_jump_table' bpf-core_clang9-**_o.txt
bpf-core_clang9-bfd_o.txt:Disassembly of section ".rodata..c_jump_table":
bpf-core_clang9-lld_o.txt:Disassembly of section ".rodata..c_jump_table":

Maybe I am looking at the wrong files or inspecting with the wrong tools and its options?
Maybe some post-processing eliminates sections?

BTW, I would like to extract a specific section from beginning to its end out of an object-file.

Peter Z. gave me this hint when checking for del_timer_sync in vmlinux:

objdump -D vmlinux | awk '/<[^>]*>:$/ { p=0; } /<del_timer_sync>:/ { p=1; } { if (p) print $0; }'

I would like to have this for .rodata..c_jump_table.

@dileks
Copy link
Author

@dileks dileks commented Aug 9, 2019

Looking at -fjump-tables/-fno-jump-tables CFLAGS:

[ arch/x86/Makefile ]

# Avoid indirect branches in kernel to deal with Spectre
ifdef CONFIG_RETPOLINE
  KBUILD_CFLAGS += $(RETPOLINE_CFLAGS)
  # Additionally, avoid generating expensive indirect jumps which
  # are subject to retpolines for small number of switch cases.
  # clang turns off jump table generation by default when under
  # retpoline builds, however, gcc does not for x86. This has
  # only been fixed starting from gcc stable version 8.4.0 and
  # onwards, but not for older ones. See gcc bug #86952.
  ifndef CONFIG_CC_IS_CLANG
    KBUILD_CFLAGS += $(call cc-option,-fno-jump-tables)
  endif
endif

For GCC yes, CLANG no.

Checking docs BPF and CLANG and -fno-jump-tables...

[ Documentation/bpf/bpf_devel_QA.rst ]

Q: In some cases clang flag ``-target bpf`` is used but in other cases the
default clang target, which matches the underlying architecture, is used.
What is the difference and when I should use which?

A: Although LLVM IR generation and optimization try to stay architecture
independent, ``-target <arch>`` still has some impact on generated code:
...
- The default target may turn a C switch statement into a switch table
  lookup and jump operation. Since the switch table is placed
  in the global readonly section, the bpf program will fail to load.
  The bpf target does not support switch table optimization.
  The clang option ``-fno-jump-tables`` can be used to disable
  switch table generation.

...thus I tried...

--- a/kernel/bpf/Makefile
+++ b/kernel/bpf/Makefile
@@ -1,5 +1,6 @@
 # SPDX-License-Identifier: GPL-2.0
 obj-y := core.o
+CFLAGS_core.o += $(call cc-option,-fno-jump-tables)
 CFLAGS_core.o += $(call cc-disable-warning, override-init)
 
 obj-$(CONFIG_BPF_SYSCALL) += syscall.o verifier.o inode.o helpers.o tnum.o

NOPE.

@jpoimboe
Copy link

@jpoimboe jpoimboe commented Aug 9, 2019

So I'd be curious which section this symbol should be in, then we can check what's the difference between GCC and Clang.

The BPF jump table is placed in .rodata, IIRC. Sorry I don't have time to look deeper into this issue at the moment.

@jpoimboe
Copy link

@jpoimboe jpoimboe commented Aug 9, 2019

The BPF jump table is placed in .rodata, IIRC. Sorry I don't have time to look deeper into this issue at the moment.

Sorry, should be ".rodata..c_jump_table" as @dileks mentioned. @dileks you can use readelf -a --wide core.o (or on vmlinux.o) to see all the sections (and a lot more interesting ELF data).

ojeda added a commit to ojeda/linux that referenced this issue Aug 29, 2019
GCC unescapes escaped string section names while Clang does not. Because
__section uses the `#` stringification operator for the section name, it
doesn't need to be escaped.

Instead, we should:
1. Prefer __section(.section_name_no_quotes).
2. Only use __attribute__((__section__(".section"))) when creating the
section name via C preprocessor (see the definition of __define_initcall
in arch/um/include/shared/init.h).

This antipattern was found with:
$ grep -e __section\(\" -e __section__\(\" -r

See the discussions in:
Link: https://bugs.llvm.org/show_bug.cgi?id=42950
Link: https://marc.info/?l=linux-netdev&m=156412960619946&w=2
Link: ClangBuiltLinux#619
Reported-by: Sedat Dilek <sedat.dilek@gmail.com>
Suggested-by: Josh Poimboeuf <jpoimboe@redhat.com>
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
Signed-off-by: Miguel Ojeda <miguel.ojeda.sandonis@gmail.com>
ojeda added a commit to ojeda/linux that referenced this issue Aug 29, 2019
GCC unescapes escaped string section names while Clang does not. Because
__section uses the `#` stringification operator for the section name, it
doesn't need to be escaped.

Instead, we should:
1. Prefer __section(.section_name_no_quotes).
2. Only use __attribute__((__section__(".section"))) when creating the
section name via C preprocessor (see the definition of __define_initcall
in arch/um/include/shared/init.h).

This antipattern was found with:
$ grep -e __section\(\" -e __section__\(\" -r

See the discussions in:
Link: https://bugs.llvm.org/show_bug.cgi?id=42950
Link: https://marc.info/?l=linux-netdev&m=156412960619946&w=2
Link: ClangBuiltLinux#619
Acked-by: Paul Burton <paul.burton@mips.com>
Reported-by: Sedat Dilek <sedat.dilek@gmail.com>
Suggested-by: Josh Poimboeuf <jpoimboe@redhat.com>
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
Signed-off-by: Miguel Ojeda <miguel.ojeda.sandonis@gmail.com>
ojeda added a commit to ojeda/linux that referenced this issue Aug 29, 2019
GCC unescapes escaped string section names while Clang does not. Because
__section uses the `#` stringification operator for the section name, it
doesn't need to be escaped.

Instead, we should:
1. Prefer __section(.section_name_no_quotes).
2. Only use __attribute__((__section__(".section"))) when creating the
section name via C preprocessor (see the definition of __define_initcall
in arch/um/include/shared/init.h).

This antipattern was found with:
$ grep -e __section\(\" -e __section__\(\" -r

See the discussions in:
Link: https://bugs.llvm.org/show_bug.cgi?id=42950
Link: https://marc.info/?l=linux-netdev&m=156412960619946&w=2
Link: ClangBuiltLinux#619
Acked-by: David S. Miller <davem@davemloft.net>
Reported-by: Sedat Dilek <sedat.dilek@gmail.com>
Suggested-by: Josh Poimboeuf <jpoimboe@redhat.com>
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
Signed-off-by: Miguel Ojeda <miguel.ojeda.sandonis@gmail.com>
ojeda added a commit to ojeda/linux that referenced this issue Aug 29, 2019
GCC unescapes escaped string section names while Clang does not. Because
__section uses the `#` stringification operator for the section name, it
doesn't need to be escaped.

Instead, we should:
1. Prefer __section(.section_name_no_quotes).
2. Only use __attribute__((__section__(".section"))) when creating the
section name via C preprocessor (see the definition of __define_initcall
in arch/um/include/shared/init.h).

This antipattern was found with:
$ grep -e __section\(\" -e __section__\(\" -r

See the discussions in:
Link: https://bugs.llvm.org/show_bug.cgi?id=42950
Link: https://marc.info/?l=linux-netdev&m=156412960619946&w=2
Link: ClangBuiltLinux#619
Reported-by: Sedat Dilek <sedat.dilek@gmail.com>
Suggested-by: Josh Poimboeuf <jpoimboe@redhat.com>
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
Signed-off-by: Miguel Ojeda <miguel.ojeda.sandonis@gmail.com>
ojeda added a commit to ojeda/linux that referenced this issue Aug 29, 2019
…ributes.h

GCC unescapes escaped string section names while Clang does not. Because
__section uses the `#` stringification operator for the section name, it
doesn't need to be escaped.

Instead, we should:
1. Prefer __section(.section_name_no_quotes).
2. Only use __attribute__((__section__(".section"))) when creating the
section name via C preprocessor (see the definition of __define_initcall
in arch/um/include/shared/init.h).

This antipattern was found with:
$ grep -e __section\(\" -e __section__\(\" -r

See the discussions in:
Link: https://bugs.llvm.org/show_bug.cgi?id=42950
Link: https://marc.info/?l=linux-netdev&m=156412960619946&w=2
Link: ClangBuiltLinux#619
Reported-by: Sedat Dilek <sedat.dilek@gmail.com>
Suggested-by: Josh Poimboeuf <jpoimboe@redhat.com>
Tested-by: Sedat Dilek <sedat.dilek@gmail.com>
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
[Added back "unused" as __maybe_unused]
[Changed title to reflect the other changes]
Signed-off-by: Miguel Ojeda <miguel.ojeda.sandonis@gmail.com>
ojeda added a commit to ojeda/linux that referenced this issue Aug 29, 2019
GCC unescapes escaped string section names while Clang does not. Because
__section uses the `#` stringification operator for the section name, it
doesn't need to be escaped.

Instead, we should:
1. Prefer __section(.section_name_no_quotes).
2. Only use __attribute__((__section__(".section"))) when creating the
section name via C preprocessor (see the definition of __define_initcall
in arch/um/include/shared/init.h).

This antipattern was found with:
$ grep -e __section\(\" -e __section__\(\" -r

See the discussions in:
Link: https://bugs.llvm.org/show_bug.cgi?id=42950
Link: https://marc.info/?l=linux-netdev&m=156412960619946&w=2
Link: ClangBuiltLinux#619
Acked-by: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
Reported-by: Sedat Dilek <sedat.dilek@gmail.com>
Suggested-by: Josh Poimboeuf <jpoimboe@redhat.com>
Tested-by: Sedat Dilek <sedat.dilek@gmail.com>
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
Signed-off-by: Miguel Ojeda <miguel.ojeda.sandonis@gmail.com>
ojeda added a commit to ojeda/linux that referenced this issue Aug 29, 2019
GCC unescapes escaped string section names while Clang does not. Because
__section uses the `#` stringification operator for the section name, it
doesn't need to be escaped.

Instead, we should:
1. Prefer __section(.section_name_no_quotes).
2. Only use __attribute__((__section__(".section"))) when creating the
section name via C preprocessor (see the definition of __define_initcall
in arch/um/include/shared/init.h).

This antipattern was found with:
$ grep -e __section\(\" -e __section__\(\" -r

See the discussions in:
Link: https://bugs.llvm.org/show_bug.cgi?id=42950
Link: https://marc.info/?l=linux-netdev&m=156412960619946&w=2
Link: ClangBuiltLinux#619
Acked-by: Will Deacon <will@kernel.org>
Reported-by: Sedat Dilek <sedat.dilek@gmail.com>
Suggested-by: Josh Poimboeuf <jpoimboe@redhat.com>
Tested-by: Sedat Dilek <sedat.dilek@gmail.com>
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
[Changed title to reflect the other changes]
Signed-off-by: Miguel Ojeda <miguel.ojeda.sandonis@gmail.com>
ojeda added a commit to ojeda/linux that referenced this issue Aug 29, 2019
GCC unescapes escaped string section names while Clang does not. Because
__section uses the `#` stringification operator for the section name, it
doesn't need to be escaped.

Instead, we should:
1. Prefer __section(.section_name_no_quotes).
2. Only use __attribute__((__section__(".section"))) when creating the
section name via C preprocessor (see the definition of __define_initcall
in arch/um/include/shared/init.h).

This antipattern was found with:
$ grep -e __section\(\" -e __section__\(\" -r

See the discussions in:
Link: https://bugs.llvm.org/show_bug.cgi?id=42950
Link: https://marc.info/?l=linux-netdev&m=156412960619946&w=2
Link: ClangBuiltLinux#619
Tested-by: Sedat Dilek <sedat.dilek@gmail.com>
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
[Improve a bit the formatting of the text to make it consistent]
Signed-off-by: Miguel Ojeda <miguel.ojeda.sandonis@gmail.com>
ojeda added a commit to ojeda/linux that referenced this issue Aug 29, 2019
…ributes.h

GCC unescapes escaped string section names while Clang does not. Because
__section uses the `#` stringification operator for the section name, it
doesn't need to be escaped.

Instead, we should:
1. Prefer __section(.section_name_no_quotes).
2. Only use __attribute__((__section__(".section"))) when creating the
section name via C preprocessor (see the definition of __define_initcall
in arch/um/include/shared/init.h).

This antipattern was found with:
$ grep -e __section\(\" -e __section__\(\" -r

See the discussions in:
Link: https://bugs.llvm.org/show_bug.cgi?id=42950
Link: https://marc.info/?l=linux-netdev&m=156412960619946&w=2
Link: ClangBuiltLinux#619
Reported-by: Sedat Dilek <sedat.dilek@gmail.com>
Suggested-by: Josh Poimboeuf <jpoimboe@redhat.com>
Tested-by: Sedat Dilek <sedat.dilek@gmail.com>
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
[Added back "unused" as __maybe_unused]
[Changed title to reflect the other changes]
Signed-off-by: Miguel Ojeda <miguel.ojeda.sandonis@gmail.com>
ojeda added a commit to ojeda/linux that referenced this issue Aug 29, 2019
GCC unescapes escaped string section names while Clang does not. Because
__section uses the `#` stringification operator for the section name, it
doesn't need to be escaped.

Instead, we should:
1. Prefer __section(.section_name_no_quotes).
2. Only use __attribute__((__section__(".section"))) when creating the
section name via C preprocessor (see the definition of __define_initcall
in arch/um/include/shared/init.h).

This antipattern was found with:
$ grep -e __section\(\" -e __section__\(\" -r

See the discussions in:
Link: https://bugs.llvm.org/show_bug.cgi?id=42950
Link: https://marc.info/?l=linux-netdev&m=156412960619946&w=2
Link: ClangBuiltLinux#619
Acked-by: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
Reported-by: Sedat Dilek <sedat.dilek@gmail.com>
Suggested-by: Josh Poimboeuf <jpoimboe@redhat.com>
Tested-by: Sedat Dilek <sedat.dilek@gmail.com>
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
Signed-off-by: Miguel Ojeda <miguel.ojeda.sandonis@gmail.com>
ojeda added a commit to ojeda/linux that referenced this issue Aug 29, 2019
GCC unescapes escaped string section names while Clang does not. Because
__section uses the `#` stringification operator for the section name, it
doesn't need to be escaped.

Instead, we should:
1. Prefer __section(.section_name_no_quotes).
2. Only use __attribute__((__section__(".section"))) when creating the
section name via C preprocessor (see the definition of __define_initcall
in arch/um/include/shared/init.h).

This antipattern was found with:
$ grep -e __section\(\" -e __section__\(\" -r

See the discussions in:
Link: https://bugs.llvm.org/show_bug.cgi?id=42950
Link: https://marc.info/?l=linux-netdev&m=156412960619946&w=2
Link: ClangBuiltLinux#619
Acked-by: Will Deacon <will@kernel.org>
Reported-by: Sedat Dilek <sedat.dilek@gmail.com>
Suggested-by: Josh Poimboeuf <jpoimboe@redhat.com>
Tested-by: Sedat Dilek <sedat.dilek@gmail.com>
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
[Changed title to reflect the other changes]
Signed-off-by: Miguel Ojeda <miguel.ojeda.sandonis@gmail.com>
ojeda added a commit to ojeda/linux that referenced this issue Aug 29, 2019
GCC unescapes escaped string section names while Clang does not. Because
__section uses the `#` stringification operator for the section name, it
doesn't need to be escaped.

Instead, we should:
1. Prefer __section(.section_name_no_quotes).
2. Only use __attribute__((__section__(".section"))) when creating the
section name via C preprocessor (see the definition of __define_initcall
in arch/um/include/shared/init.h).

This antipattern was found with:
$ grep -e __section\(\" -e __section__\(\" -r

See the discussions in:
Link: https://bugs.llvm.org/show_bug.cgi?id=42950
Link: https://marc.info/?l=linux-netdev&m=156412960619946&w=2
Link: ClangBuiltLinux#619
Tested-by: Sedat Dilek <sedat.dilek@gmail.com>
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
[Improve a bit the formatting of the text to make it consistent]
Signed-off-by: Miguel Ojeda <miguel.ojeda.sandonis@gmail.com>
@ojeda
Copy link
Member

@ojeda ojeda commented Sep 4, 2019

PR sent to Linus

torvalds pushed a commit to torvalds/linux that referenced this issue Sep 8, 2019
GCC unescapes escaped string section names while Clang does not. Because
__section uses the `#` stringification operator for the section name, it
doesn't need to be escaped.

This fixes an Oops observed in distro's that use systemd and not
net.core.bpf_jit_enable=1, when their kernels are compiled with Clang.

Link: ClangBuiltLinux#619
Link: https://bugs.llvm.org/show_bug.cgi?id=42950
Link: https://marc.info/?l=linux-netdev&m=156412960619946&w=2
Link: https://lore.kernel.org/lkml/20190904181740.GA19688@gmail.com/
Acked-by: Will Deacon <will@kernel.org>
Reported-by: Sedat Dilek <sedat.dilek@gmail.com>
Suggested-by: Josh Poimboeuf <jpoimboe@redhat.com>
Tested-by: Sedat Dilek <sedat.dilek@gmail.com>
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
[Cherry-picked from the __section cleanup series for 5.3]
[Adjusted commit message]
Signed-off-by: Miguel Ojeda <miguel.ojeda.sandonis@gmail.com>
@ojeda
Copy link
Member

@ojeda ojeda commented Sep 8, 2019

Linus preferred to take only the Oops-fixing commit for 5.3, so I re-sent him another PR which has been just merged: torvalds@983f700

For 5.4 we will send the rest of the series, i.e. the cleanup. For this, we should consider going the opposite way: we can change __section to avoid stringification. This makes it easier for everyone to understand and allows us to use __section for all use cases.

@nickdesaulniers
Copy link
Member

@nickdesaulniers nickdesaulniers commented Sep 9, 2019

I agree. Thanks @ojeda . Closing this issue as @dileks 's observed Oops should now be resolved.

PatrickRudolph added a commit to 9elements/linux that referenced this issue Sep 12, 2019
GCC unescapes escaped string section names while Clang does not. Because
__section uses the `#` stringification operator for the section name, it
doesn't need to be escaped.

This fixes an Oops observed in distro's that use systemd and not
net.core.bpf_jit_enable=1, when their kernels are compiled with Clang.

Link: ClangBuiltLinux#619
Link: https://bugs.llvm.org/show_bug.cgi?id=42950
Link: https://marc.info/?l=linux-netdev&m=156412960619946&w=2
Link: https://lore.kernel.org/lkml/20190904181740.GA19688@gmail.com/
Acked-by: Will Deacon <will@kernel.org>
Reported-by: Sedat Dilek <sedat.dilek@gmail.com>
Suggested-by: Josh Poimboeuf <jpoimboe@redhat.com>
Tested-by: Sedat Dilek <sedat.dilek@gmail.com>
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
[Cherry-picked from the __section cleanup series for 5.3]
[Adjusted commit message]
Signed-off-by: Miguel Ojeda <miguel.ojeda.sandonis@gmail.com>
DanielOgorchock added a commit to DanielOgorchock/linux that referenced this issue Sep 15, 2019
GCC unescapes escaped string section names while Clang does not. Because
__section uses the `#` stringification operator for the section name, it
doesn't need to be escaped.

This fixes an Oops observed in distro's that use systemd and not
net.core.bpf_jit_enable=1, when their kernels are compiled with Clang.

Link: ClangBuiltLinux/linux#619
Link: https://bugs.llvm.org/show_bug.cgi?id=42950
Link: https://marc.info/?l=linux-netdev&m=156412960619946&w=2
Link: https://lore.kernel.org/lkml/20190904181740.GA19688@gmail.com/
Acked-by: Will Deacon <will@kernel.org>
Reported-by: Sedat Dilek <sedat.dilek@gmail.com>
Suggested-by: Josh Poimboeuf <jpoimboe@redhat.com>
Tested-by: Sedat Dilek <sedat.dilek@gmail.com>
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
[Cherry-picked from the __section cleanup series for 5.3]
[Adjusted commit message]
Signed-off-by: Miguel Ojeda <miguel.ojeda.sandonis@gmail.com>
PatrickRudolph added a commit to 9elements/linux that referenced this issue Nov 7, 2019
GCC unescapes escaped string section names while Clang does not. Because
__section uses the `#` stringification operator for the section name, it
doesn't need to be escaped.

This fixes an Oops observed in distro's that use systemd and not
net.core.bpf_jit_enable=1, when their kernels are compiled with Clang.

Link: ClangBuiltLinux#619
Link: https://bugs.llvm.org/show_bug.cgi?id=42950
Link: https://marc.info/?l=linux-netdev&m=156412960619946&w=2
Link: https://lore.kernel.org/lkml/20190904181740.GA19688@gmail.com/
Acked-by: Will Deacon <will@kernel.org>
Reported-by: Sedat Dilek <sedat.dilek@gmail.com>
Suggested-by: Josh Poimboeuf <jpoimboe@redhat.com>
Tested-by: Sedat Dilek <sedat.dilek@gmail.com>
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
[Cherry-picked from the __section cleanup series for 5.3]
[Adjusted commit message]
Signed-off-by: Miguel Ojeda <miguel.ojeda.sandonis@gmail.com>
ACherepkov1989 added a commit to ACherepkov1989/android_kernel_sony_msm8994_kitakami that referenced this issue Aug 31, 2020
GCC unescapes escaped string section names while Clang does not. Because
__section uses the `#` stringification operator for the section name, it
doesn't need to be escaped.

This fixes an Oops observed in distro's that use systemd and not
net.core.bpf_jit_enable=1, when their kernels are compiled with Clang.

Link: ClangBuiltLinux/linux#619
Link: https://bugs.llvm.org/show_bug.cgi?id=42950
Link: https://marc.info/?l=linux-netdev&m=156412960619946&w=2
Link: https://lore.kernel.org/lkml/20190904181740.GA19688@gmail.com/
Acked-by: Will Deacon <will@kernel.org>
Reported-by: Sedat Dilek <sedat.dilek@gmail.com>
Suggested-by: Josh Poimboeuf <jpoimboe@redhat.com>
Tested-by: Sedat Dilek <sedat.dilek@gmail.com>
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
[Cherry-picked from the __section cleanup series for 5.3]
[Adjusted commit message]
Signed-off-by: Miguel Ojeda <miguel.ojeda.sandonis@gmail.com>

[Resolved context conflicts]
ACherepkov1989 added a commit to ACherepkov1989/android_kernel_sony_msm8994_kitakami that referenced this issue Aug 31, 2020
GCC unescapes escaped string section names while Clang does not. Because
__section uses the `#` stringification operator for the section name, it
doesn't need to be escaped.

This fixes an Oops observed in distro's that use systemd and not
net.core.bpf_jit_enable=1, when their kernels are compiled with Clang.

Link: ClangBuiltLinux/linux#619
Link: https://bugs.llvm.org/show_bug.cgi?id=42950
Link: https://marc.info/?l=linux-netdev&m=156412960619946&w=2
Link: https://lore.kernel.org/lkml/20190904181740.GA19688@gmail.com/
Acked-by: Will Deacon <will@kernel.org>
Reported-by: Sedat Dilek <sedat.dilek@gmail.com>
Suggested-by: Josh Poimboeuf <jpoimboe@redhat.com>
Tested-by: Sedat Dilek <sedat.dilek@gmail.com>
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
[Cherry-picked from the __section cleanup series for 5.3]
[Adjusted commit message]
Signed-off-by: Miguel Ojeda <miguel.ojeda.sandonis@gmail.com>

[Resolved context conflicts]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
6 participants