BFD: /android0/linux-next/vmlinux: unknown type [0x13] section `.relr.dyn'

[nickdesaulniers]

from a thread with @broonie , I wasn't able to hit breakpoints in gdb for a kernel under test in QEMU for a mainline aarch64 kernel build with PAC/BTI enabled. Turned out, those were red herrings.

I was also seeing

BFD: /android0/linux-next/vmlinux: unknown type [0x13] section `.relr.dyn'

(not sure why BFD is invoked from GDB)
@pcc mentioned that --use-android-relr-tags may be needed to be added to KBUILD_LDFLAGS for CONFIG_RELR=y.

When I disabled CONFIG_RELR=y, I was able to hit breakpoints again.

[pcc]

I would first of all check whether --use-android-relr-tags fixes breakpoints. If it does, it indicates that GDB doesn't like seeing unrecognized tags in the generic range. If that's the case, you could either fix GDB to recognize these sections, or change the kernel to pass the flag (although it would be strange to be passing an Android-specific flag outside of an Android context).

[nickdesaulniers]

Looks like that flag fixes the warning, and restores the ability to hit breakpoints.

(gdb) hbreak start_kernel
No hardware breakpoint support in the target.

diff --git a/Makefile b/Makefile
index ae5d8220f431..dd54412ada4c 100644
--- a/Makefile
+++ b/Makefile
@@ -966,7 +966,7 @@ LDFLAGS_vmlinux     += $(call ld-option, -X,)
 endif
 
 ifeq ($(CONFIG_RELR),y)
-LDFLAGS_vmlinux        += --pack-dyn-relocs=relr
+LDFLAGS_vmlinux        += --pack-dyn-relocs=relr --use-android-relr-tags
 endif
 
 # Align the bit size of userspace programs with the kernel

(gdb) hbreak start_kernel
Hardware assisted breakpoint 1 at 0xffff800011510790: file init/main.c, line 835.

(although it would be strange to be passing an Android-specific flag outside of an Android context).

What are the implications of setting that flag?

[nickdesaulniers]

eek, stepping produces one of these for each step

(gdb) n
Call Frame Instruction op 45 in vendor extension space is not handled on this architecture.

may be unrelated?

[pcc]

What are the implications of setting that flag?

Nothing significant from the kernel's perspective. It means that lld will use dynamic tags and section type numbers in the OS-specific range rather than the generic range.
http://llvm-cs.pcc.me.uk/tools/lld/ELF/Config.h/ruseAndroidRelrTags

The kernel itself doesn't care about these numbers; it determines the location of the RELR section using symbols defined by a linker script.
https://github.com/torvalds/linux/blob/1b5044021070efa3259f3e9548dc35d1eb6aa844/arch/arm64/kernel/vmlinux.lds.S#L202

may be unrelated?

I think so.

libunwind/src/dwarf2.h: DW_CFA_AARCH64_negate_ra_state = 0x2D

This CFI instruction is associated with PAC, which presumably isn't supported by your version of gdb.

[nickdesaulniers]

If that's the case, you could either fix GDB to recognize these sections, or change the kernel to pass the flag (although it would be strange to be passing an Android-specific flag outside of an Android context).

Didn't you attempt to send binutils patches for RELR? If they were accepted into the kernel, but not binutils, then I think it's more appropriate to add the flag to the kernel.

cc @DanielKristofKiss @kbeyls for the GDB PAC warning.

$ gdb-multiarch --version
GNU gdb (Debian 8.3.1-1) 8.3.1

great job matching the warning to the value!

[pcc]

No, I didn't send anything to binutils. @MaskRay seems to be in the habit of sending patches to binutils, so maybe he could give that a try? I would imagine that it wouldn't need to be a full RELR implementation but just defining the section type constant and teaching GDB not to blow up if it sees that constant.

That being said, I would imagine that even if we did get a change landed in binutils it would be a good idea to add the flag to the kernel in order to allow older versions of gdb to debug the kernel, so maybe we should make the kernel change anyway.

[MaskRay]

I did ask about -z relr but I do not get a response https://sourceware.org/pipermail/binutils/2020-May/111086.html

My impression is that some ChromeOS folks may have a gold RELR patch which isn't upstreamed. Note that --use-android-relr-tags is probably not a good name. There is little chance that binutils will accept an ELF specific option which isn't named -z

ChromeOS folks also have a glibc patch. I hope they can make some efforts upstreaming that as well.

It also does not look like a good idea to me that we permanently use DT_ANDROID_RELR_* instead of DT_RELR_*. See also android/ndk#909 (comment)

[pcc]

Again, I'm just talking about teaching GDB about SHT_RELR, not either of the linkers. That can be done independently of any decisions that might be made about the command line syntax.

[nickdesaulniers]

Ok, so probably if CONFIG_RELR and CONFIG_GDB_SCRIPTS are both set, then this flag should be added.

[nickdesaulniers]

submitted: https://lore.kernel.org/linux-kbuild/20210319000708.1694662-1-ndesaulniers@google.com/T/#u

[nickdesaulniers]

Pretty sure this means that GNU objdump can't be used to disassemble kernels built with llvm.

$ aarch64-linux-gnu-objdump -dr vmlinux
aarch64-linux-gnu-objdump: vmlinux.bad: unknown type [0x13] section `.relr.dyn'

vmlinux.bad:     file format elf64-little

aarch64-linux-gnu-objdump: can't disassemble for architecture UNKNOWN!

[pcc]

In my experience it's possible to work around that issue by passing -m aarch64 to objdump.

[nickdesaulniers]

accepted: https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git/commit/?id=27f2a4db76e8