Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ld.lld: warning: vmlinux.a(drivers/i2c/busses/i2c-i801.o):(.discard.retpoline_safe+0x120): has non-ABS relocation R_386_PC32 against symbol '' #1937

Closed
MaskRay opened this issue Sep 19, 2023 · 7 comments
Assignees
Labels
[ARCH] x86 This bug impacts ARCH=i386 [BUG] linux A bug that should be fixed in the mainline kernel. [TOOL] lld The issue is relevant to LLD linker

Comments

@MaskRay
Copy link
Member

MaskRay commented Sep 19, 2023

I am not familiar with objtool, but it seems that after https://git.kernel.org/linus/1c0c1faf5692c18c127d044ecc0cc92c7bab3477 (objtool: Use relative pointers for annotations) , non-SHF_ALLOC sections like .discard.retpoline_safe may contain PC-relative relocations:

% readelf -Wr arch/x86/kernel/irq_32.o
...
Relocation section '.rel.discard.retpoline_safe' at offset 0x3d8 contains 4 entries:
 Offset     Info    Type                Sym. Value  Symbol's Name
00000000  00000202 R_386_PC32             00000000   .text
00000004  00000302 R_386_PC32             00000000   .altinstr_replacement
00000008  00000202 R_386_PC32             00000000   .text
0000000c  00000302 R_386_PC32             00000000   .altinstr_replacement

In the ELF object file format, non-SHF_ALLOC sections do not occupy memory during program execution. Such sections referencing SHF_ALLOC sections (.text) do not make sense.

Currently LLD report warnings for non-relocatable links and suppresses the warnings for relocatable links.
After llvm/llvm-project#66804, in relocatable links, SHT_REL PC-relative relocations will get warnings as well.
LLD still doesn't report warnings for SHT_RELA PC-relative relocations. LLD doesn't mostly because we don't want to spend more code on the diagnostic.

@nickdesaulniers
Copy link
Member

Further background, after applying llvm/llvm-project#66804
I see a few hundred warnings like:

ld.lld: warning: vmlinux.a(drivers/i2c/busses/i2c-i801.o):(.discard.retpoline_safe+0x120): has non-ABS relocation R_386_PC32 against symbol ''

Thanks for bisecting the kernel to find a culprit. Looking at that commit, I wonder then if perhaps ANNOTATE_RETPOLINE_SAFE, ANNOTATE_NOENDBR, ANNOTATE_INTRA_FUNCTION_CALL, UNWIND_HINT, or STACK_FRAME_NON_STANDARD is being used in a non-SHF_ALLOC section then?

cc @jpoimboe

@nickdesaulniers nickdesaulniers added [BUG] linux A bug that should be fixed in the mainline kernel. [TOOL] lld The issue is relevant to LLD linker [ARCH] x86 This bug impacts ARCH=i386 labels Sep 19, 2023
@MaskRay MaskRay self-assigned this Sep 19, 2023
@nickdesaulniers
Copy link
Member

Or is it that the sections modified in 1c0c1fa that aren't "a" should not have been changed?

@nickdesaulniers nickdesaulniers changed the title "objtool: Use relative pointers for annotations" uses PC-relative relocations in non-SHF_ALLOC sections ld.lld: warning: vmlinux.a(drivers/i2c/busses/i2c-i801.o):(.discard.retpoline_safe+0x120): has non-ABS relocation R_386_PC32 against symbol '' Sep 19, 2023
MaskRay added a commit to MaskRay/linux that referenced this issue Sep 19, 2023
.discard.retpoline_safe sections do not have the SHF_ALLOC flag. These
sections referencing text sections with PC-relative relocations like
R_386_PC32 [0] are conceptually not suitable. Newer LLD will report
warnings even for relocatable links [1].

    ld.lld: warning: vmlinux.a(drivers/i2c/busses/i2c-i801.o):(.discard.retpoline_safe+0x120): has non-ABS relocation R_386_PC32 against symbol ''

Since the annotation is RELA only (therefore i386 is unsupported), we
can utilitize R_*_NONE relocations for annotation, making
.discard.retpoline_safe sections zero-sized.  If we ever support i386
REL relocations, which requires teaching reloc_addend implicit addends,
we can utilitize .long directives again.

[0]: commit 1c0c1fa

Link: ClangBuiltLinux#1937 [1]
MaskRay added a commit to MaskRay/linux that referenced this issue Sep 19, 2023
.discard.retpoline_safe sections do not have the SHF_ALLOC flag. These
sections referencing text sections with PC-relative relocations like
R_386_PC32 [0] are conceptually not suitable. Newer LLD will report
warnings even for relocatable links [1].

    ld.lld: warning: vmlinux.a(drivers/i2c/busses/i2c-i801.o):(.discard.retpoline_safe+0x120): has non-ABS relocation R_386_PC32 against symbol ''

Since the annotation is RELA only (therefore i386 is unsupported), we
can utilitize R_*_NONE relocations for annotation, making
.discard.retpoline_safe sections zero-sized.  If we ever support i386
REL relocations, which requires teaching reloc_addend implicit addends,
we can utilitize .long directives again.

[0]: commit 1c0c1fa

Link: ClangBuiltLinux#1937 [1]
Signed-off-by: Fangrui Song <maskray@google.com>
MaskRay added a commit to MaskRay/linux that referenced this issue Sep 19, 2023
.discard.retpoline_safe sections do not have the SHF_ALLOC flag.  These
sections referencing text sections with PC-relative relocations like
R_386_PC32 [0] are conceptually not suitable.  Newer LLD will report
warnings even for relocatable links [1].

    ld.lld: warning: vmlinux.a(drivers/i2c/busses/i2c-i801.o):(.discard.retpoline_safe+0x120): has non-ABS relocation R_386_PC32 against symbol ''

Since the annotation is RELA only (therefore i386 is unsupported), we
can utilitize R_*_NONE relocations for annotation, making
.discard.retpoline_safe sections zero-sized.  If we ever support i386
REL relocations, which requires teaching reloc_addend implicit addends,
we can utilitize .long directives just for REL.

[0]: commit 1c0c1fa

Link: ClangBuiltLinux#1937 [1]
Signed-off-by: Fangrui Song <maskray@google.com>
MaskRay added a commit to MaskRay/linux that referenced this issue Sep 19, 2023
.discard.retpoline_safe sections do not have the SHF_ALLOC flag.  These
sections referencing text sections with PC-relative relocations like
R_386_PC32 [0] are conceptually not suitable.  Newer LLD will report
warnings even for relocatable links [1].

    ld.lld: warning: vmlinux.a(drivers/i2c/busses/i2c-i801.o):(.discard.retpoline_safe+0x120): has non-ABS relocation R_386_PC32 against symbol ''

Since the annotation is RELA only (therefore i386 is unsupported), we
can utilitize R_*_NONE relocations for annotation, making
.discard.retpoline_safe sections zero-sized.  If we ever support i386
REL relocations, which requires teaching reloc_addend implicit addends,
we can utilitize .long directives just for REL.

[0]: commit 1c0c1fa ("objtool: Use relative pointers for
annotations")

Link: ClangBuiltLinux#1937 [1]
Signed-off-by: Fangrui Song <maskray@google.com>
MaskRay added a commit to MaskRay/linux that referenced this issue Sep 19, 2023
.discard.retpoline_safe sections do not have the SHF_ALLOC flag.  These
sections referencing text sections' STT_SECTION symbols with PC-relative
relocations like R_386_PC32 [0] are conceptually not suitable.  Newer
LLD will report warnings even for relocatable links [1].

    ld.lld: warning: vmlinux.a(drivers/i2c/busses/i2c-i801.o):(.discard.retpoline_safe+0x120): has non-ABS relocation R_386_PC32 against symbol ''

Switch to absolute relocations instead.

Note: if we decide to not support REL architectures (arm, i386), we can
utilize R_*_NONE relocations, making .discard.* sections zero-sized.

[0]: commit 1c0c1fa ("objtool: Use relative pointers for
annotations")

Link: ClangBuiltLinux#1937 [1]
Signed-off-by: Fangrui Song <maskray@google.com>
MaskRay added a commit to MaskRay/linux that referenced this issue Sep 19, 2023
.discard.retpoline_safe sections do not have the SHF_ALLOC flag.  These
sections referencing text sections' STT_SECTION symbols with PC-relative
relocations like R_386_PC32 [0] are conceptually not suitable.  Newer
LLD will report warnings for REL relocations even for relocatable links
[1].

    ld.lld: warning: vmlinux.a(drivers/i2c/busses/i2c-i801.o):(.discard.retpoline_safe+0x120): has non-ABS relocation R_386_PC32 against symbol ''

Switch to absolute relocations instead, which indicate link-time
addresses.  In a relocatable link, these addresses are also output
section offsets.  When linking vmlinux, these .discard.* sections will
be discarded, therefore it is not a problem that R_X86_64_32 cannot
represent a kernel address.

Note: if we decide to never support REL architectures (e.g. arm, i386),
we can utilize R_*_NONE relocations, making .discard.* sections
zero-sized.

[0]: commit 1c0c1fa ("objtool: Use relative pointers for
annotations")

Link: ClangBuiltLinux#1937 [1]
Signed-off-by: Fangrui Song <maskray@google.com>
intel-lab-lkp pushed a commit to intel-lab-lkp/linux that referenced this issue Sep 20, 2023
.discard.retpoline_safe sections do not have the SHF_ALLOC flag.  These
sections referencing text sections' STT_SECTION symbols with PC-relative
relocations like R_386_PC32 [0] is conceptually not suitable.  Newer
LLD will report warnings for REL relocations even for relocatable links
[1].

    ld.lld: warning: vmlinux.a(drivers/i2c/busses/i2c-i801.o):(.discard.retpoline_safe+0x120): has non-ABS relocation R_386_PC32 against symbol ''

Switch to absolute relocations instead, which indicate link-time
addresses.  In a relocatable link, these addresses are also output
section offsets, used by checks in tools/objtool/check.c.  When linking
vmlinux, these .discard.* sections will be discarded, therefore it is
not a problem that R_X86_64_32 cannot represent a kernel address.

Alternatively, we could set the SHF_ALLOC flag for .discard.* sections,
but I think non-SHF_ALLOC for sections to be discarded makes more sense.

Note: if we decide to never support REL architectures (e.g. arm, i386),
we can utilize R_*_NONE relocations (.reloc ., BFD_RELOC_NONE, sym),
making .discard.* sections zero-sized.  That said, the section content
waste is 4 bytes per entry, much smaller than sizeof(Elf{32,64}_Rel).

[0] commit 1c0c1fa ("objtool: Use relative pointers for annotations")

Link: ClangBuiltLinux#1937 [1]
Signed-off-by: Fangrui Song <maskray@google.com>
@MaskRay
Copy link
Member Author

MaskRay commented Sep 20, 2023

Or is it that the sections modified in 1c0c1fa that aren't "a" should not have been changed?

Adding "a" works as well, but I feel that non-SHF_ALLOC for sections to be discarded makes more sense. It has been a while since my previous kernel patch... [PATCH] x86/speculation, objtool: Use absolute relocations for annotations

I do not know what's a reasonable test, but the ARCH=i386 defconfig all warnings go away and /tmp/linux/x86_64/tools/objtool/objtool -r -l -i -n --link /tmp/linux/x86_64/vmlinux doesn't cause new warnings.

@nickdesaulniers
Copy link
Member

nickdesaulniers commented Sep 20, 2023 via email

@MaskRay
Copy link
Member Author

MaskRay commented Sep 20, 2023

Mind waiting to land your LLD patch until we hear back on your kernel patch?

On Tue, Sep 19, 2023, 5:23 PM Fangrui Song @.> wrote: Or is it that the sections modified in 1c0c1fa <1c0c1fa> that aren't "a" should not have been changed? Adding "a" works as well, but I feel that non-SHF_ALLOC for sections to be discarded makes more sense. It has been a while since my previous kernel patch... [PATCH] x86/speculation, objtool: Use absolute relocations for annotations @./T/#u> I do not know what's a reasonable test, but the ARCH=i386 defconfig all warnings go away and /tmp/linux/x86_64/tools/objtool/objtool -r -l -i -n --link /tmp/linux/x86_64/vmlinux doesn't cause new warnings. — Reply to this email directly, view it on GitHub <#1937 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAN5IX2WZTXR7JDBYG4Y5ILX3IZQ3ANCNFSM6AAAAAA466RMAA . You are receiving this because you commented.Message ID: @.***>

Sure. I can also work around the warning on the LLD side...

MaskRay added a commit to MaskRay/llvm-project that referenced this issue Sep 20, 2023
…debug_* referencing STT_SECTION symbols (llvm#66804)

https://reviews.llvm.org/D48929 updated addends for non-SHF_ALLOC sections
relocated by REL for -r links, but the patch did not update the addends when
--compress-debug-sections={zlib,zstd} is used (llvm#66738).

https://reviews.llvm.org/D116946 handled tombstone values in debug
sections in relocatable links. As a side effect, both
relocateNonAllocForRelocatable (using `sec->relocations`) and
relocatenonNonAlloc (using raw REL/RELA) may run.

Actually, we can adjust the condition in relocatenonAlloc to completely replace
relocateNonAllocForRelocatable. This patch implements this idea and fixes llvm#66738.

As relocateNonAlloc processes the raw relocations like copyRelocations() does,
the condition `if (config->relocatable && type != target.noneRel)` in `copyRelocations`
(commit 08d6a3f, modified by https://reviews.llvm.org/D62052)
can be made specific to SHF_ALLOC sections.

As a side effect, we can now report diagnostics for PC-relative relocations for
-r. This is a less useful diagnostic that is not worth too much code. As
ClangBuiltLinux/linux#1937 has violations, just
suppress the warning for -r. Tested by commit 561b98f.
MaskRay added a commit to llvm/llvm-project that referenced this issue Sep 20, 2023
…debug_* referencing STT_SECTION symbols (#66804)

https://reviews.llvm.org/D48929 updated addends for non-SHF_ALLOC sections
relocated by REL for -r links, but the patch did not update the addends when
--compress-debug-sections={zlib,zstd} is used (#66738).

https://reviews.llvm.org/D116946 handled tombstone values in debug
sections in relocatable links. As a side effect, both
relocateNonAllocForRelocatable (using `sec->relocations`) and
relocatenonNonAlloc (using raw REL/RELA) may run.

Actually, we can adjust the condition in relocatenonAlloc to completely replace
relocateNonAllocForRelocatable. This patch implements this idea and fixes #66738.

As relocateNonAlloc processes the raw relocations like copyRelocations() does,
the condition `if (config->relocatable && type != target.noneRel)` in `copyRelocations`
(commit 08d6a3f, modified by https://reviews.llvm.org/D62052)
can be made specific to SHF_ALLOC sections.

As a side effect, we can now report diagnostics for PC-relative relocations for
-r. This is a less useful diagnostic that is not worth too much code. As
ClangBuiltLinux/linux#1937 has violations, just
suppress the warning for -r. Tested by commit 561b98f.
intel-lab-lkp pushed a commit to intel-lab-lkp/linux that referenced this issue Sep 22, 2023
.discard.retpoline_safe sections do not have the SHF_ALLOC flag.  These
sections referencing text sections' STT_SECTION symbols with PC-relative
relocations like R_386_PC32 [0] is conceptually not suitable.  Newer
LLD will report warnings for REL relocations even for relocatable links [1]:

    ld.lld: warning: vmlinux.a(drivers/i2c/busses/i2c-i801.o):(.discard.retpoline_safe+0x120): has non-ABS relocation R_386_PC32 against symbol ''

Switch to absolute relocations instead, which indicate link-time
addresses.  In a relocatable link, these addresses are also output
section offsets, used by checks in tools/objtool/check.c.  When linking
vmlinux, these .discard.* sections will be discarded, therefore it is
not a problem that R_X86_64_32 cannot represent a kernel address.

Alternatively, we could set the SHF_ALLOC flag for .discard.* sections,
but I think non-SHF_ALLOC for sections to be discarded makes more sense.

Note: if we decide to never support REL architectures (e.g. arm, i386),
we can utilize R_*_NONE relocations (.reloc ., BFD_RELOC_NONE, sym),
making .discard.* sections zero-sized.  That said, the section content
waste is 4 bytes per entry, much smaller than sizeof(Elf{32,64}_Rel).

  [0] commit 1c0c1fa ("objtool: Use relative pointers for annotations")
  [1] ClangBuiltLinux#1937

Signed-off-by: Fangrui Song <maskray@google.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Link: https://lore.kernel.org/r/20230920001728.1439947-1-maskray@google.com
@nickdesaulniers
Copy link
Member

So we should probably make CONFIG_DEBUG_INFO_COMPRESSED unselectable if using LLD < 18 and targeting a 32b target, since it will generate invalid debug info prior to MaskRay/llvm-project@7ffdb74.

I'll send a patch for Kconfig for that; once that's landed, I think we can close out all of the bugs here. Thanks for all of your work on this @MaskRay ; it will allow Android to finally ship compressed debug info.

@nickdesaulniers nickdesaulniers self-assigned this Oct 5, 2023
DawnBreather pushed a commit to DawnBreather/linux-kernel that referenced this issue Dec 20, 2023
commit b8ec60e upstream.

.discard.retpoline_safe sections do not have the SHF_ALLOC flag.  These
sections referencing text sections' STT_SECTION symbols with PC-relative
relocations like R_386_PC32 [0] is conceptually not suitable.  Newer
LLD will report warnings for REL relocations even for relocatable links [1]:

    ld.lld: warning: vmlinux.a(drivers/i2c/busses/i2c-i801.o):(.discard.retpoline_safe+0x120): has non-ABS relocation R_386_PC32 against symbol ''

Switch to absolute relocations instead, which indicate link-time
addresses.  In a relocatable link, these addresses are also output
section offsets, used by checks in tools/objtool/check.c.  When linking
vmlinux, these .discard.* sections will be discarded, therefore it is
not a problem that R_X86_64_32 cannot represent a kernel address.

Alternatively, we could set the SHF_ALLOC flag for .discard.* sections,
but I think non-SHF_ALLOC for sections to be discarded makes more sense.

Note: if we decide to never support REL architectures (e.g. arm, i386),
we can utilize R_*_NONE relocations (.reloc ., BFD_RELOC_NONE, sym),
making .discard.* sections zero-sized.  That said, the section content
waste is 4 bytes per entry, much smaller than sizeof(Elf{32,64}_Rel).

  [0] commit 1c0c1fa ("objtool: Use relative pointers for annotations")
  [1] ClangBuiltLinux/linux#1937

Signed-off-by: Fangrui Song <maskray@google.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Link: https://lore.kernel.org/r/20230920001728.1439947-1-maskray@google.com
Cc: Nathan Chancellor <nathan@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
@MaskRay
Copy link
Member Author

MaskRay commented Jan 9, 2024

This commit is in master and linux-6.6.y branches. Closing? :)

@MaskRay MaskRay closed this as completed Jan 9, 2024
wanghao75 pushed a commit to openeuler-mirror/kernel that referenced this issue Mar 19, 2024
stable inclusion
from stable-v6.6.8
commit 06f61af8025452514945657e9b4cb1c9ba8967c5
bugzilla: https://gitee.com/openeuler/kernel/issues/I99K53

Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=06f61af8025452514945657e9b4cb1c9ba8967c5

--------------------------------

commit b8ec60e1186cdcfce41e7db4c827cb107e459002 upstream.

.discard.retpoline_safe sections do not have the SHF_ALLOC flag.  These
sections referencing text sections' STT_SECTION symbols with PC-relative
relocations like R_386_PC32 [0] is conceptually not suitable.  Newer
LLD will report warnings for REL relocations even for relocatable links [1]:

    ld.lld: warning: vmlinux.a(drivers/i2c/busses/i2c-i801.o):(.discard.retpoline_safe+0x120): has non-ABS relocation R_386_PC32 against symbol ''

Switch to absolute relocations instead, which indicate link-time
addresses.  In a relocatable link, these addresses are also output
section offsets, used by checks in tools/objtool/check.c.  When linking
vmlinux, these .discard.* sections will be discarded, therefore it is
not a problem that R_X86_64_32 cannot represent a kernel address.

Alternatively, we could set the SHF_ALLOC flag for .discard.* sections,
but I think non-SHF_ALLOC for sections to be discarded makes more sense.

Note: if we decide to never support REL architectures (e.g. arm, i386),
we can utilize R_*_NONE relocations (.reloc ., BFD_RELOC_NONE, sym),
making .discard.* sections zero-sized.  That said, the section content
waste is 4 bytes per entry, much smaller than sizeof(Elf{32,64}_Rel).

  [0] commit 1c0c1fa ("objtool: Use relative pointers for annotations")
  [1] ClangBuiltLinux/linux#1937

Signed-off-by: Fangrui Song <maskray@google.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Link: https://lore.kernel.org/r/20230920001728.1439947-1-maskray@google.com
Cc: Nathan Chancellor <nathan@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: ZhangPeng <zhangpeng362@huawei.com>
tuxedo-bot pushed a commit to tuxedocomputers/linux that referenced this issue Mar 22, 2024
BugLink: https://bugs.launchpad.net/bugs/2051924

commit b8ec60e upstream.

.discard.retpoline_safe sections do not have the SHF_ALLOC flag.  These
sections referencing text sections' STT_SECTION symbols with PC-relative
relocations like R_386_PC32 [0] is conceptually not suitable.  Newer
LLD will report warnings for REL relocations even for relocatable links [1]:

    ld.lld: warning: vmlinux.a(drivers/i2c/busses/i2c-i801.o):(.discard.retpoline_safe+0x120): has non-ABS relocation R_386_PC32 against symbol ''

Switch to absolute relocations instead, which indicate link-time
addresses.  In a relocatable link, these addresses are also output
section offsets, used by checks in tools/objtool/check.c.  When linking
vmlinux, these .discard.* sections will be discarded, therefore it is
not a problem that R_X86_64_32 cannot represent a kernel address.

Alternatively, we could set the SHF_ALLOC flag for .discard.* sections,
but I think non-SHF_ALLOC for sections to be discarded makes more sense.

Note: if we decide to never support REL architectures (e.g. arm, i386),
we can utilize R_*_NONE relocations (.reloc ., BFD_RELOC_NONE, sym),
making .discard.* sections zero-sized.  That said, the section content
waste is 4 bytes per entry, much smaller than sizeof(Elf{32,64}_Rel).

  [0] commit 1c0c1fa ("objtool: Use relative pointers for annotations")
  [1] ClangBuiltLinux/linux#1937

Signed-off-by: Fangrui Song <maskray@google.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Link: https://lore.kernel.org/r/20230920001728.1439947-1-maskray@google.com
Cc: Nathan Chancellor <nathan@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Manuel Diewald <manuel.diewald@canonical.com>
Signed-off-by: Roxana Nicolescu <roxana.nicolescu@canonical.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
[ARCH] x86 This bug impacts ARCH=i386 [BUG] linux A bug that should be fixed in the mainline kernel. [TOOL] lld The issue is relevant to LLD linker
Projects
None yet
Development

No branches or pull requests

2 participants