Skip to content

Conversation

@weliveindetail
Copy link
Member

We don't want to keep tracking allocations of debug objects in the controller after linking finished. So far, the plugin did that to free the target memory once the corresponding code is unlinked. This patch removes the tracking and instead attaches a dealloc action for it to the target memory of the code.

Copy link
Member Author

@weliveindetail weliveindetail left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a patch on top of #168518 and it's still work-in-progress. It illustrates two issues with InProcessMemoryManager. @lhames maybe you have some useful feedback?

Err = ES.getExecutorProcessControl().getBootstrapSymbols({
{RegistrationAction, rt::RegisterJITLoaderGDBAllocActionName},
{DeallocAction, rt::SimpleExecutorMemoryManagerReleaseWrapperName},
{TargetMemMgr, rt::SimpleExecutorMemoryManagerInstanceName},
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

InProcessMemoryManager does not expose bootstrap symbols yet. SimpleRemoteEPC does it in createDefaultMemoryManager() and we could do something similar in SelfExecutorProcessControl(). Or should we better enforce it in the JITLinkMemoryManager() base class ctor?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that that sounds reasonable, but we should wait until we've brought up the new ORC runtime: there might be an opportunity to define a widely implementable set of memory allocation primitives there.

// FIXME: The lookup in the segment info here is a workaround. The below
// FA->release() is supposed to provide the base address in target memory,
// but InProcessMemoryManager returns the address of a FinalizedAllocInfo
// helper instead.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is another inconsistency in InProcessMemoryManager. The workaround is ok, but not very reasonable.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a substantial further simplification that could be made here: If we're ok with the debug object living for as long as the corresponding JIT'd code then the debug object could be embedded in a special section in the graph, rather than having its own allocation. That would allow you to avoid managing the memory lifetime entirely, and you could just focus on populating the debug object bits and calling the registration and deregistration functions.

@github-actions
Copy link

🐧 Linux x64 Test Results

  • 166063 tests passed
  • 2838 tests skipped
  • 5 tests failed

Failed Tests

(click on a test name to see its output)

lldb-shell

lldb-shell.Breakpoint/jit-loader_jitlink_elf.test
Exit Code: 1

Command Output (stdout):
--
# RUN: at line 6
/home/gha/actions-runner/_work/llvm-project/llvm-project/build/bin/clang --driver-mode=g++ --target=specify-a-target-or-use-a-_host-substitution -g -S -emit-llvm -fPIC --target=x86_64-unknown-unknown-elf           -o /home/gha/actions-runner/_work/llvm-project/llvm-project/build/tools/lldb/test/Shell/Breakpoint/Output/jit-loader_jitlink_elf.test.tmp.ll /home/gha/actions-runner/_work/llvm-project/llvm-project/lldb/test/Shell/Breakpoint/Inputs/jitbp.cpp
# executed command: /home/gha/actions-runner/_work/llvm-project/llvm-project/build/bin/clang --driver-mode=g++ --target=specify-a-target-or-use-a-_host-substitution -g -S -emit-llvm -fPIC --target=x86_64-unknown-unknown-elf -o /home/gha/actions-runner/_work/llvm-project/llvm-project/build/tools/lldb/test/Shell/Breakpoint/Output/jit-loader_jitlink_elf.test.tmp.ll /home/gha/actions-runner/_work/llvm-project/llvm-project/lldb/test/Shell/Breakpoint/Inputs/jitbp.cpp
# note: command had no output on stdout or stderr
# RUN: at line 8
/home/gha/actions-runner/_work/llvm-project/llvm-project/build/bin/lldb --no-lldbinit -S /home/gha/actions-runner/_work/llvm-project/llvm-project/build/tools/lldb/test/Shell/lit-lldb-init-quiet -b -o 'settings set plugin.jit-loader.gdb.enable on' -o 'b jitbp'           -o 'run --jit-linker=jitlink /home/gha/actions-runner/_work/llvm-project/llvm-project/build/tools/lldb/test/Shell/Breakpoint/Output/jit-loader_jitlink_elf.test.tmp.ll' /home/gha/actions-runner/_work/llvm-project/llvm-project/build/bin/lli | /home/gha/actions-runner/_work/llvm-project/llvm-project/build/bin/FileCheck /home/gha/actions-runner/_work/llvm-project/llvm-project/lldb/test/Shell/Breakpoint/jit-loader_jitlink_elf.test
# executed command: /home/gha/actions-runner/_work/llvm-project/llvm-project/build/bin/lldb --no-lldbinit -S /home/gha/actions-runner/_work/llvm-project/llvm-project/build/tools/lldb/test/Shell/lit-lldb-init-quiet -b -o 'settings set plugin.jit-loader.gdb.enable on' -o 'b jitbp' -o 'run --jit-linker=jitlink /home/gha/actions-runner/_work/llvm-project/llvm-project/build/tools/lldb/test/Shell/Breakpoint/Output/jit-loader_jitlink_elf.test.tmp.ll' /home/gha/actions-runner/_work/llvm-project/llvm-project/build/bin/lli
# note: command had no output on stdout or stderr
# executed command: /home/gha/actions-runner/_work/llvm-project/llvm-project/build/bin/FileCheck /home/gha/actions-runner/_work/llvm-project/llvm-project/lldb/test/Shell/Breakpoint/jit-loader_jitlink_elf.test
# .---command stderr------------
# | /home/gha/actions-runner/_work/llvm-project/llvm-project/lldb/test/Shell/Breakpoint/jit-loader_jitlink_elf.test:14:10: error: CHECK: expected string not found in input
# | # CHECK: Process {{.*}} stopped
# |          ^
# | <stdin>:11:108: note: scanning from here
# | Process 1560053 launched: '/home/gha/actions-runner/_work/llvm-project/llvm-project/build/bin/lli' (x86_64)
# |                                                                                                            ^
# | <stdin>:12:1: note: possible intended match here
# | Process 1560053 exited with status = 0 (0x00000000)
# | ^
# | 
# | Input file: <stdin>
# | Check file: /home/gha/actions-runner/_work/llvm-project/llvm-project/lldb/test/Shell/Breakpoint/jit-loader_jitlink_elf.test
# | 
# | -dump-input=help explains the following input dump.
# | 
# | Input was:
# | <<<<<<
# |             .
# |             .
# |             .
# |             6: (lldb) settings set plugin.jit-loader.gdb.enable on 
# |             7: (lldb) b jitbp 
# |             8: Breakpoint 1: no locations (pending). 
# |             9: WARNING: Unable to resolve breakpoint to any actual locations. 
# |            10: (lldb) run --jit-linker=jitlink /home/gha/actions-runner/_work/llvm-project/llvm-project/build/tools/lldb/test/Shell/Breakpoint/Output/jit-loader_jitlink_elf.test.tmp.ll 
# |            11: Process 1560053 launched: '/home/gha/actions-runner/_work/llvm-project/llvm-project/build/bin/lli' (x86_64) 
# | check:14'0                                                                                                                X error: no match found
# |            12: Process 1560053 exited with status = 0 (0x00000000) 
# | check:14'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# | check:14'1     ?                                                    possible intended match
# | >>>>>>
# `-----------------------------
# error: command failed with exit status: 1

--

lldb-shell.Breakpoint/jit-loader_rtdyld_elf.test
Exit Code: 1

Command Output (stdout):
--
# RUN: at line 6
/home/gha/actions-runner/_work/llvm-project/llvm-project/build/bin/clang --driver-mode=g++ --target=specify-a-target-or-use-a-_host-substitution -g -S -emit-llvm --target=x86_64-unknown-unknown-elf           -o /home/gha/actions-runner/_work/llvm-project/llvm-project/build/tools/lldb/test/Shell/Breakpoint/Output/jit-loader_rtdyld_elf.test.tmp.ll /home/gha/actions-runner/_work/llvm-project/llvm-project/lldb/test/Shell/Breakpoint/Inputs/jitbp.cpp
# executed command: /home/gha/actions-runner/_work/llvm-project/llvm-project/build/bin/clang --driver-mode=g++ --target=specify-a-target-or-use-a-_host-substitution -g -S -emit-llvm --target=x86_64-unknown-unknown-elf -o /home/gha/actions-runner/_work/llvm-project/llvm-project/build/tools/lldb/test/Shell/Breakpoint/Output/jit-loader_rtdyld_elf.test.tmp.ll /home/gha/actions-runner/_work/llvm-project/llvm-project/lldb/test/Shell/Breakpoint/Inputs/jitbp.cpp
# note: command had no output on stdout or stderr
# RUN: at line 9
/home/gha/actions-runner/_work/llvm-project/llvm-project/build/bin/lldb --no-lldbinit -S /home/gha/actions-runner/_work/llvm-project/llvm-project/build/tools/lldb/test/Shell/lit-lldb-init-quiet -b -o 'settings set plugin.jit-loader.gdb.enable on' -o 'b jitbp'           -o 'run --jit-kind=mcjit /home/gha/actions-runner/_work/llvm-project/llvm-project/build/tools/lldb/test/Shell/Breakpoint/Output/jit-loader_rtdyld_elf.test.tmp.ll' /home/gha/actions-runner/_work/llvm-project/llvm-project/build/bin/lli | /home/gha/actions-runner/_work/llvm-project/llvm-project/build/bin/FileCheck /home/gha/actions-runner/_work/llvm-project/llvm-project/lldb/test/Shell/Breakpoint/jit-loader_rtdyld_elf.test
# executed command: /home/gha/actions-runner/_work/llvm-project/llvm-project/build/bin/lldb --no-lldbinit -S /home/gha/actions-runner/_work/llvm-project/llvm-project/build/tools/lldb/test/Shell/lit-lldb-init-quiet -b -o 'settings set plugin.jit-loader.gdb.enable on' -o 'b jitbp' -o 'run --jit-kind=mcjit /home/gha/actions-runner/_work/llvm-project/llvm-project/build/tools/lldb/test/Shell/Breakpoint/Output/jit-loader_rtdyld_elf.test.tmp.ll' /home/gha/actions-runner/_work/llvm-project/llvm-project/build/bin/lli
# note: command had no output on stdout or stderr
# executed command: /home/gha/actions-runner/_work/llvm-project/llvm-project/build/bin/FileCheck /home/gha/actions-runner/_work/llvm-project/llvm-project/lldb/test/Shell/Breakpoint/jit-loader_rtdyld_elf.test
# note: command had no output on stdout or stderr
# RUN: at line 12
/home/gha/actions-runner/_work/llvm-project/llvm-project/build/bin/lldb --no-lldbinit -S /home/gha/actions-runner/_work/llvm-project/llvm-project/build/tools/lldb/test/Shell/lit-lldb-init-quiet -b -o 'settings set plugin.jit-loader.gdb.enable on' -o 'b jitbp'           -o 'run --jit-linker=rtdyld /home/gha/actions-runner/_work/llvm-project/llvm-project/build/tools/lldb/test/Shell/Breakpoint/Output/jit-loader_rtdyld_elf.test.tmp.ll' /home/gha/actions-runner/_work/llvm-project/llvm-project/build/bin/lli | /home/gha/actions-runner/_work/llvm-project/llvm-project/build/bin/FileCheck /home/gha/actions-runner/_work/llvm-project/llvm-project/lldb/test/Shell/Breakpoint/jit-loader_rtdyld_elf.test
# executed command: /home/gha/actions-runner/_work/llvm-project/llvm-project/build/bin/lldb --no-lldbinit -S /home/gha/actions-runner/_work/llvm-project/llvm-project/build/tools/lldb/test/Shell/lit-lldb-init-quiet -b -o 'settings set plugin.jit-loader.gdb.enable on' -o 'b jitbp' -o 'run --jit-linker=rtdyld /home/gha/actions-runner/_work/llvm-project/llvm-project/build/tools/lldb/test/Shell/Breakpoint/Output/jit-loader_rtdyld_elf.test.tmp.ll' /home/gha/actions-runner/_work/llvm-project/llvm-project/build/bin/lli
# note: command had no output on stdout or stderr
# executed command: /home/gha/actions-runner/_work/llvm-project/llvm-project/build/bin/FileCheck /home/gha/actions-runner/_work/llvm-project/llvm-project/lldb/test/Shell/Breakpoint/jit-loader_rtdyld_elf.test
# .---command stderr------------
# | /home/gha/actions-runner/_work/llvm-project/llvm-project/lldb/test/Shell/Breakpoint/jit-loader_rtdyld_elf.test:18:10: error: CHECK: expected string not found in input
# | # CHECK: Process {{.*}} stopped
# |          ^
# | <stdin>:11:108: note: scanning from here
# | Process 1543068 launched: '/home/gha/actions-runner/_work/llvm-project/llvm-project/build/bin/lli' (x86_64)
# |                                                                                                            ^
# | <stdin>:12:1: note: possible intended match here
# | Process 1543068 exited with status = 0 (0x00000000)
# | ^
# | 
# | Input file: <stdin>
# | Check file: /home/gha/actions-runner/_work/llvm-project/llvm-project/lldb/test/Shell/Breakpoint/jit-loader_rtdyld_elf.test
# | 
# | -dump-input=help explains the following input dump.
# | 
# | Input was:
# | <<<<<<
# |             .
# |             .
# |             .
# |             6: (lldb) settings set plugin.jit-loader.gdb.enable on 
# |             7: (lldb) b jitbp 
# |             8: Breakpoint 1: no locations (pending). 
# |             9: WARNING: Unable to resolve breakpoint to any actual locations. 
# |            10: (lldb) run --jit-linker=rtdyld /home/gha/actions-runner/_work/llvm-project/llvm-project/build/tools/lldb/test/Shell/Breakpoint/Output/jit-loader_rtdyld_elf.test.tmp.ll 
# |            11: Process 1543068 launched: '/home/gha/actions-runner/_work/llvm-project/llvm-project/build/bin/lli' (x86_64) 
# | check:18'0                                                                                                                X error: no match found
# |            12: Process 1543068 exited with status = 0 (0x00000000) 
# | check:18'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# | check:18'1     ?                                                    possible intended match
# | >>>>>>
# `-----------------------------
# error: command failed with exit status: 1

--

LLVM

LLVM.ExecutionEngine/OrcLazy/debug-descriptor.ll
Exit Code: 1

Command Output (stdout):
--
# RUN: at line 3
/home/gha/actions-runner/_work/llvm-project/llvm-project/build/bin/lli --jit-linker=rtdyld --orc-lazy-debug=jit-debug-descriptor /home/gha/actions-runner/_work/llvm-project/llvm-project/llvm/test/ExecutionEngine/OrcLazy/debug-descriptor.ll 2>&1 | /home/gha/actions-runner/_work/llvm-project/llvm-project/build/bin/FileCheck /home/gha/actions-runner/_work/llvm-project/llvm-project/llvm/test/ExecutionEngine/OrcLazy/debug-descriptor.ll
# executed command: /home/gha/actions-runner/_work/llvm-project/llvm-project/build/bin/lli --jit-linker=rtdyld --orc-lazy-debug=jit-debug-descriptor /home/gha/actions-runner/_work/llvm-project/llvm-project/llvm/test/ExecutionEngine/OrcLazy/debug-descriptor.ll
# note: command had no output on stdout or stderr
# executed command: /home/gha/actions-runner/_work/llvm-project/llvm-project/build/bin/FileCheck /home/gha/actions-runner/_work/llvm-project/llvm-project/llvm/test/ExecutionEngine/OrcLazy/debug-descriptor.ll
# .---command stderr------------
# | /home/gha/actions-runner/_work/llvm-project/llvm-project/llvm/test/ExecutionEngine/OrcLazy/debug-descriptor.ll:10:10: error: CHECK: expected string not found in input
# | ; CHECK: jit_debug_descriptor 0x
# |          ^
# | <stdin>:1:40: note: scanning from here
# | jit_debug_descriptor 0x0000000000000000
# |                                        ^
# | 
# | Input file: <stdin>
# | Check file: /home/gha/actions-runner/_work/llvm-project/llvm-project/llvm/test/ExecutionEngine/OrcLazy/debug-descriptor.ll
# | 
# | -dump-input=help explains the following input dump.
# | 
# | Input was:
# | <<<<<<
# |           1: jit_debug_descriptor 0x0000000000000000 
# | check:10                                            X error: no match found
# | >>>>>>
# `-----------------------------
# error: command failed with exit status: 1

--

LLVM.ExecutionEngine/OrcLazy/debug-objects.ll
Exit Code: 2

Command Output (stdout):
--
# RUN: at line 5
/home/gha/actions-runner/_work/llvm-project/llvm-project/build/bin/lli --jit-linker=rtdyld  --orc-lazy-debug=jit-debug-objects /home/gha/actions-runner/_work/llvm-project/llvm-project/llvm/test/ExecutionEngine/OrcLazy/debug-objects.ll | /home/gha/actions-runner/_work/llvm-project/llvm-project/build/bin/llvm-dwarfdump --diff - | /home/gha/actions-runner/_work/llvm-project/llvm-project/build/bin/FileCheck /home/gha/actions-runner/_work/llvm-project/llvm-project/llvm/test/ExecutionEngine/OrcLazy/debug-objects.ll
# executed command: /home/gha/actions-runner/_work/llvm-project/llvm-project/build/bin/lli --jit-linker=rtdyld --orc-lazy-debug=jit-debug-objects /home/gha/actions-runner/_work/llvm-project/llvm-project/llvm/test/ExecutionEngine/OrcLazy/debug-objects.ll
# note: command had no output on stdout or stderr
# executed command: /home/gha/actions-runner/_work/llvm-project/llvm-project/build/bin/llvm-dwarfdump --diff -
# .---command stderr------------
# | error: -: The file was not recognized as a valid object file
# `-----------------------------
# error: command failed with exit status: 1
# executed command: /home/gha/actions-runner/_work/llvm-project/llvm-project/build/bin/FileCheck /home/gha/actions-runner/_work/llvm-project/llvm-project/llvm/test/ExecutionEngine/OrcLazy/debug-objects.ll
# .---command stderr------------
# | FileCheck error: '<stdin>' is empty.
# | FileCheck command line:  /home/gha/actions-runner/_work/llvm-project/llvm-project/build/bin/FileCheck /home/gha/actions-runner/_work/llvm-project/llvm-project/llvm/test/ExecutionEngine/OrcLazy/debug-objects.ll
# `-----------------------------
# error: command failed with exit status: 2

--

LLVM-Unit

LLVM-Unit.ExecutionEngine/Orc/_/OrcJITTests/OrcCAPITestBase/EnableDebugSupport
Script:
--
/home/gha/actions-runner/_work/llvm-project/llvm-project/build/unittests/ExecutionEngine/Orc/./OrcJITTests --gtest_filter=OrcCAPITestBase.EnableDebugSupport
--
/home/gha/actions-runner/_work/llvm-project/llvm-project/llvm/unittests/ExecutionEngine/Orc/OrcCAPITest.cpp:560
Value of: isa<ObjectLinkingLayer>(unwrap(ObjLayer))
  Actual: true
Expected: false
Error testing LLJIT debug support (triple = x86_64-unknown-linux-gnu): Symbol "__llvm_orc_SimpleExecutorMemoryManager_release_wrapper" not found in bootstrap symbols map


If these failures are unrelated to your changes (for example tests are broken or flaky at HEAD), please open an issue at https://github.com/llvm/llvm-project/issues and add the infrastructure label.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants