From 2590576843300c68ac44f5709241607512b664e7 Mon Sep 17 00:00:00 2001 From: Jason Molenda Date: Thu, 4 Sep 2025 11:42:26 -0700 Subject: [PATCH 1/7] [lldb] Add utility to create Mach-O corefile from YAML desc (#153911) I've wanted a utility to create a corefile for test purposes given a bit of memory and regsters, for a while. I've written a few API tests over the years that needed exactly this capability -- we have several one-off Mach-O corefile creator utility in the API testsuite to do this. But it's a lot of boilerplate when you only want to specify some register contents and memory contents, to create an API test. This adds yaml2mach-core, a tool that should build on any system, takes a yaml description of register values for one or more threads, optionally memory values for one or more memory regions, and can take a list of UUIDs that will be added as LC_NOTE "load binary" metadata to the corefile so binaries can be loaded into virtual address space in a test scenario. The format of the yaml file looks like ``` cpu: armv7m addressable-bits: num-bits: 39 binaries: - name: debug-binary.development uuid: 67942352-5857-3D3D-90CB-A3F80BA67B04 virtual-address: 0xfffffff01840c000 threads: - regsets: - flavor: gpr registers: [{name: sp, value: 0x2000fe70}, {name: r7, value: 0x2000fe80}, {name: pc, value: 0x0020392c}, {name: lr, value: 0x0020392d}] memory-regions: # stack memory - addr: 0x2000fe70 UInt32: [ 0x0000002a, 0x20010e58, 0x00203923, 0x00000001, 0x2000fe88, 0x00203911, 0x2000ffdc, 0xfffffff9 ] # instructions of a function - addr: 0x203910 UInt8: [ 0xf8, 0xb5, 0x04, 0xaf, 0x06, 0x4c, 0x07, 0x49, 0x74, 0xf0, 0x2e, 0xf8, 0x01, 0xac, 0x74, 0xf0 ] ``` and that's all that is needed to specify a corefile where four register values are specified (the others will be set to 0), and two memory regions will be emitted. The memory can be specified as an array of UInt8, UInt32, or UInt64, I anticipate that some of these corefiles may have stack values constructed manually and it may be simpler for a human to write them in a particular grouping of values. I needed this utility for an upcoming patch for ARM Cortex-M processors, to create a test for the change. I took the opportunity to remove two of the "trivial mach-o corefile" creator utilities I've written in the past, which also restricted the tests to only run on Darwin systems because I was using the system headers for Mach-O constant values. rdar://110663219 (cherry picked from commit a0c2d6e369a1fb4d8b3ed46baed7a2de2fb3d882) --- .../Python/lldbsuite/test/configuration.py | 11 + lldb/packages/Python/lldbsuite/test/dotest.py | 3 + .../Python/lldbsuite/test/lldbtest.py | 27 +- .../API/macosx/arm-corefile-regctx/Makefile | 6 - .../TestArmMachoCorefileRegctx.py | 17 +- .../API/macosx/arm-corefile-regctx/arm64.yaml | 30 ++ .../macosx/arm-corefile-regctx/armv7m.yaml | 36 +++ .../create-arm-corefiles.cpp | 266 ------------------ .../test/API/macosx/riscv32-corefile/Makefile | 7 - .../riscv32-corefile/TestRV32MachOCorefile.py | 17 +- .../create-empty-riscv-corefile.cpp | 116 -------- .../riscv32-corefile/riscv32-registers.yaml | 46 +++ lldb/test/CMakeLists.txt | 1 + lldb/tools/CMakeLists.txt | 2 + lldb/tools/yaml2macho-core/CMakeLists.txt | 14 + lldb/tools/yaml2macho-core/CoreSpec.cpp | 165 +++++++++++ lldb/tools/yaml2macho-core/CoreSpec.h | 80 ++++++ lldb/tools/yaml2macho-core/LCNoteWriter.cpp | 97 +++++++ lldb/tools/yaml2macho-core/LCNoteWriter.h | 33 +++ lldb/tools/yaml2macho-core/MemoryWriter.cpp | 56 ++++ lldb/tools/yaml2macho-core/MemoryWriter.h | 26 ++ lldb/tools/yaml2macho-core/ThreadWriter.cpp | 200 +++++++++++++ lldb/tools/yaml2macho-core/ThreadWriter.h | 24 ++ lldb/tools/yaml2macho-core/Utility.cpp | 22 ++ lldb/tools/yaml2macho-core/Utility.h | 18 ++ lldb/tools/yaml2macho-core/yaml2macho.cpp | 251 +++++++++++++++++ 26 files changed, 1159 insertions(+), 412 deletions(-) delete mode 100644 lldb/test/API/macosx/arm-corefile-regctx/Makefile create mode 100644 lldb/test/API/macosx/arm-corefile-regctx/arm64.yaml create mode 100644 lldb/test/API/macosx/arm-corefile-regctx/armv7m.yaml delete mode 100644 lldb/test/API/macosx/arm-corefile-regctx/create-arm-corefiles.cpp delete mode 100644 lldb/test/API/macosx/riscv32-corefile/Makefile delete mode 100644 lldb/test/API/macosx/riscv32-corefile/create-empty-riscv-corefile.cpp create mode 100644 lldb/test/API/macosx/riscv32-corefile/riscv32-registers.yaml create mode 100644 lldb/tools/yaml2macho-core/CMakeLists.txt create mode 100644 lldb/tools/yaml2macho-core/CoreSpec.cpp create mode 100644 lldb/tools/yaml2macho-core/CoreSpec.h create mode 100644 lldb/tools/yaml2macho-core/LCNoteWriter.cpp create mode 100644 lldb/tools/yaml2macho-core/LCNoteWriter.h create mode 100644 lldb/tools/yaml2macho-core/MemoryWriter.cpp create mode 100644 lldb/tools/yaml2macho-core/MemoryWriter.h create mode 100644 lldb/tools/yaml2macho-core/ThreadWriter.cpp create mode 100644 lldb/tools/yaml2macho-core/ThreadWriter.h create mode 100644 lldb/tools/yaml2macho-core/Utility.cpp create mode 100644 lldb/tools/yaml2macho-core/Utility.h create mode 100644 lldb/tools/yaml2macho-core/yaml2macho.cpp diff --git a/lldb/packages/Python/lldbsuite/test/configuration.py b/lldb/packages/Python/lldbsuite/test/configuration.py index 94172cde64279..357bbba1d9352 100644 --- a/lldb/packages/Python/lldbsuite/test/configuration.py +++ b/lldb/packages/Python/lldbsuite/test/configuration.py @@ -67,6 +67,9 @@ # Path to the yaml2obj tool. Not optional. yaml2obj = None +# Path to the yaml2macho-core tool. Not optional. +yaml2macho_core = None + # The arch might dictate some specific CFLAGS to be passed to the toolchain to build # the inferior programs. The global variable cflags_extras provides a hook to do # just that. @@ -178,3 +181,11 @@ def get_yaml2obj_path(): """ if yaml2obj and os.path.lexists(yaml2obj): return yaml2obj + + +def get_yaml2macho_core_path(): + """ + Get the path to the yaml2macho-core tool. + """ + if yaml2macho_core and os.path.lexists(yaml2macho_core): + return yaml2macho_core diff --git a/lldb/packages/Python/lldbsuite/test/dotest.py b/lldb/packages/Python/lldbsuite/test/dotest.py index bea0f9ffc2d84..24c6fe194385a 100644 --- a/lldb/packages/Python/lldbsuite/test/dotest.py +++ b/lldb/packages/Python/lldbsuite/test/dotest.py @@ -279,6 +279,9 @@ def parseOptionsAndInitTestdirs(): configuration.llvm_tools_dir = args.llvm_tools_dir configuration.filecheck = shutil.which("FileCheck", path=args.llvm_tools_dir) configuration.yaml2obj = shutil.which("yaml2obj", path=args.llvm_tools_dir) + configuration.yaml2macho_core = shutil.which( + "yaml2macho-core", path=args.llvm_tools_dir + ) if not configuration.get_filecheck_path(): logging.warning("No valid FileCheck executable; some tests may fail...") diff --git a/lldb/packages/Python/lldbsuite/test/lldbtest.py b/lldb/packages/Python/lldbsuite/test/lldbtest.py index 56c1473b66013..7271c9965e801 100644 --- a/lldb/packages/Python/lldbsuite/test/lldbtest.py +++ b/lldb/packages/Python/lldbsuite/test/lldbtest.py @@ -1715,6 +1715,29 @@ def yaml2obj(self, yaml_path, obj_path, max_size=None): command += ["--max-size=%d" % max_size] self.runBuildCommand(command) + def yaml2macho_core(self, yaml_path, obj_path, uuids=None): + """ + Create a Mach-O corefile at the given path from a yaml file. + + Throws subprocess.CalledProcessError if the object could not be created. + """ + yaml2macho_core_bin = configuration.get_yaml2macho_core_path() + if not yaml2macho_core_bin: + self.assertTrue(False, "No valid yaml2macho-core executable specified") + if uuids != None: + command = [ + yaml2macho_core_bin, + "-i", + yaml_path, + "-o", + obj_path, + "-u", + uuids, + ] + else: + command = [yaml2macho_core_bin, "-i", yaml_path, "-o", obj_path] + self.runBuildCommand(command) + def cleanup(self, dictionary=None): """Platform specific way to do cleanup after build.""" module = builder_module() @@ -2278,7 +2301,9 @@ def completions_match(self, command, completions, max_completions=-1): given list of completions""" interp = self.dbg.GetCommandInterpreter() match_strings = lldb.SBStringList() - interp.HandleCompletion(command, len(command), 0, max_completions, match_strings) + interp.HandleCompletion( + command, len(command), 0, max_completions, match_strings + ) # match_strings is a 1-indexed list, so we have to slice... self.assertCountEqual( completions, list(match_strings)[1:], "List of returned completion is wrong" diff --git a/lldb/test/API/macosx/arm-corefile-regctx/Makefile b/lldb/test/API/macosx/arm-corefile-regctx/Makefile deleted file mode 100644 index e1d0354441cd4..0000000000000 --- a/lldb/test/API/macosx/arm-corefile-regctx/Makefile +++ /dev/null @@ -1,6 +0,0 @@ -MAKE_DSYM := NO - -CXX_SOURCES := create-arm-corefiles.cpp - -include Makefile.rules - diff --git a/lldb/test/API/macosx/arm-corefile-regctx/TestArmMachoCorefileRegctx.py b/lldb/test/API/macosx/arm-corefile-regctx/TestArmMachoCorefileRegctx.py index 6754288a65e1a..a2890cdfeaa44 100644 --- a/lldb/test/API/macosx/arm-corefile-regctx/TestArmMachoCorefileRegctx.py +++ b/lldb/test/API/macosx/arm-corefile-regctx/TestArmMachoCorefileRegctx.py @@ -13,20 +13,14 @@ class TestArmMachoCorefileRegctx(TestBase): NO_DEBUG_INFO_TESTCASE = True - @skipUnlessDarwin - def setUp(self): - TestBase.setUp(self) - self.build() - self.create_corefile = self.getBuildArtifact("a.out") - self.corefile = self.getBuildArtifact("core") - def test_armv7_corefile(self): ### Create corefile - retcode = call(self.create_corefile + " armv7 " + self.corefile, shell=True) + corefile = self.getBuildArtifact("core") + self.yaml2macho_core("armv7m.yaml", corefile) target = self.dbg.CreateTarget("") err = lldb.SBError() - process = target.LoadCore(self.corefile) + process = target.LoadCore(corefile) self.assertTrue(process.IsValid()) thread = process.GetSelectedThread() frame = thread.GetSelectedFrame() @@ -50,11 +44,12 @@ def test_armv7_corefile(self): def test_arm64_corefile(self): ### Create corefile - retcode = call(self.create_corefile + " arm64 " + self.corefile, shell=True) + corefile = self.getBuildArtifact("core") + self.yaml2macho_core("arm64.yaml", corefile) target = self.dbg.CreateTarget("") err = lldb.SBError() - process = target.LoadCore(self.corefile) + process = target.LoadCore(corefile) self.assertTrue(process.IsValid()) thread = process.GetSelectedThread() frame = thread.GetSelectedFrame() diff --git a/lldb/test/API/macosx/arm-corefile-regctx/arm64.yaml b/lldb/test/API/macosx/arm-corefile-regctx/arm64.yaml new file mode 100644 index 0000000000000..fe06f2d4054b0 --- /dev/null +++ b/lldb/test/API/macosx/arm-corefile-regctx/arm64.yaml @@ -0,0 +1,30 @@ +cpu: arm64 +threads: + # (lldb) reg read + # % pbpaste | grep = | sed 's, ,,g' | awk -F= '{print "{name: " $1 ", value: " $2 "},"}' + - regsets: + - flavor: gpr + registers: [ + {name: x0, value: 0x0000000000000001}, {name: x1, value: 0x000000016fdff3c0}, + {name: x2, value: 0x000000016fdff3d0}, {name: x3, value: 0x000000016fdff510}, + {name: x4, value: 0x0000000000000000}, {name: x5, value: 0x0000000000000000}, + {name: x6, value: 0x0000000000000000}, {name: x7, value: 0x0000000000000000}, + {name: x8, value: 0x000000010000d910}, {name: x9, value: 0x0000000000000001}, + {name: x10, value: 0xe1e88de000000000}, {name: x11, value: 0x0000000000000003}, + {name: x12, value: 0x0000000000000148}, {name: x13, value: 0x0000000000004000}, + {name: x14, value: 0x0000000000000008}, {name: x15, value: 0x0000000000000000}, + {name: x16, value: 0x0000000000000000}, {name: x17, value: 0x0000000100003f5c}, + {name: x18, value: 0x0000000000000000}, {name: x19, value: 0x0000000100003f5c}, + {name: x20, value: 0x000000010000c000}, {name: x21, value: 0x000000010000d910}, + {name: x22, value: 0x000000016fdff250}, {name: x23, value: 0x000000018ce12366}, + {name: x24, value: 0x000000016fdff1d0}, {name: x25, value: 0x0000000000000001}, + {name: x26, value: 0x0000000000000000}, {name: x27, value: 0x0000000000000000}, + {name: x28, value: 0x0000000000000000}, {name: fp, value: 0x000000016fdff3a0}, + {name: lr, value: 0x000000018cd97f28}, {name: sp, value: 0x000000016fdff140}, + {name: pc, value: 0x0000000100003f5c}, {name: cpsr, value: 0x80001000} + ] + - flavor: exc + registers: [ {name: far, value: 0x0000000100003f5c}, + {name: esr, value: 0xf2000000}, + {name: exception, value: 0x00000000} + ] diff --git a/lldb/test/API/macosx/arm-corefile-regctx/armv7m.yaml b/lldb/test/API/macosx/arm-corefile-regctx/armv7m.yaml new file mode 100644 index 0000000000000..411c12a7e4074 --- /dev/null +++ b/lldb/test/API/macosx/arm-corefile-regctx/armv7m.yaml @@ -0,0 +1,36 @@ +cpu: armv7m +threads: + # (lldb) reg read + # % pbpaste | grep = | sed 's, ,,g' | awk -F= '{print "{name: " $1 ", value: " $2 "},"}' + - regsets: + - flavor: gpr + registers: [ + {name: r0, value: 0x00010000}, {name: r1, value: 0x00020000}, + {name: r2, value: 0x00030000}, {name: r3, value: 0x00040000}, + {name: r4, value: 0x00050000}, {name: r5, value: 0x00060000}, + {name: r6, value: 0x00070000}, {name: r7, value: 0x00080000}, + {name: r8, value: 0x00090000}, {name: r9, value: 0x000a0000}, + {name: r10, value: 0x000b0000}, {name: r11, value: 0x000c0000}, + {name: r12, value: 0x000d0000}, {name: sp, value: 0x000e0000}, + {name: lr, value: 0x000f0000}, {name: pc, value: 0x00100000}, + {name: cpsr, value: 0x00110000} + ] + - flavor: exc + registers: [ {name: far, value: 0x00003f5c}, + {name: esr, value: 0xf2000000}, + {name: exception, value: 0x00000000} + ] + +memory-regions: + # $sp is 0x000e0000, have bytes surrounding that address + - addr: 0x000dffe0 + UInt8: [ + 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, + 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, 0x10, 0x11, + 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18, 0x19, 0x1a, + 0x1b, 0x1c, 0x1d, 0x1e, 0x1f, 0x20, 0x21, 0x22, 0x23, + 0x24, 0x25, 0x26, 0x27, 0x28, 0x29, 0x2a, 0x2b, 0x2c, + 0x2d, 0x2e, 0x2f, 0x30, 0x31, 0x32, 0x33, 0x34, 0x35, + 0x36, 0x37, 0x38, 0x39, 0x3a, 0x3b, 0x3c, 0x3d, 0x3e, + 0x3f + ] diff --git a/lldb/test/API/macosx/arm-corefile-regctx/create-arm-corefiles.cpp b/lldb/test/API/macosx/arm-corefile-regctx/create-arm-corefiles.cpp deleted file mode 100644 index db39f12ecfb7e..0000000000000 --- a/lldb/test/API/macosx/arm-corefile-regctx/create-arm-corefiles.cpp +++ /dev/null @@ -1,266 +0,0 @@ -#include -#include -#include -#include -#include - - -// Normally these are picked up by including -// but that does a compile time check for the build host arch and -// only defines the ARM register context constants when building on -// an arm system. We're creating fake corefiles, and might be -// creating them on an intel system. -#ifndef ARM_THREAD_STATE -#define ARM_THREAD_STATE 1 -#endif -#ifndef ARM_THREAD_STATE_COUNT -#define ARM_THREAD_STATE_COUNT 17 -#endif -#ifndef ARM_EXCEPTION_STATE -#define ARM_EXCEPTION_STATE 3 -#endif -#ifndef ARM_EXCEPTION_STATE_COUNT -#define ARM_EXCEPTION_STATE_COUNT 3 -#endif -#ifndef ARM_THREAD_STATE64 -#define ARM_THREAD_STATE64 6 -#endif -#ifndef ARM_THREAD_STATE64_COUNT -#define ARM_THREAD_STATE64_COUNT 68 -#endif -#ifndef ARM_EXCEPTION_STATE64 -#define ARM_EXCEPTION_STATE64 7 -#endif -#ifndef ARM_EXCEPTION_STATE64_COUNT -#define ARM_EXCEPTION_STATE64_COUNT 4 -#endif - -union uint32_buf { - uint8_t bytebuf[4]; - uint32_t val; -}; - -union uint64_buf { - uint8_t bytebuf[8]; - uint64_t val; -}; - -void add_uint64(std::vector &buf, uint64_t val) { - uint64_buf conv; - conv.val = val; - for (int i = 0; i < 8; i++) - buf.push_back(conv.bytebuf[i]); -} - -void add_uint32(std::vector &buf, uint32_t val) { - uint32_buf conv; - conv.val = val; - for (int i = 0; i < 4; i++) - buf.push_back(conv.bytebuf[i]); -} - -std::vector armv7_lc_thread_load_command() { - std::vector data; - add_uint32(data, LC_THREAD); // thread_command.cmd - add_uint32(data, 104); // thread_command.cmdsize - add_uint32(data, ARM_THREAD_STATE); // thread_command.flavor - add_uint32(data, ARM_THREAD_STATE_COUNT); // thread_command.count - add_uint32(data, 0x00010000); // r0 - add_uint32(data, 0x00020000); // r1 - add_uint32(data, 0x00030000); // r2 - add_uint32(data, 0x00040000); // r3 - add_uint32(data, 0x00050000); // r4 - add_uint32(data, 0x00060000); // r5 - add_uint32(data, 0x00070000); // r6 - add_uint32(data, 0x00080000); // r7 - add_uint32(data, 0x00090000); // r8 - add_uint32(data, 0x000a0000); // r9 - add_uint32(data, 0x000b0000); // r10 - add_uint32(data, 0x000c0000); // r11 - add_uint32(data, 0x000d0000); // r12 - add_uint32(data, 0x000e0000); // sp - add_uint32(data, 0x000f0000); // lr - add_uint32(data, 0x00100000); // pc - add_uint32(data, 0x00110000); // cpsr - - add_uint32(data, ARM_EXCEPTION_STATE); // thread_command.flavor - add_uint32(data, ARM_EXCEPTION_STATE_COUNT); // thread_command.count - add_uint32(data, 0x00003f5c); // far - add_uint32(data, 0xf2000000); // esr - add_uint32(data, 0x00000000); // exception - - return data; -} - -std::vector arm64_lc_thread_load_command() { - std::vector data; - add_uint32(data, LC_THREAD); // thread_command.cmd - add_uint32(data, 312); // thread_command.cmdsize - add_uint32(data, ARM_THREAD_STATE64); // thread_command.flavor - add_uint32(data, ARM_THREAD_STATE64_COUNT); // thread_command.count - add_uint64(data, 0x0000000000000001); // x0 - add_uint64(data, 0x000000016fdff3c0); // x1 - add_uint64(data, 0x000000016fdff3d0); // x2 - add_uint64(data, 0x000000016fdff510); // x3 - add_uint64(data, 0x0000000000000000); // x4 - add_uint64(data, 0x0000000000000000); // x5 - add_uint64(data, 0x0000000000000000); // x6 - add_uint64(data, 0x0000000000000000); // x7 - add_uint64(data, 0x000000010000d910); // x8 - add_uint64(data, 0x0000000000000001); // x9 - add_uint64(data, 0xe1e88de000000000); // x10 - add_uint64(data, 0x0000000000000003); // x11 - add_uint64(data, 0x0000000000000148); // x12 - add_uint64(data, 0x0000000000004000); // x13 - add_uint64(data, 0x0000000000000008); // x14 - add_uint64(data, 0x0000000000000000); // x15 - add_uint64(data, 0x0000000000000000); // x16 - add_uint64(data, 0x0000000100003f5c); // x17 - add_uint64(data, 0x0000000000000000); // x18 - add_uint64(data, 0x0000000100003f5c); // x19 - add_uint64(data, 0x000000010000c000); // x20 - add_uint64(data, 0x000000010000d910); // x21 - add_uint64(data, 0x000000016fdff250); // x22 - add_uint64(data, 0x000000018ce12366); // x23 - add_uint64(data, 0x000000016fdff1d0); // x24 - add_uint64(data, 0x0000000000000001); // x25 - add_uint64(data, 0x0000000000000000); // x26 - add_uint64(data, 0x0000000000000000); // x27 - add_uint64(data, 0x0000000000000000); // x28 - add_uint64(data, 0x000000016fdff3a0); // fp - add_uint64(data, 0x000000018cd97f28); // lr - add_uint64(data, 0x000000016fdff140); // sp - add_uint64(data, 0x0000000100003f5c); // pc - add_uint32(data, 0x80001000); // cpsr - - add_uint32(data, 0x00000000); // padding - - add_uint32(data, ARM_EXCEPTION_STATE64); // thread_command.flavor - add_uint32(data, ARM_EXCEPTION_STATE64_COUNT); // thread_command.count - add_uint64(data, 0x0000000100003f5c); // far - add_uint32(data, 0xf2000000); // esr - add_uint32(data, 0x00000000); // exception - - return data; -} - -std::vector lc_segment(uint32_t fileoff, - uint32_t lc_segment_data_size) { - std::vector data; - // 0x000e0000 is the value of $sp in the armv7 LC_THREAD - uint32_t start_vmaddr = 0x000e0000 - (lc_segment_data_size / 2); - add_uint32(data, LC_SEGMENT); // segment_command.cmd - add_uint32(data, sizeof(struct segment_command)); // segment_command.cmdsize - for (int i = 0; i < 16; i++) - data.push_back(0); // segment_command.segname[16] - add_uint32(data, start_vmaddr); // segment_command.vmaddr - add_uint32(data, lc_segment_data_size); // segment_command.vmsize - add_uint32(data, fileoff); // segment_command.fileoff - add_uint32(data, lc_segment_data_size); // segment_command.filesize - add_uint32(data, 3); // segment_command.maxprot - add_uint32(data, 3); // segment_command.initprot - add_uint32(data, 0); // segment_command.nsects - add_uint32(data, 0); // segment_command.flags - - return data; -} - -enum arch { unspecified, armv7, arm64 }; - -int main(int argc, char **argv) { - if (argc != 3) { - fprintf(stderr, - "usage: create-arm-corefiles [armv7|arm64] \n"); - exit(1); - } - - arch arch = unspecified; - - if (strcmp(argv[1], "armv7") == 0) - arch = armv7; - else if (strcmp(argv[1], "arm64") == 0) - arch = arm64; - else { - fprintf(stderr, "unrecognized architecture %s\n", argv[1]); - exit(1); - } - - // An array of load commands (in the form of byte arrays) - std::vector> load_commands; - - // An array of corefile contents (page data, lc_note data, etc) - std::vector payload; - - // First add all the load commands / payload so we can figure out how large - // the load commands will actually be. - if (arch == armv7) { - load_commands.push_back(armv7_lc_thread_load_command()); - load_commands.push_back(lc_segment(0, 0)); - } else if (arch == arm64) { - load_commands.push_back(arm64_lc_thread_load_command()); - } - - int size_of_load_commands = 0; - for (const auto &lc : load_commands) - size_of_load_commands += lc.size(); - - int header_and_load_cmd_room = - sizeof(struct mach_header_64) + size_of_load_commands; - - // Erase the load commands / payload now that we know how much space is - // needed, redo it. - load_commands.clear(); - payload.clear(); - - int payload_fileoff = (header_and_load_cmd_room + 4096 - 1) & ~(4096 - 1); - - const int lc_segment_data_size = 64; - if (arch == armv7) { - load_commands.push_back(armv7_lc_thread_load_command()); - load_commands.push_back(lc_segment(payload_fileoff, lc_segment_data_size)); - } else if (arch == arm64) { - load_commands.push_back(arm64_lc_thread_load_command()); - } - - if (arch == armv7) - for (int i = 0; i < lc_segment_data_size; - i++) // from segment_command.filesize - payload.push_back(i); - - struct mach_header_64 mh; - int header_size; - if (arch == armv7) { - mh.magic = MH_MAGIC; - mh.cputype = CPU_TYPE_ARM; - mh.cpusubtype = CPU_SUBTYPE_ARM_V7M; - header_size = sizeof(struct mach_header); - } else if (arch == arm64) { - mh.magic = MH_MAGIC_64; - mh.cputype = CPU_TYPE_ARM64; - mh.cpusubtype = CPU_SUBTYPE_ARM64_ALL; - header_size = sizeof(struct mach_header_64); - } - mh.filetype = MH_CORE; - mh.ncmds = load_commands.size(); - mh.sizeofcmds = size_of_load_commands; - mh.flags = 0; - mh.reserved = 0; - - FILE *f = fopen(argv[2], "w"); - - if (f == nullptr) { - fprintf(stderr, "Unable to open file %s for writing\n", argv[2]); - exit(1); - } - - fwrite(&mh, header_size, 1, f); - - for (const auto &lc : load_commands) - fwrite(lc.data(), lc.size(), 1, f); - - fseek(f, payload_fileoff, SEEK_SET); - - fwrite(payload.data(), payload.size(), 1, f); - - fclose(f); -} diff --git a/lldb/test/API/macosx/riscv32-corefile/Makefile b/lldb/test/API/macosx/riscv32-corefile/Makefile deleted file mode 100644 index 04f268758d00c..0000000000000 --- a/lldb/test/API/macosx/riscv32-corefile/Makefile +++ /dev/null @@ -1,7 +0,0 @@ -MAKE_DSYM := NO -CXX_SOURCES := create-empty-riscv-corefile.cpp -EXE := create-empty-riscv-corefile - -all: create-empty-riscv-corefile - -include Makefile.rules diff --git a/lldb/test/API/macosx/riscv32-corefile/TestRV32MachOCorefile.py b/lldb/test/API/macosx/riscv32-corefile/TestRV32MachOCorefile.py index 8d11821d38985..449da70fb08ca 100644 --- a/lldb/test/API/macosx/riscv32-corefile/TestRV32MachOCorefile.py +++ b/lldb/test/API/macosx/riscv32-corefile/TestRV32MachOCorefile.py @@ -13,18 +13,16 @@ class TestRV32MachOCorefile(TestBase): NO_DEBUG_INFO_TESTCASE = True - @skipUnlessDarwin + @no_debug_info_test def test_riscv32_gpr_corefile_registers(self): - self.build() - create_corefile = self.getBuildArtifact("create-empty-riscv-corefile") corefile = self.getBuildArtifact("core") - call(create_corefile + " " + corefile, shell=True) + self.yaml2macho_core("riscv32-registers.yaml", corefile) target = self.dbg.CreateTarget("") process = target.LoadCore(corefile) process = target.GetProcess() - self.assertEqual(process.GetNumThreads(), 1) + self.assertEqual(process.GetNumThreads(), 2) thread = process.GetThreadAtIndex(0) self.assertEqual(thread.GetNumFrames(), 1) @@ -80,3 +78,12 @@ def test_riscv32_gpr_corefile_registers(self): val = idx | (idx << 8) | (idx << 16) | (idx << 24) self.assertEqual(gpr_regs.GetChildAtIndex(idx).GetValueAsUnsigned(), val) idx = idx + 1 + + thread = process.GetThreadAtIndex(1) + self.assertEqual(thread.GetNumFrames(), 1) + + frame = thread.GetFrameAtIndex(0) + gpr_regs = frame.registers.GetValueAtIndex(0) + + self.assertEqual(gpr_regs.GetChildAtIndex(0).GetValueAsUnsigned(), 0x90000000) + self.assertEqual(gpr_regs.GetChildAtIndex(32).GetValueAsUnsigned(), 0x90202020) diff --git a/lldb/test/API/macosx/riscv32-corefile/create-empty-riscv-corefile.cpp b/lldb/test/API/macosx/riscv32-corefile/create-empty-riscv-corefile.cpp deleted file mode 100644 index 907cca3b70b41..0000000000000 --- a/lldb/test/API/macosx/riscv32-corefile/create-empty-riscv-corefile.cpp +++ /dev/null @@ -1,116 +0,0 @@ -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include - -#define CPU_TYPE_RISCV 24 -#define CPU_SUBTYPE_RISCV_ALL 0 -#define RV32_THREAD_STATE 2 -// x0-x31 + pc, all 32-bit -#define RV32_THREAD_STATE_COUNT 33 - -union uint32_buf { - uint8_t bytebuf[4]; - uint32_t val; -}; - -union uint64_buf { - uint8_t bytebuf[8]; - uint64_t val; -}; - -void add_uint64(std::vector &buf, uint64_t val) { - uint64_buf conv; - conv.val = val; - for (int i = 0; i < 8; i++) - buf.push_back(conv.bytebuf[i]); -} - -void add_uint32(std::vector &buf, uint32_t val) { - uint32_buf conv; - conv.val = val; - for (int i = 0; i < 4; i++) - buf.push_back(conv.bytebuf[i]); -} - -std::vector lc_thread_load_command() { - std::vector data; - add_uint32(data, LC_THREAD); // thread_command.cmd - add_uint32(data, 4 + 4 + 4 + 4 + - (RV32_THREAD_STATE_COUNT * 4)); // thread_command.cmdsize - add_uint32(data, RV32_THREAD_STATE); // thread_command.flavor - add_uint32(data, RV32_THREAD_STATE_COUNT); // thread_command.count - for (int i = 0; i < RV32_THREAD_STATE_COUNT; i++) { - add_uint32(data, i | (i << 8) | (i << 16) | (i << 24)); - } - return data; -} - -int main(int argc, char **argv) { - if (argc != 2) { - fprintf(stderr, - "usage: create-empty-riscv-corefile output-corefile-name\n"); - exit(1); - } - - cpu_type_t cputype = CPU_TYPE_RISCV; - cpu_subtype_t cpusubtype = CPU_SUBTYPE_RISCV_ALL; - - // An array of load commands (in the form of byte arrays) - std::vector> load_commands; - - // An array of corefile contents (page data, lc_note data, etc) - std::vector payload; - - // First add all the load commands / payload so we can figure out how large - // the load commands will actually be. - load_commands.push_back(lc_thread_load_command()); - - int size_of_load_commands = 0; - for (const auto &lc : load_commands) - size_of_load_commands += lc.size(); - - int header_and_load_cmd_room = - sizeof(struct mach_header_64) + size_of_load_commands; - - // Erase the load commands / payload now that we know how much space is - // needed, redo it. - load_commands.clear(); - payload.clear(); - - load_commands.push_back(lc_thread_load_command()); - - struct mach_header mh; - mh.magic = MH_MAGIC; - mh.cputype = cputype; - - mh.cpusubtype = cpusubtype; - mh.filetype = MH_CORE; - mh.ncmds = load_commands.size(); - mh.sizeofcmds = size_of_load_commands; - mh.flags = 0; - - FILE *f = fopen(argv[1], "w"); - - if (f == nullptr) { - fprintf(stderr, "Unable to open file %s for writing\n", argv[1]); - exit(1); - } - - fwrite(&mh, sizeof(struct mach_header), 1, f); - - for (const auto &lc : load_commands) - fwrite(lc.data(), lc.size(), 1, f); - - fseek(f, header_and_load_cmd_room, SEEK_SET); - - fwrite(payload.data(), payload.size(), 1, f); - - fclose(f); -} diff --git a/lldb/test/API/macosx/riscv32-corefile/riscv32-registers.yaml b/lldb/test/API/macosx/riscv32-corefile/riscv32-registers.yaml new file mode 100644 index 0000000000000..81c725f1a4f0e --- /dev/null +++ b/lldb/test/API/macosx/riscv32-corefile/riscv32-registers.yaml @@ -0,0 +1,46 @@ +cpu: riscv +threads: + # (lldb) reg read + # % pbpaste | grep = | sed 's, ,,g' | awk -F= '{print "{name: " $1 ", value: " $2 "},"}' + - regsets: + - flavor: gpr + registers: [ + {name: zero, value: 0x00000000}, {name: ra, value: 0x01010101}, + {name: sp, value: 0x02020202}, {name: gp, value: 0x03030303}, + {name: tp, value: 0x04040404}, {name: t0, value: 0x05050505}, + {name: t1, value: 0x06060606}, {name: t2, value: 0x07070707}, + {name: fp, value: 0x08080808}, {name: s1, value: 0x09090909}, + {name: a0, value: 0x0a0a0a0a}, {name: a1, value: 0x0b0b0b0b}, + {name: a2, value: 0x0c0c0c0c}, {name: a3, value: 0x0d0d0d0d}, + {name: a4, value: 0x0e0e0e0e}, {name: a5, value: 0x0f0f0f0f}, + {name: a6, value: 0x10101010}, {name: a7, value: 0x11111111}, + {name: s2, value: 0x12121212}, {name: s3, value: 0x13131313}, + {name: s4, value: 0x14141414}, {name: s5, value: 0x15151515}, + {name: s6, value: 0x16161616}, {name: s7, value: 0x17171717}, + {name: s8, value: 0x18181818}, {name: s9, value: 0x19191919}, + {name: s10, value: 0x1a1a1a1a}, {name: s11, value: 0x1b1b1b1b}, + {name: t3, value: 0x1c1c1c1c}, {name: t4, value: 0x1d1d1d1d}, + {name: t5, value: 0x1e1e1e1e}, {name: t6, value: 0x1f1f1f1f}, + {name: pc, value: 0x20202020} + ] + - regsets: + - flavor: gpr + registers: [ + {name: zero, value: 0x90000000}, {name: ra, value: 0x91010101}, + {name: sp, value: 0x92020202}, {name: gp, value: 0x93030303}, + {name: tp, value: 0x94040404}, {name: t0, value: 0x95050505}, + {name: t1, value: 0x96060606}, {name: t2, value: 0x97070707}, + {name: fp, value: 0x98080808}, {name: s1, value: 0x99090909}, + {name: a0, value: 0x9a0a0a0a}, {name: a1, value: 0x9b0b0b0b}, + {name: a2, value: 0x9c0c0c0c}, {name: a3, value: 0x9d0d0d0d}, + {name: a4, value: 0x9e0e0e0e}, {name: a5, value: 0x9f0f0f0f}, + {name: a6, value: 0x90101010}, {name: a7, value: 0x91111111}, + {name: s2, value: 0x92121212}, {name: s3, value: 0x93131313}, + {name: s4, value: 0x94141414}, {name: s5, value: 0x95151515}, + {name: s6, value: 0x96161616}, {name: s7, value: 0x97171717}, + {name: s8, value: 0x98181818}, {name: s9, value: 0x19191919}, + {name: s10, value: 0x9a1a1a1a}, {name: s11, value: 0x9b1b1b1b}, + {name: t3, value: 0x9c1c1c1c}, {name: t4, value: 0x9d1d1d1d}, + {name: t5, value: 0x9e1e1e1e}, {name: t6, value: 0x9f1f1f1f}, + {name: pc, value: 0x90202020} + ] diff --git a/lldb/test/CMakeLists.txt b/lldb/test/CMakeLists.txt index 555a6eddd7fea..4eab922a32205 100644 --- a/lldb/test/CMakeLists.txt +++ b/lldb/test/CMakeLists.txt @@ -230,6 +230,7 @@ add_lldb_test_dependency( llvm-pdbutil llvm-readobj llvm-ar + yaml2macho-core ) if(TARGET lld) diff --git a/lldb/tools/CMakeLists.txt b/lldb/tools/CMakeLists.txt index 17d2dfda0c6c0..edbc855f49cae 100644 --- a/lldb/tools/CMakeLists.txt +++ b/lldb/tools/CMakeLists.txt @@ -22,6 +22,8 @@ if (LLDB_CAN_USE_LLDB_SERVER) add_lldb_tool_subdirectory(lldb-server) endif() +add_lldb_tool_subdirectory(yaml2macho-core) + # BEGIN Swift Mods if (LLDB_ENABLE_SWIFT_SUPPORT) add_subdirectory(repl/swift) diff --git a/lldb/tools/yaml2macho-core/CMakeLists.txt b/lldb/tools/yaml2macho-core/CMakeLists.txt new file mode 100644 index 0000000000000..5d8ee3645d52f --- /dev/null +++ b/lldb/tools/yaml2macho-core/CMakeLists.txt @@ -0,0 +1,14 @@ +add_lldb_tool(yaml2macho-core + CoreSpec.cpp + LCNoteWriter.cpp + MemoryWriter.cpp + ThreadWriter.cpp + Utility.cpp + yaml2macho.cpp + + LINK_COMPONENTS + Support + LINK_LIBS + lldbUtility + ${LLDB_SYSTEM_LIBS} +) diff --git a/lldb/tools/yaml2macho-core/CoreSpec.cpp b/lldb/tools/yaml2macho-core/CoreSpec.cpp new file mode 100644 index 0000000000000..f898b30641ddf --- /dev/null +++ b/lldb/tools/yaml2macho-core/CoreSpec.cpp @@ -0,0 +1,165 @@ +//===-- CoreSpec.cpp ------------------------------------------------------===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// + +#include "CoreSpec.h" +#include "llvm/BinaryFormat/MachO.h" +#include "llvm/Support/YAMLTraits.h" +#include +#include + +using llvm::yaml::Input; +using llvm::yaml::IO; +using llvm::yaml::MappingTraits; + +template <> struct MappingTraits { + static void mapping(IO &io, RegisterNameAndValue &name_value) { + io.mapRequired("name", name_value.name); + io.mapRequired("value", name_value.value); + } +}; +LLVM_YAML_IS_SEQUENCE_VECTOR(RegisterNameAndValue) + +template <> struct llvm::yaml::ScalarEnumerationTraits { + static void enumeration(IO &io, RegisterFlavor &flavor) { + io.enumCase(flavor, "gpr", RegisterFlavor::GPR); + io.enumCase(flavor, "fpr", RegisterFlavor::FPR); + io.enumCase(flavor, "exc", RegisterFlavor::EXC); + } +}; + +template <> struct MappingTraits { + static void mapping(IO &io, RegisterSet ®set) { + io.mapRequired("flavor", regset.flavor); + io.mapRequired("registers", regset.registers); + } +}; +LLVM_YAML_IS_SEQUENCE_VECTOR(RegisterSet) + +template <> struct MappingTraits { + static void mapping(IO &io, Thread &thread) { + io.mapRequired("regsets", thread.regsets); + } +}; +LLVM_YAML_IS_SEQUENCE_VECTOR(Thread) + +template <> struct MappingTraits { + static void mapping(IO &io, MemoryRegion &memory) { + io.mapRequired("addr", memory.addr); + io.mapOptional("UInt8", memory.bytes); + io.mapOptional("UInt32", memory.words); + io.mapOptional("UInt64", memory.doublewords); + + if (memory.bytes.size()) { + memory.type = MemoryType::UInt8; + memory.size = memory.bytes.size(); + } else if (memory.words.size()) { + memory.type = MemoryType::UInt32; + memory.size = memory.words.size() * 4; + } else if (memory.doublewords.size()) { + memory.type = MemoryType::UInt64; + memory.size = memory.doublewords.size() * 8; + } + } +}; +LLVM_YAML_IS_SEQUENCE_VECTOR(MemoryRegion) + +template <> struct MappingTraits { + static void mapping(IO &io, Binary &binary) { + io.mapOptional("name", binary.name); + io.mapRequired("uuid", binary.uuid); + std::optional va, slide; + io.mapOptional("virtual-address", va); + io.mapOptional("slide", slide); + if (va && *va != UINT64_MAX) { + binary.value_is_slide = false; + binary.value = *va; + } else if (slide && *slide != UINT64_MAX) { + binary.value_is_slide = true; + binary.value = *slide; + } else { + fprintf(stderr, + "No virtual-address or slide specified for binary %s, aborting\n", + binary.uuid.c_str()); + exit(1); + } + } +}; +LLVM_YAML_IS_SEQUENCE_VECTOR(Binary) + +template <> struct llvm::yaml::MappingTraits { + static void mapping(IO &io, AddressableBits &addr_bits) { + std::optional addressable_bits; + io.mapOptional("num-bits", addressable_bits); + if (addressable_bits) { + addr_bits.lowmem_bits = *addressable_bits; + addr_bits.highmem_bits = *addressable_bits; + } else { + io.mapOptional("lowmem-num-bits", addr_bits.lowmem_bits); + io.mapOptional("highmem-num-bits", addr_bits.highmem_bits); + } + } +}; + +template <> struct MappingTraits { + static void mapping(IO &io, CoreSpec &corespec) { + std::string cpuname; + io.mapRequired("cpu", cpuname); + if (cpuname == "armv7m") { + corespec.cputype = llvm::MachO::CPU_TYPE_ARM; + corespec.cpusubtype = llvm::MachO::CPU_SUBTYPE_ARM_V7M; + } else if (cpuname == "armv7") { + corespec.cputype = llvm::MachO::CPU_TYPE_ARM; + corespec.cpusubtype = llvm::MachO::CPU_SUBTYPE_ARM_ALL; + } else if (cpuname == "riscv") { + corespec.cputype = llvm::MachO::CPU_TYPE_RISCV; + corespec.cpusubtype = llvm::MachO::CPU_SUBTYPE_RISCV_ALL; + } else if (cpuname == "arm64") { + corespec.cputype = llvm::MachO::CPU_TYPE_ARM64; + corespec.cpusubtype = llvm::MachO::CPU_SUBTYPE_ARM64_ALL; + } else { + fprintf(stderr, "Unrecognized cpu name %s, exiting.\n", cpuname.c_str()); + exit(1); + } + io.mapOptional("threads", corespec.threads); + io.mapOptional("memory-regions", corespec.memory_regions); + if (corespec.cputype == llvm::MachO::CPU_TYPE_ARM || + corespec.cputype == llvm::MachO::CPU_TYPE_RISCV) + corespec.wordsize = 4; + else if (corespec.cputype == llvm::MachO::CPU_TYPE_ARM64) + corespec.wordsize = 8; + else { + fprintf(stderr, + "Unrecognized cputype, could not set wordsize, exiting.\n"); + exit(1); + } + io.mapOptional("addressable-bits", corespec.addressable_bits); + io.mapOptional("binaries", corespec.binaries); + if (corespec.addressable_bits) { + if (!corespec.addressable_bits->lowmem_bits) + corespec.addressable_bits->lowmem_bits = corespec.wordsize * 8; + if (!corespec.addressable_bits->highmem_bits) + corespec.addressable_bits->highmem_bits = corespec.wordsize * 8; + } + } +}; + +CoreSpec from_yaml(char *buf, size_t len) { + llvm::StringRef file_corespec_strref(buf, len); + + Input yin(file_corespec_strref); + + CoreSpec v; + yin >> v; + + if (yin.error()) { + fprintf(stderr, "Unable to parse YAML, exiting\n"); + exit(1); + } + + return v; +} diff --git a/lldb/tools/yaml2macho-core/CoreSpec.h b/lldb/tools/yaml2macho-core/CoreSpec.h new file mode 100644 index 0000000000000..5c27cc96bdaad --- /dev/null +++ b/lldb/tools/yaml2macho-core/CoreSpec.h @@ -0,0 +1,80 @@ +//===----------------------------------------------------------------------===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// +/// +/// \file +/// CoreSpec holds the internal representation of the data that will be +/// written into the corefile. Theads, register sets within threads, registers +/// within register sets. Block of memory. Metadata about the CPU or binaries +/// that were present. +//===----------------------------------------------------------------------===// + +#ifndef YAML2MACHOCOREFILE_CORESPEC_H +#define YAML2MACHOCOREFILE_CORESPEC_H + +#include +#include +#include +#include + +struct RegisterNameAndValue { + std::string name; + uint64_t value; +}; + +enum RegisterFlavor { GPR = 0, FPR, EXC }; + +struct RegisterSet { + RegisterFlavor flavor; + std::vector registers; +}; + +struct Thread { + std::vector regsets; +}; + +enum MemoryType { UInt8 = 0, UInt32, UInt64 }; + +struct MemoryRegion { + uint64_t addr; + MemoryType type; + uint32_t size; + // One of the following formats. + std::vector bytes; + std::vector words; + std::vector doublewords; +}; + +struct AddressableBits { + std::optional lowmem_bits; + std::optional highmem_bits; +}; + +struct Binary { + std::string name; + std::string uuid; + bool value_is_slide; + uint64_t value; +}; + +struct CoreSpec { + uint32_t cputype; + uint32_t cpusubtype; + int wordsize; + + std::vector threads; + std::vector memory_regions; + + std::optional addressable_bits; + std::vector binaries; + + CoreSpec() : cputype(0), cpusubtype(0), wordsize(0) {} +}; + +CoreSpec from_yaml(char *buf, size_t len); + +#endif diff --git a/lldb/tools/yaml2macho-core/LCNoteWriter.cpp b/lldb/tools/yaml2macho-core/LCNoteWriter.cpp new file mode 100644 index 0000000000000..824a1de7ddfaa --- /dev/null +++ b/lldb/tools/yaml2macho-core/LCNoteWriter.cpp @@ -0,0 +1,97 @@ +//===-- LCNoteWriter.cpp --------------------------------------------------===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// + +#include "LCNoteWriter.h" +#include "Utility.h" +#include "lldb/Utility/UUID.h" +#include "llvm/BinaryFormat/MachO.h" +#include +#include + +void create_lc_note_binary_load_cmd(const CoreSpec &spec, + std::vector &cmds, + const Binary &binary, + std::vector &payload_bytes, + off_t data_offset) { + + // Add the payload bytes to payload_bytes. + size_t starting_payload_size = payload_bytes.size(); + add_uint32(payload_bytes, 1); // version + lldb_private::UUID uuid; + uuid.SetFromStringRef(binary.uuid); + for (size_t i = 0; i < uuid.GetBytes().size(); i++) + payload_bytes.push_back(uuid.GetBytes().data()[i]); + if (binary.value_is_slide) { + add_uint64(payload_bytes, UINT64_MAX); // address + add_uint64(payload_bytes, binary.value); // slide + } else { + add_uint64(payload_bytes, binary.value); // address + add_uint64(payload_bytes, UINT64_MAX); // slide + } + if (binary.name.empty()) { + payload_bytes.push_back(0); // name_cstring + } else { + size_t len = binary.name.size(); + for (size_t i = 0; i < len; i++) + payload_bytes.push_back(binary.name[i]); + payload_bytes.push_back(0); // name_cstring + } + + size_t payload_size = payload_bytes.size() - starting_payload_size; + // Pad out the entry to a 4-byte aligned size. + if (payload_bytes.size() % 4 != 0) { + size_t pad_bytes = + ((payload_bytes.size() + 4 - 1) & ~(4 - 1)) - payload_bytes.size(); + for (size_t i = 0; i < pad_bytes; i++) + payload_bytes.push_back(0); + } + + // Add the load command bytes to cmds. + add_uint32(cmds, llvm::MachO::LC_NOTE); + add_uint32(cmds, sizeof(struct llvm::MachO::note_command)); + char cmdname[16]; + memset(cmdname, '\0', sizeof(cmdname)); + strcpy(cmdname, "load binary"); + for (int i = 0; i < 16; i++) + cmds.push_back(cmdname[i]); + add_uint64(cmds, data_offset); + add_uint64(cmds, payload_size); +} + +void create_lc_note_addressable_bits(const CoreSpec &spec, + std::vector &cmds, + const AddressableBits &addr_bits, + std::vector &payload_bytes, + off_t data_offset) { + // Add the payload bytes to payload_bytes. + size_t starting_payload_size = payload_bytes.size(); + add_uint32(payload_bytes, 4); // version + + add_uint32(payload_bytes, *addr_bits.lowmem_bits); // low memory + add_uint32(payload_bytes, *addr_bits.highmem_bits); // high memory + add_uint32(payload_bytes, 0); // reserved + size_t payload_size = payload_bytes.size() - starting_payload_size; + // Pad out the entry to a 4-byte aligned size. + if (payload_bytes.size() % 4 != 0) { + size_t pad_bytes = + ((payload_bytes.size() + 4 - 1) & ~(4 - 1)) - payload_bytes.size(); + for (size_t i = 0; i < pad_bytes; i++) + payload_bytes.push_back(0); + } + + // Add the load command bytes to cmds. + add_uint32(cmds, llvm::MachO::LC_NOTE); + add_uint32(cmds, sizeof(struct llvm::MachO::note_command)); + char cmdname[16]; + memset(cmdname, '\0', sizeof(cmdname)); + strcpy(cmdname, "addrable bits"); + for (int i = 0; i < 16; i++) + cmds.push_back(cmdname[i]); + add_uint64(cmds, data_offset); + add_uint64(cmds, payload_size); +} diff --git a/lldb/tools/yaml2macho-core/LCNoteWriter.h b/lldb/tools/yaml2macho-core/LCNoteWriter.h new file mode 100644 index 0000000000000..ae04faabe828b --- /dev/null +++ b/lldb/tools/yaml2macho-core/LCNoteWriter.h @@ -0,0 +1,33 @@ +//===----------------------------------------------------------------------===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// +/// \file +/// Functions to add an LC_NOTE load command to the corefile's load commands, +/// and supply the payload of that LC_NOTE separately. +//===----------------------------------------------------------------------===// + +#ifndef YAML2MACHOCOREFILE_LCNOTEWRITER_H +#define YAML2MACHOCOREFILE_LCNOTEWRITER_H + +#include "CoreSpec.h" + +#include +#include + +void create_lc_note_binary_load_cmd(const CoreSpec &spec, + std::vector &cmds, + const Binary &binary, + std::vector &payload_bytes, + off_t data_offset); + +void create_lc_note_addressable_bits(const CoreSpec &spec, + std::vector &cmds, + const AddressableBits &addr_bits, + std::vector &payload_bytes, + off_t data_offset); + +#endif diff --git a/lldb/tools/yaml2macho-core/MemoryWriter.cpp b/lldb/tools/yaml2macho-core/MemoryWriter.cpp new file mode 100644 index 0000000000000..4ace2894b6be5 --- /dev/null +++ b/lldb/tools/yaml2macho-core/MemoryWriter.cpp @@ -0,0 +1,56 @@ +//===-- MemoryWriter.cpp --------------------------------------------------===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// + +#include "MemoryWriter.h" +#include "CoreSpec.h" +#include "Utility.h" +#include "llvm/BinaryFormat/MachO.h" + +void create_lc_segment_cmd(const CoreSpec &spec, std::vector &cmds, + const MemoryRegion &memory, off_t data_offset) { + if (spec.wordsize == 8) { + // Add the bytes for a segment_command_64 from + add_uint32(cmds, llvm::MachO::LC_SEGMENT_64); + add_uint32(cmds, sizeof(struct llvm::MachO::segment_command_64)); + for (int i = 0; i < 16; i++) + cmds.push_back(0); + add_uint64(cmds, memory.addr); // segment_command_64.vmaddr + add_uint64(cmds, memory.size); // segment_command_64.vmsize + add_uint64(cmds, data_offset); // segment_command_64.fileoff + add_uint64(cmds, memory.size); // segment_command_64.filesize + } else { + // Add the bytes for a segment_command from + add_uint32(cmds, llvm::MachO::LC_SEGMENT); + add_uint32(cmds, sizeof(struct llvm::MachO::segment_command)); + for (int i = 0; i < 16; i++) + cmds.push_back(0); + add_uint32(cmds, memory.addr); // segment_command_64.vmaddr + add_uint32(cmds, memory.size); // segment_command_64.vmsize + add_uint32(cmds, data_offset); // segment_command_64.fileoff + add_uint32(cmds, memory.size); // segment_command_64.filesize + } + add_uint32(cmds, 3); // segment_command_64.maxprot + add_uint32(cmds, 3); // segment_command_64.initprot + add_uint32(cmds, 0); // segment_command_64.nsects + add_uint32(cmds, 0); // segment_command_64.flags +} + +void create_memory_bytes(const CoreSpec &spec, const MemoryRegion &memory, + std::vector &buf) { + if (memory.type == MemoryType::UInt8) + for (uint8_t byte : memory.bytes) + buf.push_back(byte); + + if (memory.type == MemoryType::UInt32) + for (uint32_t word : memory.words) + add_uint32(buf, word); + + if (memory.type == MemoryType::UInt64) + for (uint64_t word : memory.doublewords) + add_uint64(buf, word); +} diff --git a/lldb/tools/yaml2macho-core/MemoryWriter.h b/lldb/tools/yaml2macho-core/MemoryWriter.h new file mode 100644 index 0000000000000..2bef00cdb764f --- /dev/null +++ b/lldb/tools/yaml2macho-core/MemoryWriter.h @@ -0,0 +1,26 @@ +//===----------------------------------------------------------------------===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// +/// \file +/// Functions to emit the LC_SEGMENT load command, and to provide the bytes +/// that appear later in the corefile. +//===----------------------------------------------------------------------===// + +#ifndef YAML2MACHOCOREFILE_MEMORYWRITER_H +#define YAML2MACHOCOREFILE_MEMORYWRITER_H + +#include "CoreSpec.h" + +#include + +void create_lc_segment_cmd(const CoreSpec &spec, std::vector &cmds, + const MemoryRegion &memory, off_t data_offset); + +void create_memory_bytes(const CoreSpec &spec, const MemoryRegion &memory, + std::vector &buf); + +#endif diff --git a/lldb/tools/yaml2macho-core/ThreadWriter.cpp b/lldb/tools/yaml2macho-core/ThreadWriter.cpp new file mode 100644 index 0000000000000..40c70d92970c7 --- /dev/null +++ b/lldb/tools/yaml2macho-core/ThreadWriter.cpp @@ -0,0 +1,200 @@ +//===-- ThreadWriter.cpp --------------------------------------------------===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// + +#include "ThreadWriter.h" +#include "CoreSpec.h" +#include "Utility.h" +#include "llvm/BinaryFormat/MachO.h" +#include +#include + +#define ARM_THREAD_STATE 1 +#define ARM_THREAD_STATE_COUNT 17 +#define ARM_EXCEPTION_STATE 3 +#define ARM_EXCEPTION_STATE_COUNT 3 + +std::vector::const_iterator +find_by_name(std::vector::const_iterator first, + std::vector::const_iterator last, + const char *name) { + for (; first != last; ++first) + if (first->name == name) + return first; + return last; +} + +void add_reg_value(CoreSpec &spec, std::vector &buf, + const std::vector ®isters, + const char *regname, int regsize) { + const auto it = find_by_name(registers.begin(), registers.end(), regname); + if (it != registers.end()) { + if (regsize == 8) + add_uint64(buf, it->value); + else + add_uint32(buf, it->value); + } else { + if (regsize == 8) + add_uint64(buf, 0); + else + add_uint32(buf, 0); + } +} + +void add_reg_value_32(CoreSpec &spec, std::vector &buf, + const std::vector ®isters, + const char *regname) { + add_reg_value(spec, buf, registers, regname, 4); +} + +void add_reg_value_64(CoreSpec &spec, std::vector &buf, + const std::vector ®isters, + const char *regname) { + add_reg_value(spec, buf, registers, regname, 8); +} + +void add_lc_threads_armv7(CoreSpec &spec, + std::vector> &load_commands) { + for (const Thread &th : spec.threads) { + std::vector lc; + int size_of_all_flavors = 0; + for (const RegisterSet &rs : th.regsets) { + if (rs.flavor == RegisterFlavor::GPR) + size_of_all_flavors += (ARM_THREAD_STATE_COUNT * 4); + if (rs.flavor == RegisterFlavor::EXC) + size_of_all_flavors += (ARM_EXCEPTION_STATE_COUNT * 4); + } + int cmdsize = 4 * 2; // cmd, cmdsize + cmdsize += 4 * 2 * th.regsets.size(); // flavor, count (per register flavor) + cmdsize += size_of_all_flavors; // size of all the register set data + + add_uint32(lc, llvm::MachO::LC_THREAD); // thread_command.cmd + add_uint32(lc, cmdsize); // thread_command.cmdsize + for (const RegisterSet &rs : th.regsets) { + if (rs.flavor == RegisterFlavor::GPR) { + add_uint32(lc, ARM_THREAD_STATE); // thread_command.flavor + add_uint32(lc, ARM_THREAD_STATE_COUNT); // thread_command.count + const char *names[] = {"r0", "r1", "r2", "r3", "r4", "r5", + "r6", "r7", "r8", "r9", "r10", "r11", + "r12", "sp", "lr", "pc", "cpsr", nullptr}; + for (int i = 0; names[i]; i++) + add_reg_value_32(spec, lc, rs.registers, names[i]); + } + if (rs.flavor == RegisterFlavor::EXC) { + add_uint32(lc, ARM_EXCEPTION_STATE); // thread_command.flavor + add_uint32(lc, ARM_EXCEPTION_STATE_COUNT); // thread_command.count + const char *names[] = {"far", "esr", "exception", nullptr}; + for (int i = 0; names[i]; i++) + add_reg_value_32(spec, lc, rs.registers, names[i]); + } + } + load_commands.push_back(lc); + } +} + +#define ARM_THREAD_STATE64 6 +#define ARM_THREAD_STATE64_COUNT 68 +#define ARM_EXCEPTION_STATE64 7 +#define ARM_EXCEPTION_STATE64_COUNT 4 + +void add_lc_threads_arm64(CoreSpec &spec, + std::vector> &load_commands) { + for (const Thread &th : spec.threads) { + std::vector lc; + int size_of_all_flavors = 0; + for (const RegisterSet &rs : th.regsets) { + if (rs.flavor == RegisterFlavor::GPR) + size_of_all_flavors += (ARM_THREAD_STATE64_COUNT * 4); + if (rs.flavor == RegisterFlavor::EXC) + size_of_all_flavors += (ARM_EXCEPTION_STATE64_COUNT * 4); + } + int cmdsize = 4 * 2; // cmd, cmdsize + cmdsize += 4 * 2 * th.regsets.size(); // flavor, count (per register flavor) + cmdsize += size_of_all_flavors; // size of all the register set data + + add_uint32(lc, llvm::MachO::LC_THREAD); // thread_command.cmd + add_uint32(lc, cmdsize); // thread_command.cmdsize + + for (const RegisterSet &rs : th.regsets) { + if (rs.flavor == RegisterFlavor::GPR) { + add_uint32(lc, ARM_THREAD_STATE64); // thread_command.flavor + add_uint32(lc, ARM_THREAD_STATE64_COUNT); // thread_command.count + const char *names[] = {"x0", "x1", "x2", "x3", "x4", "x5", "x6", + "x7", "x8", "x9", "x10", "x11", "x12", "x13", + "x14", "x15", "x16", "x17", "x18", "x19", "x20", + "x21", "x22", "x23", "x24", "x25", "x26", "x27", + "x28", "fp", "lr", "sp", "pc", nullptr}; + for (int i = 0; names[i]; i++) + add_reg_value_64(spec, lc, rs.registers, names[i]); + + // cpsr is a 4-byte reg + add_reg_value_32(spec, lc, rs.registers, "cpsr"); + // the 4 bytes of zeroes + add_uint32(lc, 0); + } + if (rs.flavor == RegisterFlavor::EXC) { + add_uint32(lc, ARM_EXCEPTION_STATE64); // thread_command.flavor + add_uint32(lc, + ARM_EXCEPTION_STATE64_COUNT); // thread_command.count + add_reg_value_64(spec, lc, rs.registers, "far"); + add_reg_value_32(spec, lc, rs.registers, "esr"); + add_reg_value_32(spec, lc, rs.registers, "exception"); + } + } + load_commands.push_back(lc); + } +} + +#define RV32_THREAD_STATE 2 +#define RV32_THREAD_STATE_COUNT 33 + +void add_lc_threads_riscv(CoreSpec &spec, + std::vector> &load_commands) { + for (const Thread &th : spec.threads) { + std::vector lc; + int size_of_all_flavors = 0; + for (const RegisterSet &rs : th.regsets) { + if (rs.flavor == RegisterFlavor::GPR) + size_of_all_flavors += (RV32_THREAD_STATE_COUNT * 4); + } + int cmdsize = 4 * 2; // cmd, cmdsize + cmdsize += 4 * 2 * th.regsets.size(); // flavor, count (per register flavor) + cmdsize += size_of_all_flavors; // size of all the register set data + + add_uint32(lc, llvm::MachO::LC_THREAD); // thread_command.cmd + add_uint32(lc, cmdsize); // thread_command.cmdsize + for (const RegisterSet &rs : th.regsets) { + if (rs.flavor == RegisterFlavor::GPR) { + add_uint32(lc, RV32_THREAD_STATE); // thread_command.flavor + add_uint32(lc, RV32_THREAD_STATE_COUNT); // thread_command.count + const char *names[] = {"zero", "ra", "sp", "gp", "tp", "t0", "t1", + "t2", "fp", "s1", "a0", "a1", "a2", "a3", + "a4", "a5", "a6", "a7", "s2", "s3", "s4", + "s5", "s6", "s7", "s8", "s9", "s10", "s11", + "t3", "t4", "t5", "t6", "pc", nullptr}; + for (int i = 0; names[i]; i++) + add_reg_value_32(spec, lc, rs.registers, names[i]); + } + } + load_commands.push_back(lc); + } +} + +void add_lc_threads(CoreSpec &spec, + std::vector> &load_commands) { + if (spec.cputype == llvm::MachO::CPU_TYPE_ARM) + add_lc_threads_armv7(spec, load_commands); + else if (spec.cputype == llvm::MachO::CPU_TYPE_ARM64) + add_lc_threads_arm64(spec, load_commands); + else if (spec.cputype == llvm::MachO::CPU_TYPE_RISCV) + add_lc_threads_riscv(spec, load_commands); + else { + fprintf(stderr, + "Unrecognized cputype, could not write LC_THREAD. Exiting.\n"); + exit(1); + } +} diff --git a/lldb/tools/yaml2macho-core/ThreadWriter.h b/lldb/tools/yaml2macho-core/ThreadWriter.h new file mode 100644 index 0000000000000..58ce4abdf0d15 --- /dev/null +++ b/lldb/tools/yaml2macho-core/ThreadWriter.h @@ -0,0 +1,24 @@ +//===----------------------------------------------------------------------===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// +/// \file +/// Functions to emit LC_THREAD bytes to the corefile's Mach-O load commands, +/// specifying the threads, the register sets ("flavors") within those threads, +/// and all of the registers within those register sets. +//===----------------------------------------------------------------------===// + +#ifndef YAML2MACHOCOREFILE_THREADWRITER_H +#define YAML2MACHOCOREFILE_THREADWRITER_H + +#include "CoreSpec.h" + +#include + +void add_lc_threads(CoreSpec &spec, + std::vector> &load_commands); + +#endif diff --git a/lldb/tools/yaml2macho-core/Utility.cpp b/lldb/tools/yaml2macho-core/Utility.cpp new file mode 100644 index 0000000000000..2414c2f039323 --- /dev/null +++ b/lldb/tools/yaml2macho-core/Utility.cpp @@ -0,0 +1,22 @@ +//===-- Utility.cpp -------------------------------------------------------===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// + +#include "Utility.h" +#include "CoreSpec.h" + +void add_uint64(std::vector &buf, uint64_t val) { + uint8_t *p = reinterpret_cast(&val); + for (int i = 0; i < 8; i++) + buf.push_back(*p++); +} + +void add_uint32(std::vector &buf, uint32_t val) { + uint8_t *p = reinterpret_cast(&val); + for (int i = 0; i < 4; i++) + buf.push_back(*p++); +} diff --git a/lldb/tools/yaml2macho-core/Utility.h b/lldb/tools/yaml2macho-core/Utility.h new file mode 100644 index 0000000000000..ff05858c4dee2 --- /dev/null +++ b/lldb/tools/yaml2macho-core/Utility.h @@ -0,0 +1,18 @@ +//===----------------------------------------------------------------------===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// + +#ifndef YAML2MACHOCOREFILE_UTILITY_H +#define YAML2MACHOCOREFILE_UTILITY_H + +#include "CoreSpec.h" +#include + +void add_uint64(std::vector &buf, uint64_t val); +void add_uint32(std::vector &buf, uint32_t val); + +#endif diff --git a/lldb/tools/yaml2macho-core/yaml2macho.cpp b/lldb/tools/yaml2macho-core/yaml2macho.cpp new file mode 100644 index 0000000000000..91445e191969f --- /dev/null +++ b/lldb/tools/yaml2macho-core/yaml2macho.cpp @@ -0,0 +1,251 @@ +//===-- main.cppp ---------------------------------------------------------===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// + +#include "CoreSpec.h" +#include "LCNoteWriter.h" +#include "MemoryWriter.h" +#include "ThreadWriter.h" +#include "Utility.h" +#include "llvm/BinaryFormat/MachO.h" +#include "llvm/Support/CommandLine.h" +#include +#include +#include +#include + +std::vector get_fields_from_delimited_string(std::string str, + const char delim) { + std::vector result; + std::string::size_type prev = std::string::npos; + std::string::size_type next = str.find(delim); + if (str.empty()) { + return result; + } + if (next == std::string::npos) { + result.push_back(str); + } else { + result.push_back(std::string(str, 0, next)); + prev = next; + while ((next = str.find(delim, prev + 1)) != std::string::npos) { + result.push_back(std::string(str, prev + 1, next - prev - 1)); + prev = next; + } + result.push_back(std::string(str, prev + 1)); + } + return result; +} + +llvm::cl::opt InputFilename("i", llvm::cl::Required, + llvm::cl::desc("input yaml filename"), + llvm::cl::value_desc("input")); +llvm::cl::opt + OutputFilename("o", llvm::cl::Required, + llvm::cl::desc("output core filenames"), + llvm::cl::value_desc("output")); +llvm::cl::list + UUIDs("u", llvm::cl::desc("uuid of binary loaded at slide 0"), + llvm::cl::value_desc("uuid")); +llvm::cl::list + UUIDAndVAs("L", llvm::cl::desc("UUID,virtual-address-loaded-at"), + llvm::cl::value_desc("--uuid-and-load-addr")); +llvm::cl::opt + AddressableBitsOverride("A", + llvm::cl::desc("number of bits used in addressing"), + llvm::cl::value_desc("--address-bits")); + +int main(int argc, char **argv) { + llvm::cl::ParseCommandLineOptions(argc, argv); + + if (InputFilename.empty() || OutputFilename.empty()) { + fprintf(stderr, "Missing input or outpur file.\n"); + exit(1); + } + + struct stat sb; + + if (stat(InputFilename.c_str(), &sb) == -1) { + fprintf(stderr, "Unable to stat %s, exiting\n", InputFilename.c_str()); + exit(1); + } + + FILE *input = fopen(InputFilename.c_str(), "r"); + if (!input) { + fprintf(stderr, "Unable to open %s, exiting\n", InputFilename.c_str()); + exit(1); + } + auto file_corespec = std::make_unique(sb.st_size); + if (fread(file_corespec.get(), sb.st_size, 1, input) != 1) { + fprintf(stderr, "Unable to read all of %s, exiting\n", + InputFilename.c_str()); + exit(1); + } + CoreSpec spec = from_yaml(file_corespec.get(), sb.st_size); + fclose(input); + + for (const std::string &uuid : UUIDs) { + Binary binary; + binary.uuid = uuid; + binary.value = 0; + binary.value_is_slide = true; + spec.binaries.push_back(binary); + } + + for (const std::string &uuid_and_va : UUIDAndVAs) { + std::vector parts = + get_fields_from_delimited_string(uuid_and_va, ','); + + std::string uuid = parts[0]; + uint64_t va = std::strtoull(parts[1].c_str(), nullptr, 16); + Binary binary; + binary.uuid = uuid; + binary.value = va; + binary.value_is_slide = false; + spec.binaries.push_back(binary); + } + + if (AddressableBitsOverride) { + AddressableBits bits; + bits.lowmem_bits = bits.highmem_bits = AddressableBitsOverride; + spec.addressable_bits = bits; + } + + // An array of load commands + std::vector> load_commands; + + // An array of corefile contents (memory regions) + std::vector payload; + + // First add all the load commands / payload so we can figure out how large + // the load commands will be. + + add_lc_threads(spec, load_commands); + for (size_t i = 0; i < spec.memory_regions.size(); i++) { + std::vector segment_command_bytes; + create_lc_segment_cmd(spec, segment_command_bytes, spec.memory_regions[i], + 0); + load_commands.push_back(segment_command_bytes); + } + + if (spec.binaries.size() > 0) + for (const Binary &binary : spec.binaries) { + std::vector segment_command_bytes; + std::vector payload_bytes; + create_lc_note_binary_load_cmd(spec, segment_command_bytes, binary, + payload_bytes, 0); + load_commands.push_back(segment_command_bytes); + } + if (spec.addressable_bits) { + std::vector segment_command_bytes; + std::vector payload_bytes; + create_lc_note_addressable_bits(spec, segment_command_bytes, + *spec.addressable_bits, payload_bytes, 0); + load_commands.push_back(segment_command_bytes); + } + + off_t size_of_load_commands = 0; + for (const auto &lc : load_commands) + size_of_load_commands += lc.size(); + + off_t header_and_load_cmd_room = + sizeof(llvm::MachO::mach_header_64) + size_of_load_commands; + off_t initial_payload_fileoff = header_and_load_cmd_room; + initial_payload_fileoff = (initial_payload_fileoff + 4096 - 1) & ~(4096 - 1); + off_t payload_fileoff = initial_payload_fileoff; + + // Erase the load commands / payload now that we know how much space is + // needed, redo it with real values. + load_commands.clear(); + payload.clear(); + + add_lc_threads(spec, load_commands); + for (size_t i = 0; i < spec.memory_regions.size(); i++) { + std::vector segment_command_bytes; + create_lc_segment_cmd(spec, segment_command_bytes, spec.memory_regions[i], + payload_fileoff); + load_commands.push_back(segment_command_bytes); + payload_fileoff += spec.memory_regions[i].size; + payload_fileoff = (payload_fileoff + 4096 - 1) & ~(4096 - 1); + } + + off_t payload_fileoff_before_lcnotes = payload_fileoff; + std::vector lc_note_payload_bytes; + if (spec.binaries.size() > 0) + for (const Binary &binary : spec.binaries) { + std::vector segment_command_bytes; + std::vector payload_bytes; + create_lc_note_binary_load_cmd(spec, segment_command_bytes, binary, + lc_note_payload_bytes, payload_fileoff); + payload_fileoff = + payload_fileoff_before_lcnotes + lc_note_payload_bytes.size(); + load_commands.push_back(segment_command_bytes); + } + if (spec.addressable_bits) { + std::vector segment_command_bytes; + std::vector payload_bytes; + create_lc_note_addressable_bits(spec, segment_command_bytes, + *spec.addressable_bits, + lc_note_payload_bytes, payload_fileoff); + payload_fileoff = + payload_fileoff_before_lcnotes + lc_note_payload_bytes.size(); + load_commands.push_back(segment_command_bytes); + } + + // Realign our payload offset if we added any LC_NOTEs. + if (lc_note_payload_bytes.size() > 0) + payload_fileoff = (payload_fileoff + 4096 - 1) & ~(4096 - 1); + + FILE *f = fopen(OutputFilename.c_str(), "w"); + if (f == nullptr) { + fprintf(stderr, "Unable to open file %s for writing\n", + OutputFilename.c_str()); + exit(1); + } + + std::vector mh; + // Write the fields of a mach_header_64 struct + if (spec.wordsize == 8) + add_uint32(mh, llvm::MachO::MH_MAGIC_64); // magic + else + add_uint32(mh, llvm::MachO::MH_MAGIC); // magic + add_uint32(mh, spec.cputype); // cputype + add_uint32(mh, spec.cpusubtype); // cpusubtype + add_uint32(mh, llvm::MachO::MH_CORE); // filetype + add_uint32(mh, load_commands.size()); // ncmds + add_uint32(mh, size_of_load_commands); // sizeofcmds + add_uint32(mh, 0); // flags + if (spec.wordsize == 8) + add_uint32(mh, 0); // reserved + + fwrite(mh.data(), mh.size(), 1, f); + + for (const auto &lc : load_commands) + fwrite(lc.data(), lc.size(), 1, f); + + // Reset the payload offset back to the first one. + payload_fileoff = initial_payload_fileoff; + if (spec.memory_regions.size() > 0) { + for (size_t i = 0; i < spec.memory_regions.size(); i++) { + std::vector bytes; + create_memory_bytes(spec, spec.memory_regions[i], bytes); + fseek(f, payload_fileoff, SEEK_SET); + fwrite(bytes.data(), bytes.size(), 1, f); + + payload_fileoff += bytes.size(); + payload_fileoff = (payload_fileoff + 4096 - 1) & ~(4096 - 1); + } + } + + if (lc_note_payload_bytes.size() > 0) { + fseek(f, payload_fileoff, SEEK_SET); + fwrite(lc_note_payload_bytes.data(), lc_note_payload_bytes.size(), 1, f); + payload_fileoff += lc_note_payload_bytes.size(); + payload_fileoff = (payload_fileoff + 4096 - 1) & ~(4096 - 1); + } + + fclose(f); +} From 59ebdac0075f1ba71334fb8e33fa9923bed6b619 Mon Sep 17 00:00:00 2001 From: Jason Molenda Date: Thu, 4 Sep 2025 12:32:55 -0700 Subject: [PATCH 2/7] [lldb][yaml2machocore] Give explicit namespace for compile err A couple of the ubuntu CI bots failed to compile saying that MappingTraits was unqualified despite a `using llvm::yaml::MappingTraits` earlier in the file. For PR https://github.com/llvm/llvm-project/pull/153911 (cherry picked from commit fffd6da2b4f05e4b7987eb4360f06a56d9313c09) --- lldb/tools/yaml2macho-core/CoreSpec.cpp | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/lldb/tools/yaml2macho-core/CoreSpec.cpp b/lldb/tools/yaml2macho-core/CoreSpec.cpp index f898b30641ddf..22551a60b3a5e 100644 --- a/lldb/tools/yaml2macho-core/CoreSpec.cpp +++ b/lldb/tools/yaml2macho-core/CoreSpec.cpp @@ -16,7 +16,7 @@ using llvm::yaml::Input; using llvm::yaml::IO; using llvm::yaml::MappingTraits; -template <> struct MappingTraits { +template <> struct llvm::yaml::MappingTraits { static void mapping(IO &io, RegisterNameAndValue &name_value) { io.mapRequired("name", name_value.name); io.mapRequired("value", name_value.value); @@ -32,7 +32,7 @@ template <> struct llvm::yaml::ScalarEnumerationTraits { } }; -template <> struct MappingTraits { +template <> struct llvm::yaml::MappingTraits { static void mapping(IO &io, RegisterSet ®set) { io.mapRequired("flavor", regset.flavor); io.mapRequired("registers", regset.registers); @@ -40,14 +40,14 @@ template <> struct MappingTraits { }; LLVM_YAML_IS_SEQUENCE_VECTOR(RegisterSet) -template <> struct MappingTraits { +template <> struct llvm::yaml::MappingTraits { static void mapping(IO &io, Thread &thread) { io.mapRequired("regsets", thread.regsets); } }; LLVM_YAML_IS_SEQUENCE_VECTOR(Thread) -template <> struct MappingTraits { +template <> struct llvm::yaml::MappingTraits { static void mapping(IO &io, MemoryRegion &memory) { io.mapRequired("addr", memory.addr); io.mapOptional("UInt8", memory.bytes); @@ -68,7 +68,7 @@ template <> struct MappingTraits { }; LLVM_YAML_IS_SEQUENCE_VECTOR(MemoryRegion) -template <> struct MappingTraits { +template <> struct llvm::yaml::MappingTraits { static void mapping(IO &io, Binary &binary) { io.mapOptional("name", binary.name); io.mapRequired("uuid", binary.uuid); @@ -105,7 +105,7 @@ template <> struct llvm::yaml::MappingTraits { } }; -template <> struct MappingTraits { +template <> struct llvm::yaml::MappingTraits { static void mapping(IO &io, CoreSpec &corespec) { std::string cpuname; io.mapRequired("cpu", cpuname); From 818473a1cc35dced7ba3f90125b60053e84c1864 Mon Sep 17 00:00:00 2001 From: Jason Molenda Date: Thu, 4 Sep 2025 13:14:59 -0700 Subject: [PATCH 3/7] [lldb][yaml2macho-core] remove unnecessary include Fixes a CI fail on https://github.com/llvm/llvm-project/pull/153911 (cherry picked from commit eb0c97717cce8c02e11d7dc04f45a42fbb81281e) --- lldb/tools/yaml2macho-core/yaml2macho.cpp | 1 - 1 file changed, 1 deletion(-) diff --git a/lldb/tools/yaml2macho-core/yaml2macho.cpp b/lldb/tools/yaml2macho-core/yaml2macho.cpp index 91445e191969f..b01f61d491079 100644 --- a/lldb/tools/yaml2macho-core/yaml2macho.cpp +++ b/lldb/tools/yaml2macho-core/yaml2macho.cpp @@ -13,7 +13,6 @@ #include "Utility.h" #include "llvm/BinaryFormat/MachO.h" #include "llvm/Support/CommandLine.h" -#include #include #include #include From 38cf78833ee794f4bc60d133799f22f8ff0d542b Mon Sep 17 00:00:00 2001 From: Jason Molenda Date: Thu, 4 Sep 2025 17:15:18 -0700 Subject: [PATCH 4/7] [lldb][yaml2macho-core] Disable two API tests on Windows I changed two tests that were previously @skipUnlessDarwin to run on all systems, using yaml2macho-core. I'm seeing failures on these tests on the Windows bots. I'm skipping them on Windows for now to unblock the CI, while I try to understand if this is a yaml2macho-core bug, or an existing problem with lldb's macho corefile reading on windows bug. v. https://github.com/llvm/llvm-project/pull/153911 (cherry picked from commit 53efe0a892199570aef9823cf0b1bc50bf4d480b) --- .../API/macosx/arm-corefile-regctx/TestArmMachoCorefileRegctx.py | 1 + lldb/test/API/macosx/riscv32-corefile/TestRV32MachOCorefile.py | 1 + 2 files changed, 2 insertions(+) diff --git a/lldb/test/API/macosx/arm-corefile-regctx/TestArmMachoCorefileRegctx.py b/lldb/test/API/macosx/arm-corefile-regctx/TestArmMachoCorefileRegctx.py index a2890cdfeaa44..e11f569c4f281 100644 --- a/lldb/test/API/macosx/arm-corefile-regctx/TestArmMachoCorefileRegctx.py +++ b/lldb/test/API/macosx/arm-corefile-regctx/TestArmMachoCorefileRegctx.py @@ -13,6 +13,7 @@ class TestArmMachoCorefileRegctx(TestBase): NO_DEBUG_INFO_TESTCASE = True + @skipIfWindows # CI fail on Windows, lr has value 0x0F000000? def test_armv7_corefile(self): ### Create corefile corefile = self.getBuildArtifact("core") diff --git a/lldb/test/API/macosx/riscv32-corefile/TestRV32MachOCorefile.py b/lldb/test/API/macosx/riscv32-corefile/TestRV32MachOCorefile.py index 449da70fb08ca..0516cea481ac6 100644 --- a/lldb/test/API/macosx/riscv32-corefile/TestRV32MachOCorefile.py +++ b/lldb/test/API/macosx/riscv32-corefile/TestRV32MachOCorefile.py @@ -13,6 +13,7 @@ class TestRV32MachOCorefile(TestBase): NO_DEBUG_INFO_TESTCASE = True + @skipIfWindows # windows CI failure, says only 1 thread in corefile @no_debug_info_test def test_riscv32_gpr_corefile_registers(self): corefile = self.getBuildArtifact("core") From adf333c32f8d59079563ba4d6d4cd6aeb0784196 Mon Sep 17 00:00:00 2001 From: Jason Molenda Date: Thu, 4 Sep 2025 19:51:44 -0700 Subject: [PATCH 5/7] [lldb][yaml2macho-core] change the two API tests to skipUnlessDarwin These two 32-bit macho corefile tests are still failing on the lldb-remote-linux-win bot, I'd expect the @skipIfWindows decorator to skip the tests there until I can debug this, but somehow because it's a remote windows execution it's not working. Change these tests back to skipUnlessDarwin until I can figure out whether it's yaml2macho-core or lldb's 32-bit mach-o corefile reader support on windows which is the problem. (cherry picked from commit 4c17749b6d26093f33efd13f798d47b893252b15) --- .../macosx/arm-corefile-regctx/TestArmMachoCorefileRegctx.py | 2 +- lldb/test/API/macosx/riscv32-corefile/TestRV32MachOCorefile.py | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/lldb/test/API/macosx/arm-corefile-regctx/TestArmMachoCorefileRegctx.py b/lldb/test/API/macosx/arm-corefile-regctx/TestArmMachoCorefileRegctx.py index e11f569c4f281..acb6956ec478b 100644 --- a/lldb/test/API/macosx/arm-corefile-regctx/TestArmMachoCorefileRegctx.py +++ b/lldb/test/API/macosx/arm-corefile-regctx/TestArmMachoCorefileRegctx.py @@ -13,7 +13,7 @@ class TestArmMachoCorefileRegctx(TestBase): NO_DEBUG_INFO_TESTCASE = True - @skipIfWindows # CI fail on Windows, lr has value 0x0F000000? + @skipUnlessDarwin # CI fail on Windows, lr has value 0x0F000000? def test_armv7_corefile(self): ### Create corefile corefile = self.getBuildArtifact("core") diff --git a/lldb/test/API/macosx/riscv32-corefile/TestRV32MachOCorefile.py b/lldb/test/API/macosx/riscv32-corefile/TestRV32MachOCorefile.py index 0516cea481ac6..ed39a3ca9d7e4 100644 --- a/lldb/test/API/macosx/riscv32-corefile/TestRV32MachOCorefile.py +++ b/lldb/test/API/macosx/riscv32-corefile/TestRV32MachOCorefile.py @@ -13,7 +13,7 @@ class TestRV32MachOCorefile(TestBase): NO_DEBUG_INFO_TESTCASE = True - @skipIfWindows # windows CI failure, says only 1 thread in corefile + @skipunlessDarwin # windows CI failure, says only 1 thread in corefile @no_debug_info_test def test_riscv32_gpr_corefile_registers(self): corefile = self.getBuildArtifact("core") From ce7a944dda7484be96dff5ff9c6860e832268148 Mon Sep 17 00:00:00 2001 From: Jason Molenda Date: Thu, 4 Sep 2025 19:57:05 -0700 Subject: [PATCH 6/7] [lldb][yaml2macho-core] fix typeo in API test (cherry picked from commit e7572d73b5fabe69844c943b7799996495121774) --- lldb/test/API/macosx/riscv32-corefile/TestRV32MachOCorefile.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lldb/test/API/macosx/riscv32-corefile/TestRV32MachOCorefile.py b/lldb/test/API/macosx/riscv32-corefile/TestRV32MachOCorefile.py index ed39a3ca9d7e4..f791c01313fb1 100644 --- a/lldb/test/API/macosx/riscv32-corefile/TestRV32MachOCorefile.py +++ b/lldb/test/API/macosx/riscv32-corefile/TestRV32MachOCorefile.py @@ -13,7 +13,7 @@ class TestRV32MachOCorefile(TestBase): NO_DEBUG_INFO_TESTCASE = True - @skipunlessDarwin # windows CI failure, says only 1 thread in corefile + @skipUnlessDarwin # windows CI failure, says only 1 thread in corefile @no_debug_info_test def test_riscv32_gpr_corefile_registers(self): corefile = self.getBuildArtifact("core") From 00b779cc57ab0f70fc241300c97fc29aad23ef83 Mon Sep 17 00:00:00 2001 From: Jason Molenda Date: Mon, 8 Sep 2025 10:34:33 -0700 Subject: [PATCH 7/7] [lldb][yaml2macho-core] Address bug when run on Windows The two API tests I converted to use yaml2macho-core were failing on windows. I was writing the output corefile with fopen(filename, "w"), and it turns out there was some newline conversion happening on Windows if a linefeed character was present. Charles Zablit helped to identify the issue and confirm the fix. Re-enabling the two tests on all platforms. https://github.com/llvm/llvm-project/pull/153911 (cherry picked from commit 693146dcc0a1d653827cc88993acf52428af1a94) --- .../macosx/arm-corefile-regctx/TestArmMachoCorefileRegctx.py | 1 - lldb/test/API/macosx/riscv32-corefile/TestRV32MachOCorefile.py | 1 - lldb/tools/yaml2macho-core/yaml2macho.cpp | 2 +- 3 files changed, 1 insertion(+), 3 deletions(-) diff --git a/lldb/test/API/macosx/arm-corefile-regctx/TestArmMachoCorefileRegctx.py b/lldb/test/API/macosx/arm-corefile-regctx/TestArmMachoCorefileRegctx.py index acb6956ec478b..a2890cdfeaa44 100644 --- a/lldb/test/API/macosx/arm-corefile-regctx/TestArmMachoCorefileRegctx.py +++ b/lldb/test/API/macosx/arm-corefile-regctx/TestArmMachoCorefileRegctx.py @@ -13,7 +13,6 @@ class TestArmMachoCorefileRegctx(TestBase): NO_DEBUG_INFO_TESTCASE = True - @skipUnlessDarwin # CI fail on Windows, lr has value 0x0F000000? def test_armv7_corefile(self): ### Create corefile corefile = self.getBuildArtifact("core") diff --git a/lldb/test/API/macosx/riscv32-corefile/TestRV32MachOCorefile.py b/lldb/test/API/macosx/riscv32-corefile/TestRV32MachOCorefile.py index f791c01313fb1..449da70fb08ca 100644 --- a/lldb/test/API/macosx/riscv32-corefile/TestRV32MachOCorefile.py +++ b/lldb/test/API/macosx/riscv32-corefile/TestRV32MachOCorefile.py @@ -13,7 +13,6 @@ class TestRV32MachOCorefile(TestBase): NO_DEBUG_INFO_TESTCASE = True - @skipUnlessDarwin # windows CI failure, says only 1 thread in corefile @no_debug_info_test def test_riscv32_gpr_corefile_registers(self): corefile = self.getBuildArtifact("core") diff --git a/lldb/tools/yaml2macho-core/yaml2macho.cpp b/lldb/tools/yaml2macho-core/yaml2macho.cpp index b01f61d491079..85979a37d1676 100644 --- a/lldb/tools/yaml2macho-core/yaml2macho.cpp +++ b/lldb/tools/yaml2macho-core/yaml2macho.cpp @@ -198,7 +198,7 @@ int main(int argc, char **argv) { if (lc_note_payload_bytes.size() > 0) payload_fileoff = (payload_fileoff + 4096 - 1) & ~(4096 - 1); - FILE *f = fopen(OutputFilename.c_str(), "w"); + FILE *f = fopen(OutputFilename.c_str(), "wb"); if (f == nullptr) { fprintf(stderr, "Unable to open file %s for writing\n", OutputFilename.c_str());