New build tool + host toolchain integration #113
Conversation
Also, ensure no install paths have a leading '/' character.
Note: The dbuild submodule currently points to my own fork of PureDarwin/dbuild. This will need to be fixed if this is merged back to PureDarwin/PureDarwin
Rename cctools-973.0.1 --> cctools
Use in-tree dtrace sources to build the three ctf tools with rc
All cctools binaries now build into the host toolchain. The main missing component is now the linker
The imported tools (versions) are: - bison (bison-14) - bootstrap_cmds (bootstrap_cmds-122) - developer_cmds (developer_cmds-66) - gm4 (gm4-16) - gnumake (gnumake-132) - gperf (gperf-13)
flex-26
This adds a few binaries, and moves the unifdef tool
Added bison, gm4, gnumake, gperf, and a few utilities from the ld64 sources
Also import CommonCrypto-60191.100.1, as this is needed for a specific header. TAPI support has been removed from the linker, but other than that it builds just fine.
There are still some issues, I need to go through the kernel build process in more depth, and then it's just getting everything to build together...
Use compiler internals headers from toolchain in libsystem_kernel
The following versions of these libraries are imported: libSystem-1311.100.3 libm-2026 libnotify-301 libinfo-554 libclosure-79.1 libunwind-201 removefile-60 keymgr-31 copyfile-180.100.3 UserNotification-31 configd-1163.100.19.0.2 syslog-392.100.2
dbuild --> rc
This will allow us to remove the host dependency on Darwin, as well as better segregate toolchain and host source
This reverts commit 877c5b3. With the new build system, architecture configuration is handled automatically by rc, so this is not necessary.
|
A few notes since I noticed how absolutely giant this pr is. This effectively implements the design we talked about previously here. Most of the imported files and such are all just pulling in Apple's code for development tools. These repos have been placed in
Other changes are mainly small and meaningless |
|
I have just pulled your changes into the Since I’m not merging this PR into |
This is a bit of a giant pr, but I'll do my best to explain. See
tools/rc/README.mdfor a description of the newsetup.shandrcscripts.Effectively, this commit does two major things:
I've added the tool
rcwhich is to build the host toolchain and invoke system builds/rebuilds, keeping track of where all the host tools are installed to. This pr includes a full system building toolchain, including the full llvm-project, cctools, linker, and a few misc. tools Apple includes with their toolchain. The toolchain is by default installed to theproducts/toolchaindirectory, and the system root is installed to theproducts/rootdirectory.The new tools can be used by sourcing
tools/rc/setup.shonce per session, configuring any environment variables desired, and simply invokingrc buildto build and install the target system sources. Note that if the toolchain doesn't exist, it may be necessary to invoke firstrc --make-host-toolchainto build it.Included with this pr there are now two submodules: the
ninjabuild tool and Apple's clone ofllvm-project.Ensure these two submodules are properly checked out before runningrc.This needs more testing. It currently works very well on my M1 Macbook Pro (host tools target native system; target system is built for x86_64 arch), but it hasn't been tested in other setups. We've nearly gotten it to build on an older macOS version in the discord, but we could use some further testing on other people's macs.
Currently, macOS is required, but with this new setup, it shouldn't be too hard to port the build system to other systems, since we would need only to modify the build scripting for the host toolchain, and the target build would remain the same.
More changes to come,
-- Tyler