Replace build-ldc-runtime.sh script by D tool #2253
Conversation
kinke
force-pushed
the
build-ldc-runtime
branch
5 times, most recently
from
7f53cef
to
bd01264
Compare
kinke
force-pushed
the
build-ldc-runtime
branch
2 times, most recently
from
a264bfd
to
296c75b
Compare
|
Heh, very nice, looking forward to trying this. |
Primarily for portability (Windows) and to pave the way for a simpler interface. Already comes with ninja and testrunners support and a consistent command-line interface wrt. D/C/linker flags. To be used like this: bin\build-ldc-runtime [--ldcSrcDir=..\ldc] [--ninja] [--testrunners] [--dFlags=-mtriple=x86_64-pc-windows-msvc;...] [--cFlags=...] [--linkerFlags=...] [CMAKE_VAR=value ...]
The main object apparently needs to come first for TLS on Android.
|
Testing it now, works well! I like that there is a |
|
Trying to help: I've added the defaults to the help message. You can merge this if you are happy with it. (how about starting the name of the program with "ldc", like "ldc-profdata", "ldc2", "ldc-prune-cache", so they are nicely grouped in file listings?) |
|
Alright, finally tried this out in a git repo after building a cross-compiler for Android/ARM. All the D files built fine with the following commands, confirmed by comparing md5 hashes with my normal build: I only had to make two changes to the runtime CMake config, removing the I initially tried running the I'm able to set the C flags, but I notice that the C defines and includes are empty. Here's what they are for my default build of the Not sure if that matters, but the md5 hashes of the resulting C object files differ. Finally, the two assembly files, threadasm.S and eh_asm.S, from druntime aren't built with your approach, not sure why. I tried setting One last request, it'd be nice to be able to supply specific targets to make, rather than only |
Thanks Johan, I'll have to correct them though. ;)
Thanks, will do.
I was too lazy to handle this case (non-packaged tool and not built manually from tagged source) so far.
That's basically CMake's job. I guess using a gcc toolchain (still via CC env variable) instead of a clang cross-compiler (requiring
Hmm, if you come up with a simple interface, feel free to add it to your PR then. For non-trivial use cases, running the tool once and then executing |
Not a big deal, just thought I'd let you know.
Looking further, it appears these C defines/includes are set from the top-level CMake config for ldc, then passed down to the runtime config also. I stripped both versions of the druntime C object files and then compared the md5 hashes and there's no difference, as we'd expect for all those includes of ddmd/ldc/llvm directories, so these defines/includes likely don't matter. I'll look into the assembly issue now. |
|
Found it, I needed to add this line to the runtime's CMake Config to get it to include the assembly files. It automatically uses the C compiler for ASM too, was probably only setting that manually before because I wasn't using |
|
Perfect. I don't think we need asm for LDC itself, so maybe just moving it to the runtime's CMake script will do, I'll check. Wrt. |
|
Yeah, don't worry about |
|
Trying out |
|
[Ninja exploits parallelization out of the box.] Updated and tried to incorporate |
|
OK, I did see that about ninja, as I tried porting it to run on Android, but it relies on I just installed ninja in my linux/x64 VPS where I'm cross-compiling for Android/ARM, right before you mentioned it, looking forward to trying it out. This PR is looking good, thanks for adding the parallel jobs flag. I'll let you know the results of cross-compiling the test runners to Android/ARM soon. |
|
All tests from druntime and phobos pass when cross-compiled to Android/ARM (except for the one from |
|
Perfect, so this is good to go, right? I guess you had to add some |
|
This is the full command I ran to build the test runners using ninja: I want to have a list of supported platforms in the runtime CMake config, so that we can stick all these flags in there and people can just build the runtime by setting the target OS and Arch. Now that you got the linker changes in, that's basically all my Android PR will consist of. @redstar can add his flags for linux/PPC and so on, let me know what you think. As for this PR, I want to try it with a cross-compiler build of ldc 1.3, the one I put up releases for. I'll have to modify the runtime CMake config a bit to apply parts of this patch, but I'll get that running today and get back to you with the results. |
|
I was thinking about putting up a Wiki page about this tool and how to use it to a) generate LTO-able runtime libs for best performance and b) cross-compile them for other targets. I hope a few compact examplary command lines on the Wiki page are enough for people to get it to work with their particular setup. Have you tried compiling via your cross-gcc or is using Clang mandatory? I think the list of flags could at least be halved by using gcc.
Not sure what this is supposed to prove; you'll probably need a bunch of more |
|
A wiki page would be good, but there's no reason not to stick all these flags we're using in the runtime CMake config and let people use them easily with just two config options, target OS and arch. That way, they can easily cross-compile, but override the flags if wanted. I haven't used gcc in years. The Android NDK and Termux use clang and are switching to libc++, and the NDK will remove gcc next year. Some flags might not be needed with gcc but most affect codegen, so you'd want them regardless. I want to try this PR with ldc 1.3 to see how well it works with a release, as opposed to from git. Probably the same, but testing will verify that. I may have to pull in earlier PRs of yours too, let's see. |
|
(just popping in here to say that having this tool already has been very useful for me, being able to rebuild the runtime with specific D options (-fsanitize), great stuff.) |
runtime/ldc-build-runtime.d.in
Outdated
| arraySep = ";"; | ||
| auto helpInformation = getopt( | ||
| args, | ||
| "j", "Number of parallel build jobs", &config.numBuildJobs, |
There was a problem hiding this comment.
When viewing this in the console, it looks like this:
-j Number of parallel build jobs
--ldc Path to LDC executable (default: '/home/joakim/ldc/build14/bin/ldc2')
--buildDir Path to temporary build directory (default: './ldc-build-runtime.tmp')
---snip---
--linkerFlags C linker flags for shared libraries and testrunner executables (separated by ';')
-h --help This help information.
Apparently, single-character flags go all the way on the left, while full-word flags go in the middle. It looks weird like this: maybe move -j to the end before -h, so the single-character ones are on the bottom.
There was a problem hiding this comment.
Yeah I found it pretty weird too; I just put it at the top as it's probably the most important switch when using make. ;) It'd probably look a bit better below.
|
OK, trying this PR with my ldc 1.3 cross-compiler, by simply building a Sure enough, Not sure why this fails when run with an ldc binary as opposed to a source install, some CMake magic? Just tried Ninja, it spits out a long warning about this, but puts in a phony rule and goes ahead and builds anyway. |
|
I was aware of |
|
Ah wait, so the script first ran CMake with the original CMake script, failed, and then you replaced the CMake script and ran the tool again? That's not good; you should remove everything but the (patched) ldc-src dir from the build dir before running the tool a 2nd time. CMake's cache can be tricky, resetting the build dir sometimes works wonders. |
|
[A |
|
Yep, thats it: I ran all the ldc 1.3 stdlib tests built with your binary, both release and debug, with make and ninja, on Android/ARM: everything passes. |
runtime/ldc-build-runtime.d.in
Outdated
| "ninja", "Use Ninja as CMake build system", &config.ninja, | ||
| "testrunners", "Build the testrunner executables too", &config.buildTestrunners, | ||
| "dFlags", "LDC flags for the D modules (separated by ';')", &config.dFlags, | ||
| "cFlags", "C compiler flags for the C modules (separated by ';')", &config.cFlags, |
There was a problem hiding this comment.
Maybe mention that it's used for the assembly files too, "C/ASM compiler flags for the handful of C/ASM files in the stdlib" or something like that.
|
Thanks a lot for testing. Updated. |
|
Three new commits LGTM, will merge in an hour if nobody objects. |
|
Semaphore and Travis CI are failing because |
kinke
force-pushed
the
build-ldc-runtime
branch
from
be461cd
to
41848a8
Compare
|
Fixed. Too bad these methods' constness depends on the OS, but oh well. |
Primarily for portability (Windows) and to pave the way for a simpler interface. Already comes with ninja and testrunners support and a consistent command-line interface wrt. D/C/linker flags.
To be used like this: