Autodetect triple if the default triple is not compatible with the rootfs #655
Conversation
…otfs This allows to inject a statically built x86_64 Crystal compiler into a 32-bit x86 Linux rootfs and successfully use it there. The compiler will see that the x86_64 dynamic linker is not present in the system. And automatically override the triple configuration with the tuple string obtained by probing the (hopefully) installed GCC or Clang. It also makes porting Crystal to ARM, MIPS or other non-x86 systems much easier. Because the statically built x86_64 Crystal compiler binary can work nicely in the QEMU user chroot and also pick the right target triple automatically. More info: https://wiki.debian.org/Multiarch/Tuples https://wiki.debian.org/Multiarch/LibraryPathOverview
ssvb
changed the title
|
Looks good, thanks! But how can a x86_64 Crystal compiler run in a 32 bits environment? (Sorry, I'm not familiar with rootfs). As a side note, I found this, which should probably also ask the |
|
A x86_64 kernel is just fine executing i686 code, thus if you chroot into a i686 rootfs you can run everything just fine. However it's a valid standpoint that one should use In any case I think we need to get the ARM parts out of this PR until we have do have official ARM support. And re-adding them at that point shouldn't be much of a hassle either. |
If x86_64 Crystal compiler is statically built, then it already contains everything and can run in 32-bit userland, assuming that the system uses x86_64 kernel. However it tries to compile 64-bit binaries and link them with 64-bit libraries by default and this fails (unless crosscompilation related command line options are provided). Dynamically linked executables in ELF format are relying on dynamic loader (which is listed as INTERP header in the
That's a good find. Probably this explains why using a 32-bit static Crystal compiler binary in a 64-bit rootfs produced broken 64-bit binaries when I tried this test. |
This is a chicken-egg problem. Full ARM support needs a set of fixes in various places. This particular part is about the only thing, where we can be reasonably confident that it works as expected on ARM too (assuming that this approach is useful on x86 / x86_64). |
|
I'd still prefer the patches in one, doing the steps inside a fork shouldn't make a difference. |
|
@asterite Yes, it looks like @ysbaddaden used a similar approach with a static binary for bootstrapping. But my patch is slightly different in a way that it uses autodetection and does not need any special cross-compilation command line options or code patching. I'm not sure what is the current 32-bit x86 bootstrapping support story, but if everyone is already happy with the way how it works, then my pull request can be just closed. Since there is an "all or nothing" policy regarding ARM support, it indeed makes more sense to have this code as a part of the ARM patchset in my private branch without showing it to anybody until everything is perfect :-) |
| @@ -240,7 +240,7 @@ module Crystal | |||
|
|
|||
| private def codegen_single_unit(unit, target_triple, multithreaded) | |||
| unit.llvm_mod.target = target_triple | |||
| ifdef x86_64 | |||
| if target_triple =~ /^x86_64/ | |||
There was a problem hiding this comment.
Flags are now set from the target triple (default or specified). I suppose this is no longer required?
There was a problem hiding this comment.
Currently if you have a 64-bit static build of the Crystal compiler binary, then it never uses DataLayout32 regardless of what target triple is set via the command line because the relevant code branch is simply not compiled in. Replacing 'ifdef' with 'if' allows to make the selection between the use of DataLayout32 and DataLayout64 at runtime.
There was a problem hiding this comment.
@ssvb @ysbaddaden I understand the issue now. You are right, ifdef shouldn't be used inside the compiler. I just pushed this commit.
At first I thought: if you aren't cross-compiling and you are on a 64-bits machine then you want 64-bits executables. But it never occurred to me that you could have a 64-bits compiler inside a 32-bits machine.
Do you think the above commit solves the problem?
|
I had your patch in mind when i made the uname to target triple change, but decided to keep things simple for now and let the developer set This patch may come handy when we start supporting the Android NDK toolchain, thought. |
|
I'm closing this since a lot has happened since this. It's probably fixed, but I don't know. Please reopen if something still not working. |
|
Yes, this already has landed in a bit reworked form. It was necessary for ARM architecture support. I can submit a new pull request if anything is still missing. |
This allows to inject a statically built x86_64 Crystal compiler into
a 32-bit x86 Linux rootfs and successfully use it there. The compiler
will see that the x86_64 dynamic loader is not present in the system.
And automatically override the triple configuration with the tuple
string obtained by probing the (hopefully) installed GCC or Clang.
It also makes porting Crystal to ARM, MIPS or other non-x86 systems
much easier. Because the statically built x86_64 Crystal compiler
binary can work nicely in the QEMU user chroot and also pick the
right target triple automatically.
More info:
https://wiki.debian.org/Multiarch/Tuples
https://wiki.debian.org/Multiarch/LibraryPathOverview