ldc2 fails to run simple main example on ppc64el

[LocutusOfBorg]

Debian Buster:

$ ldc2  -v
binary    /usr/bin/ldc2
version   1.3.0 (DMD v2.073.2, LLVM 4.0.1)
config    /etc/ldc2.conf (powerpc64le-unknown-linux-gnu)
OVERVIEW: LDC - the LLVM D compiler

ldc 1.3.0-2 is fine:
echo "void main() {}" > foo.d
ldc2 foo.d
./foo
echo $?

Debian sid, ldc 1.4

$ ldc2 -v
binary    /usr/bin/ldc2
version   1.4.0 (DMD v2.074.1, LLVM 5.0.0)
config    /etc/ldc2.conf (powerpc64le-unknown-linux-gnu)
OVERVIEW: LDC - the LLVM D compiler

$ echo "void main() {}" > foo.d
$ ldc2 foo.d
$ ./foo
Segmentation fault
$ echo $?
139
$ gdb foo
GNU gdb (Debian 7.12-6) 7.12.0.20161007-git
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "powerpc64le-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from foo...(no debugging symbols found)...done.
(gdb) r 
Starting program: /home/locutusofborg/foo 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/powerpc64le-linux-gnu/libthread_db.so.1".

Program received signal SIGSEGV, Segmentation fault.
0x00003fffb799c3a0 in rt.util.container.array.Array!(void[]).Array.opSlice() inout () from /usr/lib/powerpc64le-linux-gnu/libdruntime-ldc.so.74
(gdb) bt
#0  0x00003fffb799c3a0 in rt.util.container.array.Array!(void[]).Array.opSlice() inout () from /usr/lib/powerpc64le-linux-gnu/libdruntime-ldc.so.74
#1  0x00003fffb798eafc in rt.sections_elf_shared.DSO.gcRanges() inout () from /usr/lib/powerpc64le-linux-gnu/libdruntime-ldc.so.74
#2  0x00003fffb7988c24 in rt.memory.initStaticDataGC().__foreachbody1(ref rt.sections_elf_shared.DSO) () from /usr/lib/powerpc64le-linux-gnu/libdruntime-ldc.so.74
#3  0x00003fffb798e96c in rt.sections_elf_shared.DSO.opApply(scope int(ref rt.sections_elf_shared.DSO) delegate) () from /usr/lib/powerpc64le-linux-gnu/libdruntime-ldc.so.74
#4  0x00003fffb7988bd4 in rt.memory.initStaticDataGC() () from /usr/lib/powerpc64le-linux-gnu/libdruntime-ldc.so.74
#5  0x00003fffb797f3a4 in rt_init () from /usr/lib/powerpc64le-linux-gnu/libdruntime-ldc.so.74
#6  0x00003fffb797f9d8 in _d_run_main () from /usr/lib/powerpc64le-linux-gnu/libdruntime-ldc.so.74
#7  0x0000000020000c34 in main ()
#8  0x00003fffb758289c in ?? () from /lib/powerpc64le-linux-gnu/libc.so.6
#9  0x00003fffb7582ab8 in __libc_start_main () from /lib/powerpc64le-linux-gnu/libc.so.6
#10 0x0000000000000000 in ?? ()
(gdb) 

as @ximion suggested in #2300 I'm opening a new ticket

[redstar]

Does not happen with latest beta 1.5.0-beta1 and LLVM 4.0.1:

ubuntu@ldc:~$ build-ldc/bin/ldc2 --version
LDC - the LLVM D compiler (1.5.0-beta1):
  based on DMD v2.075.1 and LLVM 4.0.1git
  built with LDC - the LLVM D compiler (0.17.6git-52205e9)
  Default target: powerpc64le-unknown-linux-gnu
  Host CPU: pwr8
  http://dlang.org - http://wiki.dlang.org/LDC

  Registered Targets:
    ppc32   - PowerPC 32
    ppc64   - PowerPC 64
    ppc64le - PowerPC 64 LE
ubuntu@ldc:~$ echo "void main() {}" > foo.d
ubuntu@ldc:~$  build-ldc/bin/ldc2 foo.d
ubuntu@ldc:~$ ./foo
ubuntu@ldc:~$ echo $?
0
ubuntu@ldc:~$

I will check with LLVM 5.0, too.

[ximion]

This does still happen with LDC 1.5.0, LLVM 5.0 - we just upgraded ozr LDC and the issue doesn't seem to be gone: https://buildd.debian.org/status/fetch.php?pkg=gir-to-d&arch=ppc64el&ver=0.13.0-1&stamp=1510197266&raw=0

[ximion]

Any news on this? What additional information do you need?
We kind of need this fixed to get LDC back into Debian Testing/Buster (was removed today).

[dnadlinger]

We kind of need this fixed to get LDC back into Debian Testing/Buster (was removed today).

I feel like I'm having a déjà vu. LDC is currently only officially supported on a handful of platforms. PPC, of whatever bitness, is not one of them. While I understand the excitement for and importance of non-x86 platforms, we (LDC) like doing a proper engineering job and only advertise support for those targets where we can offer a self-hosted compiler, with a full set of features, of what we consider production quality. This entails passing the full test suite, and in particular doing that in a fashion that is covered by CI, so we can guarantee the same quality release after release while staying nimble as a project (allowing us to react quickly to new front-end releases, etc.).

(Notably, GDC follows a different strategy – Iain often talks about supporting many platforms and means by this that one can build a GDC cross-compiler executable for the given target triple – which may or may not even be able to compile Phobos.)

Thanks to the excellent work of many contributors, on LLVM as well as on LDC and the shared frontend and standard libraries, LDC is currently the compiler which can target the largest share of popular target platforms with a wide set of features (LTO, PGO, dynamic libraries, --gc-sections, fully static linking, sanitizers, etc. to various degrees). Some of them – e.g. Windows/Linux/macOS on x86_64 – are first-tier platforms that we commit to supporting in the above sense, provide binary releases for, and the status of which currently defines the release cycle to a large extent. Another few targets are "almost" first-tier, but currently provided on a best-effort basis or a separate release cycle still, for example Linux/arm{el, hf} and Android. The only thing only missing for the latter to graduate to first-tier targets is integration into the main CI system and release workflows.

Beyond that, there are a number of targets which have received (in some case significant) work in the past, but which are not and have never been supported to the same level by the main project. These – including, for instance, anything PPC and AArch64 – might or might not compile, produce working binaries, or pass the test suite at any point in time. There is of course interest in supporting them, but no-one has committed the necessary amount of work and time to bring them to first-tier status yet.

I am not very familiar with the formal aspects of Debian packaging policy, but at least from a user perspective, it is certainly a distribution that values stability, in terms of software quality at a certain point but also compatibility over time. Given that, it seems pretty clear to me that packaging a toolchain for a target not supported by the upstream project is not a desirable thing to do. (If one had the resources to maintain the port, this would best be done in upstream.) Trying to do so nevertheless is only doing users a disservice; those on unsupported platforms who expect that an ldc package in Debian would give them a stable compiler, and as things currently seem to be, also those on x86, as the availability of (recent) packages suffers without good reason. Of course, it also makes the packager's life more complicated than necessary, as I suppose you have been discovering.

I assume that the Debian packaging model does not accommodate keeping a package in experimental for only some archs. Thus, can we please just exclude anything but the officially supported targets (i386/amd64 and Linux kernel) from building, and be done with this once and for all? I certainly hope the (Debian) powers that be would recognise that the current situation is a result of past misunderstandings and would swiftly restore the package to Buster. If and when the level of upstream support improves, we can add back some of the other archs (32-bit ARM is getting there).

If necessary, we could also add a check to CMake that explicitly gives an error message nagging the user about instability on unsupported targets unless some sort of "developer mode" is explicitly enabled (cmake -DLDC_DEVELOPER_MODE=ON <...>). It really seems like this shouldn't be an issue that needs a technological solution, though.

[ximion]

Hey :)
Thanks for your reply! There are a couple of misconceptions on how Debian works though that I would like to address.

Given that, it seems pretty clear to me that packaging a toolchain for a target not supported by the upstream project is not a desirable thing to do. (If one had the resources to maintain the port, this would best be done in upstream.) Trying to do so nevertheless is only doing users a disservice

I would disagree with that - having a not-well-supported compiler on a particular architecture is much better than having no compiler at all.
We can put a disclaimer for that on LDC for that architecture, even.

In terms of Debian's workflow, there are a couple of architectures that are considered "release architectures" - issues that happen on these architectures are considered release-critical and have to be addressed before the release, or the package has to be removed from the particular architecture. This is the case for ppc64el.
If a package gets released from testing (Buster) it doesn't mean that it will be gone from the final release - we can still drop architectures for that package prior to the release deadline and have the package migrate to testing again. Those issues do add a certain level of urgency to the issue though, because they make or break whether a package will be available on a particular architecture at release time.

That being said, if LDC on ppc64el isn't able to compile another package in Debian, that is also not an issue for us - that particular package will just not be available on that architecture. What we do care about though is regressions, which is what this bug report is about.
This particular code compiled without issues in a previous release of LDC, but doesn't build anymore with the current version, and that is treated as release-critical, not the overall case of LDC on ppc64el not being at the same level of the amd64 version. If something never worked, it's okay. But if something once worked and broke later, we treat that as an issue.

So, to proceed here, I could ask for the removal of the gir-to-d package on ppc64el and all its reverse dependencies from Debian, which would allow LDC to migrate to testing again while keeping a ppc64el version of it in the archive for people to test.
If you, however, really think that the quality of the ppc64el port is so bad that people shouldn't even have it available as an option to built stuff for that architecture, I could remove the port from Debian. This would also mean though that we will not know of any regressions anymore, which might actually make things worse for that architecture.

[ximion]

@klickverbot @LocutusOfBorg Debian now contains a d-compiler metapackage that will install the default D compiler in Debian that we build packages with for each architecture. This seems to be a good solution for me, but needs some testing. To make it permanent and more robust, gdc might need to drop its "d-compiler" virtual package, or we need to rename the d-compiler metapackage to e.g. default-d-compiler (I forgot that a virtual package of the same name existed when I created the metapackage).

Anyway, do you think that solution is okay for you?
@klickverbot Do you think LDC could get some alignment with compiler flags used by GCC? At the moment, D packages in Debian use a macro to set the right DFLAGS based on which compiler we are building with, which is really annoying to do and prevents us from just setting these flags globally. With Clang supporting GCC flags, rebuilding Debian packages with a different C/C++ compiler is really easy, for D code that doesn't work though, because each compiler has very different flags.
(to really resolve this, I guess some coordination between dmd, ldc and gdc is needed, and nobody will want to drop existing flags to not break compatibility, so I guess this is a tricky issue)

[ximion]

Also, and interesting obervation: GDC appears to suffer from the same or a similar bug like LDC on ppc64el, so it also isn't able to build a simple void main() {} program...

[redstar]

The root cause seems to be LLVM 5.0+ support. As soon as I switch from LLVM 4 to LLVM 5, the generated binaries crash. This is interesting, because there is no problem with LLVM 5+ on x86. I try to investigate this further. Current work around: use LLVM 4.