dmd,dtools,dub: 2.079.0 -> 2.079.1 and wrap ldc2 binary with $CC #40228

ThomasMader · 2018-05-09T15:08:38Z

Motivation for this change

Fix dmd build on Master and update to new version.
Wrap ldc2 binary with C compiler because it's needed for linking.

Things done

Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)
- NixOS
- macOS
- other Linux distributions
Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/)
Fits CONTRIBUTING.md.

ThomasMader · 2018-05-09T15:20:51Z

The reason why this is WIP is that the resulting compiler (dmd) is not working properly.
The build of dmd is successful but it can't even build a simple hello world program anymore.
The compilation results in various linker errors.
This problem was hidden because dmd wasn't building anymore for some time now.
The merge commit which introduced the problem into master was ee6894c.
I don't know the exact problem commit but I suspect that it is 273e58e which made glibc 2.27 the default.
This problem only occurs on master, on 18.03 there should be no problem. At least not the last time I checked.

The error output of the hello world compilation can be seen at https://forum.dlang.org/post/ooxytxcurfzakjpqhvkt@forum.dlang.org

@shlevy Do you have a clue what might be wrong? I am having a hard time here.

To compile the hello world program with dmd just run 'dmd hello.d'
The contents of hello.d looks like this:

import std.stdio;

void main()
{
	writeln("test");
}

ThomasMader · 2018-05-11T05:01:52Z

@shlevy I did a very time consuming bisect and found the error commit to be 3027bca which bumped the binutils to 2.30.

Since this PR fixes the build of dmd and the problem with binutils doesn't exist on 18.03 I remove the WIP label and consider it ready to be merged as it also improves the situation on master.

This should be cherry picked to 18.03 too to update the packages.

ThomasMader · 2018-05-11T05:08:05Z

@GrahamcOfBorg build dmd

I don't trigger the build of dtools and dub because they can not build due to the binutils problem.

GrahamcOfBorg · 2018-05-11T05:09:41Z

No attempt on aarch64-linux (full log)

The following builds were skipped because they don't evaluate on aarch64-linux: dmd

Partial log (click to expand)


a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.

GrahamcOfBorg · 2018-05-11T05:14:46Z

Failure on x86_64-linux (full log)

Attempted: dmd

Partial log (click to expand)

3.247s PASS release64 std.parallelism
16.010s PASS release64 std.file
installing
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/540f42qs5p6xsrvbhkr1j0v60clh066b-phobosUnittests-2.067.1
strip is /nix/store/j75dgadrff2d1fyc4fczmcgqkid2imdx-binutils-2.30/bin/strip
patching script interpreter paths in /nix/store/540f42qs5p6xsrvbhkr1j0v60clh066b-phobosUnittests-2.067.1
checking for references to /build in /nix/store/540f42qs5p6xsrvbhkr1j0v60clh066b-phobosUnittests-2.067.1...
warning: path '/nix/store/540f42qs5p6xsrvbhkr1j0v60clh066b-phobosUnittests-2.067.1' claims to be content-addressed but isn't
error: unexpected end-of-file

ThomasMader · 2018-05-11T16:26:46Z

The error
warning: path '/nix/store/540f42qs5p6xsrvbhkr1j0v60clh066b-phobosUnittests-2.067.1' claims to be content-addressed but isn't error: unexpected end-of-file
was first mentioned in #39951. -> cc @voobscout

Removing the phobosUnittests resolves this.
Because I disabled all tests while bisecting the first error I didn't see this second error.
I don't want to just remove the tests, so I will try to bisect this problem too.

ThomasMader · 2018-05-12T08:56:53Z

The error
warning: path '/nix/store/540f42qs5p6xsrvbhkr1j0v60clh066b-phobosUnittests-2.067.1' claims to be content-addressed but isn't error: unexpected end-of-file
was first mentioned in #39951. -> cc @voobscout

This seems to be a regression in nix. Opened NixOS/nix#2155 for it.

ThomasMader · 2018-05-12T10:26:22Z

I also opened #40397 for the binutils problem.

While testing I saw that the ldc package can't compile if there is no gcc installed. I fixed this by wrapping the ldc2 binary like it was already done with the dmd binary.

Because the other remaining issues are outside the scope of this PR and this PR fixes the build problems on master, this can now be finally merged as it improves the current situation.

Ericson2314 · 2018-05-13T01:31:14Z

pkgs/development/compilers/ldc/default.nix

    '';

+    postInstall = ''
+      wrapProgram $out/bin/ldc2 \
+          --prefix PATH ":" "${stdenv.cc}/bin" \


If you are care about cross compilation, or might eventually care about cross compilation, it's better if you do targetPackages.stdenv.cc.

Thanks, also changed it in the dmd package.

Ericson2314 · 2018-05-13T01:36:19Z

pkgs/development/compilers/ldc/default.nix

+    postInstall = ''
+      wrapProgram $out/bin/ldc2 \
+          --prefix PATH ":" "${stdenv.cc}/bin" \
+          --set-default CC "$CC"


likewise

--set-default CC "${targetPackages.stdenv.cc}/bin/cc"

Thanks, also changed it in the dmd package.

GrahamcOfBorg · 2018-05-16T16:42:45Z

No attempt on aarch64-linux (full log)

The following builds were skipped because they don't evaluate on aarch64-linux: dmd, dtools, dub, ldc

Partial log (click to expand)


a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.

GrahamcOfBorg · 2018-05-16T17:21:29Z

Failure on x86_64-linux (full log)

Attempted: dmd, dtools, dub, ldc

Partial log (click to expand)


Total Test time (real) =  16.05 sec
installing
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/0hj4zi4v1hfhcclh3zxhlykrx3g8frzg-ldcUnittests-0.17.5
strip is /nix/store/j75dgadrff2d1fyc4fczmcgqkid2imdx-binutils-2.30/bin/strip
patching script interpreter paths in /nix/store/0hj4zi4v1hfhcclh3zxhlykrx3g8frzg-ldcUnittests-0.17.5
checking for references to /build in /nix/store/0hj4zi4v1hfhcclh3zxhlykrx3g8frzg-ldcUnittests-0.17.5...
warning: path '/nix/store/0hj4zi4v1hfhcclh3zxhlykrx3g8frzg-ldcUnittests-0.17.5' claims to be content-addressed but isn't
error: unexpected end-of-file

ThomasMader · 2018-05-19T21:26:57Z

I found the linking problem.
It's because of of a changed/broken strip in binutils 2.30.

Since it is better anyway not to strip the dmd package I put a dontStrip = true into the package which fixes the problem immediately. See also #40397 (comment).

This means that the only thing left is to merge nix 2.0.2 into release-18.03. See #40652.

GrahamcOfBorg · 2018-05-19T21:41:09Z

No attempt on aarch64-linux (full log)

The following builds were skipped because they don't evaluate on aarch64-linux: dmd, dtools, dub, ldc

Partial log (click to expand)


a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.

GrahamcOfBorg · 2018-05-19T22:25:46Z

Success on x86_64-linux (full log)

Attempted: dmd, dtools, dub, ldc

Partial log (click to expand)


Total Test time (real) =  34.47 sec
installing
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/zrw5hxv62gmc8gh75wiqy6053xi7f0i7-ldcUnittests-1.8.0
strip is /nix/store/j75dgadrff2d1fyc4fczmcgqkid2imdx-binutils-2.30/bin/strip
patching script interpreter paths in /nix/store/zrw5hxv62gmc8gh75wiqy6053xi7f0i7-ldcUnittests-1.8.0
checking for references to /build in /nix/store/zrw5hxv62gmc8gh75wiqy6053xi7f0i7-ldcUnittests-1.8.0...
building '/nix/store/1fdwk5p7jc2ggwjcp11927sxi5xca308-ldc-1.8.0.drv'...
installing

ElvishJerricco · 2018-05-23T01:55:40Z

@ThomasMader Looks like that's merged. Is this PR good to go then?

ThomasMader · 2018-05-23T05:57:14Z

Yes can be merged.

dmd,dtools,dub: 2.079.0 -> 2.079.1 and wrap ldc2 binary with $CC (cherry picked from commit 4aa04d1) Reason: This bumps the version to a newer release and fixes package issues.

GrahamcOfBorg added 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 labels May 9, 2018

ThomasMader changed the title ~~[WIP] dmd,dtools,dub: 2.079.0 -> 2.079.1~~ dmd,dtools,dub: 2.079.0 -> 2.079.1 May 11, 2018

ThomasMader changed the title ~~dmd,dtools,dub: 2.079.0 -> 2.079.1~~ [WIP] dmd,dtools,dub: 2.079.0 -> 2.079.1 May 11, 2018

ThomasMader mentioned this pull request May 12, 2018

Unable to build dmd with newer nix versions NixOS/nix#2155

Closed

ThomasMader mentioned this pull request May 12, 2018

binutils: Since Version 2.30 strip is broken #40397

Closed

ThomasMader force-pushed the dlang-update branch from 9b1a1db to d877c56 Compare May 12, 2018 10:23

ThomasMader changed the title ~~[WIP] dmd,dtools,dub: 2.079.0 -> 2.079.1~~ dmd,dtools,dub: 2.079.0 -> 2.079.1 and wrap ldc2 binary with $CC May 12, 2018

Ericson2314 reviewed May 13, 2018

View reviewed changes

ThomasMader force-pushed the dlang-update branch from d877c56 to 4bc4943 Compare May 16, 2018 16:30

ThomasMader mentioned this pull request May 16, 2018

DMD or related packages are broken on NixOS 18.09 #40461

Closed

ThomasMader added 2 commits May 19, 2018 23:16

ldc: wrap ldc2 binary with C compiler because it's needed for linking

893ba5d

dmd,dtools,dub: 2.079.0 -> 2.079.1

e762dca

ThomasMader force-pushed the dlang-update branch from 4bc4943 to e762dca Compare May 19, 2018 21:18

xeji merged commit 4aa04d1 into NixOS:master May 23, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

dmd,dtools,dub: 2.079.0 -> 2.079.1 and wrap ldc2 binary with $CC #40228

dmd,dtools,dub: 2.079.0 -> 2.079.1 and wrap ldc2 binary with $CC #40228

ThomasMader commented May 9, 2018 •

edited

ThomasMader commented May 9, 2018

ThomasMader commented May 11, 2018

ThomasMader commented May 11, 2018

GrahamcOfBorg commented May 11, 2018

GrahamcOfBorg commented May 11, 2018

ThomasMader commented May 11, 2018

ThomasMader commented May 12, 2018

ThomasMader commented May 12, 2018 •

edited

Ericson2314 May 13, 2018

ThomasMader May 16, 2018

Ericson2314 May 13, 2018

ThomasMader May 16, 2018

GrahamcOfBorg commented May 16, 2018

GrahamcOfBorg commented May 16, 2018

ThomasMader commented May 19, 2018

GrahamcOfBorg commented May 19, 2018

GrahamcOfBorg commented May 19, 2018

ElvishJerricco commented May 23, 2018

ThomasMader commented May 23, 2018

dmd,dtools,dub: 2.079.0 -> 2.079.1 and wrap ldc2 binary with $CC #40228

dmd,dtools,dub: 2.079.0 -> 2.079.1 and wrap ldc2 binary with $CC #40228

Conversation

ThomasMader commented May 9, 2018 • edited

Motivation for this change

Things done

ThomasMader commented May 9, 2018

ThomasMader commented May 11, 2018

ThomasMader commented May 11, 2018

GrahamcOfBorg commented May 11, 2018

GrahamcOfBorg commented May 11, 2018

ThomasMader commented May 11, 2018

ThomasMader commented May 12, 2018

ThomasMader commented May 12, 2018 • edited

Ericson2314 May 13, 2018

Choose a reason for hiding this comment

ThomasMader May 16, 2018

Choose a reason for hiding this comment

Ericson2314 May 13, 2018

Choose a reason for hiding this comment

ThomasMader May 16, 2018

Choose a reason for hiding this comment

GrahamcOfBorg commented May 16, 2018

GrahamcOfBorg commented May 16, 2018

ThomasMader commented May 19, 2018

GrahamcOfBorg commented May 19, 2018

GrahamcOfBorg commented May 19, 2018

ElvishJerricco commented May 23, 2018

ThomasMader commented May 23, 2018

ThomasMader commented May 9, 2018 •

edited

ThomasMader commented May 12, 2018 •

edited