Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pass `ld-options` through to GHC #5451

Open
wants to merge 4 commits into
base: master
from

Conversation

Projects
None yet
4 participants
@nh2
Copy link
Member

commented Jul 20, 2018

Tentative. Fixes #4925

Based on #5446; that one should be merged first and then this one rebased.

To review, only review the top commits (this change is much smaller than it looks here).

Please include the following checklist in your PR:

  • Patches conform to the coding conventions.
  • Any changes that could be relevant to users have been recorded in the changelog.
  • The documentation has been updated, if necessary.
  • If the change is docs-only, [ci skip] is used to avoid triggering the build bots.

Please also shortly describe how you tested your change. Bonus points for added tests!

  • I'll be testing it by using a cabal init mini project and checking that it "fails the right way" (linker errors) when --ld-option=-static is given.

nh2 added some commits Jul 18, 2018

Add `--enable-executable-static` flag. Fixes #391.
Also update the docs for `--enable-executable-dynamic`
as they were slightly misleading.
@nh2

This comment has been minimized.

Copy link
Member Author

commented Jul 20, 2018

Please let me know if there are docs I should update for this.

@nh2

This comment has been minimized.

Copy link
Member Author

commented Jul 21, 2018

Hmm, this isn't working yet. The flags are passed to the linker, but they appear to early in the cc invocation; I intended them to appear at the very end but they appear before -o and before all the Haskell libraries.

Since with linker flags later -l flags fill in "holes" of symbols left by earlier -l flags, this isn't good yet.

I want them to appear at the end. Any hints?

Is this because of the position inside GHC flags as given in

, [ "-optl" ++ opt | opt <- ghcOptLinkOptions opts]
or is this a decision of GHC itself?

@nh2

This comment has been minimized.

Copy link
Member Author

commented Jul 21, 2018

Is this because of the position inside GHC flags as given in ... or is this a decision of GHC itself?

I tried permuting the flags directly in the GHC invocation cabal makes, but the results are the same. So it seems like ghc sticks -optl options at the very front of the linker invocation (as the 5th argument for me), before it passes -lHS... flags.

@nh2

This comment has been minimized.

Copy link
Member Author

commented Jul 21, 2018

I found a very hacky workaround: Pass -optl-Wl,--start-group to GHC first, without closing it with --end-group.

This makes the order unimportant: See https://linux.die.net/man/1/ld

--start-group archives --end-group
The archives should be a list of archive files. They may be either explicit file names, or -l options.

The specified archives are searched repeatedly until no new undefined references are created. Normally, an archive is searched only once in the order that it is specified on the command line. If a symbol in that archive is needed to resolve an undefined symbol referred to by an object in an archive that appears later on the command line, the linker would not be able to resolve that reference. By grouping the archives, they all be searched repeatedly until all possible references are resolved.

Using this option has a significant performance cost. It is best to use it only when there are unavoidable circular references between two or more archives.

I don't want to keep this as the final solution though, as it can make linking slower (and you lose the ability to specify which symbols exactly should be picked by exploiting the order; but this ability you lose anyway if GHC gives you only one location to put your symbols).

At least this workaround is independent from Cabal itself: It works with the current PR without modifications, only users would have to use it.

@nh2

This comment has been minimized.

Copy link
Member Author

commented Jul 21, 2018

Urgh, trouble. You cannot pass --ld-option=-Wl,--start-group to cabal configure:

  • Cabal will pass that flag directly to ld like this:
/nix/store/3a74d03pq6d0cg3crkfw965k5s58hbn4-binutils-2.30/bin/ar -r dist/build/objs-27195/libHSdarcs-2.14.1-ARuU0cypkJyGiHVcj2sKnE.a '@dist/build/objs-27195/ar27195-1.rsp'
/nix/store/3a74d03pq6d0cg3crkfw965k5s58hbn4-binutils-2.30/bin/ar: creating dist/build/objs-27195/libHSdarcs-2.14.1-ARuU0cypkJyGiHVcj2sKnE.a
/nix/store/pgsjpn6hp8cgcbrvvabk6vnza2i7l7sg-binutils-wrapper-2.30/bin/ld -r -o dist/build/HSdarcs-2.14.1-ARuU0cypkJyGiHVcj2sKnE.o '@dist/build/ld27195-2.rsp' -Wl,--start-group -lgssapi_krb5 -lcom_err -lkrb5support -lkrb5 -lkrb5support -lk5crypto -lssl -lcrypto -lnghttp2 -lssh2
/nix/store/3a74d03pq6d0cg3crkfw965k5s58hbn4-binutils-2.30/bin/ld: unrecognized option '-Wl,--start-group'

But you also can't pass --ld-option=--start-group, because then at a later stage Cabal will pass -optl--start-group to gcc (which it uses as the linker), which doesn't understand --start-group and wants -Wl,--start-group instead.

This problem is created by the fact that GHC uses gcc for linking instead of ld.

Why is that? From #ghc IRC:

geekosaur: because the C scaffolding is needed for libc to work, and it's not always clear how to set it up right; the C compiler knows how

@nh2

This comment has been minimized.

Copy link
Member Author

commented Jul 21, 2018

I have researched how the various linkers do it:

GNU ld

Implements --start-group and the workaround works conceptually (I have tried it with ghc directly).

GNU gold

Implements --start-group; I have not tried it.

LLVM lld

(See https://stackoverflow.com/questions/11893996/why-does-the-order-of-l-option-in-gcc-matter#comment89874319_11894098)

The lld linker of the LLVM project does not implement the POSIX behaviour (see R..'s comment):

Here is how LLD approaches the problem. Instead of memorizing only undefined symbols, we program LLD so that it memorizes all symbols. When it sees an undefined symbol that can be resolved by extracting an object file from an archive file it previously visited, it immediately extracts the file and link it. It is doable because LLD does not forget symbols it have seen in archive files.

@nh2 nh2 force-pushed the nh2:issue-4925-pass-ld-options-to-ghc branch from 6840cdf to eeb4aeb Oct 16, 2018

@nh2

This comment has been minimized.

Copy link
Member Author

commented Dec 7, 2018

Is this PR good?

@l29ah

This comment has been minimized.

Copy link

commented Dec 17, 2018

How can i test it without messing up my working ghc installation?

@nh2

This comment has been minimized.

Copy link
Member Author

commented Dec 17, 2018

How can i test it without messing up my working ghc installation?

You should be able to test it by just cloning from git, checking out the branch, and building it with cabal or stack. You don't need to install it globally.

@jkachmar

This comment has been minimized.

Copy link

commented Jan 28, 2019

Is there anything that an interested passerby like myself could do to help get this merged?

Statically linked Haskell executables are something I'm very much interested, so I'd like to help unblock anything if it's at all within my capability 😄

@sboosali

This comment has been minimized.

Copy link
Collaborator

commented Feb 14, 2019

any update?

@jkachmar

This comment has been minimized.

Copy link

commented Apr 6, 2019

Ping. #5446 has been merged, so it would be really nice if this could be merged in as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.