Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stack 1.6 linking issues on Arch Linux #3518

Closed
ruuda opened this issue Oct 26, 2017 · 27 comments
Closed

Stack 1.6 linking issues on Arch Linux #3518

ruuda opened this issue Oct 26, 2017 · 27 comments

Comments

@ruuda
Copy link
Contributor

ruuda commented Oct 26, 2017

General summary/comments

I am testing the Stack 1.6.0 pre-release. I am trying to build ruuda/christmas-tree, commit a48012e. I can build it fine with Stack 1.5.1, but not with the 1.6.0 pre-release; the output is as follows:

Linking /home/ruud/.stack/setup-exe-cache/x86_64-linux-tinfo6-nopie/tmp-Cabal-simple_mPHDZzAJ_1.24.0.0_ghc-8.0.1 ...

In file included from /tmp/ghc2706_0/ghc_1.c:1:0: error: 

In file included from /home/ruud/.stack/programs/x86_64-linux/ghc-tinfo6-nopie-8.0.1/lib/ghc-8.0.1/include/Rts.h:217:0: error:
    

/home/ruud/.stack/programs/x86_64-linux/ghc-tinfo6-nopie-8.0.1/lib/ghc-8.0.1/include/rts/storage/ClosureMacros.h:505:5: error:
     warning: macro expansion producing 'defined' has undefined behavior [-Wexpansion-to-defined]
#if ZERO_SLOP_FOR_LDV_PROF || ZERO_SLOP_FOR_SANITY_CHECK
    ^

... (more of those, it's a GHC bug which I thought was fixed at some point) ...

/usr/bin/ld: /home/ruud/.stack/setup-exe-src/setup-shim-mPHDZzAJ.o: relocation R_X86_64_32S against symbol `stg_bh_upd_frame_info' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /tmp/ghc2706_0/ghc_2.o: relocation R_X86_64_32 against symbol `ZCMain_main_closure' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /home/ruud/.stack/programs/x86_64-linux/ghc-tinfo6-nopie-8.0.1/lib/ghc-8.0.1/Cabal-1.24.0.0/libHSCabal-1.24.0.0.a(Simple__35.o): relocation R_X86_64_32S against `.text' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /home/ruud/.stack/programs/x86_64-linux/ghc-tinfo6-nopie-8.0.1/lib/ghc-8.0.1/Cabal-1.24.0.0/libHSCabal-1.24.0.0.a(Simple__48.o): relocation R_X86_64_32S against `.text' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /home/ruud/.stack/programs/x86_64-linux/ghc-tinfo6-nopie-8.0.1/lib/ghc-8.0.1/Cabal-1.24.0.0/libHSCabal-1.24.0.0.a(Simple__33.o): relocation R_X86_64_32S against symbol `stg_upd_frame_info' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /home/ruud/.stack/programs/x86_64-linux/ghc-tinfo6-nopie-8.0.1/lib/ghc-8.0.1/Cabal-1.24.0.0/libHSCabal-1.24.0.0.a(Command__75.o): relocation R_X86_64_32S against symbol `stg_upd_frame_info' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /home/ruud/.stack/programs/x86_64-linux/ghc-tinfo6-nopie-8.0.1/lib/ghc-8.0.1/Cabal-1.24.0.0/libHSCabal-1.24.0.0.a(Command__24.o): relocation R_X86_64_32S against symbol `stg_upd_frame_info' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /home/ruud/.stack/programs/x86_64-linux/ghc-tinfo6-nopie-8.0.1/lib/ghc-8.0.1/Cabal-1.24.0.0/libHSCabal-1.24.0.0.a(Command__185.o): relocation R_X86_64_32S against `.text' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /home/ruud/.stack/programs/x86_64-linux/ghc-tinfo6-nopie-8.0.1/lib/ghc-8.0.1/Cabal-1.24.0.0/libHSCabal-1.24.0.0.a(Command__66.o): relocation R_X86_64_32 against symbol `Cabalzm1zi24zi0zi0_DistributionziSimpleziCommand_zdLrbGjpolyzugo3_closure' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /home/ruud/.stack/programs/x86_64-linux/ghc-tinfo6-nopie-8.0.1/lib/ghc-8.0.1/Cabal-1.24.0.0/libHSCabal-1.24.0.0.a(Command__55.o): relocation R_X86_64_32S against symbol `stg_bh_upd_frame_info' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /home/ruud/.stack/programs/x86_64-linux/ghc-tinfo6-nopie-8.0.1/lib/ghc-8.0.1/Cabal-1.24.0.0/libHSCabal-1.24.0.0.a(Command__65.o): relocation R_X86_64_32 against symbol `Cabalzm1zi24zi0zi0_DistributionziSimpleziCommand_zdLrbGipolyzugo2_closure' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /home/ruud/.stack/programs/x86_64-linux/ghc-tinfo6-nopie-8.0.1/lib/ghc-8.0.1/Cabal-1.24.0.0/libHSCabal-1.24.0.0.a(Configure__329.o): relocation R_X86_64_32S against `.text' can not be used when making a shared object; recompile with -fPIC

... thousands of those ...

/usr/bin/ld: /home/ruud/.stack/programs/x86_64-linux/ghc-tinfo6-nopie-8.0.1/lib/ghc-8.0.1/rts/libCffi.a(ffi64.o): relocation R_X86_64_32S against `.rodata' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Nonrepresentable section on output
clang-5.0: error: linker command failed with exit code 1 (use -v to see invocation)
`clang' failed in phase `Linker'. (Exit code: 1)
Exit code ExitFailure 1 while running ["ghc","-rtsopts","-threaded","-clear-package-db","-global-package-db","-hide-all-packages","-package","base","-main-is","StackSetupShim.mainOverride","-package","Cabal-1.24.0.0","/home/ruud/.stack/setup-exe-src/setup-mPHDZzAJ.hs","/home/ruud/.stack/setup-exe-src/setup-shim-mPHDZzAJ.hs","-o","/home/ruud/.stack/setup-exe-cache/x86_64-linux-tinfo6-nopie/tmp-Cabal-simple_mPHDZzAJ_1.24.0.0_ghc-8.0.1"] in /tmp/stack2688/

Steps to reproduce

git clone https://github.com/ruuda/christmas-tree
cd christmas-tree
git checkout a48012e
stack build

I removed my ~/.stack/config.yaml but that does not resolve the issue. The project's stack.yaml looks like this:

resolver: lts-7.13
extra-deps:
  - serialport-0.4.7

I think Stack might have downloaded the wrong compiler here; it downloaded a nopie compiler but then complains about -fPIC. I am using Arch Linux.

Expected

The build should succeed.

Actual

Thousands of linker complaints.

Stack version

Version 1.6.0.20171022, Git revision 7bddfaf7f9f8cd9ec1c710fa83e77262e494eee4 (5285 commits) x86_64 hpack-0.18.1

Method of installation

  • Adjusted the stack-static AUR pkgbuild for 1.6.0 pre-release. It downloads the tar from Github.
@mgsloan
Copy link
Contributor

mgsloan commented Oct 26, 2017

Thanks for the report! Is it possible that this is related to changes in arch and not changes in stack? In other words, does the problem still reproduce now if you downgrade to 1.5.1?

Haven't seen this particular issue, but there's been lots of bug reports recently about ncurses / nopie and arch linux, which largely seem to be from upstream changes:

@mgsloan mgsloan added this to the Support milestone Oct 26, 2017
@ruuda
Copy link
Contributor Author

ruuda commented Oct 27, 2017

If I downgrade to 1.5.1 I can build again.

I did have trouble with ncurses and nopie in the past, the first was resolved by installing an AUR package, the second was resolved by somebody uploading the right bindist so Stack could download it. There was a comment about that in an issue here somewhere, but I cannot find it at the moment.

@mgsloan
Copy link
Contributor

mgsloan commented Oct 28, 2017

Hmm, can you please try with this patch? #3521 It might solve the issue here.

@ruuda
Copy link
Contributor Author

ruuda commented Oct 28, 2017

When I build Stack 53a0e618 and then run stack setup, I get the following message:

No information found for ghc-8.0.1.
Supported versions for OS key 'linux64-ncurses6-nopie': GhcVersion 8.0.2, GhcVersion 8.2.1

When I change the resolver to the GHC 8.0.2-based lts-9.9, it downloads ghc-ncurses6-nopie-8.0.2, which works fine.

@henrique-almeida
Copy link

henrique-almeida commented Oct 28, 2017

sed -i 's/-fno-PIE/-no-pie/g' ~/.stack/programs/x86_64-linux/ghc-tinfo6-nopie-8.2.1/lib/ghc-8.2.1/settings
fixed it for me (issue #2712)

@mgsloan
Copy link
Contributor

mgsloan commented Oct 29, 2017

Interesting! Pinging @borsboom

@borsboom
Copy link
Contributor

I think @ruuda's approach is right, it's just that I guess nobody built a compatible ghc-8.0.1 bindist. Probably best to upgrade to ghc-8.0.2 anyway.

@Earnestly
Copy link

Arch Linux has adopted --enable-default-pie for their gcc package. I believe other distributions are slowly adopting the same disposition as well. If you want to continue building static packages you'll need to use -no-pie on those distributions as it is no longer the default behaviour.

@borsboom
Copy link
Contributor

There seems to be so much contradictory information coming in. Starting with a totally fresh Arch system, the following commands worked for me, so I think selection of the tinfo6-nopie variant is right for what should be the most common Arch case:

pacman -Syu make gcc
curl -sSL https://get.haskellstack.org/ |sh
stack new test
cd test
stack build
stack exec test-exe

However, building network, bindings-uname, and others failed with a linker error mentioning "recompile with -fPIC". I edited the GHC settings file and added -fPIC to the "C compiler link flags" and then building those worked as well

So that leaves me with a couple of questions:

  1. How can we automatically determine whether -fPIC is needed?
  2. Why are others having what appear to be completely different problems? Perhaps it's related to the presence of the ncurses5-compat-lib package, and we should stop recommending it?

@borsboom
Copy link
Contributor

One more observation: downgrading to stack-1.5.1 does not solve the problem for me.

@borsboom
Copy link
Contributor

@NorfairKing told me that adding -no-pie to the "C compiler link flags" in settings works for him and is the better approach than using -fPIC. I can confirm that works for me as well. Someone mentioned that older versions of gcc don't support -no-pie, but I wonder if we can assume that any distribution that's using PIE by default is also using a newer version of gcc (it at least ought to work on more cases).

What's weird is that it looks like GHC should be passing in -no-pie automatically (introduced in http://git.haskell.org/ghc.git/commitdiff/fefe02c0324a25a52455a61f7f6e48be6d82d1ab), but haven't dug into it yet.

@borsboom borsboom changed the title Stack 1.6 pre-release linking issues Stack 1.6 linking issues on Arch Dec 24, 2017
@borsboom borsboom changed the title Stack 1.6 linking issues on Arch Stack 1.6 linking issues on Arch Linux Dec 24, 2017
@borsboom
Copy link
Contributor

Relevant information also in #3648

@borsboom
Copy link
Contributor

borsboom commented Dec 24, 2017

It turns out the regular tinfo6 configuration (without nopie) seems to work fine on Arch since GHC 8.0.2! Perhaps the extra nopie settings are actually interfering GHC's own logic. I switched to that by adding to ~/.stack/config.yaml:

ghc-build: tinfo6

I'll see if this also works on other PIE-by-default distributions, in which case we may be able to simply remove the extra configure options from the nopie variants in stackage-content/stack/stack-setup-2.yaml

@borsboom
Copy link
Contributor

More potentially relevant information in #3630.

@borsboom
Copy link
Contributor

I've tested without nopie on a number of distributions now and the only one that seems to be a problem is Gentoo/Sabayon, which actually has quite a different gcc setup (it's Gentoo Hardened but not --enable-default-pie, which unfortunately we combined into 'nopie').

My current thinking is that we should remove the configure-envs for GHC >= 8.0.2 in stack-setup-2.yaml since that should simplify things for more people. Unfortunately, this will complicate things for Gentoo users but we can document that and add a special case for Gentoo in the next Stack release (any thoughts @rdnetto?)

@borsboom
Copy link
Contributor

borsboom commented Dec 24, 2017

Alright, I think I have something that's going to work for both Arch and Gentoo (and also tested on Ubuntu 18.04 and Fedora 27). I've removed the configure-envs from the nopie "builds" so that Stack will stop trying to set the flags and let GHC's own logic (since 8.0.2) do it properly. But since that fails on Gentoo, I've patched the GHC bindists and added a check to their configure script for the Gentoo case. GHC's configure is really the right place for this sort of thing (rather than extra logic in Stack) and also means we can potentially contribute fixes upstream. Please give this a try using:

stack setup --setup-info-yaml=https://raw.githubusercontent.com/fpco/stackage-content/nopie-fixes-arch-gentoo/

I've only updated the tinfo6 bindists for GHC 8.0.2 and 8.2.2 for now.

Strangely, Gentoo seems to need --no-pie instead of -no-pie (note the number of dashes).

@rdnetto
Copy link
Contributor

rdnetto commented Dec 26, 2017

I've confirmed that Stack 1.6.1 works with the modified YAML under Sabayon with GCC 5.4.0, though I had to change the command to:

stack --resolver=lts-10.0 setup --setup-info-yaml=https://raw.githubusercontent.com/fpco/stackage-content/nopie-fixes-arch-gentoo/stack/stack-setup-2.yaml

(I think you might have truncated the URL)

I'm happy with any solution that works seamlessly, particularly if we can upstream it into GHC.

@ruuda
Copy link
Contributor Author

ruuda commented Dec 27, 2017

With the --setup-info-yaml= from the above comment, I observe the same issues as in the original report:

...
relocation R_X86_64_32S against ``.rodata' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Nonrepresentable section on output
clang-5.0: error: linker command failed with exit code 1 (use -v to see invocation)
`clang' failed in phase `Linker'. (Exit code: 1)

My CC and CXX are set to clang and clang++. This is with Stack version 1.6.3, Git revision b27e629 (5454 commits) x86_64 hpack-0.20.0.

I still have the ncurses5-compat-libs package installed.

@borsboom
Copy link
Contributor

@ruuda You might be on your own for this one... we endeavour to have stack setup work out-of-the-box for standard configurations of common distros, but we can't support every possible deviation (especially when many of these are more properly handled by GHC itself than Stack; in fact much of the reason we're having this current issue is because Stack tried to work around missing support in GHC and ended up interfering on newer versions of GHC).

Did you have CC and CXX set this way when Stack first installed GHC? It not, GHC's configure script may have set itself up for gcc instead of clang. In that case, try running stack setup --reinstall so that it'll be set up properly for clang.

@ruuda
Copy link
Contributor Author

ruuda commented Dec 27, 2017

Yes, I did have those set during initial install. I cleaned my ~/.stack directory before doing the stack setup. stack setup failed to compile a sanity check after downloading and installing GHC. I also tried with CC=gcc CXX=g++, but this fails with the same error, except that it now says “gcc failed in phase Linker” rather than “clang failed”.

Doing a stack setup of LTS 9.10 with GHC 8.0.2 does work fine with the --setup-info-yaml; it was LTS 7.13 with GHC 8.0.1 that failed. But LTS 9.10 also works fine without passing the custom --setup-info-yaml.

It is not a big issue for me, I can upgrade all my projects to a more recent LTS.

@borsboom
Copy link
Contributor

@ruuda Ah, that explains it. GHC didn't add -no-pie support until 8.0.2.

borsboom added a commit to commercialhaskell/stackage-content that referenced this issue Dec 27, 2017
borsboom added a commit to commercialhaskell/stackage-content that referenced this issue Dec 27, 2017
* Removes GHC <8.0.2 for all Linux `-nopie` builds, since those don't work
  reliably.
* Removes `configure-env` for all remaining Linux `-nopie` builds, since GHC
  >=8.0.2 autodetects GCCs that support `-no-pie` (and the
  configure-envs interfered with the auto-detection causing it to fail
  on some distros)
* Patches Linux GHC >= 8.0.2 bindists so that their `configure`
  script detects Gentoo Hardened GCC and adds `--no-pie`
  to linker arguments.
* Adds ncurses6 GHC 8.2.2 bindist for Void Linux

Relates to commercialhaskell/stack#3518 commercialhaskell/stack#3636
@borsboom
Copy link
Contributor

I've merged commercialhaskell/stackage-content#34, which should fix this for new stack setups, at least for GHC >= 8.0.2 (older GHC versions don't support pie-by-default gcc properly). Please try stack setup --reinstall if you're experiencing these linking errors.

Closing this issue, but if it's not solved for you feel free to re-open.

@borsboom
Copy link
Contributor

I've updated the FAQ entry here: #3725

@theolaurent
Copy link

Hi guys.
What about GHCJS, which according to https://docs.haskellstack.org/en/stable/ghcjs/ seems to be based at most on 8.0.1 ?
Is there an (even ugly) workaround to make it work ?

@borsboom
Copy link
Contributor

borsboom commented Jan 6, 2018

@theolaurent: It would be very nice to get updated "repacks" of GHCJS that use GHC 8.0.2 or newer (for reasons other than this as well). That deserves a separate issue.

In terms of workaround for now, see https://docs.haskellstack.org/en/stable/faq/#i-get-strange-ld-errors-about-recompiling-with-fpic.

@kindaro
Copy link

kindaro commented Jan 24, 2018

I will leave a brief report of my success building things with all four of ghc-7.10.3 ghc-8.0.1 ghc-8.0.2 ghc-8.2.2 on the same ArchLinux machine.

Perhaps it is redundant, but actually I don't think the FAQ article linked above is comprehensive. Though I do find it insightful, perhaps it might be improved, or otherwise someone may stumble on this issue as I did and find my report helpful.

  • The build with ghc-8.0.2 & ghc-8.2.2 is straightforward:

    1. This haskell-stack-git package from aur builds like a charm.
    2. As a dependency it lists ncurses-full or ncurses-full-git. It is indeed necessary to install one of these (at your preference), as it provides shared libraries such as libtinfo.so of version 6.
  • The build with ghc-7.10.3 & ghc-8.0.1 will not succeed. These older compilers will not even get installed with stack setup. I advise the following measures:

    1. Prepare stack.yaml files that declare an appropriate resolver and, additionally, ghc-build: standard.
    2. The older compilers need ncurses 5 rather than 6. So, obtain ncurses5-compat-libs from aur and makepkg it. You will not be able to install the package with pacman -U without removing ncurses-full or ncurses-full-git first, which will render the newer ghc-8.0.2 & ghc-8.2.2 compilers unusable. So, you will have to manually extract usr/lib/libncursesw.so.5.9 and place it at /usr/lib/libtinfo.so.5. (There is a system of symlinks to be found in ncurses5-compat-libs that maps from one library path to another and then another that I neglected to reconstruct awhole but I guess you may if you wish.) Now you should be able to stack setup the older compilers.
    3. After installing the compilers, fix the corresponding ~/.stack/programs/.../lib/.../settings files, adding -no-pie at C compiler flags, just as described in the Stack FAQ. At this point your projects should start to compile. Hooray!

The minimal stack.yaml files for the older compilers:

ghc-7.10.3

resolver: lts-6.35
ghc-build: standard

ghc-8.0.1

resolver: lts-7.24
ghc-build: standard

I hope this somewhat longish installation is not too out of place here.

@fosskers
Copy link
Contributor

During an attempt at compiling a git clone of ghcjs:

colin@yumi ~/c/h/ghcjs> stack build
No information found for ghc-8.0.1.
Supported versions for OS key 'linux64-tinfo6-nopie': GhcVersion 8.0.2, GhcVersion 8.2.1, GhcVersion 8.2.2

The same thing occurs if following the usual instructions for using GHCjs w/ stack found here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

9 participants