stack install fails on FreeBSD 12.0-Current

[noinia]

General summary/comments (optional)

I'm having trouble building/installing packages with stack (1.5.1, installed using the official port) on FreeBSD 12.0-Current. Essentially, building any package fails with:

    Cabal-simple_mPHDZzAJ_1.24.2.0_ghc-8.0.2: No cabal file found.
    Please create a package description file <pkgname>.cabal

It does not seem to matter whether I try to build a local project using 'stack build', or install a particular package with 'stack install '. Nor does it seem matter which stackage release/version of ghc I use: I've tried with lts-9.* and ghc-8.0.2 (installed by stack), lts-* and ghc installed using the port, and the nightly snapshots with ghc 8.2.1 (installed by stack). Any idea what is going on here?

Steps to reproduce

rm ~/.stack
stack setup
stack install fgl --verbose

Expected

stack installs the package fgl.

Actual

See the full output here

Stack version

frank@UU# stack --version
Version 1.5.1 x86_64
Compiled with:
<see full output linked above>

Method of installation

• Via the freebsd port in devel/stack

frank@UU# pkg info stack
stack-1.5.1
Name           : stack
Version        : 1.5.1
Installed on   : Tue Oct 24 16:00:14 2017 CEST
Origin         : devel/stack
Architecture   : FreeBSD:12:amd64
Prefix         : /usr/local
Categories     : haskell devel
Licenses       : BSD3CLAUSE
Maintainer     : tobik@FreeBSD.org
WWW            : http://www.haskellstack.org/
Comment        : Cross-platform program for developing Haskell programs
Shared Libs required:
	libgmp.so.10
	libiconv.so.2
	libcharset.so.1
Annotations    :
	repo_type      : binary
	repository     : local
Flat size      : 49.2MiB
Description    :
Stack is a cross-platform program for developing Haskell projects.
It is aimed at Haskellers both new and experienced. 

WWW: http://www.haskellstack.org/

system:

frank@UU# uname -a 
FreeBSD UU.FStaals.net 12.0-CURRENT FreeBSD 12.0-CURRENT #0 r324496: Wed Oct 11 09:22:17 CEST 2017     root@UU.FStaals.net:/usr/obj/usr/src/sys/GENERIC  amd64

[mgsloan]

Curious! What happens if you clone a package with no deps, say, http://github.com/aelve/microlens ?

Please post the log of it running with -v. It may also be helpful to copy out the invocation of Cabal-simple_mPHDZzAJ_1.24.2.0_ghc-8.0.2 and try it manually. Guessing it's failing on configure.

[noinia]

I cloned the microlens repo and tried to run 'stack install -v' in the microlens/microlens directory. (I did not blow away my entire .stack folder again before testing, but somehow I doubt that would make a difference). The full output is in this gist. Seems to be the same problem.

Running Cabal-simple manually just produces the same output as in the stack invocation:

frank@UU# /home/frank/.stack/setup-exe-cache/x86_64-freebsd/Cabal-simple_mPHDZzAJ_1.24.2.0_ghc-8.0.2 --builddir=.stack-work/dist/x86_64-freebsd/Cabal-1.24.2.0 configure --with-ghc=/home/frank/.stack/programs/x86_64-freebsd/ghc-8.0.2/bin/ghc --with-ghc-pkg=/home/frank/.stack/programs/x86_64-freebsd/ghc-8.0.2/bin/ghc-pkg --user --package-db=clear --package-db=global --package-db=/home/frank/.stack/snapshots/x86_64-freebsd/lts-9.10/8.0.2/pkgdb --package-db=/home/frank/tmp/microlens/microlens/.stack-work/install/x86_64-freebsd/lts-9.10/8.0.2/pkgdb --libdir=/home/frank/tmp/microlens/microlens/.stack-work/install/x86_64-freebsd/lts-9.10/8.0.2/lib --bindir=/home/frank/tmp/microlens/microlens/.stack-work/install/x86_64-freebsd/lts-9.10/8.0.2/bin --datadir=/home/frank/tmp/microlens/microlens/.stack-work/install/x86_64-freebsd/lts-9.10/8.0.2/share --libexecdir=/home/frank/tmp/microlens/microlens/.stack-work/install/x86_64-freebsd/lts-9.10/8.0.2/libexec --sysconfdir=/home/frank/tmp/microlens/microlens/.stack-work/install/x86_64-freebsd/lts-9.10/8.0.2/etc --docdir=/home/frank/tmp/microlens/microlens/.stack-work/install/x86_64-freebsd/lts-9.10/8.0.2/doc/microlens-0.4.8.1 --htmldir=/home/frank/tmp/microlens/microlens/.stack-work/install/x86_64-freebsd/lts-9.10/8.0.2/doc/microlens-0.4.8.1 --haddockdir=/home/frank/tmp/microlens/microlens/.stack-work/install/x86_64-freebsd/lts-9.10/8.0.2/doc/microlens-0.4.8.1 --dependency=base=base-4.9.1.0 --enable-tests --enable-benchmarks
Cabal-simple_mPHDZzAJ_1.24.2.0_ghc-8.0.2: No cabal file found.
Please create a package description file <pkgname>.cabal

anything else I can try?

[mgsloan]

@noinia Strange! Must be a cabal bug, I suggest reporting it to https://github.com/haskell/cabal

[myfreeweb]

I've been seeing this for a while… just realized — not seeing files! That must be the awesome new 64-bit inodes (and related changes) in 12-CURRENT that break everything. (I recently fixed that in Crystal; hacks are required in Rust because they don't have OS version conditionals yet.)

This is not a cabal bug.

Looks like GHC picks up the structures from C headers so there are no Rust-like problems. Stack just needs to provide GHC builds compiled on 12 for >=12.0 users. All 11 binaries are broken in terms of filesystem access. Changing inode size does that :)

[myfreeweb]

Oh — and stack itself should be built on 12 too! duh. That's why it didn't work when @noinia tried ghc from ports.

[myfreeweb]

So stack from pkg/ports should be built on 12 already, but just in case, my build (xz compressed)

And!! You need to run rm -rf ~/.stack/setup-exe-cache to clean up the old cached setup executables. I had them from before the inode changes landed.

[noinia]

Ah indeed, if I use stack with system-ghc, both built locally from ports, then I can successfully build and install packages.

Thanks @myfreeweb !

[hxw]

Looks like stack setup downloads a 9.3 compiled ghc (on TrueOS) so that is probably why I get the same error

% uname -rsv
FreeBSD 12.0-CURRENT FreeBSD 12.0-CURRENT #1 67a4643eb(trueos-master): Wed Nov  8 18:09:27 UTC 2017     root@chimera:/usr/obj/usr/src/amd64.amd64/sys/GENERIC
% cd ~/.stack/programs/x86_64-freebsd/ghc-8.0.2/lib/ghc-8.0.2/bin
% file ghc
ghc: ELF 64-bit LSB executable, x86-64, version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1, for FreeBSD 9.3, not stripped

[myfreeweb]

@hxw yeah yeah. use system ghc.

[simonmichael]

Thanks for figuring this out. I wonder where it needs to be fixed. The stack installer ? The ghc binaries stack installs ? Two sets of binaries required for old and new freebsd ? setup-exe-cache to be flushed automatically if found to contain too-old freebsd executables ?

[myfreeweb]

@simonmichael yes, 12 requires ghc compiled on 12, and cache should be flushed.

[myfreeweb]

also #1026 would've helped a lot (stack setup building a new ghc from source, using system ghc)

[borsboom]

It sounds like a first step might be include the FreeBSD version in the setup-info OS key, like we do for OpenBSD (see 3cd92c8). Then we could provide different bindists for different FreeBSD versions (and, since the caches are indexed by OS key, this would also invalidate the cache). That should be pretty easily to implement, and a pull request would be most welcome!

[myfreeweb]

Oh that's already implemented for OpenBSD? Nice.

bindings-uname-0.1 seems broken?? sysname returns FreeBSD but release returns f and nodename returns E. Nothing in the source has hardcoded offsets or anything…

[arrowd]

I have fixed bindings-uname on FreeBSD and sent the patch to author, but didn't get response yet. Maybe we can push this some other way?

[myfreeweb]

I guess you can release it as a new package. Or send a PR to stack that puts the module right here in this repo, since it's a tiny module.

[arrowd]

The proper solution for this is just building bindists on least supported FreeBSD version (10.4 ATM). I reported this to GHC upstream and 8.6.1 should already be fine.

[myfreeweb]

Hm. They were compiled on some 9.x, and now they will be on 10.4, how would this fix 12?

You can still use old functions that take old (32-bit-inode based) structs by linking to versioned (@FBSD_1.x) symbols, e.g.: https://github.com/rust-lang/libc/blob/41944d5c37ef53647c09ac92cdb4bf636840dfab/src/unix/bsd/freebsdlike/mod.rs#L1096

BUT all built binaries, including Stack itself should be done like that. When the Haskell C FFI stuff would look at system headers, it'd find the new (64-bit inode) structs!

The real solution should be building binaries on 12 for >=12 users.

[arrowd]

Hm. They were compiled on some 9.x, and now they will be on 10.4, how would this fix 12?

GHC bootstraps that are used to compile lang/ghc port are built on 10.4 and work on 12. But when you are compiling GHC on 12, you need to apply the workaround you mention. It is done automatically in ports Makefile: https://svnweb.freebsd.org/ports/head/lang/ghc/Makefile?view=markup#l230

[myfreeweb]

bootstraps that are used to compile lang/ghc port

Oh, sure.

But this thread is about the official binaries downloaded by Stack.

[arrowd]

Stack gets these binaries from GHC upstream, but we (FreeBSD) can also provide them. However, judging by bindist names at https://downloads.haskell.org/~ghc/8.6.1/ the upstream also uses ports to build GHC.

[noinia]

It all seems to boil down to two things:

1. There need to separate builds of GHC for (<12) and for (>= 12) that are available for download somewhere (preferably from the GHC website I guess).
2. Stack needs to figure out on which version of FreeBSD it is running, and use that information to figure out which version of GHC it needs to download.

as myfreeweb also stated; I don't think changing the GHC builder to FreeBSD 10.4 does anything that helps here.

[arrowd]

Again, GHC compiled on 10.4 works everywhere. Why do we need separate builds?

[myfreeweb]

GHC compiled on 10.4 works everywhere

For very limited definitions of "works" I think. You mentioned:

when you are compiling GHC [with the 10.4 as bootstrap] on 12, you need to apply the workaround

— this is because the 10.4 compiler does not produce correct binaries on 12 — and this applies not just to bootstrapping GHC, but to compiling anything.

[arrowd]

Ah, you are right. I somehow missed that wrap hack is applied to the bootstrap compiler, not resulting one.

Ok, then I either need to reroll #4269 or come up with solution that would allow running pre-ino64 GHC on 12 too.

[duncan-bayne]

Can I help in any way? I don't have any Haskell but do know FreeBSD reasonably well, and I can set up a development environment for anyone working on this issue to use.

[arrowd]

This is already fixed, I'm just waiting a new Stack release.

[dbaynard]

@duncan-bayne If you want to get involved I'm sure @arrowd can find something for you!