New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[nix-local-build] Replacement needed for `cabal copy` #3473

Open
acfoltzer opened this Issue May 31, 2016 · 8 comments

Comments

@acfoltzer
Collaborator

acfoltzer commented May 31, 2016

When preparing binary distributions of Cabalized software, I use a combination of configure and copy to prepare the staging directory that will be turned into a tarball, .msi, etc. For example:

% cabal configure --prefix=/usr/local
% cabal build
% cabal copy --destdir=$(pwd)/foo-1.0
% tar -czf foo-1.0.tar.gz foo-1.0 # for example

It looks like we can still specify a --prefix argument, but there is no new version of copy. I imagine a solution for this might overlap with #3332 but I want to make a separate ticket since the use cases are distinct.

@dcoutts

This comment has been minimized.

Member

dcoutts commented Jun 1, 2016

Yeah, honestly I'm not sure yet what the solution for this looks like. If the goal is to produce an installable exe then there's two cases I think

  • you're only producing a statically linked exe and you're lucky enough to not depend on any libraries that use data files
  • all other cases, producing dynamically linked exe or depend on libs that have data files

In the former case you can get away with just copying the output of this one package, and build it against deps from the store, but in the latter case you really need to also copy things from those deps. So in the latter case if we're not yet able to do relocatable builds then in fact we need to build the deps into some local space where we've configured all the deps for the user's chosen --prefix and then finally copy all of them to the target image dir.

So to be clear, I think the replacement here is not for copy which had a very limited (single-package) scope, but a command to produce an installable image of the target (and if necessary its deps).

@acfoltzer

This comment has been minimized.

Collaborator

acfoltzer commented Jun 6, 2016

So to be clear, I think the replacement here is not for copy which had a very limited (single-package) scope, but a command to produce an installable image of the target (and if necessary its deps).

Agreed. We structure our builds carefully to avoid problems with data files in dependencies, but this is shaky at best. It would be much nicer to have a more robust solution.

Does it make sense to try the "build deps with --prefix for staging" approach in the short term? Unless relocatable builds are coming soon, I don't want this to be a blocker for us adopting the new commands.

@ezyang

This comment has been minimized.

Contributor

ezyang commented Jul 21, 2016

Hello @acfoltzer,

Here's a workaround to get copy: run cabal copy --destdir=$(pwd)/foo-1.0 --builddir=dist-newstyle/build/foo-1.0/. The directory formats are compatible so this should just work.

Can you see if this does work? And if so, does this move this off the hotpath?

@acfoltzer

This comment has been minimized.

Collaborator

acfoltzer commented Jul 21, 2016

This doesn't seem to work entirely, though I might be Doing It Wrong.

The root problem seems to be that --prefix just isn't handled properly. This is probably worth splitting out into a separate ticket, but I end up with a destdir with the relevant parts of my ~/.cabal grafted in:

cryptol-2.4.0-Darwin-x86_64_2e15d4f
├── Users
│   └── acfoltzer
│       └── .cabal
│           ├── bin
│           │   └── cryptol
│           ├── lib
│           │   └── x86_64-osx-ghc-7.10.3
│           │       └── .fake.cryptol-2.4.0
│           │           ├── Cryptol
│           │           │   ├── Eval
│           │           │   │   ├── Arch.dyn_hi
│           │           │   │   ├── Arch.hi
│           │           │   │   ├── Env.dyn_hi
│           │           │   │   ├── Env.hi
...
@acfoltzer

This comment has been minimized.

Collaborator

acfoltzer commented Jul 21, 2016

Ah, it looks like that copy was picking up the --prefix of a previous cabal configure. It looks like new-configure and new-build don't handle --prefix at all; again an issue for a separate ticket.

@dcoutts

This comment has been minimized.

Member

dcoutts commented Sep 8, 2016

So our latest thinking on this issue is that we can configure with a custom --prefix etc and still install into the store. Then the replacement for copy (perhaps "image", as in create an installable image) would just copy the relevant files from deps from the store. For static linking that'd just be data files, but for dynamic linking it'd also be shared libs. That works for exes. Doing copy/image for a lib rather than an exe is still "interesting" but should be doable. Fortunately it's not that common to want to build a lib here and deploy it over there so we can postpone that for a bit (though some Haskell progs do rely on building code and so do use libs, e.g. xmonad).

One adjustment that would be necessary for packages with custom prefix in the store is to adjust the paths that get registered with in the ghc-pkg db so that the registration actually works.

@ezyang ezyang added this to the Default nix-local-build milestone Sep 8, 2016

@ezyang

This comment has been minimized.

Contributor

ezyang commented Sep 8, 2016

@acfoltzer I assigned you to this ticket because it's in Galois' bucket list. When Galois does get an engineer working on this, reassign it to them!

@acfoltzer

This comment has been minimized.

Collaborator

acfoltzer commented Sep 8, 2016

@ezyang cool; feel free to do this on others where appropriate!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment