New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backpack doesn't work with TemplateHaskell #4755

Closed
int-index opened this Issue Sep 10, 2017 · 8 comments

Comments

Projects
None yet
2 participants
@int-index
Contributor

int-index commented Sep 10, 2017

The error:

Build profile: -w ghc-8.2.1 -O1
In order, the following will be built (use -v for more details):
 - slay-gtk-0 (lib) (file src/Slay/Gtk.hs changed)
 - slay-gtk-0 (exe:slay-gtk-example) (dependency rebuilt)
Preprocessing library for slay-gtk-0..
Building library for slay-gtk-0..
[1 of 1] Compiling Slay.Gtk         ( src/Slay/Gtk.hs, /home/int-index/Projects/slay/dist-newstyle/build/x86_64-linux/ghc-8.2.1/slay-gtk-0/build/Slay/Gtk.o )
<command line>: can't load .so/.DLL for: /home/int-index/Projects/slay/dist-newstyle/build/x86_64-linux/ghc-8.2.1/slay-core-0/slay-core-0-inplace+JACzl5parSWAjZnfXRg086/build/slay-core-0-inplace+JACzl5parSWAjZnfXRg086/libHSslay-core-0-inplace+JACzl5parSWAjZnfXRg086-ghc8.2.1.so (/home/int-index/Projects/slay/dist-newstyle/build/x86_64-linux/ghc-8.2.1/slay-core-0/slay-core-0-inplace+JACzl5parSWAjZnfXRg086/build/slay-core-0-inplace+JACzl5parSWAjZnfXRg086/libHSslay-core-0-inplace+JACzl5parSWAjZnfXRg086-ghc8.2.1.so: undefined symbol: slayzmnumberzmdoublezm0zminplace_SlayziNumberziDouble_zdwzdcshowsPrec_info)
cabal: Failed to build slay-gtk-0 (which is required by exe:slay-gtk-example
from slay-gtk-0).

System configuration: Xubuntu, ghc-8.2.1 and cabal-head installed from ppa:hvr/ghc (including ghc-8.2.1-prof).

The contents of cabal.project.local:

with-compiler: /opt/ghc/8.2.1/bin/ghc

Steps to reproduce:

  1. clone https://github.com/int-index/slay
  2. checkout 3fe9e38
  3. apply the patch below
  4. cabal new-build all

The patch:

diff --git a/gtk/src/Slay/Gtk.hs b/gtk/src/Slay/Gtk.hs
index b791fdf..6da3b6c 100644
--- a/gtk/src/Slay/Gtk.hs
+++ b/gtk/src/Slay/Gtk.hs
@@ -29,14 +29,11 @@ data PreMatrix = PreMatrix
     pmOffset :: (Integer, Integer)
   }
 
-pmScaleL :: Lens' PreMatrix Centi
-pmScaleL = lens pmScale (\pm x -> pm { pmScale = x })
-
-pmRotateL :: Lens' PreMatrix Integer
-pmRotateL = lens pmRotate (\pm x -> pm { pmRotate = x })
-
-pmOffsetL :: Lens' PreMatrix (Integer, Integer)
-pmOffsetL = lens pmOffset (\pm x -> pm { pmOffset = x })
+makeLensesFor
+  [ ("pmScale", "pmScaleL"),
+    ("pmRotate", "pmRotateL"),
+    ("pmOffset", "pmOffsetL") ]
+  ''PreMatrix
 
 prepareMatrix1 :: PreMatrix -> Cairo.Matrix
 prepareMatrix1 PreMatrix{..} =

Without the patch the project builds fine. (It also used to build fine with this use of TH before I started using Backpack).

@ezyang

This comment has been minimized.

Show comment
Hide comment
@ezyang

ezyang Sep 10, 2017

Contributor

Curiously, although I can reproduce when I follow your reproduction instructions, when I try to make a synthetic example involving just lens deriving Template Haskell (even one of the fields in the record I'm deriving from comes from a signature), things seem to work fine.

Based on the error message, in particular, the reference to symbol slayzmnumberzmdoublezm0zminplace_SlayziNumberziDouble_zdwzdcshowsPrec_info, it is possible that, somehow, in your program, the TH function is attempting to 'show' a Slay.Number.Double (which is a signature, so there's no way we can actually do this.) But I am not sure where in your program this could be happening; certainly I wouldn't expect makeLenses to do this. Any ideas?

BTW, it would be nice if the error message here was better. See http://ghc.haskell.org/trac/ghc/ticket/14212

P.S. If you are feeling up for it, a minimized test case would help a lot! Though I suspect that in the end, it will be a TH function attempting to call show on an abstract type...

Contributor

ezyang commented Sep 10, 2017

Curiously, although I can reproduce when I follow your reproduction instructions, when I try to make a synthetic example involving just lens deriving Template Haskell (even one of the fields in the record I'm deriving from comes from a signature), things seem to work fine.

Based on the error message, in particular, the reference to symbol slayzmnumberzmdoublezm0zminplace_SlayziNumberziDouble_zdwzdcshowsPrec_info, it is possible that, somehow, in your program, the TH function is attempting to 'show' a Slay.Number.Double (which is a signature, so there's no way we can actually do this.) But I am not sure where in your program this could be happening; certainly I wouldn't expect makeLenses to do this. Any ideas?

BTW, it would be nice if the error message here was better. See http://ghc.haskell.org/trac/ghc/ticket/14212

P.S. If you are feeling up for it, a minimized test case would help a lot! Though I suspect that in the end, it will be a TH function attempting to call show on an abstract type...

@int-index

This comment has been minimized.

Show comment
Hide comment
@int-index

int-index Sep 10, 2017

Contributor

the TH function is attempting to 'show' a Slay.Number.Double (which is a signature, so there's no way we can actually do this

Interesting hypothesis, but I don't think that's right. Slay.Number is a signature, Slay.Number.Double is the concrete implementation.

when I try to make a synthetic example involving just lens deriving Template Haskell (even one of the fields in the record I'm deriving from comes from a signature), things seem to work fine

Yes, it's quite tricky to reproduce. Since 6c3f77 and before 3fe9e38 I was using Template Haskell together with Backpack, but instantiated Slay.Number to Slay.Number.Integer, and it worked fine. But when I tried to instantiate to Slay.Number.Double, the linking started to fail, and the only remedy I could find was to avoid TH.

a minimized test case would help a lot

Right, I know :( but I could barely find time to report it at all. What I can tell, though, is that the issue occurs only when the concrete implementation has its own data/function declarations. Here's an alternative implementation of Slay.Number.Double that doesn't cause issues:

module Slay.Number.Double
  ( Signed
  , Unsigned
  , unsafeToUnsigned
  , toSigned
  ) where

type Signed = Double
type Unsigned = Double

unsafeToUnsigned :: Double -> Double
unsafeToUnsigned = id

toSigned :: Double -> Double
toSigned = id
Contributor

int-index commented Sep 10, 2017

the TH function is attempting to 'show' a Slay.Number.Double (which is a signature, so there's no way we can actually do this

Interesting hypothesis, but I don't think that's right. Slay.Number is a signature, Slay.Number.Double is the concrete implementation.

when I try to make a synthetic example involving just lens deriving Template Haskell (even one of the fields in the record I'm deriving from comes from a signature), things seem to work fine

Yes, it's quite tricky to reproduce. Since 6c3f77 and before 3fe9e38 I was using Template Haskell together with Backpack, but instantiated Slay.Number to Slay.Number.Integer, and it worked fine. But when I tried to instantiate to Slay.Number.Double, the linking started to fail, and the only remedy I could find was to avoid TH.

a minimized test case would help a lot

Right, I know :( but I could barely find time to report it at all. What I can tell, though, is that the issue occurs only when the concrete implementation has its own data/function declarations. Here's an alternative implementation of Slay.Number.Double that doesn't cause issues:

module Slay.Number.Double
  ( Signed
  , Unsigned
  , unsafeToUnsigned
  , toSigned
  ) where

type Signed = Double
type Unsigned = Double

unsafeToUnsigned :: Double -> Double
unsafeToUnsigned = id

toSigned :: Double -> Double
toSigned = id
@ezyang

This comment has been minimized.

Show comment
Hide comment
@ezyang

ezyang Sep 10, 2017

Contributor

Right, I know :( but I could barely find time to report it at all.

Thanks a lot for giving a reproduction! I'll definitely try minimizing it myself; it just might take some time :)

Contributor

ezyang commented Sep 10, 2017

Right, I know :( but I could barely find time to report it at all.

Thanks a lot for giving a reproduction! I'll definitely try minimizing it myself; it just might take some time :)

@ezyang

This comment has been minimized.

Show comment
Hide comment
@ezyang

ezyang Sep 11, 2017

Contributor

OK, I have a hypothesis, which is that the error has to do with our algorithm for determining what objects to link in when TH is involved...

Contributor

ezyang commented Sep 11, 2017

OK, I have a hypothesis, which is that the error has to do with our algorithm for determining what objects to link in when TH is involved...

@ezyang

This comment has been minimized.

Show comment
Hide comment
@ezyang

ezyang Oct 1, 2017

Contributor

I minimized the test case.

-- p.cabal
name: p
version: 1.0
build-type: Simple
cabal-version: >= 2.0

library
    build-depends: base
    exposed-modules: P

-- P.hs
module P where
type B = Foo
newtype Foo = Foo Double

-- indef.cabal
name: indef
version: 1.0
build-type: Simple
cabal-version: >= 2.0

library
    build-depends: base
    signatures: Sig
    exposed-modules: Indef

-- Sig.hsig
signature Sig where
data B

-- Indef.hs
module Indef where
import Sig
data T = MkT B

-- th.cabal
name: th
version: 1.0
build-type: Simple
cabal-version: >= 2.0

library
    build-depends: p, indef, base
    mixins: p (P as Sig)
    exposed-modules: TH

-- TH.hs
{-# LANGUAGE TemplateHaskell #-}
module TH where
$( return [] )

Pretty intruiging. It only triggers on a "fresh" build: if you edit a file, rebuild, and then edit it back, the problem goes away.

Contributor

ezyang commented Oct 1, 2017

I minimized the test case.

-- p.cabal
name: p
version: 1.0
build-type: Simple
cabal-version: >= 2.0

library
    build-depends: base
    exposed-modules: P

-- P.hs
module P where
type B = Foo
newtype Foo = Foo Double

-- indef.cabal
name: indef
version: 1.0
build-type: Simple
cabal-version: >= 2.0

library
    build-depends: base
    signatures: Sig
    exposed-modules: Indef

-- Sig.hsig
signature Sig where
data B

-- Indef.hs
module Indef where
import Sig
data T = MkT B

-- th.cabal
name: th
version: 1.0
build-type: Simple
cabal-version: >= 2.0

library
    build-depends: p, indef, base
    mixins: p (P as Sig)
    exposed-modules: TH

-- TH.hs
{-# LANGUAGE TemplateHaskell #-}
module TH where
$( return [] )

Pretty intruiging. It only triggers on a "fresh" build: if you edit a file, rebuild, and then edit it back, the problem goes away.

@ezyang

This comment has been minimized.

Show comment
Hide comment
@ezyang

ezyang Oct 1, 2017

Contributor

@int-index I believe the problem is that when we compile the instantiated version of a package, we don't correctly link it with the library it was instantiated with; i.e., the library doesn't show up as an ldd dependency. However, the ghc-pkg entry does have the correct deps, so things mostly work OK... except for Template Haskell, which relies on accurate ldd information for performing linking.

The correct place to fix this is GHC but there might be a hotfix for Cabal which may be helpful.

Contributor

ezyang commented Oct 1, 2017

@int-index I believe the problem is that when we compile the instantiated version of a package, we don't correctly link it with the library it was instantiated with; i.e., the library doesn't show up as an ldd dependency. However, the ghc-pkg entry does have the correct deps, so things mostly work OK... except for Template Haskell, which relies on accurate ldd information for performing linking.

The correct place to fix this is GHC but there might be a hotfix for Cabal which may be helpful.

@ezyang

This comment has been minimized.

Show comment
Hide comment
@ezyang
Contributor

ezyang commented Oct 2, 2017

bgamari added a commit to bgamari/ghc that referenced this issue Oct 3, 2017

Include libraries which fill holes as deps when linking.
Fixes the issue reported at haskell/cabal#4755
and fixes #14304 in the GHC tracker.

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Test Plan: validate

Reviewers: bgamari, austin, goldfire

Reviewed By: bgamari

Subscribers: rwbarton, thomie

GHC Trac Issues: #14304

Differential Revision: https://phabricator.haskell.org/D4057

bgamari added a commit to bgamari/ghc that referenced this issue Oct 3, 2017

Include libraries which fill holes as deps when linking.
Fixes the issue reported at haskell/cabal#4755
and fixes #14304 in the GHC tracker.

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Test Plan: validate

Reviewers: bgamari, austin, goldfire

Reviewed By: bgamari

Subscribers: rwbarton, thomie

GHC Trac Issues: #14304

Differential Revision: https://phabricator.haskell.org/D4057

bgamari added a commit to ghc/ghc that referenced this issue Oct 3, 2017

Include libraries which fill holes as deps when linking.
Fixes the issue reported at haskell/cabal#4755
and fixes #14304 in the GHC tracker.

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Test Plan: validate

Reviewers: bgamari, austin, goldfire

Reviewed By: bgamari

Subscribers: rwbarton, thomie

GHC Trac Issues: #14304

Differential Revision: https://phabricator.haskell.org/D4057

bgamari added a commit to ghc/ghc that referenced this issue Oct 3, 2017

Include libraries which fill holes as deps when linking.
Fixes the issue reported at haskell/cabal#4755
and fixes #14304 in the GHC tracker.

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Test Plan: validate

Reviewers: bgamari, austin, goldfire

Reviewed By: bgamari

Subscribers: rwbarton, thomie

GHC Trac Issues: #14304

Differential Revision: https://phabricator.haskell.org/D4057

(cherry picked from commit f3f624a)
@ezyang

This comment has been minimized.

Show comment
Hide comment
@ezyang

ezyang Nov 17, 2017

Contributor

Closing this because 8.2.2 is going to have this fix.

Contributor

ezyang commented Nov 17, 2017

Closing this because 8.2.2 is going to have this fix.

@ezyang ezyang closed this Nov 17, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment