Cabal could not resolve dependencies

[wd15]

I'm having some problems installing hasktorch.

$ cabal new-build all
Resolving dependencies...
cabal: Could not resolve dependencies:
[__0] trying: hasktorch-codegen-0.0.1.0 (user goal)
[__1] next goal: base (dependency of hasktorch-codegen)
[__1] rejecting: base-4.9.1.0/installed-4.9... (constraint from project config
TODO requires ==4.11.1.0)
[__1] rejecting: base-4.11.1.0 (constraint from non-upgradeable package
requires installed instance)
[__1] rejecting: base-4.11.0.0, base-4.10.1.0, base-4.10.0.0, base-4.9.1.0,
base-4.9.0.0, base-4.8.2.0, base-4.8.1.0, base-4.8.0.0, base-4.7.0.2,
base-4.7.0.1, base-4.7.0.0, base-4.6.0.1, base-4.6.0.0, base-4.5.1.0,
base-4.5.0.0, base-4.4.1.0, base-4.4.0.0, base-4.3.1.0, base-4.3.0.0,
base-4.2.0.2, base-4.2.0.1, base-4.2.0.0, base-4.1.0.0, base-4.0.0.0,
base-3.0.3.2, base-3.0.3.1 (constraint from project config TODO requires
==4.11.1.0)
[__1] fail (backjumping, conflict set: base, hasktorch-codegen)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: base, hasktorch-codegen

Any idea what I'm doing wrong? I'm using

$ cabal --version
cabal-install version 2.3.0.0
compiled using version 2.3.0.0 of the Cabal library 
$ ls vendor/build
libATen.so  libEHX.so
$ echo $LD_LIBRARY_PATH
/home/dwheeler/git/hasktorch/vendor/build/

[austinvhuang]

Most commonly when I've run into this it's an issue with matching cabal/ghc versions. Could you try with ghc 8.4.2?

We definitely need to document a golden path for the build to avoid this as it comes up frequently.

[wd15]

Thanks for responding. I went ahead and updated ghc and things are better.

$ cabal-new  new-build all
Resolving dependencies...
Build profile: -w ghc-8.4.2 -O1
In order, the following will be built (use -v for more details):
 - alex-3.2.4 (exe:alex) (requires download & build)
 - base-compat-0.10.1 (lib) (requires download & build)
 - base-orphans-0.7 (lib) (requires download & build)

...

Downloading cuda-0.9.0.3...
Downloading pretty-show-1.7...
Building happy-1.19.9 (all, legacy fallback)...

Failed to build gd-3000.7.3. The failure occurred during the configure step.
Build log (
/home/wd15/.cabal/logs/ghc-8.4.2/gd-3000.7.3-0c58361cb016f66ce0013fd25aa74dac64f2cbe0f9c32b2ab38d34218587c797.log
):
Configuring gd-3000.7.3...
cabal: Missing dependency on a foreign library:
* Missing (or bad) header file: gd.h
* Missing (or bad) C library: gd
This problem ...

...

cabal: Failed to build gd-3000.7.3 (which is required by exe:lenet-cifar10
from hasktorch-examples-0.0.1.0). See the build log above for details.

$

Looks like some sort of issue with the gd library. The same error occurs if I try cabal install gd.

My current configuration

$ cabal --version
cabal-install version 2.3.0.0
compiled using version 2.3.0.0 of the Cabal library 
$ echo $LD_LIBRARY_PATH
/home/wd15/git/hasktorch/vendor/aten/build/lib
$ ls /home/wd15/git/hasktorch/vendor/aten/build/lib
libATen.so  libATen.so.1
$ ghc --version
The Glorious Glasgow Haskell Compilation System, version 8.4.2

I noticed that there is a Nix recipe on the way. I'm happy to wait for that if necessary. What is the timeline on the Nix recipe?

[stites]

@austinvhuang is correct in that this is probably a ghc-8.4.2 problem -- hasktorch-codegen is just a plain parser file, and the failed constraint is base.

A future problem you may have is that cabal new-build seems to have less interest in the LD_LIBRARY_PATH. We haven't really made a release, but we'll make sure to document all of this beforehand. If you ping Austin or I on gitter, we can help you out. In the meantime, check out how circle-ci builds ATen (you can skip the cabal section since you have 2.3) and also make sure that your cabal.project.local looks something like the cabal.project.local-example file (written for CUDA-based systems, though, so comment out irrelevant lines now defaulting to CPU-based systems).

Update: It looks like cabal new-builds don't really pay attention to the linked-library path, so you will need to correctly configure your command line or use the cabal.project.local. However, if you want to run any examples, you will still need to have libATen.so in your LD-path (DYLIB for osx, I think).

[austinvhuang]

Regarding gd, it's not the haskell library that's the issue, it's the system library libgd (can be installed w/ the package manager, gd-3000 is just a haskell binding). The gd is a library used for loading images in the examples, but we could potentially move back to juicypixels which is native haskell to reduce the dependencies. As Sam says, we're working towards cleaning up a simplified build process for the release.

[stites]

@wd15 GHC-8.4.3 support was just added to master

[wd15]

Thanks for all the support. I installed libgd and it got me passed the previous error, but I wasn't building on a machine with GPU / Cuda support. I'll be back in touch after building on a more appropriate platform.

[austinvhuang]

Hi @wd15 there's a flag to disable cuda that should make it build for cpus, try cabal new-build all --flags=-cuda. You can also add this flag to the cabal.project.local file. Others have run into this, so we'll be making a few changes so that CPU compilation is the default. See https://github.com/hasktorch/hasktorch/blob/master/cabal.project.local-example for an example of a cabal.project.local file if you haven't worked with them previously.

[wd15]

I tried with the --flags=-cuda and still throwing the "The configuration process failed to locate your CUDA installation" error, see attached.

[wd15]

Sorry about the last comment. I'd failed to update to the current master (c67fae6). After updating to master and ghc 8.4.3 and modifying cabal.project.local things seem to work. I can compile Main.hs and it runs.

One last thing though, it seems that ghci is broken,

$ ghci
GHCi, version 8.4.3: http://www.haskell.org/ghc/  :? for help
Loaded package environment from /home/wd15/git/hasktorch/.ghc.environment.x86_64-linux-8.4.3
<command line>: can't load .so/.DLL for: /home/wd15/git/hasktorch/dist-newstyle/build/x86_64-linux/ghc-8.4.3/hasktorch-raw-thc-0.0.1.0/noopt/build/libHShasktorch-raw-thc-0.0.1.0-inplace-ghc8.4.3.so (/home/wd15/git/hasktorch/dist-newstyle/build/x86_64-linux/ghc-8.4.3/hasktorch-raw-thc-0.0.1.0/noopt/build/libHShasktorch-raw-thc-0.0.1.0-inplace-ghc8.4.3.so: undefined symbol: THNN_CudaDoubleVolumetricUpSamplingTrilinear_updateGradInput)

Not sure what's going on there.

BTW It would be awesome to have a nix expression for hasktorch. Is that in progress?

Thanks for all the help!

[austinvhuang]

Do you run into the same issue with cabal new-repl? I haven't used nix myself, so will let @stites speak to that Q.

Thanks for your early feedback, troubleshooting now will pave the path for a clean onboarding for the initial release.

[stites]

cabal new-repl should be the way to go (according to https://typedr.at/posts/what-i-did-on-my-summer-vacation this may turn into cabal v2-repl).

I believe the main issue with the vanilla ghci call is that GHCi will attempt to load all of the symbols for your cabal file with your system GHC, but is not smart enough to take into account the packages cabal manages. cabal is also taking advantage of system-GHC-friendly behavior, however, so ghci is trying a best-effort build from the generated .ghc.environment.x86_64-linux-8.4.3.

--

On nix support, after consulting with @tstat: this wont be possible until either nix supports cabal new-build, or cabal new-build becomes cabal build. I have a feeling the latter will happen first. In the meantime, you can see tstat#1 which contains a nix formula for ATen -- I'm actually supposed to remove the redundant build-vendor.nix.

To consolidate threads: #103

[austinvhuang]

Closing this as the original issue is resolved per #105 (comment)

@wd15 I'll send a slack invite - free to join there if you have other Qs. Cheers!