require 'cunn' fails on OSX 10.10

[alexbw]

I'm on OS X 10.10

th> require 'nn'
true
                                                                      [0.0603s]
th> require 'cunn'
/usr/local/share/lua/5.1/trepl/init.lua:318: loop or previous error loading module 'cunn'
stack traceback:
    [C]: in function 'error'
    /usr/local/share/lua/5.1/trepl/init.lua:318: in function 'require'
    [string "require 'cunn'"]:1: in main chunk
    [C]: in function 'xpcall'
    /usr/local/share/lua/5.1/trepl/init.lua:587: in function 'repl'
    /usr/local/lib/luarocks/rocks/trepl/scm-1/bin/th:185: in main chunk
    [C]: at 0x010d8957b0
                                                                      [0.1432s]
th>

looking deeper, seems to be an issue with the libcunn.so file

th> require 'libtorch'
{
  ceil : function: 0x0dd82c58
  tan : function: 0x0dd82b90
  test : function: 0x0dd9ca38
  sinh : function: 0x0dd82b68
  loadobj : function: 0x0dd75408
... 

th> require 'libcunn'
/usr/local/share/lua/5.1/trepl/init.lua:318: /usr/local/share/lua/5.1/luarocks/loader.lua:117: error loading module 'libcunn' from file '/usr/local/lib/lua/5.1/libcunn.so':
    dlopen(/usr/local/lib/lua/5.1/libcunn.so, 6): no suitable image found.  Did find:
    /usr/local/lib/lua/5.1/libcunn.so: malformed mach-o image: load command #23 length (0) too small in /usr/local/lib/lua/5.1/libcunn.so
stack traceback:
    [C]: in function 'error'
    /usr/local/share/lua/5.1/trepl/init.lua:318: in function 'require'
    [string "require 'libcunn'"]:1: in main chunk
    [C]: in function 'xpcall'
    /usr/local/share/lua/5.1/trepl/init.lua:587: in function 'repl'
    /usr/local/lib/luarocks/rocks/trepl/scm-1/bin/th:185: in main chunk
    [C]: at 0x010d8957b0

I've run into an issue like this before with installing libraries like OpenCV, which required some .so surgery, but I'm a Lua newbie, so I'm not sure exactly how to proceed. OS X 10.10.1, Retina MBP 15".

Any suggestions or further information for debugging required would be appreciated! Would love to get to the bottom of this.

[alexbw]

Should also mention cutorch imports

th> require 'cutorch'
{
  withDevice : function: 0x0df21dd0
  seed : function: 0x0df11c08
  _state : userdata: 0x7fb619752680
  getDeviceCount : function: 0x0df11b30
  getDeviceProperties : function: 0x0df11c70
  deviceReset : function: 0x0df11b90
  test : function: 0x0df21d88
  getDevice : function: 0x0df11b68
  synchronize : function: 0x0df11b08
  manualSeed : function: 0x0df11cc8
  setDevice : function: 0x0df11be0
  seedAll : function: 0x0df11d90
  initialSeed : function: 0x0df11db8
  getRNGState : function: 0x0df11d40
  manualSeedAll : function: 0x0df11cf0
  setRNGState : function: 0x0df11d68
}

[soumith]

for cutorch, @szagoruyko pushed a fix for OSX 10.10:
https://github.com/torch/cutorch/pull/34/files

we might have to do the same for cunn.
unfortunately i do not have OSX 10.10 with a GPU.
You might have to adapt the cutorch PR that I pointed out and patch it to cunn.

[szagoruyko]

For cutorch and cunn on OS X 10.10 I delete cutorch/cmake folder and let cmake do it's job, since the FindCUDA files are included in cmake (since v2.8 aclually). The latest cmake 3.1 from homebrew doesn't have an issue with malformed libs.

[alexbw]

Can you explain that process a little more mechanically?
After downloading torch-distro, delete those particular folders, and then
run ./install.sh?

On Sun, Jan 11, 2015 at 5:28 PM, Sergey Zagoruyko notifications@github.com
wrote:

For cutorch and cunn on OS X 10.10 I delete cutorch/cmake folder and let
cmake do it's job, since the FindCUDA files are included in cmake (since
v2.8 aclually). The latest cmake 3.1 from homebrew doesn't have an issue
with malformed libs.

—
Reply to this email directly or view it on GitHub
#4 (comment).

[szagoruyko]

At first check that your cmake is updated to the latest version. Maybe it will handle everything itself.

[alexbw]

I'm at 3.1, installed via homebrew. The errors I posted above were built with that version.

[szagoruyko]

okay delete then extra/cunn/cmake and try to rerun ./install.sh

[alexbw]

Alright, sorry for the noise on the thread. There is a conflict when continuum.io's anaconda python distribution is on the path. It has its own MKL and BLAS which seem to interfere in ways I'm not going to try to disentangle. During an install, I just move anaconda aside,

mv ~/anaconda ~/anaconda-backup

Other torch installer guides online ask for gcc-4.9. I had tried these other torch installers before finding this repo (and use and like GCC 4.9 for other projects), so that was my default compiler. CUDA 6.5 (the most up-to-date) on OS X 10.10 does not play nicely with gcc, I think of any version. So, make sure that gcc --version says that you're using clang. Here's what an Xcode command line tools build provides:

$ gcc --version
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 6.0 (clang-600.0.56) (based on LLVM 3.5svn)
Target: x86_64-apple-darwin14.0.0
Thread model: posix

Then, use @szagoruyko's trick to remove the cmake folder

rm ~/torch-build/extra/cunn/cmake

And then build everything with

./install.sh

The install script as it is uses which nvidia-smi to detect of CUDA is installed. That doesn't show up on CUDA 6.5 on OS X 10.10, but which nvcc does. I'll try to put together a pull request to this effect soon. In any case, just manually run the three CUDA library install lines after everything finishes.

export currdir=$HOME/torch-distro; export PREFIX=${currdir}/install
cd ${currdir}/extra/cutorch && $PREFIX/bin/luarocks make rocks/cutorch-scm-1.rockspec
cd ${currdir}/extra/cunn && $PREFIX/bin/luarocks make rocks/cunn-scm-1.rockspec
cd ${currdir}/extra/cudnn && $PREFIX/bin/luarocks make cudnn-scm-1.rockspec

I don't have the CUDA so file for cudnn, so I haven't tested that, but everything else at least imports. I will run actual code soon.

[soumith]

thanks a lot @alexbw for making this work on OSX 10.10, will look forward to the PR

[szagoruyko]

just to note, torch-distro installs without modifications on my OS X 10.10

[alexbw]

Yeah, I think the tricks I described are how to get it to install on a
modified system, with a non-standard installed, which I had to turn off,
and a special Python library, which I had to take off the path.
On Mon, Jan 12, 2015 at 3:46 AM Sergey Zagoruyko notifications@github.com
wrote:

just to note, torch-distro installs without modifications on my OS X 10.10

—
Reply to this email directly or view it on GitHub
#4 (comment).

[alexbw]

Question — 
At Whetlab we're beginning to make quite a lot of use of Lua internally, so I've forked this repo so that we can have an easy-to-install Lua distribution. I'll be fleshing it out with a lot more packages, almost all of which you've written, or were written at Facebook. Would you like a separate PR that just contains the OS X 10.10 fixes, or do you want all of the improvements I'll have finished in the next week?

Just FYI, I'm looking to blend the style of continuum.io's Anaconda installer with this, to make it really easy to get a torch install up and running in a half hour, as opposed to a half week.

[soumith]

@alexbw that is awesome. I would love the 10.10 fixes if it happens in the short-term (today hopefully), as we're going to see an influx of users today and tomorrow because of some press coverage.

I'm excited that you're using it at Whetlab, an Anaconda style installer would be out of the world in terms of usability