Trouble installing on Mac 10.9.5

[dloyer]

Installed latest cuda, have java 7.

Installed the tarball per instructions from the download site.

If I run the scripts/getdata.sh

dougs-mbp:BIDMach_0.9.5-osx-x86_64 dloyer$ ./scripts/getdata.sh
./scripts/getdata.sh: line 20: /Users/dloyer/Downloads/BIDMach_0.9.5-osx-x86_64/getrcv1.sh: No such file or directory
./scripts/getdata.sh: line 22: /Users/dloyer/Downloads/BIDMach_0.9.5-osx-x86_64/getuci.sh: No such file or directory
./scripts/getdata.sh: line 24: /Users/dloyer/Downloads/BIDMach_0.9.5-osx-x86_64/getuci.sh: No such file or directory
./scripts/getdata.sh: line 28: /Users/dloyer/Downloads/BIDMach_0.9.5-osx-x86_64/getdigits.sh: No such file or directory

If I cd to scripts and run getrcv1.sh, I get further, but get a different error message...

....
Scanning lyrl2004_tokens_train.dat.gz
171542 lines
Writing Dictionary
2606875 lines processed
/var/folders/3l/s60hgztj5_zc_chmj8hvy4gm0000gn/T/scalacmd2525326315882625671.scala:1: error: not found: value BIDMat
import BIDMat.{CMat,CSMat,DMat,Dict,IDict,Image,FMat,FND,GMat,GIMat,GSMat,HMat,IMat,Mat,SMat,SBMat,SDMat}
^
/var/folders/3l/s60hgztj5_zc_chmj8hvy4gm0000gn/T/scalacmd2525326315882625671.scala:2: error: not found: value BIDMat
import BIDMat.MatFunctions._
^
....

[dloyer]

The 0.9.7 version (just posted) gets much further. The file not found errors are fixed.

However, the getdata.sh script still fails further down.

[jcanny]

Could you elaborate? Its working on Windows and Linux.

If the getdata scripts fails it may leave some junk lying around that breaks the script later on. Try clearing out BIDMach/data/rcv1/* and BIDMach/data/uci/* etc.

[dloyer]

I purged the install and did a fresh untar. I found several errors. I tried to work around them, but I ran it several different ones. The first place it gets stuck is..

Scanning lyrl2004_tokens_train.dat.gz
171542 lines
Writing Dictionary
2606875 lines processed
1 CUDA device found, CUDA version 6.5
Preprocessing
Making Sparse Data Matrix
Making Category Matrix
Making Sparse Test Data Matrix
Making Test Category Matrix
Loading nips data
Uncompressing docword.nips.txt.gz
746316 lines processed

No progress after a long time. Tried running a few times.

Also... the script getmnist.sh has this error:

( 232, 0) 254
... ... ...

c2: BIDMat.IMat = 7,2,1,0,4,1,4,9,5,9,0,6,9,0,1,5,9,7,3,4,9,6,6,5,4,0,7,4,0,1,3,1,3,4,7,2,7,1,2,1,1,7,4,2,3,5,1,2,4,4,6,3,5,5,6,0,4,1,9,5,7,8,9,3,7,4,6,4,3,0,7,0,2,9,1,7,3,2,9,7,7,6,2,7,8,4,7,3,6,1,3,6,9,3,1,4,1,7,6,9,6,0,5,4,9,9,2,1,9,4,8,7,3,9,7,4,4,4,9,2,5,4,7,6,7,9,0,5,8,5,6,6,5,7,8,1,0,1,6,4,6,7,3,1,7,1,8,2,0,2,9,9,5,5,1,5,6,0,3,4,4,6,5,4,6,5,4,5,1,4,4,7,2,3,2,7,1,8,1,8,1,8,5,0,8,9,2,5,0,1,1,1,0,9,0,3,1,6,4,2,3,6,1,1,1,3,9,5,2,9,4,5,9,3,9,0,3,6,5,5,7,2,2,7,1,2,8,4,1,7,3,3,8,8,7,9,2,2,4,1,5,9,8,7,2,3,0,4,4,2,4,1,9,5,7,7,2,8,2,6,8,5,7,7,9,1,8,1,8,0,3,0,1,9,9,4,1,8,2,1,2,9,7,5,9,2,6,4,1,5,8,2,9,2,0...:27: error: not found: value exit
exit
^

The script getuci.sh has the following errors:

Loading data
--2014-11-20 10:56:01-- https://archive.ics.uci.edu/ml/machine-learning-databases/bag-of-words/docword..txt.gz
Resolving archive.ics.uci.edu... 128.195.1.87
Connecting to archive.ics.uci.edu|128.195.1.87|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2014-11-20 10:56:02 ERROR 404: Not Found.

--2014-11-20 10:56:02-- https://archive.ics.uci.edu/ml/machine-learning-databases/bag-of-words/vocab..txt
Resolving archive.ics.uci.edu... 128.195.1.87
Connecting to archive.ics.uci.edu|128.195.1.87|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2014-11-20 10:56:02 ERROR 404: Not Found.

Additionally... I tried to install this on a windows 7 box. I installed the cuda dependency, but running bidmach has an error that it is not able to find jcuda. Looks like some kind of path problem. I tried with both cygwin (64bit) and windows shell.

This looks like a very cool tool. The FM and GPU support is very interesting. However, there are lots of these little install/path/dependency problems.

I was able to run some of the tutorial examples on the mac. It found the gpu, but seemed to leak memory on the gpu. One of the examples had no learning and had a flat roc curve. It is running on the latest cuda driver (6.5)

I was able to launch the ami, but havent done much with it yet. That might be the most stable environment for prototyping/evaluation.

[jcanny]

We just migrated to Scala 2.11. This seems to have broken the scripts in several places. In particular "exit" no longer exists in Scala 2.11 the "-e" command now requires the -cp option to use external jars. This was actually fixed several days ago (doing a git pull will get the updates) but not yet into the distributions. i.e. the errors you reported above were fixed already. I just pushed all the changes to the 0.9.7 distribution.

You'll see from the issues list that apart from the Scala 2.11 migration, the system has been quite stable over time. We normally fix issues within a couple of days.

I just tried the Windows version from cygwin and its working fine. Usually with windows installs, its a path problem. Are the CUDA dlls in your %PATH%? We havent used the windows .cmd in a long time and I'm sure it needs updating.

We have never heard of a memory leak problem on Macs. If you can document it, it will get fixed.

[dloyer]

Thanks for following up. Just to be clear, I am using the 0.9.7 distribution. I started with the 0.9.5 and saw that there was a new version later the same day. The problems listed above still manifest in 0.9.7

I can check the path on the windows box when I get home.

[jcanny]

I'm going to leave 0.9.5 since we've buried it in the previous release page. The 0.9.7 distributions have been updated (today) with the fixes above.

[dloyer]

Better.

With a fresh install, using yesterdays 0.9.7 distribution, it runs for awhile, then gets stuck at...

2014-11-20 22:22:32 (350 KB/s) - 'docword.nips.txt.gz' saved [2288169/2288169]

--2014-11-20 22:22:32-- https://archive.ics.uci.edu/ml/machine-learning-databases/bag-of-words/vocab.nips.txt
Resolving archive.ics.uci.edu... 128.195.1.87
Connecting to archive.ics.uci.edu|128.195.1.87|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 100807 (98K) [text/plain]
Saving to: 'vocab.nips.txt'

100%[===================================================================================================================================================================================================>] 100,807 277KB/s in 0.4s

2014-11-20 22:22:33 (277 KB/s) - 'vocab.nips.txt' saved [100807/100807]

Uncompressing docword.nips.txt.gz
746316 lines processed

It gets stuck here for several hours with no progress. The cpu is at 100%. This is after purging the old install and unzipping from a fresh download.

I ran some of the other download scripts. Some seem to leave you stuck in scala rather than exit back to the shell.

[jcanny]

Well, that bug was well and truly fixed a while ago, but the tar.gz file for Macs was not correctly uploaded. I'm guessing you never actually got the update to unpack. Sorry we dont test regularly with Macs.
Try it now. Inside /scripts/getuci.sh you should see the line:
wget https://archive.ics.uci.edu/ml/machine-learning-databases/bag-of-words/vocab.${1}.txt

I checked and that's what in the current version. The old version used to point to
wget https://archive.ics.uci.edu/ml/machine-learning-databases/bag-of-words/vocab.${1}.txt.gz

which doesnt exist, so wget just sits there. Once you get the current version the other problems should go away. They were caused by, and fixed after the 2.11 update.

[jcanny]

PS, you can also do a git pull to get the latest fixes after you clone the source distribution. These are script issues and dont require a recompile.