helpers.c cannot be compiled on Mac OSX

[randy3k]

I successfully installed pqR (with --disable-helper-threads ) on Mac

In test.r, two random 1000x1000 matrices are multiplied together 10 times. The improvement is encouraging.

However, when I tried to install it with the helper threads, the follows happened. I have no idea what it is. Hope this piece of information will help you.

make[4]: `liblzma.a' is up to date.
gcc -std=gnu99 -I. -I./../helpers -I. -I../../../src/include -I../../../src/include -I/usr/local/include -DHAVE_CONFIG_H   -fopenmp -fPIC  -g -O2   -c piped-matprod.c -o piped-matprod.o
rm -f libmatprod.a
ar cr libmatprod.a matprod.o piped-matprod.o
ranlib libmatprod.a
gcc -std=gnu99 -I. -I. -I../../../src/include -I../../../src/include -I/usr/local/include -DHAVE_CONFIG_H   -fopenmp -fPIC  -g -O2   -c helpers.c -o helpers.o
helpers.c:313: sorry, unimplemented: threadprivate variables not supported in this target
make[4]: *** [helpers.o] Error 1
make[3]: *** [R] Error 2
make[2]: *** [make.helpers] Error 2
make[1]: *** [R] Error 1
make: *** [R] Error 1

[radfordneal]

Thanks for trying it out on a Mac. It's encouraging that it works without helper threads. The speed up on the matrix multiply test is a result of pqR using C routines that are a bit more sophisticated than the BLAS supplied with R-2.15.0. I'll be writing a blog post on matrix multiplies in pqR soon.

The problem when helper threads are enabled appears to be that openMP is not fully implemented with your toolset on the Mac, which surprises me. A bit of a search revealed a thread at http://openmp.org/forum/viewtopic.php?f=3&t=698 about this. It seems that there are solutions, but not having gotten into Mac's development environment yet, I can't say much more.

[randy3k]

Thanks. The thread solves my problem! The Apple gcc compiler doesn't work but GNU gcc compiler does!

[radfordneal]

Great! I guess the new compiler must also generate better code, since the test has speeded up!

You can try something like the following to see if the helper threads are actually working:

A <- matrix(1.1,1000,1000)
B <- matrix(2.2,1000,1000)
system.time({ X <- A %*% B + B %*% A; print(X[1000,1000]) })
system.time({ X <- A %*% B + B %*% A; print(X[1000,1000]) })

You can compare results with and without a --helpers=1 argument to the R command. You should see a "Using 1 helper threads" line printed in the startup message with the --helpers=1 argument. The first system.time may be different from later times, both because memory allocation may not have settled down, and because of a somewhat strange feature of pqR (which I may eliminate so I won't feel a need to explain it...).

Don't try it without the "print", since then pqR will see no need to wait for anything to actually finish before the system.time is done. (Maybe I should change system.time to wait...?)

[randy3k]

I compared the R, pqR and pqR -- helpers==1 with the below example.

The time elapsed are 18.823,12.836 and 9.928 respectively.

f = function(n){
    for(i in 1:n){
    A = matrix(rnorm(1000^2), 1000,1000)
    B = matrix(rnorm(1000^2), 1000,1000)
    list(A%*%B,B%*%A)
    }
}
system.time(f(10))

Very promising!

[sritchie73]

@randy3k What did you have to do to get it to compile on Mac OSX? I'm also interested in trying it out

[randy3k]

I am also able to compile the R GUI for the non-helper distribution.

But for the helper distribution,
I got a runtime error. Same problem? #2

[randy3k]

For the non-helper distribution

1. Get any gcc compiler, either from Xcode or GNU.
2. follow http://radfordneal.github.io/pqR/R-admin.html#Installing-R-under-Unix_002dalikes

./configure --disable-helper-threads

For the helper distribution

1. First get a GNU gcc compiler.
2. follow http://radfordneal.github.io/pqR/R-admin.html#Installing-R-under-Unix_002dalikes

./configure CC=/path/to/GNU/gcc

[sritchie73]

What version of gcc did you use? I downloaded gcc-4.9 through homebrew and got the following error:

arima.c: In function 'getQ0':
arima.c:819:6: internal compiler error: Segmentation fault: 11
SEXP getQ0(SEXP sPhi, SEXP sTheta)
^

arima.c:819:6: internal compiler error: Abort trap: 6

[randy3k]

i am using gcc4.8, also from homebrew.

rmbp:~ Randy$ brew list gcc48
/usr/local/Cellar/gcc48/4.8.1/bin/x86_64-apple-darwin12.4.0-gcc-ranlib-4.8
/usr/local/Cellar/gcc48/4.8.1/bin/x86_64-apple-darwin12.4.0-gcc-nm-4.8
/usr/local/Cellar/gcc48/4.8.1/bin/x86_64-apple-darwin12.4.0-gcc-ar-4.8
/usr/local/Cellar/gcc48/4.8.1/bin/x86_64-apple-darwin12.4.0-gcc-4.8.1
/usr/local/Cellar/gcc48/4.8.1/bin/x86_64-apple-darwin12.4.0-gcc-4.8
/usr/local/Cellar/gcc48/4.8.1/bin/x86_64-apple-darwin12.4.0-g++-4.8
/usr/local/Cellar/gcc48/4.8.1/bin/x86_64-apple-darwin12.4.0-c++-4.8
/usr/local/Cellar/gcc48/4.8.1/bin/gcov-4.8
/usr/local/Cellar/gcc48/4.8.1/bin/gcc-ranlib-4.8
/usr/local/Cellar/gcc48/4.8.1/bin/gcc-nm-4.8
/usr/local/Cellar/gcc48/4.8.1/bin/gcc-ar-4.8
/usr/local/Cellar/gcc48/4.8.1/bin/gcc-4.8
/usr/local/Cellar/gcc48/4.8.1/bin/g++-4.8
/usr/local/Cellar/gcc48/4.8.1/bin/cpp-4.8
/usr/local/Cellar/gcc48/4.8.1/bin/c++-4.8
/usr/local/Cellar/gcc48/4.8.1/gcc/include/ (669 files)
/usr/local/Cellar/gcc48/4.8.1/gcc/lib/ (268 files)
/usr/local/Cellar/gcc48/4.8.1/gcc/libexec/ (10 files)
/usr/local/Cellar/gcc48/4.8.1/share/gcc-4.8.1/ (3 files)
/usr/local/Cellar/gcc48/4.8.1/share/man/ (4 files)

In particularly, I used
./configure CC=/usr/local/Cellar/gcc48/4.8.1/bin/gcc-4.8

[sritchie73]

Confirming that I've successfully been able to install using gcc-4.4 and make check comes up fine. Speed tests are as follows for the function above:

>pqR - no helpers:
   user  system elapsed 
 13.421   0.102  13.523 

> pqR - 1 helper
   user  system elapsed 
 18.758   0.092  10.126 

> R 3.0.1
   user  system elapsed 
 85.334   0.178  85.618