clang built julia is unstable

[ViralBShah]

Building with make USECLANG=1 doesn't pass all tests. For me, it gets stuck here - and is perhaps related to the ccall issue discussed in #978. Also, the tests seem to take much longer when julia is built with clang.

$ make testall
    PERL test/unicode/UTF-32BE.txt
    PERL test/unicode/UTF-32LE.txt
    PERL test/unicode/UTF-16BE.txt
    PERL test/unicode/UTF-16LE.txt
    PERL test/unicode/UTF-8.txt
    JULIA test/all
     * all
     * core
     * numbers
     * strings
     * unicode
     * corelib
     * hashing
     * arrayops
     * lapack
     * factorizations
     * fft
     * sparse
     * arpack
     * bitarray
     * random
     * special
     * functional
     * bigint
     * distributions
     * combinatorics
     * statistics
     * integers
     * glpk

[StefanKarpinski]

Same issue (also on latest OS X).

[vtjnash]

Very strange. The failing function appears to be stat().

sometime during the call to uv_fs_stat in jl_stat ret = uv_fs_stat(uv_default_loop(), &req, path, NULL); the stack can be seen to be destroyed

It's not clear why this would be a problem, but the allocation of uv_fs_t req seems to be generating broken code.
I tried switching to alloca, but that still failed. However, switching to malloc or making this a static pointer (or presumably, but untested, refactoring to pass around a uv_fs_t buffer instead of a struct stat) would seem to make this OK. The real question is why are these failing?

The answer lies in a few missing complier flags:
CFLAGS=" -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_DARWIN_USE_64_BIT_INODE=1 "
that are used by libuv but not by julia and alter the size required by the stat struct. I don't know what they mean, I just know that adding one or more of them will likely solve the issue.

(sorry if that reads a bit like a story, but I kept typing this as I kept finding new details)

[pao]

USE_64_BIT_INODE sounds very much like it would affect the size of the stat structure, which contains the inode number.

[ViralBShah]

I am trying out the suggested CFLAGS, but currently, make testall dies when julia is built with clang, and openblas is used.

$ make test-lapack
    JULIA test/lapack
     * lapack
assertion failed: norm(-(*(l,u),ref(a,p,:)))<Eps
 in load at util.jl:234
 in load at util.jl:246
 in runtests at ./runtests.jl:3
 in include at boot.jl:197
 in process_options at client.jl:172
 in _start at client.jl:214
at /Users/viral/julia-clang/test/lapack.jl:4
 in load at util.jl:257
 in runtests at ./runtests.jl:3
 in include at boot.jl:197
 in process_options at client.jl:172
 in _start at client.jl:214
at ./runtests.jl:48
 in include at boot.jl:197
 in process_options at client.jl:172
 in _start at client.jl:214
make[1]: *** [lapack] Error 1
make: *** [test-lapack] Error 2

[ViralBShah]

I should add that all openblas tests pass for me. I wonder if this is some clang/ccall related issue, or some other corruption.

[nolta]

@ViralBShah, are you running on sandybridge? If so, i'm seeing a similar error, and compiling openblas with TARGET=NEHALEM or USE_THREAD=0 seems to fix it.

OpenMathLib/OpenBLAS#125

[ViralBShah]

Yes, I am on sandybridge. Just seeing if openblas' sandybridge support works well enough, when built with Apple clang. I suspect this is going to take some time to stabilize.

[ViralBShah]

Apart from the openblas+lapack tests, which seems to be an openblas issue, all other tests pass, with the flags that @vtjnash mentions.

[nolta]

This is not just a problem with clang. The lapack test also fails on my sandybridge ubuntu 10.04 box, for both gcc 4.4.3 and 4.7.1.

[ViralBShah]

I am closing this bug and opening #1056 for the openblas issue.

[ViralBShah]

Does it also make julia tests fail? The LU decomposition is incorrect - so it is not a case of just some torture tests failing. I haven't tried with gcc yet.

-viral

On 15-Jul-2012, at 8:41 AM, Mike Nolta wrote:

This is not just a problem with clang. The lapack test also fails on my sandybridge ubuntu 10.04 box, for both gcc 4.4.3 and 4.7.1.

---

Reply to this email directly or view it on GitHub:
#1013 (comment)

[nolta]

Yes, i get the exact same error as you do. It's what prompted me to open the openblas issue.