Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

work around OpenBLAS inability to correctly detect CPU #13842

Closed
StefanKarpinski opened this issue Oct 31, 2015 · 27 comments
Closed

work around OpenBLAS inability to correctly detect CPU #13842

StefanKarpinski opened this issue Oct 31, 2015 · 27 comments
Labels
domain:building Build system, or building Julia or its dependencies

Comments

@StefanKarpinski
Copy link
Sponsor Member

It's gotten to the point where every time I build Julia from scratch, building OpenBLAS fails. For a while, I had hoped that OpenBLAS's hardware detection might get better, but this has been going on for a long time now. Can we work around this somehow so that people building Julia from source have a better experience?

@yuyichao
Copy link
Contributor

Which CPU do you have?

@tkelman
Copy link
Contributor

tkelman commented Oct 31, 2015

What exactly are the error messages. Unless you have a brand new skylake and haven't built 0.2.15 since we started using it, this might be something different. The only thing we can do differently about this is re-implement openblas with llvm calls.

@Keno
Copy link
Member

Keno commented Nov 1, 2015

I just had an OpenBLAS error as well that looked like an architecture misdetect, but cleaning and rebuilding fixed it, so I assume it was just some stale state from getting an updated version. @StefanKarpinski did this happen just after updating OpenBLAS?

@tkelman
Copy link
Contributor

tkelman commented Nov 1, 2015

We even give an exact error message here telling you to do make -C deps clean-openblas.

@StefanKarpinski
Copy link
Sponsor Member Author

This happens on fresh clones. The error message is excellent but not having the error in the first place is better. You used to reliably clone Julia and run make and get a working system. Have we given up on that ideal?

@tkelman
Copy link
Contributor

tkelman commented Nov 2, 2015

cat /proc/cpuinfo and more detailed information please, I've rarely seen issues on fresh clones.

@StefanKarpinski
Copy link
Sponsor Member Author

This now happens on multiple different systems: iMac from 2011 and my MacBook from this year. Will post cpu details when I'm back home at the end of the week. The newer machine is somewhat understandable but the older machine used to work so this is regression.

@tkelman
Copy link
Contributor

tkelman commented Nov 2, 2015

Have you upgraded software or toolchain recently? Sounds like #13836 or a similar local misconfiguration, otherwise this would be reported more often. Need full logs of output to say anything conclusive.

@kshyatt kshyatt added the domain:building Build system, or building Julia or its dependencies label Nov 2, 2015
@accopeland
Copy link

I'm seeing the same issue consistently of late on both on OSX 10.8 (MacBookPro10,2 ; Intel Core i7) and a Linux server (Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz) when following the build instructions here https://github.com/JuliaLang/julia with a fresh clone.

Want the make logs as an attachment or is a gist preferable?

@tkelman
Copy link
Contributor

tkelman commented Nov 18, 2015

gist please. much easier to look through when on mobile, can stay in browser

@accopeland
Copy link

Here's the gist: https://gist.github.com/d607ada268ddba9e696e

I had to run make a number of times to get to the openblas error. The gist contains the full output of the various make commands. The system details are

Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
Linux 2.6.32-431.46.2.el6.nersc.x86_64 #1 SMP  x86_64 GNU/Linux

> git --version
git version 1.7.2.5

> gcc --version
gcc (GCC) 4.6.3

@yuyichao
Copy link
Contributor

Looks like #7653 ? binutils too old?

@yuyichao
Copy link
Contributor

@accopeland
Copy link

Yes, a workaround is documented, thanks. This issue has to do with avoiding the need for the workaround . To quote from the initial report: "... this has been going on for a long time now. Can we work around this somehow so that people building Julia from source have a better experience?"

@nalimilan
Copy link
Member

Actually we're supposed to require gcc 4.7 to build Julia itself. At least that's what README.md says, and while it works without it for now, it should stop soon when porting to a newer LLVM.

@tkelman
Copy link
Contributor

tkelman commented Nov 18, 2015

We can implement #9372.

Note that when we upgrade LLVM it will no longer be possible to build Julia with gcc older than 4.7 anyway. If you're on NERSC, can't you get a newer toolchain via some module load PrgEnv-gnu invocation?

@accopeland
Copy link

Sure, I can easily try another compile from a clean install using gcc-4.7,
gcc-4.8 or gcc-4.9 if you think this will address the openblas issue. I'll
follow up with what I find.

On Wed, Nov 18, 2015 at 1:52 AM, Tony Kelman notifications@github.com
wrote:

We can implement #9372 #9372.

Note that when we upgrade LLVM it will no longer be possible to build
Julia with gcc older than 4.7 anyway. If you're on NERSC, can't you get a
newer toolchain via some module load prgenv gnu invocation?


Reply to this email directly or view it on GitHub
#13842 (comment).

@tkelman
Copy link
Contributor

tkelman commented Nov 18, 2015

Technically it's the binutils version that should matter here. So check that first, via as --version or similar

@accopeland
Copy link

Good that you mentioned this. The NERSC binutils module is old (GNU
assembler (GNU Binutils for Debian) 2.20.1-system.20100303) and the module
for a more recent version is broken I've just discovered. I'll get that
fixed then try again.

On Wed, Nov 18, 2015 at 10:32 AM, Tony Kelman notifications@github.com
wrote:

Technically it's the binutils version that should matter here. So check
that first, via as --version or similar


Reply to this email directly or view it on GitHub
#13842 (comment).

@accopeland
Copy link

No openblas error this time with (new) binutils ((GNU Binutils) 2.25) and gcc-4.8.1 and a fresh clone.
For the record, the full make log is here: https://gist.github.com/accopeland/88065beecd50b73274b6

As an aside, and totally unrelated I think, I saw quite a few other failures which were "fixable" by simply rerunning make:

make[1]: *** [/scratch/copeland/julia/deps/srccache/libuv/configure] Error 129
make[1]: *** [build/openlibm/Makefile] Error 128
make[1]: *** [build/openspecfun/Makefile] Error 129
make[1]: *** [build/openblas/Makefile] Error 128
make[1]: *** [/scratch/copeland/julia/deps/srccache/libgit2/CMakeLists.txt] Error 128
make[1]: *** [build/utf8proc/Makefile] Error 128

and all due to variations on this

warning: Remote branch v0.2.15 not found in upstream origin, using HEAD instead
fatal: Couldn't find remote ref v0.2.15
warning: Remote branch v0.2.15 not found in upstream origin, using HEAD instead
fatal: Couldn't find remote ref v0.2.15
fatal: Couldn't find remote ref remotes/origin/v0.2.15
fatal: The remote end hung up unexpectedly

@tkelman
Copy link
Contributor

tkelman commented Nov 19, 2015

Not positive, but those might be due to your old git version. We used to require 1.7.3 or newer for Pkg, but the makefiles might be using newer features now.

@StefanKarpinski you were OP here, can you update what was happening on your machine?

@accopeland
Copy link

I'll update git and redo to confirm.

On Thu, Nov 19, 2015 at 3:16 AM, Tony Kelman notifications@github.com
wrote:

Not positive, but those might be due to your old git version. We used to
require 1.7.3 or newer for Pkg, but the makefiles might be using newer
features now.

@StefanKarpinski https://github.com/StefanKarpinski you were OP here,
can you update what was happening on your machine?


Reply to this email directly or view it on GitHub
#13842 (comment).

@accopeland
Copy link

Thanks for the good advice. Using the following tool versions, I got a clean (though long) make with no openblas problems for release-0.4 on Xeon(R) CPU E5-2670 0 @ 2.60GHz (Sandy bridge) / Linux 2.6.32. Updating the Platform specific build notes @ https://github.com/JuliaLang/julia would be very helpful for newcomers.

git --version
git version 2.0.0
gcc --version
gcc (GCC) 4.8.1
as --version
GNU assembler (GNU Binutils) 2.25
git clone git://github.com/JuliaLang/julia.git
Cloning into 'julia'...
cd julia
git checkout release-0.4
Branch release-0.4 set up to track remote branch release-0.4 from origin.
Switched to a new branch 'release-0.4'
make &>> make.log

@tkelman
Copy link
Contributor

tkelman commented Nov 19, 2015

Thanks for the report. Would you like to take a crack at incorporating what you learned via a pull request?

@accopeland
Copy link

Sure, I'll give it a try.

On Thu, Nov 19, 2015 at 12:01 PM, Tony Kelman notifications@github.com
wrote:

Thanks for the report. Would you like to take a crack at incorporating
what you learned via a pull request?


Reply to this email directly or view it on GitHub
#13842 (comment).

@svaksha
Copy link

svaksha commented Apr 19, 2017

Since this bug is still open, here is another openblas error report on the dell latitude 7480 with an Intel Core i7-7600U CPU built on zesty (ubuntu 17.04) released last week, https://gist.github.com/svaksha/cc30047982e9b603e1a27d99eedcf703

I tried to build Julia with :

[1] the latest unstable version of Julia - commit ceccddf, and

[2] the last stable version of Julia - release-0.5

When both failed with the openblas error, I tried building it by running 'make -C deps clean-openblas', 'make OPENBLAS_USE_THREAD=0' and 'make OPENBLAS_TARGET_ARCH=NEHALEM'. No luck.

@ViralBShah
Copy link
Member

I believe this hasn't been seen for a while now. Please reopen if still an issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
domain:building Build system, or building Julia or its dependencies
Projects
None yet
Development

No branches or pull requests

9 participants