New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
NTL 9.8.1 does not build on OS X versions 10.8 and 10.9, nor CentOS 6.8 #20779
Comments
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Changed keywords from none to MacOS AVX no such instruction |
comment:3
Is this something we should report upstream? (And is the work-around using clang's assembler instead possible on all affected MacOS X systems? In fact, it's better than dropping |
comment:4
We could ask Victor for his opinion, but it probably works out of the box on an OS X system. The
But I would go for adding the |
comment:5
As mentioned on #20563, we could also configure Sage's GCC (which is built and used on MacOS X in any case?) to use clang's assembler on MacOS X by default. Not using AVX (when building from source) on capable CPUs is perhaps too much punishment, or do you think they get what they deserve? |
comment:6
Replying to @nexttime:
That would actually be the best solution in my opinion and would mean no changes are needed in other packages. ntl and SnapPy are probably just the tip of the iceberg (linbox and co will want to do that kind of stuff too once we finish their upgrade ticket). Stuff will start to break. If a single fix in gcc can save us trouble in the future we should go for it. |
comment:7
I agree that we shouldn't use clang's assembler just for NTL --- I compiled |
comment:8
Well, only one way to do that, let's try it. How do we do that. |
comment:9
Replying to @kiwifb:
https://gcc.gnu.org/install/configure.html#with-as ? An even easier way would be to install a wrapper script |
comment:10
I am not in favor of the wrapper unless unless we cannot do otherwise. I am more concerned about whether clang needs options to tell it to act as an assembler. |
comment:11
Replying to @kiwifb:
Well, more precisely, it's not clang's assembler, but LLVM's, hence presumably |
comment:12
I don't seem to have |
comment:13
We may have to go the wrapper route unfortunately
I am not sure what configure expect, |
comment:14
Replying to @NathanDunfield:
Sure? It's usually not in
Hmmm, the |
comment:15
Well same results from gcc's stage 1 configure script with or without |
comment:16
clang -print-prog-name=llvm-as perhaps? (Dumps the full pathname at least in case it is non-standard; maybe That's a GCC(-compatible) option btw. |
comment:17
Hum...
And
so the |
comment:18
Replying to @kiwifb:
What version of OS X is this? On 10.8 and 10.9, the
I think Apple switched to a clang-based |
comment:19
Replying to @NathanDunfield:
So, in fact, Sage has been using clang's |
comment:20
It is 10.11.5.
|
comment:21
W.r.t. script solutions: Putting one of the following into #!/bin/sh
exec /usr/bin/as -q "$@" #!/bin/sh
exec `clang -print-prog-name=as` "$@" I'm not sure whether the first at all works if |
comment:22
P.S.: At least both versions avoid using |
comment:23
Testing stuff on 10.11.5.... |
comment:24
It's taking more time to do it right than I had expected. Especially when you realize you messed up after a full build on this little laptop... |
comment:25
Testing on 10.9... |
comment:28
Yes, it gives
which is again the gcc
No, or at least not as far as I can tell. In particular, there no On 10.10 and newer, the version of I realized my replacement script doesn't work either --- I guess we have do something with the |
comment:29
We could check whether |
comment:30
P.S.: I didn't know it is at all possible to install the command line tools without installing Xcode. Does Sage build without having Xcode installed? |
comment:31
Replying to @nexttime:
I'll try
On 10.9
This has been possible for a while. On clean 10.9 machine if you type "clang" in Terminal.app, then a dialog pops up to ask if you want to install the command line tools, though there is also a button to install the whole of Xcode.
I'm about to find out ;-). It should, I think. At the very least, it can build everything up to NTL. |
comment:32
FWIW, on Linux at least, |
comment:33
Ok, with Sage 7.3.beta5, which includes this version of NTL, the following worked to build Sage on 10.9:
I'm running the doctests now, and will report back when those are done. It looks good so far, though. |
comment:34
Replying to @NathanDunfield:
Yep, we could create such a script in Sage's I still don't know whether Not sure whether we have to require Apple's "command line tools" and Xcode then (at least for builds from source and "developing" with bdists). All of this is IMHO still orthogonal to fixing NTL's |
comment:35
P.S.: Where "developing with bdists" includes installing optional Sage packages that involves compilation of C/C++ files, cf. #20563. |
comment:36
I checked that on 10.10 and 10.11 that
Definitely just the command-line tools --- I didn't have Xcode installed on the 10.9 system referred to above.
I think it's reasonable on NTL's part to assume that if the compiler accepts the option then it generates working code --- I would argue that the issue here on OS X is a mismatch or if you prefer a misconfiguration between the compiler and the assembler. I haven't tested yet, but I think |
comment:37
Replying to @NathanDunfield:
:-) It does generate "working" code in the sense that the assembly output is valid (even in the case of #20563 btw.). But the test should check whether the whole toolchain works, and -- unless cross-compiling, where Besides hardware bugs, there've also frequently been issues with VMs. (And note that for example AVX requires operating system support, not just an AVX-capable CPU.)
The (newer) LLVM assembler certainly complies with Intel's AVX specification... ;-) |
comment:38
Replying to @NathanDunfield:
All tests pass! I did skip the long ones, will run those now. Tomorrow, I will test on 10.11 as well. |
comment:39
I actually already done so on 10.11. build succeeded and sage starts. Haven't run doctests though but I would think it is all ok. |
comment:40
Replying to @NathanDunfield:
There was one failure in the long doctests, but probably presumably not related to the change at hand:
So assuming we want to add this |
comment:41
This now hits us on CentOS (6.8, with apparently ancient binutils) as well:
(From sage-devel.) |
comment:42
Replying to @nexttime:
Because this now hits us on older Linux distros as well (where Sage's GCC gets built), I think we should fix the NTL package (or any package which tries to use |
comment:43
Replying to @nexttime:
I think it would be fine to use the current ticket to fix NTL on both Linux and OS X, assuming it's essentially the same fix on both platforms. Messing with clang's asm on OS X can go under #20563. |
comment:44
Replying to @NathanDunfield:
From the ticket's title, yes, but the comments here mainly deal with |
comment:45
Replying to @nexttime:
Ok, that makes sense. If you put me in the "cc" for the new ticket I can test on OS X. |
comment:46
Hmmm, still haven't pushed a branch to #21064 (still playing a little with improvements)... I wonder what Does it print its own version, or does it invoke LLVM's (At least older versions of LLVM's assembler on Linux, probably not on MacOS X, don't know/accept And does |
comment:47
It's LLVM's
Yes, it does. The
|
comment:48
P.S.: A funny interpretation of |
comment:49
FYI (in case you don't already know it): This nice list maps Xcode versions to (Apple) clang versions at least. We could use it to get the Xcode version from (I haven't found any information on the assembler versions / when Apple's GAS vanished, though.) According to that, Travis' friend (with |
comment:50
Replying to @nexttime:
I guess it was removed when switching from Xcode 6.x to 7.x; if that's true, the
Xcode versions 7.0 - 7.2.1 require at least Yosemite, Xcode 7.3 and later El Capitan. It seems the Xcode and "Apple LLVM" major version numbers are (meanwhile) derived from the minor version of upstream LLVM (with some fuzz), while the versions in "clang-x.y.z" are more weird (x = 100 * Xcode major version + n with a "random" n < 10, y being a small minor version number and z being the patchlevel). |
comment:51
Replying to @nexttime:
According to this post on sage-support, my guess was a bit wrong. Apple's GAS was removed / LLVM's assembler made the default in (still presumably) Xcode 7.3, not 7.0 or 7.1 (but perhaps already 7.2, which in contrast to 7.3 is available for MacOS X 10.10). |
comment:52
I've opened #21116 for improvements to Sage's top-level For binary distributions, we'd probably have to include such a script as well, at least when they're used also on older versions of MacOS X than the buildbot has, or when the Xcode version (if any) on the user's machine is less recent than that with which the binary got built (if the user wants to install packages requiring compilation, or probably rebuild parts of Sage). |
comment:53
Sage now ships NTL 10.3.0. Are there any outstanding issues from this ticket? Or was it solved, maybe by #21064? |
The problem is that the flag "-march=native" is now given to the compiler, causing GCC to generate assembly code that is not accepted by the ancient version of "as" that ships with 10.8 and 10.9. See
https://groups.google.com/forum/#!topic/sage-devel/FloU3fAgQLM
for a discussion and some potential solutions. Currently, it seems unknown if this NTL builds on 10.10 but it is known to do so on 10.11.
From the above mentioned thread:
If you pass
gcc
the flag-Wa,-q
, then it will use clang's assembler rather than the ancient version of the GNU assembler that comes with these versions of OS X. Seehttp://stackoverflow.com/questions/9840207
''I can confirm that adding this flag allowed me to compile all of NTL on these machines. Specifically, changing line 78 of
build/pkgs/ntl/spkg-install
tocaused NTL to build smoothly.
Nathan
CC: @kcrisman @unhyperbolic @kiwifb
Component: algebra
Keywords: MacOS AVX no such instruction
Issue created by migration from https://trac.sagemath.org/ticket/20779
The text was updated successfully, but these errors were encountered: