OS X v10.11.x 'El Capitan,' Xcode 8, and the Command Line Tools (Or, Rather, the Lack Thereof…) (Tracking)

[RandomDSdevel]

Please follow the general troubleshooting steps first:

• Ran brew update and retried your prior step? N/A (See below…)
• Ran brew doctor, fixed as many issues as possible and retried your prior step? N/A (See below…)
• If you're seeing permission errors tried running sudo chown -R $(whoami) $(brew --prefix)? N/A (See below…)

N. b.: The items in the previous checklist are not applicable to this issue because it describes a problem with Homebrew's installation environment.

Bug reports:

Well, it appears that Apple may have botched the deployment of Xcode 8's command-line tools for OS X v10.11.x 'El Capitan,' as the App Store has not yet presented me with an update notification to install them and they don't seem to be on Apple Developer Connection's manual 'Downloads for Apple Developers' page, either. I imagine that this might end up causing some problems for Homebrew's user base, so I filed this issue. I've also posted a question on the Apple Developer Forums. It appears that I'm not the only person who's noticed this deficiency, either, as I was also able to find another user's forum post earlier this afternoon.

N. b.: Accessing most of the ADC links and all of the Apple Developer Forum links I included in the paragraph above requires a valid free or paid Apple Developer account.

---

P. S.: Apparently, I'm a bit late to the party on this one! Some of the relevant issues and pull requests were actually filed before I posted this one! In any case, here's a list of the other relevant issues:

• Homebrew/brew:
  • Pull Request #721: 'Set -no_weak_imports linker flag for Xcode 8'
  • Pull Request #741: 'Add permit_weak_imports directive'
  • Pull Request #782: 'Make -no_weak_imports opt-in.'
  • Pull Request #965: 'Warn when using Xcode 8 without CLT on 10.11'
  • Issue #969: '"Empty archive" error when install formulas with tar.xz archives, after upgrading to Xcode 8'
  • Pull Request #970: 'superenv: help Autotools with 10.12 SDK on 10.11'
• Homebrew/homebrew-core:
  • Issue #1957: 'macOS 10.12 and Xcode 8.0 broken formulae'
  • Issue #2674: 'xz broken on OSX 10.11.6 Beta (15G24b) with XCode 8 Beta'
  • Issue #3727: 'Set -no_weak_imports linker flag for Xcode 8 on OS X 10.11'
  • Issue #4809: 'python3 failed to build on 10.11.6 with Xcode 8' (Specifically, this comment on that thread.)

[RandomDSdevel]

I also just found another report of this in the form of a post on the MacRumors forums.

[zmwangx]

I don't have time to type out details right now, but rest assured, we're well aware of the problems, and a huge chunk of those were solved by #970. You'll find more information if you look at the recent issues filed here or in homebrew-core.

[RandomDSdevel]

@zmwangx: OK, glad you guys know what's going on with respect to this. Hopefully Apple releases the 10.11 CLT package soon, though…

[RandomDSdevel]

@zmwangx: OK, I've started adding a list of the pull requests and issues that are relevant to this tracking issue. It's still incomplete, though, so bear with me here…

[zmwangx]

Generally it's just the xcode 8 tag. (I mean over at homebrew-core; there's no such tag here.)

[RandomDSdevel]

@zmwangx: Ah, heh, I should have guessed. That should save me some time following links around in a wild goose chase. I've got everything from doing that added to the list, though, so I thought it was safe to remove my notices that certain parts of that list were a work in progress. Since I forgot to look through all the things tagged 'Xcode 8,' I'll add that back now…; whoops! 😆

P. S.: OK, there we go: I just added the 'work in progress' notices back to the list!

[RandomDSdevel]

@zmwangx: All right, I think that's all the major related issues and pull requests (there are a few more, but I omitted them because they basically just link back to the listed issues and PRs,) so I removed the 'work in progress' markers from both sub-lists. Hopefully this issue ends up being helpful!

[RandomDSdevel]

OK, I just got a response from somebody who sounds like they know what they're talking about (hopefully they work at Apple) on the discussion thread I started on the Apple Developer Forums earlier. Keep watching that thread for updates if you can't find anything else that looks relevant to the situation here.

[mistydemeo]

Oh dear. Thanks for passing that on - I hope they act on that soon.

[zmwangx]

I don't think the sole response was from an Apple engineer. Listen to this:

It doesn't sound like this will be fixed in time for the Xcode 8.0 release.

(emphasis by me)

[draeath]

EDIT: I can't read, disregard. Sorry!

[zmwangx]

That's not for El Capitan.

[draeath]

I followed your instructions here and followed up with sudo xcode-select --install - this seems to have done the trick (on my end). I did have to do the brew unlink $(brew ls); brew link $(brew ls) bit as well. Finally I had a whole pile of unlinked kegs reported by brew doctor (which I don't recall seeing previously) - so linked all those. I did have some json-c library conflicts to resolve, simply because some third-party software also installed such libraries in /usr/local/lib. Mea culpa.

Anyway. Adding these notes here to help direct folks to the above fixwork, so they don't have to follow the same trail I did.

[zmwangx]

@draeath Um I think you're on the wrong issue.

[draeath]

How so? Issue title is "OS X v10.11.x 'El Capitan,' Xcode 8, and the Command Line Tools (Or, Rather, the Lack Thereof…)" and I just explained what path I had to follow to get them installed.

[zmwangx]

@draeath

I followed your instructions here

Those instructions are for #991 and has nothing to do with how to install CLT...

[draeath]

"and followed up with"

I had to do both. And my post with the full path I followed was for anyone else who came around trying to figure out how to fix things after installing xcode 8. They'd hit both problems just the same...

[zmwangx]

how to fix things after installing xcode 8.

Problems in #991 are not related to Xcode 8 update.

[draeath]

Oh, so the migration isn't related, just a coincidence in timing?

Is it possible to rename issues then? You might consider renaming #991 then, it reads as "Homebrew Cannot Update, Doctor, Unlink, etc. since Xcode update?" currently.

If it mislead me, it can mislead others.

[RandomDSdevel]

@zmwangx:

I don't think the sole response was from an Apple engineer. Listen to this:

It doesn't sound like this will be fixed in time for the Xcode 8.0 release.

(emphasis by me)

See my reply to that post on that thread. (EDIT: Never mind, as you're probably right that the 'jhowarth' guy who first replied to the discussion thread I started over on the Apple Developer Forums doesn't work for Apple, seeing as his account there is only 'Level 1.') Sigh. This is starting to get more than just annoying, though…

[draeath]

@RandomDSdevel it is fixable now, though, it's just another command you need to run (in addition to accepting the license) - in fact a current instance of 'brew' will tell you this with brew doctor. Just in case it's missed, though, you can install the CLI tools after installing xcode8 as such:

sudo xcode-select --install

This will pop up a small GUI that'll prompt you to download/install them. Just whack the button and come back in a few minutes.

I have successfully built stuff after doing this (fontconfig specifically, which involved it patching, a ./configure pass, and a make pass - vs just pouring a bottle).

[zmwangx]

you can install the CLI tools after installing xcode8 as such:

No, that installs CLT 7.3.1.0.1.1461711523. Check your brew config.

[zmwangx]

I have successfully built stuff after doing this (fontconfig specifically

As for fontconfig, pouring bottle is definitely fine. Problems only manifest when you build things from source, and we have fixed most of those too so you shouldn't notice.

[draeath]

Oh. OH. Well that's just hilarious... yep, "7.3.1.0.1.1461711523" here.

Well, thank you for all the fixes you've already put in there, since things seem to be working OK on my end. I haven't gone and built anything substantial, but the simpler stuff seems happy.

[zmwangx]

since things seem to be working OK on my end.

Good to know. We know a few outstanding issues, but hopefully you won't hit those (pouring bottles is almost universally safe).

[RandomDSdevel]

@draeath, @zmwangx:
From @draeath:

@RandomDSdevel it is fixable now, though, it's just another command you need to run (in addition to accepting the license) - in fact a current instance of 'brew' will tell you this with brew doctor. Just in case it's missed, though, you can install the CLI tools after installing xcode8 as such:

sudo xcode-select --install

This will pop up a small GUI that'll prompt you to download/install them. Just whack the button and come back in a few minutes.

I have successfully built stuff after doing this (fontconfig specifically, which involved it patching, a ./configure pass, and a make pass - vs just pouring a bottle).

From @zmwangx:

[From @draeath:]

you can install the CLI tools after installing xcode8 as such:

No, that installs CLT 7.3.1.0.1.1461711523. Check your brew config.

In this case, I'm not really comfortable with using Xcode v8.0 with the version of the Command Line Tools intended for use with Xcode v7.3.1 and am not exactly sure how this mismatch would affect command-line developer-tool usage, so I'm pretty much stuck without any development tools until Apple fixes this and, since I tend to --build-from-source, can't brew upgrade anything in the meantime. sudo xcode-select --install spews the following error at me, too, …:

xcode-select: error: command line tools are already installed, use "Software Update" to install updates

…so that, quite annoyingly, doesn't help me at all. Grrrrrrr…

[RandomDSdevel]

There's been a bit of action with respect to the reason I created this tracking issue on that Apple Developer Forum thread I started (along with some others,) so perhaps Apple will release a corrected version of the installer for the El Capitan Xcode 8 command-line tools relatively soon…? Anyway, I've got my fingers crossed here.

[RandomDSdevel]

And Apple has now officially acknowledged the existence of this issue by including the following note in Xcode v8.1's first beta's release notes under the 'Known Issues in Xcode 8.1 beta - Swift and Apple LLVM Compilers' heading's 'Command Line Tools' sub-heading:

• There is no Command Line Tools (OS X 10.11) for Xcode 8 package. Xcode 8 contains SDKs that are incompatible with earlier toolchains. Developers who want to make use of the Xcode 8 SDKs from the command line must choose the SDK with xcode-select. Developers on OS X El Capitan who have installed versions of the Command Line Tools (OS X 10.11) for Xcode 8 Beta should install Command Line Tools (OS X 10.11) for Xcode 7.3.1. (28234439)

• On OS X El Capitan, running Swift from the command line can fail if any Command Line Tools (OS X 10.11) package is installed on the system. (28234754)

  Workaround: Execute Swift through from the command line using xcrun, . Choose the macOS SDK using the -sdk. For example, the following command compiles the file main.swift:

  xcrun -sdk macosx swiftc main.swift

Should Homebrew have any special handling for this? What I get from this is that the version of the command-line tools installed inside the Xcode bundle were updated properly if you are on OS X v10.11.x 'El Capitan' and installed Xcode v8, so xcodebuild, xcrun, and friends still run properly even though the stand-alone command-line developer tools don't exist for Xcode v8. Whether this means we'll get a proper CLT package for El Capitan with Xcode v8.1 isn't so clear…

[fscofield]

Has anyone had success using homebrew after downgrading to xcode & xcode CLT 7.3.1 on (OS X 10.11)? I keep running into this issue when installing mysql with homebrew 1.0.3, Xcode 8.0 and CLT 7.3.1. As a result I tried downgrading Xcode to 7.3.1 and now get the following issue from homebrew:

Error: Your Xcode (7.3.1) is outdated
Please update to Xcode 8.0.
Xcode can be updated from the App Store.

Is there a way to bypass this error? Or is 8.0 imperative for running homebrew?

[ellio167]

I found that the cli 7.3.1 are incompatible with the xcode 8 SKD. In particular when I tried to do

brew install matplotlib --build-from-source

I got lots of syntax errors directly from one of the main system frameworks.

Once I downgraded back to xcode 7.3.1 all is fine...

[DomT4]

Is there a way to bypass this error? Or is 8.0 imperative for running homebrew?

For now at least, this will no longer happen on < Sierra.

[DomT4]

Or at least, it won't once the next tag is cut if you're a standard user. That will likely land fairly soon.

[RandomDSdevel]

@rscottm:

This worked for me:

sudo xcode-select -r

OK, but why, exactly…? Does installing the command-line tools suite set a custom developer tools path…?

---

@DomT4:

From @fscofield:

…

Is there a way to bypass this error? Or is 8.0 imperative for running home-brew?

For now at least, this will no longer happen on < Sierra.

Also from @DomT4:

Or at least, it won't once the next tag is cut if you're a standard user. That will likely land fairly soon.

Great, that's wonderful!

[RandomDSdevel]

Hmm, according to the following posts on the 'fink-users' mailing list, …:

• 'Re: [fink-users] fink fails to recognize updated CLT'
• 'Re: [fink-users] Fink on El Capitan'

…somebody at Apple confirmed that it's OK for end-users to use Xcode v8.0 with the Command Line Tools suite intended for use with Xcode v7.3.1. I couldn't find a source for this, though, so can somebody else confirm this?

[zmwangx]

Martin noticed a few days ago that according to Xcode 8.1 beta release notes,

There is no Command Line Tools (OS X 10.11) for Xcode 8 package. Xcode 8 contains SDKs that are incompatible with earlier toolchains. Developers who want to make use of the Xcode 8 SDKs from the command line must choose the SDK with xcode-select. Developers on OS X El Capitan who have installed versions of the Command Line Tools (OS X 10.11) for Xcode 8 Beta should install Command Line Tools (OS X 10.11) for Xcode 7.3.1.

[RandomDSdevel]

@zmwangx: Oh, yeah! I noticed that statement, too (see here for when I mentioned it,) but didn't really read it closely enough for that to register with my brain (😆!) Now that makes me wonder what SDKs xcode-select recognizes as installed under such a configuration…

[RandomDSdevel]

     To answer the question I myself posed in my next most recent reply to this thread, here's what I get from xcodebuild -showsdks:

iOS SDKs:
    iOS 10.0                        -sdk iphoneos10.0

iOS Simulator SDKs:
    Simulator - iOS 10.0            -sdk iphonesimulator10.0

macOS SDKs:
    macOS 10.12                     -sdk macosx10.12

tvOS SDKs:
    tvOS 10.0                       -sdk appletvos10.0

tvOS Simulator SDKs:
    Simulator - tvOS 10.0           -sdk appletvsimulator10.0

watchOS SDKs:
    watchOS 3.0                     -sdk watchos3.0

watchOS Simulator SDKs:
    Simulator - watchOS 3.0         -sdk watchsimulator3.0

As such, it appears that the only OS X 10.11 SDK I might have left on my machine could be the one hanging around in my installation of the version of the Xcode command-line tools suite originally intended for use only with version 7.3.1 of the IDE, but that's not showing up here because xcodebuild only knows about SDKs installed inside of the current version of Xcode installed on my machine, which would be v8.0, and not any SDKs installed outside of Xcode —i. e.: as part of a CLT suite. On that note, can I assume that we're all done here?

[MikeMcQuaid]

Closing this as most issues have now been addressed and the remaining ones can be tackled in more specific issues.

[Beshario]

Yea, i found the CLT at the manual Downloads for Apple Developers page

[RandomDSdevel]

@Beshario: If you're on El Cap, DON'T INSTALL THOSE COMMAND-LINE TOOLS, AS THEY AREN'T THE RIGHT ONES!!! The problem here is that the CLT suite was released for Xcode 8 users running macOS Sierra (v10.12.x,) but not for Xcode 8 users running OS X v10.11.x El Capitan. Given the discussions referenced in this thread and something we've spotted in the release notes for the Xcode v8.1 betas, this should finally really get fixed when that version of the IDE is released. In the meantime, Apple says it's perfectly fine to use Xcode 8 with the CLT suite that the company released for use with Xcode 7.3.1, and Homebrew has been changed so that it doesn't freak out under that configuration since it would usually scream at you that the versions of your copies of Xcode and the accompanying command-line tools don't match up, which might normally be a problem but isn't in this situation.

[artforlife]

I am experiencing this on Sierra + XCode 7.3.1. It is quite annoying. Is there a fix for this? How does one go about hacking through this?

[RandomDSdevel]

@artforlife: Um, I don't think you're supposed to use a version of Xcode prior to version 8.0 on macOS Sierra. Just download the Sierra CLT suite designated by Apple for use with the relevant/latest version of Xcode (that, unlike the El Capitan one, is available.)

[JustinTArthur]

@rscottm you ran sudo xcode-select -r from El Capitan after upgrading to Xcode 8.x? Didn't seem to do much for me. Had you manually downloaded the CLT 7.3.1 like the release notes suggested first?

[MikeMcQuaid]

Yes. You need to use Xcode 8 on Sierra or many things are broken.

[dreizehnelf]

Couldn't build my python libs after upgrading to Xcode 8.x on my OSX 10.11. Luckily I still had my old CLT 7.3.1 lying around at /Library/Developer/CommandLineTools, so I ran sudo xcode-select --switch /Library/Developer/CommandLineTools and things started working again.

[RandomDSdevel]

@dreizehnelf: If your Python libraries were built with Homebrew, then your issue sounds like a new one that you need to file either here in Homebrew/brew or over in Homebrew/homebrew-core. Using Xcode 8 with the Xcode 7.3.1 command-line tools suite on OS X v10.11.x 'El Capitan' is fully supported by Apple.

[RandomDSdevel]

@zmwangx, @mistydemeo, @DomT4, @MikeMcQuaid: Please also note that the changes you guys made to Homebrew to work around this issue will, as noted in Apple's beta release notes for the software¹, no longer be needed once Xcode v8.2 has been released.

¹: Note that Apple Developer Connection requires users to have an account there in order to read these release notes.

[dl111122]

The Mac App Store now has: Command Line Tools (macOS El Capitan version 10.11) for Xcode 8.2

[RandomDSdevel]

Yup! I've installed it, too. This issue has already been closed, though, @dl111122. Should this issue perhaps be locked to prevent further noise on it (at least directly in this issue rather than in new ones, though I doubt any more of those will arise any time soon unless Apple messes up like this again, which I hope they don't) now, @MikeMcQuaid, @zmwangx, and/or @mistydemeo?

[MikeMcQuaid]

Should this issue perhaps be locked to prevent further noise on it

Yes, done.