Support for Xcode 4.3

[xose]

The upcoming Xcode 4.3 release will be distributed as a self-contained application bundle. It will no longer require an extra installation step for easy distribution and faster delta updates on the AppStore. This means that command line tools will NOT be installed on /usr/bin and SDKs/headers/related tools will not be on /Developer, so Homebrew should be updated with the new paths.

On a default installation of Xcode, the new location of the command line tools will be /Applications/Xcode.app/Contents/Developer/usr/bin. This includes the compilers, as well as git, svn and other development tools. Other references to /Developer on Homebrew and the formulae should also be updated.

There is already a xcode_prefix function on Homebrew/utils.rb that works great for finding the correct Developer folder on Xcode 4.3. It just needs to be used for the compiler paths, git and other tools, and the formulae that reference /Developer.

[camillol]

They could still symlink stuff when you launch Xcode... they don't even do that?

[Sharpie]

I don't think any of the core team has access to developer previews of XCode. So there is not much we can do until we get our hands on a copy of XCode.app to see what the new paths are.

[xose]

For previous versions of Xcode, this can be tested by deselecting the "UNIX Development" package during installation, or uninstalling it with sudo /Developer/Library/uninstall-devtools --mode=unixdev. If Homebrew works without requiring the UNIX tools to be installed, it will work fine on Xcode 4.3.

The correct path is already provided by xcode_prefix in utils.rb. The tools are located in {xcode_prefix}/usr/bin, no matter which version of Xcode is installed.

I tried to fix it myself, but I'm not familiar with the internals. The compiler paths are easy to update, as well as formulae that hardcode the path. But there are way too many calls to git with no path, and there may be other required changes that I'm not aware of.

They could still symlink stuff when you launch Xcode... they don't even do that?

The first time you launch Xcode it will request to install a couple packages, but not the UNIX tools.

[jacknagel]

there are way too many calls to git with no path

FYI, we don't hard code the path to git because we still support OS X / XCode combinations that did not ship with git.

[adamv]

Is there at least a link to a reference on these changes?

[ashgti]

I just downloaded Xcode 4.3.

It does say the following in the release notes:

This preview release does not include Unix development tools placed into /usr. To perform makefile- based and config-based builds, either install Xcode 4.2.1 or use xcrun to execute the tools provided within Xcode.

I do not know if this means the final release will not include the Unix development tools placed in /usr or not, but I'd be happy to test anything out that needs testing since I do have Xcode 4.3 developer preview 2 installed.

[Sharpie]

If Homebrew works without requiring the UNIX tools to be installed, it will work fine on Xcode 4.3.

Most tasks executed by Homebrew require the UNIX tools. Because of this, it will be hard to make progress without a copy of XCode 4.3 to test against.

[camillol]

I think we should simply ask users to add /Applications/Xcode.app/Contents/Developer/usr/bin to their PATH in .profile. Maybe provide a script to do it.

[jacknagel]

We can probably use xcrun in some capacity. Though it is really slow with a cold cache.

[xose]

Most tasks executed by Homebrew require the UNIX tools. Because of this, it will be hard to make progress without a copy of XCode 4.3 to test against.

UNIX tools have always been installed with Xcode in Developer. Until now there was an optional installation of a duplicate of the tools in /usr, but this is no longer the case on Xcode 4.3.

I don't know how will Apple handle the updates from 4.2, but the tools in /usr might be stuck with the current version and never updated again, so they shouldn't be relied upon. There is always a copy of the tools in Developer which is guaranteed to be the latest version.

I think we should simply ask users to add /Applications/Xcode.app/Contents/Developer/usr/bin to their PATH in .profile. Maybe provide a script to do it.

This will not fix anything, compilers and other tools still use absolute paths in Homebrew.

---

There is also a new Toolchains directory for pluggable toolchains (compiler, assembler, linker, ...). Clang and related tools have been moved there, LLVM and other UNIX tools remain on Developer.

Here is a file listing of the new Developer folder, as well as the Toolchains folder on DP2: https://gist.github.com/1502152

[MikeMcQuaid]

We should do the smart thing and auto detect the path when we have access to 4.3. This should not require any manual user intervention.

[ashgti]

xcode-select -print-path can be used print the path to the current version of Xcode, just so you know.

[ashgti]

So, I made a VM with OS X 10.7.2 and Xcode 4.3 installed and nothing else. Some thing worth noting:

• /Applications/Xcode.app/Contents/Developer/usr/bin has a lot of things, but not clang, ld, nm, etc. Note, this directory contains a symbolic link 'cc -> clang' and 'c++ -> clang++' but clang isn't in that directory and they are not absolute paths to the location of the actual clang executable's location.
• /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin contains clang, ld, nm, etc.

This is causing problems in brew because of all of the hard coded paths (eg. /usr/bin/clang , /usr/bin/cc , /usr/bin/gcc , etc.)

If there is anything specific you guys want me to test let me know, but I can tell a number of hard coded paths will need to be updated for Xcode 4.3 to work.

[xose]

This is the file listing as of Xcode 4.3 DP3: https://gist.github.com/1591451

Notable changes:

• Autotools are gone! autoconf/automake and all related tools have been removed from Xcode.
• bison/yacc and m4 have been moved to Toolchains.
• cc and c++ symlinks are now on Toolchains too.

[jacknagel]

Autotools are gone! autoconf/automake and all related tools have been removed from Xcode.

That's really disappointing.

[jacknagel]

Can I get a list of things that XCode 4.3 does put in /usr/bin? Is there anything other than xcrun?

[xose]

These are the only files installed/updated to /usr/bin on first run: https://gist.github.com/1592925

Those ~1000 bytes files are actually scripts that call xcode-select to find the proper Xcode folder, and then run the tool.

[MikeMcQuaid]

So it's released. I'm going to try and sort it.

[samueljohn]

• First I called sudo /Library/Developer/Shared/uninstall-devtools to get a clean start.
• Then no gcc nor clang or whatever was in my path
• Installed XCode 4.3 from App store. Launched it for the first time and restarted my Mac.
• Still no gcc nor clang in my path.
• xcode-select -print-path tells me I have not selected any Xcode.
• So I did sudo xcode-select -select /Applications/Xcode.app/Contents/Developer/. I hope this is just a bug and not everyone has to do this. Perhaps Apple forgot to set this on install?
• Now I can find gcc and clang and the other with xcrun -find <toolname>

$ xcrun -find gcc
/Applications/Xcode.app/Contents/Developer/usr/bin/gcc
$ xcrun -find clang
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang

Is this the way to go?
Do you also have to teach xcode-select manually about the location?

[xose]

The command line tools can now be installed manually: Xcode > Preferences > Downloads > Components

[sorin-ionescu]

I'm investigating Xcode 4.3 to see if gcc-without-xcode is still relevant. Where before programs were compiled to be run from /usr/bin, now, I suspect, that they can only be run from /Applications/Xcode.app/Contents/Developer/usr/bin/ and /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ respectively, and they cannot be copied to /usr/bin. Hence why Xcode installs those dummy scripts in /usr/bin.

I am leaning towards depending on Xcode 4.3 for the SDK headers only and installing our own gcc, autotools, etc.

[sorin-ionescu]

@xose Is it actually installing the tools or dummy scripts that in turn call the command line tools in the Xcode application directory?

[etehtsea]

https://developer.apple.com/downloads/index.action#
Didn't tried yet, but "Command line tools for Xcode" is the only package needed at the moment.

It could be downloaded directly or installed from xcode download section in preferences.

[samueljohn]

@xose and @etehtsea you need a Developer ID to download these. That's no way to go.
I cannot even open the package to see it's contents.

Resorting to xcrun is better in my opinion.

My question to you would be, if xcode-select -print-path does this right out of the box, or did you need to "-switch" it first?

[sorin-ionescu]

I have had a look in command_line_tools_for_xcode.dmg_ that can be downloaded from @etehtsea's link.

The good news is that they have not yet removed llvm-gcc4.2.pkg since not everything compiles with clang. The bad news is that they have indeed removed Autotools.

gcc-without-xcode still seems relevant since there is no uninstall script in the disk image.

Present Packages

• BluetoothSDK.pkg
• clang.pkg
• CoreAudioSDK.pkg
• DeveloperToolsCLI.pkg
• DevSDK.pkg
• FireWireSDK.pkg
• JavaSDK.pkg
• llvm-gcc4.2.pkg
• OpenGLSDK.pkg
• QuickTimeSDK.pkg
• WebKitSDK.pkg
• X11Documentation.pkg
• X11SDK.pkg

Missing Packages

• DeveloperToolsSystemSupport.pkg
• DistributedBuildsSupport.pkg

[jacknagel]

xcode download section in preferences.

Does this mean any Xcode 4.3 user can install the UNIX tools from preferences?

Resorting to xcrun is better in my opinion.

I messed around with this and xcrun is just painfully slow on a cold cache. It will make everything in Homebrew lag.

The best solution (short of finding a way to install the dev tools in their proper location) is to maintain an internal list of paths where the tools are found, and use a wrapper method that calls system with a custom PATH.

[MikeMcQuaid]

Yep, seems reasonable @jacknagel. Will have a look at this later. Requiring preferences or intervention like that is a pain. At least the App Store defaults to installing Xcode in the same place.

[2bits]

sounds good. it has an odd dep on libsigsegv. not sure why.

[adamv]

Reopening since discussion is still going on here...

[camillol]

@mxcl so the current autotools require GNU libtool?

BTW, the current libtool seems to be 2.4.2, not 2.4.

[mxcl]

Apple don't provide libtoolize. If I somehow didn't provide the most current libtool, patches are welcome. Thanks.

[anatol]

@mxcl Does it mean that using xcodebuild for project (such as fuse4x) is discouraged? If so what is the best build system alternative for these projects?

fuse4x-kext uses xcodebuild because it provides a lot of plumbing for KEXT building, e.g. creates kext directory structure, sets required compile/link flags. Are there any good examples of KEXT projects that do not use xcodebuild?

[camillol]

@anatol if you need xcodebuild, use xcodebuild. Won't most users install the binary packages for fuse4x anyway?

[mxcl]

@anatol using xcodebuild should be fine. It's just that by default the CLI tools don't work with it. I'll talk with the Xcode guys and figure out if they are going to fix that. Maybe xcodebuild can only be used if you have Xcode installed.

For now we may have to error out with a better error message.

[mxcl]

I believe everything we can do is fixed.

[jfirebaugh]

@mxcl Now that Xcode 4.3 CLT no longer includes gcc, would you consider including the apple-gcc42 formula from homebrew-alt? It's needed to compile Ruby 1.8.7 (in fact, any Ruby version < 1.9.3-p125).

[mxcl]

@jfirebaugh I believe there's a ticket for this already. So yes.