Extract 10.12 SDK from Xcode 8.3.3 (to do before release of macOS 10.13/Xcode 9)

[devernay]

Xcode 8.3.3 is distributed as a xip archive.
I managed to extract something from it by:

• compiling the pbzx tool (source code can be found at various places, I took the one from http://www.newosxbook.com/src.jl?tree=listings&file=pbzx.c)
• running the following commands:

xar -xvf Xcode8.3.3.xip
pbzx Content 2>/dev/null | tar xf - ./Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs

• the SDK is in ./Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk (/not/ MacOSX10.12.sdk)

There seems to be errors during the extraction. Are these fatal?

That also means XcodeLegacy.sh has to rely on the pbzx tool (if not available, the script could give the recipe to compile and install it in /usr/local/bin - it does not seem to be available from MacPorts or homebrew)

As noted in #24 (comment) we cannot distribute the SDKs. They are available here but this infringes the Xcode licence. Maybe we could just let the user download the SDK from that source?

see also:
https://stackoverflow.com/questions/40414645/how-to-extract-xip-archive-using-command-line

[EricFromCanada]

My findings were similar. The script would have to supply a tool that can decode pbzx streams, since macOS doesn't provide one.

Besides pbzx.c, another option is parse_pbzx2.py, which works with the system's Python. This successfully extracted the entire archive:

xar -xf Xcode8.3.3.xip
python parse_pbzx2.py Content
cpio -idm --quiet -F Content.part00.cpio.xz

Ideally the script would extract just the path we need, and both cpio and tar will do so if given a path as an argument. But in this case, both tools throw lots of random "Can't create [file]" errors without giving a reason why, and the resulting SDK has missing files. If no argument is given, they'll still throw the same errors, but (on my machine) the resulting SDK within the new Xcode.app is complete, according to a comparison with a known-good copy using FileMerge.

[EricFromCanada]

Solved it: as confirmed by ls -i, it turns out that Apple hard-links common files between everything in the Platforms folder, so the script needs to extract it entirely to get a complete SDK. Both of these commands work despite their errors, although the second throws far fewer of them:

cpio -idm --quiet -F Content.part00.cpio.xz ./Xcode.app/Contents/Developer/Platforms
xz --decompress --stdout Content.part00.cpio.xz | pax -r ./Xcode.app/Contents/Developer/Platforms

(see also this Q&A)

[EricFromCanada]

This version of parse_pbzx.py supports decompression and pipes, allowing the final command sequence to be:

xar -xf Xcode8.3.3.xip
./parse_pbzx.py Content | pax -r ./Xcode.app/Contents/Developer/Platforms 2>/dev/null

resulting in a 9GB package from which the SDK can be extracted. Since it takes a while to perform the extraction, even on a modern SSD-equipped system, it might be smart to wrap the command in a function that provides a progress bar or spinner in the terminal output.

[sstillwell]

Any further progress or comment on this issue, or is this a showstopper?

[devernay]

You can still download it from https://github.com/phracker/MacOSX-SDKs/releases .
Sorry, I don't have much time to work on this.

[EricFromCanada]

I've had success elsewhere with live-downloading-and-patching another tool within a script to accomplish a similar task. I can attempt to put a PR together.

[sstillwell]

@devernay I can manually download the SDK, no problem. I guess I'm questioning what to do WITH the SDK once I have it...will the existing script patch everything in Xcode 9.x with the exception of the 10.12 SDK and I just copy that into Xcode after patching?

[devernay]

use the latest github version. 6956e79 installs the file at the right place

[devernay]

temporary fix in XcodeLegacy 2.1

[sstillwell]

Evidently you also need to re-download 10.7-10.11 from the same github link above - the current XcodeLegacy.sh won't recognize the SDKs already extracted from older Xcode versions (due to naming, likely). Was that intended?

[devernay]

no it wasn't, it should check for both

[sstillwell]

I re-ran the -buildpackages section, and the XcodeSDK* files are there, but -install would not use them. Downloading the SDKs from the github site and placing the .xz files in the same folder did work, however.

[devernay]

juste fixed it. re-download and execute XcodeLegacy install, it should recognize the old packages again