zlib skipped when building 2.7 on macOS without /usr/include installed

[ronaldoussoren]

BPO	19960
Nosy	@ronaldoussoren, @ned-deily, @hynek
PRs	[2.7] bpo-19960: Fix building of zlib on macOS without installed headers #14257
Files	python2.7-alternative-sdk.txt

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/ronaldoussoren'
closed_at = <Date 2019-07-01.23:19:24.892>
created_at = <Date 2013-12-12.12:07:05.718>
labels = ['build']
title = 'zlib skipped when building 2.7 on macOS without /usr/include installed'
updated_at = <Date 2019-07-01.23:19:24.891>
user = 'https://github.com/ronaldoussoren'

bugs.python.org fields:

activity = <Date 2019-07-01.23:19:24.891>
actor = 'ned.deily'
assignee = 'ronaldoussoren'
closed = True
closed_date = <Date 2019-07-01.23:19:24.892>
closer = 'ned.deily'
components = ['Build']
creation = <Date 2013-12-12.12:07:05.718>
creator = 'ronaldoussoren'
dependencies = []
files = ['33101']
hgrepos = []
issue_num = 19960
keywords = ['patch', 'needs review']
message_count = 4.0
messages = ['205952', '228476', '346102', '347087']
nosy_count = 3.0
nosy_names = ['ronaldoussoren', 'ned.deily', 'hynek']
pr_nums = ['14257']
priority = 'high'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'compile error'
url = 'https://bugs.python.org/issue19960'
versions = ['Python 2.7']

[ronaldoussoren]

When you build python 2.7 on an OSX 10.9 machine with Xcode but without the command-line tools installed that build mostly succeeds, but doesn't detect a number of dependencies (in particular zlib).

The attached patch teaches "add_dir_to_list" about the SDK sysroot on OSX and with that the build succeeds.

Aside: I also noticed problems with build tinter, with 2.7 but also with 3.3 and 3.4. I'll file a separate issue about that.

[BreamoreBoy]

Just a reminder that a patch review is needed here guys.

[ned-deily]

Now that Apple, as of 10.14, has removed the option to install system header files into their traditional locations (like /usr/include), thus making every build a build from an SDK, this problem is now more noticeable. At least, I finally noticed it while fixing the more general problems that setup.py had on all branches (see bpo-36231). This issue is in addition and is specific to 2.7. As can be seen in Ronald's patch, which I didn't find until after I had written my own, the problem is that, in 2.7's setup.py, add_dir_to_list() does not take the SDK manipulation into account and, in particular, skips adding /usr/include because that directory doen't exist in the root filesystem anymore. On my systems, besides zlib, the dbm and nis modules were also being skipped. Curiously, some other extension modules with system library dependencies do get built properly, like _sqlite3, because their build step in setup.py explicitly added /usr/include to the SDK search paths.

Anyway, the PR I produced is somewhat more complicated that Ronald's original patch. It's more careful about what paths it checks for and it also stores the directory name, if found, without the prepended SDK path. That may be important in other steps in setup.py where it is doing path comparisons. The SDK will eventually get added again, either explicitly or implicitly by the compiler tool chain.

[ned-deily]

New changeset da7f6db by Ned Deily in branch '2.7':
bpo-19960: Fix building of zlib on macOS without installed headers (GH-14257)
da7f6db