php-build is breaking on Mojave

[mizunashi-mana]

Now, php-build is breaking on Mojave.
This is the upstream issue of PHP. However, php-build supports many PHP versions ended of supports. So, we should discussion the road map to fix.

Reasons

TL;DR:

• Mojave changed the policy of library paths.
• ./configure of PHP includes some bugs; on older macOS, these are not appeared.
• Therefore, PHP ./configure with current php-build options cannot detect some necessary library on Mojave.

Details:

Mojave changed the policy (https://developer.apple.com/documentation/xcode_release_notes/xcode_10_release_notes#3035624):

The command line tools will search the SDK for system headers by default. However, some software may fail to build correctly against the SDK and require macOS headers to be installed in the base system under /usr/include. If you are the maintainer of such software, we encourage you to update your project to work with the SDK or file a bug report for issues that are preventing you from doing so. As a workaround, an extra package is provided which will install the headers to the base system. In a future release, this package will no longer be provided.

In the short term, we can use below command for fallback:

sudo installer -pkg /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg -target /

On the other hand,

• current configure of PHP does not support recommended macOS SDK paths.
• PHP does not support current X11 paths of macOS.
• macOS SDK use .tbd extension for library, but, PHP does not support.
• (LDAP library bundled Mojave is not included some necessary symbols for PHP.)

On macOS < Mojave, we use libxml2/zlib/bzip2/libiconv/libedit/libpng/libjpeg/sqlite/libcurl bundled on macOS at /usr/include.
However, on Mojave, we cannot use these libraries bundled at /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include because PHP cannot detect them.

Now, Homebrew give up to use bundled libiconv/libpng/etc. on Mojave and use Homebrew's ones. We also can do same. (Fortunately, because some libraries such as zlib/bzip2 avoid checking of PHP configure and are linked by PHP make, we can use them with just specifing /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr as prefix path.)

We can select another way; patch php/php-src#3616 and remove =/usr on default configure options.

See also

Filed bugs:

• All latest versions fail to build/install on OSX 10.14 Mojave phpenv/phpenv#90
• https://bugs.php.net/bug.php?id=76433
• https://bugs.php.net/bug.php?id=77011
• LDAP issues with Mojave PHP7.0+ Homebrew/homebrew-core#32916

Patches:

• Support shared libraries of macOS SDKs php/php-src#3616
• php: update zlib detection on Mojave Homebrew/homebrew-core#31498
• Fix #76403: MacOS compilation incorrectly linking to system libs php/php-src#3472

Related:

• Install failed, "zlib not available" on macOS Mojave pyenv/pyenv#1219
• php: update zlib detection on Mojave Homebrew/homebrew-core#31498
• Mojave: use the CLT SDK where necessary Homebrew/brew#4335

[GrahamCampbell]

Thanks for looking into this. What would be your recommended fix for this codebase?

[mizunashi-mana]

My suggestions:

• Recommend using /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg for older versions (PHP < 7.1)
• I expect Support shared libraries of macOS SDKs php/php-src#3616 will be merged to all active PHP versions (PHP >= 7.1 and maybe >= 5.6). If it will be merged, we can create patches based it for ealier versions and apply them by using patch_file.

[GrahamCampbell]

Could you prepare a PR please?

[mizunashi-mana]

@GrahamCampbell Sorry for late response. I created a prototype #544 which supports for only PHP 7.2. I will complete it for other versions if we will finish discussions.

However, I do not like it. The PR will include many patches which are also too big and not testable finally.
Are there anyone having better ideas?

And, should we patch for snapshots and master?

[mizunashi-mana]

We may wait to resolve this issue.

Now, Apple distributed XCode 10.1 (includes security updates):
https://download.developer.apple.com/Developer_Tools/Command_Line_Tools_macOS_10.14_for_Xcode_10.1/Command_Line_Tools_macOS_10.14_for_Xcode_10.1.dmg
This update removes /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg and restores /usr/include for UNIX style applications.

Therefore, we can build PHP without patches on latest Mojave.

[GrahamCampbell]

I think it's possibly acceptable if we don't build on an old patch release, but on the latest in each major series?

[mizunashi-mana]

I think it's possibly acceptable if we don't build on an old patch release, but on the latest in each major series?

I agree.
However, I don't know which versions are most used; maybe one to match production (e.g. on Debian apt (7.0.30) which is not the latest of 7.0).

P.S.
Unfortunately, I lost test instances because I installed XCode 10.1 to my PC 😱
So, I cannot test #544 now...

[GrahamCampbell]

Seems to be working now then?

[mizunashi-mana]

Yeah, phpenv works now on Mojave with latest Command Line Tools.

Someone said this is a temporary glitch: php/php-src#3616 (comment) .
However, there are not official announce and I don't know reliable information about it.

I afraid the patch break phpenv on next macOS version although it fix this issue.
And, on current macOS, phpenv works well without the patch.

So, I suggest to wait any official announcement or the next version of macOS, and then complete or reject the patch.
If Apple decides to remove Unix-style headers according to release notes, this issue will return again on next macOS.

[GrahamCampbell]

Right. Best keep a tab on this then.

[ottlinger]

Is this bug possibly related to the currently broken build for
https://travis-ci.org/php-build/php-build/jobs/503369367
PHP 5.1.2?

The disadvantage of the broken build is that services like Travis do not pick up newer PHP versions.
@GrahamCampbell Should I file a PR to disable above version to get the build up'n'running again?

WDYT - thanks.

[mizunashi-mana]

@ottlinger No. Build failed was caused by other reason. PHP <= 5.2 needs mysql@5.7 and php-build use xcode8 for CI build now. Unfortunately, mysql@5.7 formula of Homebrew supports only >= macOS Sierra for bottle build: https://github.com/Homebrew/homebrew-core/blob/master/Formula/mysql@5.7.rb. So, Homebrew need to build mysql@5.7 for PHP 5.2.17 matrix. Building times is too too long then travis CI give up building and report the failure.

[ottlinger]

In order to have newer PHP versions on upstream services I removed 5.2.17 in a separate issue and PR:

• Disable build of version 5.2.17 to get newer versions on upstream projects #562
• Issue #562: Remove v5.2.17 from travis yml #563

HTH