macOS build issues (M1) #840

nikspyratos · 2024-06-02T08:38:31Z

What happened?

Heyo. Been playing around with static builds. Got the Linux one with Docker working fine - though haven't yet tested the binary, and the build was suuuuper slow (#835). Next I wanted to try out the macOS build. Been fighting through a few issues and I'm at the point where this stuff is above my head. If nothing else I'd like to report this I'm also using Laravel Herd, which AFAICT is a normal build of PHP.

Issue 1:
composer check-platform-reqs (added in 404086d) has a chance to put builtin PHP extensions in the PHP_EXTENSIONS var, and ./bin/spc seems to not be happy with that (on macOS at least). Stripping out the builtins from the extensions list seems to solve the problem.

Issue 2:
Once the extensions thing was fixed, the go build failed because something about how the CGO_FLAGS variable is built duplicates search paths for the linker. After some consultation with Claude AI, it suggested removing the sed part, which worked for me.
The error was:

ld: warning: search path '/Users/nik/Documents/dev/niksoftware-projects/frankenphp/dist/static-php-cli/buildroot//Users/nik/Documents/dev/niksoftware-projects/frankenphp/dist/static-php-cli/buildroot/lib' not found
ld: warning: search path '/Users/nik/Documents/dev/niksoftware-projects/frankenphp/dist/static-php-cli/buildroot//Users/nik/Documents/dev/niksoftware-projects/frankenphp/dist/static-php-cli/buildroot/lib' not found
ld: warning: search path '/Users/nik/Documents/dev/niksoftware-projects/frankenphp/dist/static-php-cli/buildroot//Users/nik/Documents/dev/niksoftware-projects/frankenphp/dist/static-php-cli/buildroot/lib' not found
Undefined symbols for architecture arm64:
  "_BZ2_bzCompress", referenced from:
      _process in libzip.a[114](zip_algorithm_bzip2.c.o)
  "_BZ2_bzCompressEnd", referenced from:
      _end in libzip.a[114](zip_algorithm_bzip2.c.o)
  "_BZ2_bzCompressInit", referenced from:
      _start in libzip.a[114](zip_algorithm_bzip2.c.o)
  "_BZ2_bzDecompress", referenced from:
      _process in libzip.a[114](zip_algorithm_bzip2.c.o)
  "_BZ2_bzDecompressEnd", referenced from:
      _end in libzip.a[114](zip_algorithm_bzip2.c.o)
  "_BZ2_bzDecompressInit", referenced from:
      _start in libzip.a[114](zip_algorithm_bzip2.c.o)
ld: symbol(s) not found for architecture arm64

Issue 3:
Lastly, I'm getting the same ./frankenphp.h:4:10: fatal error: 'Zend/zend_types.h' file not found error as in #825. At this point I gave up. From that issue and the compile docs linked, it suggests that the static builds are an alternative to compiling from sources, so my understanding is that I shouldn't need to go as deep as downloading the PHP source for this?

Build Type

Custom (tell us more in the description)

Worker Mode

No

Operating System

macOS

CPU Architecture

Apple Silicon

PHP configuration

Relevant log output

+ type git
++ uname -m
+ arch=arm64
++ uname -s
++ tr '[:upper:]' '[:lower:]'
+ os=darwin
+ md5binary=md5sum
+ '[' darwin = darwin ']'
+ os=mac
+ md5binary='md5 -q'
+ '[' mac = linux ']'
+ '[' -z '' ']'
+ '[' -n /Users/nik/Documents/dev/niksoftware-projects/toybox ']'
+ '[' -f /Users/nik/Documents/dev/niksoftware-projects/toybox/composer.json ']'
+ cd /Users/nik/Documents/dev/niksoftware-projects/toybox
++ composer check-platform-reqs --no-dev
++ grep '^ext'
++ sed -e 's/^ext-//' -e 's/ .*//'
++ xargs
++ tr ' ' ,
+ PHP_EXTENSIONS=ctype,curl,dom,fileinfo,filter,hash,iconv,intl,json,libxml,mbstring,openssl,pcre,session,tokenizer,xmlreader,zip
+ export PHP_EXTENSIONS
+ cd -
/Users/nik/Documents/dev/niksoftware-projects/frankenphp
+ '[' -z '' ']'
+ export PHP_EXTENSION_LIBS=bzip2,freetype,libavif,libjpeg,liblz4,libwebp,libzip
+ PHP_EXTENSION_LIBS=bzip2,freetype,libavif,libjpeg,liblz4,libwebp,libzip
+ echo bzip2,freetype,libavif,libjpeg,liblz4,libwebp,libzip
+ grep -q '\bbrotli\b'
+ export PHP_EXTENSION_LIBS=bzip2,freetype,libavif,libjpeg,liblz4,libwebp,libzip,brotli
+ PHP_EXTENSION_LIBS=bzip2,freetype,libavif,libjpeg,liblz4,libwebp,libzip,brotli
+ '[' -z '' ']'
+ export PHP_VERSION=8.3
+ PHP_VERSION=8.3
+ '[' -z '' ']'
++ git rev-parse --verify HEAD
+ FRANKENPHP_VERSION=0b4a427cac1da044518ada44345bbafd98a0bc04
+ export FRANKENPHP_VERSION
+ bin=frankenphp-mac-arm64
+ '[' -n '' ']'
+ '[' -f dist/static-php-cli/buildroot/lib/libphp.a ']'
+ mkdir -p dist/
+ cd dist/
+ '[' -d static-php-cli/ ']'
+ cd static-php-cli/
+ git pull
Already up to date.
+ type brew
+ type composer
+ type go
+ '[' -n '' ']'
+ '[' -n '' ']'
+ composer install --no-dev -a
Installing dependencies from lock file
Verifying lock file contents can be installed on current platform.
Nothing to install, update or remove
Generating optimized autoload files
8 packages you are using are looking for funding.
Use the `composer fund` command to find out more!
+ '[' mac = linux ']'
+ '[' -n '' ']'
+ ./bin/spc doctor --auto-fix
     _        _   _                 _
 ___| |_ __ _| |_(_) ___      _ __ | |__  _ __
/ __| __/ _` | __| |/ __|____| '_ \| '_ \| '_ \
\__ \ || (_| | |_| | (_|_____| |_) | | | | |_) |
|___/\__\__,_|\__|_|\___|    | .__/|_| |_| .__/   v2.2.2
                             |_|         |_|
Checking if current OS are supported ... Darwin arm64, supported
Checking if homebrew has installed ... ok
Checking if necessary tools are installed ... ok
Doctor check complete !
+ ./bin/spc download --with-php=8.3 --for-extensions=ctype,curl,dom,fileinfo,filter,hash,iconv,intl,json,libxml,mbstring,openssl,pcre,session,tokenizer,xmlreader,zip --for-libs=bzip2,freetype,libavif,libjpeg,liblz4,libwebp,libzip,brotli --ignore-cache-sources=php-src

In DependencyUtil.php line 169:

  Extension [hash] not exist !


download [--shallow-clone] [--with-openssl11] [--with-php WITH-PHP] [--clean] [-A|--all] [-U|--custom-url CUSTOM-URL] [-Z|--from-zip FROM-ZIP] [-e|--for-extensions FOR-EXTENSIONS] [-l|--for-libs FOR-LIBS] [--without-suggestions] [--ignore-cache-sources IGNORE-CACHE-SOURCES] [-R|--retry RETRY] [--debug] [--no-motd] [--] <sources>

The text was updated successfully, but these errors were encountered:

withinboredom · 2024-06-02T10:38:32Z

I'm much more inclined to say there is something wrong with your environment since static builds in CI appear to work fine. That being said, getting an environment in the right conditions when you're trying to do it on a computer you actually need to work on can be a bit frustrating (from experience). I highly recommend using a build environment specific to building this (such as a VM or CI).

That being said, it seems you resolved most of your issues? I'm unclear what the actual issue is here.

You say the issue is ./frankenphp.h:4:10: fatal error: 'Zend/zend_types.h' file not found but your logs indicate you're trying to install an extension that doesn't exist on the current platform: Extension [hash] not exist !

Can you clarify what issue you're running into?

dunglas · 2024-06-02T11:34:34Z

It's maybe a bug in the extension detection system of our shell script. Could you try to explicitly set PHP_EXTENSIONS? Could you also copy your composer.json please?

For the slow builds, try to disable UPX.

nikspyratos · 2024-06-03T09:48:41Z

Can you clarify what issue you're running into?

I've run into three different problems here, the logs for the GH template are just the first one.

That being said, it seems you resolved most of your issues? I'm unclear what the actual issue is here.

I didn't, I gave up on the third and though it might be worth opening this issue as I surely can't be the only one with this problem. I get the impression macOS support isn't a priority, and I agree it shouldn't really be since prod is most likely going to be the linux build for most, but it's still worth having this report at least posted so others can follow the same trail.

Could you try to explicitly set PHP_EXTENSIONS

Sorry I wasn't clear in my description, I did do that and stripped out the defaults by hand. Then I got down to issue #2. Yes on my todo list for next time is not using UPX.

dunglas · 2024-06-03T10:23:41Z

The warnings are not important and can be safely ignored.

Issue 2 is weird and looks like a static-php-cli issue when building the zip extension (it happens). Could you try deleting entirely the dist/ (or starting with a fresh install) with PHP_EXTENSIONS and NO_COMPRESS explicitly set?

Anyway, we need to do a patch to filter out builtin extensions (like hash in your case).

nikspyratos · 2024-06-04T11:29:50Z

The warnings are not important and can be safely ignored.

The search path ones? It wouldn't proceed with the build until I resolved that. The bzip and search path issue went away when I removed the sed part of line 123.

Anyway, we need to do a patch to filter out builtin extensions (like hash in your case).

Yep. I wouldn't mind taking a crack at it over the weekend with some bash-fu as well.

Could you try deleting entirely the dist/ (or starting with a fresh install) with PHP_EXTENSIONS and NO_COMPRESS explicitly set?

Same as issue 3 before - still goes all the way to ./frankenphp.h:4:10: fatal error: 'Zend/zend_types.h' file not found on the go build command. The zend_types.h file is definitely there though 🤔 If the problem is related to my aforementioned sed change then it's going to be a bit of a catch-22 to build.

crazywhalecc · 2024-06-05T07:10:02Z

@nikspyratos

Issue 1: static-php-cli can consider these builtin extensions (always enabled) as redundant input, and put them as warning messages if needed. (next version of spc)
Issue 2:

Undefined symbols for architecture arm64:
  "_BZ2_bzCompress", referenced from:
      _process in libzip.a[114](zip_algorithm_bzip2.c.o)

It seems it's compiling zip extension with bzip2 support, and compile again without bzip2 problem. Just make sure every different combination of extensions has a clean buildroot (rm -rf buildroot source before different build). And also duplicates search paths for the linker will not affect anything.

If you still get the same error when compiling, you can turn on full output logging and find all the arguments to the bin/spc build command line and print them out.

nikspyratos · 2024-06-05T12:45:28Z

It seems it's compiling zip extension with bzip2 support, and compile again without bzip2 problem. Just make sure every different combination of extensions has a clean buildroot (rm -rf buildroot source before different build). And also duplicates search paths for the linker will not affect anything.

🤷‍♂️ That error only happens when those linker warnings are also present, and again removing sed s#-I/#-I"${PWD}"/buildroot/#g) from CGO_CFLAGS in the build script gets rid of both issues. If that doesn't make sense, I've got no idea, that's just what I'm experiencing.

here's the full output that I have, starting from a clean buildroot.

If you still get the same error when compiling, you can turn on full output logging and find all the arguments to the bin/spc build command line and print them out.

--debug is already an enabled option on the spc commands. Is there another arg for fuller logging?

crazywhalecc · 2024-06-05T14:58:26Z

@nikspyratos Could you provide this line of log (argv: ****)? This will show what arguments are passed to static-php-cli.

$ bin/spc build $PHP_EXTENSIONS --build-cli --build-micro --debug --with-libs=$PHP_EXTENSION_LIBS
     _        _   _                 _           
 ___| |_ __ _| |_(_) ___      _ __ | |__  _ __  
/ __| __/ _` | __| |/ __|____| '_ \| '_ \| '_ \ 
\__ \ || (_| | |_| | (_|_____| |_) | | | | |_) |
|___/\__\__,_|\__|_|\___|    | .__/|_| |_| .__/   v2.2.3
                             |_|         |_|    
[14:56:56] [DEBU] argv: bin/spc build hash,opcache,mbstring,zip --build-cli --build-micro --debug --with-libs=bzip2,freetype,libavif,libjpeg,liblz4,libwebp,libzip

EDIT: Issue 1 Extension [hash] not exist ! bug has been fixed via crazywhalecc/static-php-cli#473 .

nikspyratos · 2024-06-05T15:29:19Z

Nice!

Here's the log:

[15:27:41] [DEBU] argv: ./bin/spc build --debug --enable-zts --build-embed ctype,curl,dom,fileinfo,filter,iconv,intl,libxml,mbstring,openssl,session,tokenizer,xmlreader,zip --with-libs=bzip2,freetype,libavif,libjpeg,liblz4,libwebp,libzip,brotli

crazywhalecc · 2024-06-05T16:04:13Z

I just figured out what was causing the bzip2 link error.

build static php with zip,xxxx extension (not include bz2 and zlib extension), with additional libs bzip2,yyy, if you build --build-cli, the libzip requires bzip2, and zip does not require it for now.
It will cause configure check item zip_compression_method_supported -> no, and zip extension will not support any additional algorithm.
if build with --build-embed, same as --build-cli, but this error only occurs when linking.
The solution is also simple: include bz2,zlib,zip extensions at the same time. But I'll also adjust spc's deps config to prevent it.

nikspyratos · 2024-06-05T17:08:57Z

That worked! Thanks!

For this specific issue, I think it can be closed now, but I believe there are some lingering issues about how extensions are pulled/listed from the embedded application (at least on the local app)

The build was successful, but since it was completely relying on the output of composer check-platform-reqs, it was missing extensions that should have been listed but weren't - for example PDO, as any artisan command I ran first had issues of not finding the PDO class.

So perhaps trying to auto-parse the extensions from a project isn't going to be as accurate as originally hoped, since many project dependencies don't include accurate extension requirements.

dunglas · 2024-06-05T20:20:55Z

That's a good opportunity to fix the composer.json of these projects to add the missing requirements 😃

It's always possible to use the environnement variable for specific needs.

nikspyratos · 2024-06-06T07:12:24Z

100%. Will keep playing around with this, thanks for all the help!

nikspyratos added the bug Something isn't working label Jun 2, 2024

dunglas mentioned this issue Jun 5, 2024

Build Error [Linux PopOs] #850

Closed

nikspyratos closed this as completed Jun 6, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

macOS build issues (M1) #840

macOS build issues (M1) #840

nikspyratos commented Jun 2, 2024

withinboredom commented Jun 2, 2024

dunglas commented Jun 2, 2024

nikspyratos commented Jun 3, 2024

dunglas commented Jun 3, 2024

nikspyratos commented Jun 4, 2024

crazywhalecc commented Jun 5, 2024 •

edited

Loading

nikspyratos commented Jun 5, 2024 •

edited

Loading

crazywhalecc commented Jun 5, 2024 •

edited

Loading

nikspyratos commented Jun 5, 2024

crazywhalecc commented Jun 5, 2024 •

edited

Loading

nikspyratos commented Jun 5, 2024

dunglas commented Jun 5, 2024

nikspyratos commented Jun 6, 2024

macOS build issues (M1) #840

macOS build issues (M1) #840

Comments

nikspyratos commented Jun 2, 2024

What happened?

Build Type

Worker Mode

Operating System

CPU Architecture

PHP configuration

Relevant log output

withinboredom commented Jun 2, 2024

dunglas commented Jun 2, 2024

nikspyratos commented Jun 3, 2024

dunglas commented Jun 3, 2024

nikspyratos commented Jun 4, 2024

crazywhalecc commented Jun 5, 2024 • edited Loading

nikspyratos commented Jun 5, 2024 • edited Loading

crazywhalecc commented Jun 5, 2024 • edited Loading

nikspyratos commented Jun 5, 2024

crazywhalecc commented Jun 5, 2024 • edited Loading

nikspyratos commented Jun 5, 2024

dunglas commented Jun 5, 2024

nikspyratos commented Jun 6, 2024

crazywhalecc commented Jun 5, 2024 •

edited

Loading

nikspyratos commented Jun 5, 2024 •

edited

Loading

crazywhalecc commented Jun 5, 2024 •

edited

Loading

crazywhalecc commented Jun 5, 2024 •

edited

Loading