Support for prebuilt binaries for extensions

[asgrim]

Discussed in #295

^{Originally posted by godruoyi March 18, 2025}

Description

Thank you for creating such a nice tool that allows us to avoid using PECL any more.

I would like to ask if there are any plans to support the direct installation of extension files like .dylib, .dll, and .so, without the need for building them when we run pie install . It would be simpler since we can build the extension in another language and publish the artifacts to GitHub, so that we can install them directly without having to build them again. I would be interested in supporting this feature if it is feasible.

The plan

The proposal is that extensions can provide a matrix pre-built binaries, attached to immutable GH releases, that cover the "80% use case" - i.e. most common combinations. If the combination that is trying to be installed is not available pre-built, then the fallback mechanism of downloading the source & compiling it the traditional way is employed (although the maintainer can disable that behaviour, e.g. if the source code is not open source).

My own notes (subject to change! - do NOT quote me on any of this)

• Ext can specify they provide prebuilt binaries with DownloadUrlMethod::PrePackagedBinaryDownload
• Ext MUST provide a format to build the expected names
• If we find the asset:
  • Download it
  • Use gh attestations to verify the downloaded asset
    • note: depends on implementing Support release attestations ThePHPF/attestation#15
  • Install it (skip build)
  • What to do with metadata? We can generate some (i.e. install metadata), but not the build metadata. Check for reliance on the build metadata
• If we do NOT find the asset:
• Fall back to compiling from source (should this be configurable? e.g. DownloadUrlMethod::PrePackagedBinaryDownloadWithoutSourceFallback)
• Could probably help to create a GH action that builds an ext and attaches prebuilt binaries, to help adoption

Approximate matrix of important fields to differentiate builds

• OS (Windows/Linux/OSX)
• Architecture (arm64/aarch64, x86_64, x86)
• libc flavour (gnu, musl)
• PHP version OR PHP API version?
• Debug (debug/not)
• ZTS (-zts, -nts/omit)

Questions:

• how do we detect gnu vs musl libc?
• why is DD using PHP API version instead of PHP version?

[tillkruss]

It would be amazing if this could call a script.post-install-cmd.

[henderkes]

ZTS/NTS and GNU/musl detection are simple. PHP API version is also simple. Glibc version is also simple, other than it bloating up the matrix to at least 4 different debian flavour versions and 4 different fedora/rgel favour versions.

The real question is how you make sure that users get correct versions of dependencies like OpenSSL. Ar that point you're running into something that is impossible to resolve, because it will force recompiles in sync with different distrtibution streams (ubuntu, debian, fedora and rhel at the very least). And you need a way to force users to install the current version and either lock it in place or force users to update without manual input. A dnf or apt update could otherwise make your precompiled extensions unusable.

And that's just library dependencies - what about suggested extensions? Compile 16 different versions of swoole for 4 different glibc operating systems for 2-3 different library dependency versions for 3 different php versions for 2 different architectures.

[asgrim]

Yes, it's a good point; this solution won't work for everything (and isn't intended to solve every use case); but is a model used by companies like Data Dog, New Relic, etc. (especially the latter, as their extension is not open source), and is already working - so it makes sense to adopt this into PIE. This solution does not replace the "traditional" method that PIE uses (i.e. download source, compile, etc.), just adds another option for extension maintainers to use; if they use it, then it's their responsibility to handle their dependencies (if any) appropriately.

[henderkes]

The reason this works for e.g. NewRelic or Blackfire is because they have no external dependencies and only focus on a common libc version. For many extension maintainers this would be much more difficult without tools to facilitate the build process and possibly link in (library) dependencies statically.

Does PIE already provide a way to declare that extension X depends on extensions Y and Z?

[asgrim]

Does PIE already provide a way to declare that extension X depends on extensions Y and Z?

Yes, it is done in the extension's composer.json, e.g.

    "require": {
        "php": ">=7.4,^8.0",
        "ext-foobar": "*"
        "lib-whatever": "*"
    },

[henderkes]

That should make it much more realistic then. E.g. imagick-x86_64-linux-gnu.2.17.so. Perhaps extension maintainers could set up their builds with https://github.com/crazywhalecc/static-php-cli to ease their lives a bit.