New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Extension installer #2898

Open
wants to merge 3 commits into
from

Conversation

Projects
None yet
@naderman
Member

naderman commented Apr 10, 2014

Supports PHP & HHVM.

Todo:

  • Need to come up with a solution for ext-* packages actually being installable now, so extensions can be compiled automatically to satisfy dependencies on extensions.
  • Documentation
  • Use a different downloader to download binary builds from github releases for windows (or pecl for PHP)
  • Output information on configuring the server to include extension.so

Replaces the earlier PR #498

@naderman naderman referenced this pull request Apr 10, 2014

Closed

ExtensionInstaller #498

@Seldaek

This comment has been minimized.

Show comment
Hide comment
@Seldaek

Seldaek Apr 11, 2014

Member

Todo: add a pecl.php.net composer repo and to make all these packages available and be able to download auto-built dlls on windows :)

Member

Seldaek commented Apr 11, 2014

Todo: add a pecl.php.net composer repo and to make all these packages available and be able to download auto-built dlls on windows :)

@Seldaek

This comment has been minimized.

Show comment
Hide comment
@Seldaek

Seldaek Apr 11, 2014

Member

Ah I missed the third point, but my point still stands, I think we need a way to get pecl stuff as a composer repository, and then maybe also support extension-typed packages on packagist? Could include a binary download url or something.

Member

Seldaek commented Apr 11, 2014

Ah I missed the third point, but my point still stands, I think we need a way to get pecl stuff as a composer repository, and then maybe also support extension-typed packages on packagist? Could include a binary download url or something.

@naderman

This comment has been minimized.

Show comment
Hide comment
@naderman

naderman Apr 11, 2014

Member

Yes my idea was to have a way to specify alternative download urls based on architecture somehow.

Member

naderman commented Apr 11, 2014

Yes my idea was to have a way to specify alternative download urls based on architecture somehow.

@mnapoli

This comment has been minimized.

Show comment
Hide comment
@mnapoli

mnapoli Apr 11, 2014

This will install the extension globally on the machine right?

mnapoli commented Apr 11, 2014

This will install the extension globally on the machine right?

@staabm

This comment has been minimized.

Show comment
Hide comment
@staabm

staabm Apr 13, 2014

Is it intended that here all files are copied and in the hhvm branch only the .so?

Is it intended that here all files are copied and in the hhvm branch only the .so?

This comment has been minimized.

Show comment
Hide comment
@naderman

naderman Apr 14, 2014

Owner

@igorw 's php extension code was copying all files from the modules/ directory. hhvm builds them into the root, so there's lots of other files around which definitely do not want. Not sure if there are any other file types worth copying from PHP modules extensions or hhvm extensions. We may want to use make install with a configure prefix in the PHP case anyway. The HHVM cmake for extensions doesn't seem to define an install target at all.

Owner

naderman replied Apr 14, 2014

@igorw 's php extension code was copying all files from the modules/ directory. hhvm builds them into the root, so there's lots of other files around which definitely do not want. Not sure if there are any other file types worth copying from PHP modules extensions or hhvm extensions. We may want to use make install with a configure prefix in the PHP case anyway. The HHVM cmake for extensions doesn't seem to define an install target at all.

@mcuadros

This comment has been minimized.

Show comment
Hide comment
@mcuadros

mcuadros Apr 24, 2014

About HNI extensions:

The HHVM HNI extension has the very similar installation process of the PHP extension, so this not should a problem.

hphpize
cmake .
make
make install

As I see, is very complex install extension from composer due to:

1a) From source: C/C++ library dependencies, for example Mongofill extension for HHVM requires libbson installed on the system.
1b) From binary, will require a horde of workers and config for compile every extension for every arch, os and PHP/HHVM version. (The sweet dream of a puppet guy)
2) The extension are installed globally, not locally like a common PHP package.
3) In the most of the cases will require root access to be installed
4) Some extensions for example pthreads requires a PHP version compiled with a specific option

Looks a great idea but with a bunch of problems in the back.

About HNI extensions:

The HHVM HNI extension has the very similar installation process of the PHP extension, so this not should a problem.

hphpize
cmake .
make
make install

As I see, is very complex install extension from composer due to:

1a) From source: C/C++ library dependencies, for example Mongofill extension for HHVM requires libbson installed on the system.
1b) From binary, will require a horde of workers and config for compile every extension for every arch, os and PHP/HHVM version. (The sweet dream of a puppet guy)
2) The extension are installed globally, not locally like a common PHP package.
3) In the most of the cases will require root access to be installed
4) Some extensions for example pthreads requires a PHP version compiled with a specific option

Looks a great idea but with a bunch of problems in the back.

@naderman

This comment has been minimized.

Show comment
Hide comment
@naderman

naderman May 16, 2014

Member

@dzuelke didn't you want to write down what we discussed on this issue? ;-)

Member

naderman commented May 16, 2014

@dzuelke didn't you want to write down what we discussed on this issue? ;-)

@dzuelke

This comment has been minimized.

Show comment
Hide comment
@dzuelke

dzuelke May 16, 2014

Contributor

Will do, @naderman, thanks for the reminder :)

Contributor

dzuelke commented May 16, 2014

Will do, @naderman, thanks for the reminder :)

@pierrejoye

This comment has been minimized.

Show comment
Hide comment

For the record here:

see https://github.com/pierrejoye/pickle

@dzuelke

This comment has been minimized.

Show comment
Hide comment
@dzuelke

dzuelke Jun 17, 2014

Contributor

Summary of my discussion with @naderman a few weeks ago:

There should be a composer install --extensions or similar command; it uses whatever plugin is installed (by default, that could be Pickle) to install missing extensions.

If an extension isn't available during a regular composer install, it'll error (probably with a slightly nicer message than currently) and advise users that they can try composer install --extensions and then try again.

Debian etc could ship a php5-composer package that has a different extension installer plugin enabled by default which tries an apt-get install php5-foobarz.

This is the only sane way, as one can only enable extensions globally in PHP by copying a small .ini to whatever init.d folder that was (hopefully!) defined at ./configure time. The alternative is to use a fully replaced php.ini. The folder where PHP scans for additional files can't be re-defined at runtime.

So an approach where Composer on regular install writes out extra php.ini files clearly won't work, because a composer install works on an individual project, but it would have to modify this global state, and that state wouldn't change either when jumping around between different projects.

It also greatly simplifies the amount of work to be done ;) Much less complicated approach overall; installer plugin developers can figure out the gritty details for their respective platform(s) themselves.

Contributor

dzuelke commented Jun 17, 2014

Summary of my discussion with @naderman a few weeks ago:

There should be a composer install --extensions or similar command; it uses whatever plugin is installed (by default, that could be Pickle) to install missing extensions.

If an extension isn't available during a regular composer install, it'll error (probably with a slightly nicer message than currently) and advise users that they can try composer install --extensions and then try again.

Debian etc could ship a php5-composer package that has a different extension installer plugin enabled by default which tries an apt-get install php5-foobarz.

This is the only sane way, as one can only enable extensions globally in PHP by copying a small .ini to whatever init.d folder that was (hopefully!) defined at ./configure time. The alternative is to use a fully replaced php.ini. The folder where PHP scans for additional files can't be re-defined at runtime.

So an approach where Composer on regular install writes out extra php.ini files clearly won't work, because a composer install works on an individual project, but it would have to modify this global state, and that state wouldn't change either when jumping around between different projects.

It also greatly simplifies the amount of work to be done ;) Much less complicated approach overall; installer plugin developers can figure out the gritty details for their respective platform(s) themselves.

@kmiller68 kmiller68 referenced this pull request Jun 23, 2014

Closed

HHVM Extension Support #21

@pierrejoye

This comment has been minimized.

Show comment
Hide comment
@pierrejoye

pierrejoye Jul 9, 2014

Just a status about pickle:

We are close to the 1st release, it supports:

  • 100% compatible json file
  • binary install on windows
  • pickle install apcu will fetch from pecl.php.net when available
  • pickle install supports git, http(s), etc.
  • packaging of release packages
  • fully compatible with current package PECL format (install or conversion, from package or repository)

About the roadmap, I will update the README and the wiki later this week but the main next points are:

  • automatic installation of deps extensions
  • plugins to support distributions packages (apt, rpm, etc.), to avoid to have to many tools for the same goal

Feedback, PRs, etc welcome :)

Just a status about pickle:

We are close to the 1st release, it supports:

  • 100% compatible json file
  • binary install on windows
  • pickle install apcu will fetch from pecl.php.net when available
  • pickle install supports git, http(s), etc.
  • packaging of release packages
  • fully compatible with current package PECL format (install or conversion, from package or repository)

About the roadmap, I will update the README and the wiki later this week but the main next points are:

  • automatic installation of deps extensions
  • plugins to support distributions packages (apt, rpm, etc.), to avoid to have to many tools for the same goal

Feedback, PRs, etc welcome :)

@kmiller68

This comment has been minimized.

Show comment
Hide comment
@kmiller68

kmiller68 Jul 23, 2014

I am interested in helping out with getting an extension installer working. Are there any thoughts on how the extension installer plugin interface would look? I see a couple of design decisions:

  1. How to specify which plugin to use, such as pickle or an apt-get based installer (if the user does not want the default). I was thinking something along the lines of composer install --extensions apt-get or composer install --extensions pickle

  2. When people build the extension from source, how should they specify the C/C++ library dependencies. Based on the current system there is not an obvious way to do this. Additionally, if the dependency is installed via apt-get, rpm, etc the creator may have differing dependencies based on the distro/release.

Any thoughts?

I am interested in helping out with getting an extension installer working. Are there any thoughts on how the extension installer plugin interface would look? I see a couple of design decisions:

  1. How to specify which plugin to use, such as pickle or an apt-get based installer (if the user does not want the default). I was thinking something along the lines of composer install --extensions apt-get or composer install --extensions pickle

  2. When people build the extension from source, how should they specify the C/C++ library dependencies. Based on the current system there is not an obvious way to do this. Additionally, if the dependency is installed via apt-get, rpm, etc the creator may have differing dependencies based on the distro/release.

Any thoughts?

@pierrejoye

This comment has been minimized.

Show comment
Hide comment
@pierrejoye

pierrejoye Jul 23, 2014

hi,

On Wed, Jul 23, 2014 at 8:20 PM, Keith Miller notifications@github.com
wrote:

I am interested in helping out with getting an extension installer
working. Are there any thoughts on how the extension installer plugin
interface would look? I see a couple of design decisions:

  1. How to specify which plugin to use, such as pickle or an apt-get based
    installer (if the user does not want the default). I was thinking something
    along the lines of composer install --extensions apt-get or composer
    install --extensions pickle

Please not that distros package support will be added in pickle (too?). As
we support binaries on windows, it just makes sense to have them as well.

  1. When people build the extension from source, how should they specify
    the C/C++ library dependencies. Based on the current system there is not an
    obvious way to do this. Additionally, if the dependency is installed via
    apt-get, rpm, etc the creator may have differing dependencies based on the
    distro/release.

Any thoughts?

We can do what we do on windows as well. Check the package dependencies and
fetch them. On *nix, either using packages or fetching from sources. We do
not yet implement that nor think about how to do it but raise an error on
configure, given that the config.m4 has the validations in place (99.99% of
them have it).


Reply to this email directly or view it on GitHub
#2898 (comment).

Pierre

@pierrejoye | http://www.libgd.org

hi,

On Wed, Jul 23, 2014 at 8:20 PM, Keith Miller notifications@github.com
wrote:

I am interested in helping out with getting an extension installer
working. Are there any thoughts on how the extension installer plugin
interface would look? I see a couple of design decisions:

  1. How to specify which plugin to use, such as pickle or an apt-get based
    installer (if the user does not want the default). I was thinking something
    along the lines of composer install --extensions apt-get or composer
    install --extensions pickle

Please not that distros package support will be added in pickle (too?). As
we support binaries on windows, it just makes sense to have them as well.

  1. When people build the extension from source, how should they specify
    the C/C++ library dependencies. Based on the current system there is not an
    obvious way to do this. Additionally, if the dependency is installed via
    apt-get, rpm, etc the creator may have differing dependencies based on the
    distro/release.

Any thoughts?

We can do what we do on windows as well. Check the package dependencies and
fetch them. On *nix, either using packages or fetching from sources. We do
not yet implement that nor think about how to do it but raise an error on
configure, given that the config.m4 has the validations in place (99.99% of
them have it).


Reply to this email directly or view it on GitHub
#2898 (comment).

Pierre

@pierrejoye | http://www.libgd.org

@ovr

This comment has been minimized.

Show comment
Hide comment
@ovr

ovr Mar 1, 2015

Any news?

ovr commented Mar 1, 2015

Any news?

@pierrejoye

This comment has been minimized.

Show comment
Hide comment
@pierrejoye

pierrejoye Mar 1, 2015

Pickle is almost feature complete now.

Composer part is being worked on.
On Mar 1, 2015 5:51 PM, "Дмитрий Пацура" notifications@github.com wrote:

Any news?


Reply to this email directly or view it on GitHub
#2898 (comment).

Pickle is almost feature complete now.

Composer part is being worked on.
On Mar 1, 2015 5:51 PM, "Дмитрий Пацура" notifications@github.com wrote:

Any news?


Reply to this email directly or view it on GitHub
#2898 (comment).

@brunoric

This comment has been minimized.

Show comment
Hide comment
@brunoric

brunoric Mar 10, 2016

+1 from Pickle being integrated on Composer. <3

+1 from Pickle being integrated on Composer. <3

@sbuzonas sbuzonas referenced this pull request Mar 23, 2016

Open

Project status #3

@Seldaek Seldaek added this to the Nice To Have milestone Apr 12, 2016

@jean-pasqualini

This comment has been minimized.

Show comment
Hide comment
@jean-pasqualini

jean-pasqualini Jul 22, 2016

you planned to finish?

you planned to finish?

@mgdm mgdm referenced this pull request Jul 28, 2016

Closed

PHP Composer support? #41

@DavertMik

This comment has been minimized.

Show comment
Hide comment
@DavertMik

DavertMik Oct 12, 2016

Any update on this from Composer or Pickle team?

Any update on this from Composer or Pickle team?

@jippi

This comment has been minimized.

Show comment
Hide comment
@jippi

jippi Oct 28, 2016

bump? :)

jippi commented Oct 28, 2016

bump? :)

@dzuelke

This comment has been minimized.

Show comment
Hide comment
@dzuelke

dzuelke Oct 29, 2016

Contributor

Still heavily unsure how useful this would really be... the issue remains that people run many different projects, and extensions would have to be installed for these, potentially even in different versions.

Two years ago I wrote:

The folder where PHP scans for additional files can't be re-defined at runtime.

That's actually not true; PHP_INI_SCAN_DIR can be used for that, and it can even include an empty segment to represent the ./configure time path. Still, the whole thing is a massive bag of hurt.

Contributor

dzuelke commented Oct 29, 2016

Still heavily unsure how useful this would really be... the issue remains that people run many different projects, and extensions would have to be installed for these, potentially even in different versions.

Two years ago I wrote:

The folder where PHP scans for additional files can't be re-defined at runtime.

That's actually not true; PHP_INI_SCAN_DIR can be used for that, and it can even include an empty segment to represent the ./configure time path. Still, the whole thing is a massive bag of hurt.

@cyrrill

This comment has been minimized.

Show comment
Hide comment
@cyrrill

cyrrill Jul 31, 2017

Any updates on this PR? It's been open for 3 years now...

cyrrill commented Jul 31, 2017

Any updates on this PR? It's been open for 3 years now...

@sepehr

This comment has been minimized.

Show comment
Hide comment
@sepehr

sepehr Jul 31, 2017

Would be great to finally see this live.

sepehr commented Jul 31, 2017

Would be great to finally see this live.

@chris-kruining

This comment has been minimized.

Show comment
Hide comment
@chris-kruining

chris-kruining Nov 6, 2017

Any news? I would love to rewrite my lib with php-cpp to ship as an extension to improve performance :D But I refuse to ship the lib without composer support, just to stay modern and easy-to-use ;)

chris-kruining commented Nov 6, 2017

Any news? I would love to rewrite my lib with php-cpp to ship as an extension to improve performance :D But I refuse to ship the lib without composer support, just to stay modern and easy-to-use ;)

@atrauzzi

This comment has been minimized.

Show comment
Hide comment
@atrauzzi

atrauzzi Mar 20, 2018

Would love to see this completed!

Specifically in my case for: https://github.com/arnaud-lb/php-rdkafka

Would love to see this completed!

Specifically in my case for: https://github.com/arnaud-lb/php-rdkafka

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment