Audit & Clean roles' PHP 7.x & 8.0 PHP module dependencies [MediaWiki, Moodle, MySQL, Nextcloud, NGINX, FreePBX]

[holta]

Tested on 32-bit RaspiOS Lite (PHP 7.3), Ubuntu Server 21.04 (PHP 7.4) and Ubuntu Server 21.10 pre-release (PHP 8.0).

Explanations are added to push role authors to be aware of AND declare their own evolving PHP dependencies — instead of blindly assuming things that were installed MANY years ago for "God knows what reason" (!) but too often nobody seems to remember why.

Forcing apps/services to live on their own 2 feet helps IIAB implementers installing diverse and unexpected sets of apps/services, rather than the proverbial kitchen sink (BIG-sized) which might make for really great "Get to Know" demos — but is just not appropriate in the field.

PHP modules that had no good reason to be there were removed. As we all have enough security/complexity nightmares in our lives as it stands.

Further testing is ongoing for the heck of it.

FYI this PR includes and supersedes PR #2830 to accommodate PHP 8.0's minor changes.

ASIDE: Admin Console should declare its own dependency on php-pear for Line 19 of https://github.com/iiab/iiab-admin-console/blob/master/roles/cmdsrv/tasks/main.yml#19 (likely by installing apt package php-pear in the stanza just above that). An interim patch is included until that happens: https://github.com/holta/iiab/blob/php7-and-php8-cleanup/roles/nginx/tasks/install.yml#L15 (LATER MOVED TO https://github.com/holta/iiab/blob/php7-and-php8-cleanup/roles/9-local-addons/tasks/main.yml#L33-L36)

Somewhat Related:

• PR Initial stab at nextcloud playbook running on nginx #2119
• PR @m-anish plz help us come to a decision to get Nextcloud's php-* dependencies in place [non-php pkgs might also be added/removed?] #2258
• PR Nextcloud 21 recommends php{{ php_version }}-bcmath #2684
• Testing IIAB with PHP 8.0, part of new OS's like Ubuntu 21.10 pre-releases #2814
• Test IIAB on Ubuntu 21.10 (RPi+PC) x (Server+Desktop). Does RPi WiFi firmware 7.45.18.0 work well? [OS released 2021-10-14] [JupyterHub install pip fails intermittently?] #2818
• Moodle failed to install repeatedly: possibly automate the workaround in future? #2829

[holta]

Also tested with:

• MIN-sized fresh install on 32-bit RaspiOS on RPi 4.
• MEDIUM-sized fresh install on 32-bit RaspiOS on RPi 4.
• BIG-sized fresh install on 32-bit RaspiOS on RPi 4.
• A stripped-down /etc/iiab/local_vars.yml that contained essentially no apps except for Moodle. Fresh-installed onto Ubuntu Desktop 21.04 in this case.

[jvonau]

28279bd time would be better spent on dropping apache as a dependency for moodle than adding this spaghetti code that would go away when apache does. Have you even tried https://github.com/iiab/iiab/blob/master/roles/moodle/templates/moodle-nginx.conf.j2.native as discussed in #2762?

[jvonau]

ubuntu@test:~$ apt depends php-imagick
php-imagick
PreDepends: php-common (>= 2:69~)
Depends:
libapache2-mod-php7.4
libphp7.4-embed
php7.4-cgi
php7.4-cli
php7.4-fpm
php7.4-phpdbg
Depends: libc6 (>= 2.17)
Depends: libmagickcore-6.q16-6 (>= 8:6.9.10.2)
Depends: libmagickwand-6.q16-6 (>= 8:6.9.10.2)
Recommends: ghostscript
Recommends:

Given php-imagick drags in apache perhaps omitting php-imagick and in its place use the dependencies list: libmagickcore, libmagickwand, libphpX.Y-embed, phpX.Y-cgi, phpX.Y-cli, phpX.Y-fpm, phpX.Y-phpdbg? This was not an issue in the past when apache was the primary webserver but with the switch to nginx better defined dependencies might be the way to go?

This is more or less the approach that moodlebox, which is running nginx, uses where php-imagick is not installed at all and ghostscript looks like it replaces the libmagick* libraries.

[holta]

Great news IIAB Apps like the following can now be installed truly independently (i.e. without blindly throwing the entire Kitchen Sink at dependency problems) which up until now was impossible:

• MediaWiki (and essentially nothing else) in local_vars.yml
• Moodle (and essentially nothing else) in local_vars.yml
• Nextcloud (and essentially nothing else) in local_vars.yml

In short: the above 3 IIAB fresh install tests (and many similar, especially on the latest pre-release of Ubuntu Server 21.10) confirm that IIAB Apps/Services' dependencies are becoming increasingly coherent — towards helping all contributors work together to sanity-check and put an end to Dependency Hell Spaghetti — allowing for increasingly "antifragile" roles/apps/services going forward.

[holta]

This is more or less the approach that moodlebox, which is running nginx, uses where php-imagick is not installed at all and ghostscript looks like it replaces the libmagick* libraries.

Such an approach could be extremely promising.

Not that we can solve all of Nextcloud's "codependency" nightmares — and likewise Moodle's, etc — when so many of these products have very nebulous/shifty and imprecise (sometimes downright contradictory ;) specs around what PHP modules (etc) they truly require.

Some are more forthcoming than others for sure (e.g. at least Nextcloud admits that php-imagick Ain't Paradise, doesn't work with SVG's, etc). And certainly nobody is saying that "budget reconciliations" — i.e. dependency specs spelled out rather precisely, and maintained with sincerity — are a walk in the park.

But if http://MoodleBox.net has done solid work reconciling some of these mystery meat / potluck soups, let's definitely investigate and/or collaborate with open education community leaders like this.

[holta]

I wouldn't have believed it staring at this 4 days ago. But in the end it's certainly edifying to have learned that almost every line below from www_base/tasks/main.yml was misleading — providing a quite false sense of security and structure — arising from years of well-meaning but increasingly bogus and bitrotting assumptions:

- name: 'Install ~10 PHP packages (debuntu)'
  package:
    name:
      - php{{ php_version }}-common
      - php{{ php_version }}-curl
      - php{{ php_version }}-gd
      - php{{ php_version }}-imap
      - php{{ php_version }}-ldap
      - php{{ php_version }}-mysql
      - php{{ php_version }}-odbc
      - php-pear
      - php{{ php_version }}-xml
      - php{{ php_version }}-xmlrpc
    state: present

This PR is certainly not the final word to "Spring Cleaning" CODEpendencies writ large. But it does clean out several years of abandonware (disentangling & clarifying PHP dependencies for common IIAB Apps/Sevices). So I'm merging it after having tested it intensively on 32-bit Raspberry Pi OS (PHP 7.3), Ubuntu 21.04 (PHP 7.4) and Ubuntu 21.10 (PHP 8.0) — including Desktop and Server versions — with an assortment of different local_vars.yml files to get at some very different scenarios.

At least we now have a substantially more sane foundation, to better understanding what depends on what going forward. So IIAB App authors (and maintainers, most important!) can be more in tune with underlying dependencies, assumptions, reliability, and ever-changing ground truths just over the horizon~

ASIDE: in the 3.5 months until Ubuntu 21.10 is released on 2021-10-14, here are a couple IIAB App glitches we might keep an eye on:

• Elgg, Lokole [also on 21.04] & Node-RED [nodered] don't yet work on Ubuntu 21.10 [Impish Indri] ? #2833

[holta]

One could remove uwsgi & uwsgi-plugin-python3 from nginx, add to captiveportal and...

Things like that sound incredibly reasonable. However it happens.

e.g. roles/captiveportal and Admin Console (etc) should really install (or at least declare) their own dependencies a.k.a. requirements.

And not just to keep things from spinning out of control as months become years. But also (equally important) to provide an inviting/open/understandable IIAB platform for devs and schools/communities to get to know and trust, where we can.

new stage9 entry [...] should be conditional on admin_console_install

Done: https://github.com/holta/iiab/blob/php7-and-php8-cleanup/roles/9-local-addons/tasks/main.yml#L33-L37

[jvonau]

Isn't wordpress a heavy user of php? I don't see any dependencies listed within wordpress's role.

[jvonau]

https://wpassist.me/wordpress-required-php-extensions/ has bundled/external notes

[holta]

Isn't wordpress a heavy user of php? I don't see any dependencies listed within wordpress's role.

Great question. Unfortunately I don't know the answer. In the past I had the impression WordPress has all its own PHP dependencies built in here somehow??

root@box:/library/wordpress# ls
index.php           wp-comments-post.php  wp-keys.php        wp-settings.php
license.txt         wp-config.php         wp-keys.php.BAK    wp-signup.php
readme.html         wp-config-sample.php  wp-links-opml.php  wp-trackback.php
wp-activate.php     wp-content            wp-load.php        xmlrpc.php
wp-admin            wp-cron.php           wp-login.php
wp-blog-header.php  wp-includes           wp-mail.php

But that may be nonsense. I don't know. Great if we can get an answer.

[holta]

/library/wordpress contains these 3 directories FWIW:

root@box:/library/wordpress# ls -p | grep / | xargs du -hsc
10M     wp-admin/
8.1M    wp-content/
39M     wp-includes/
57M     total

[holta]

Note to self; Nextcloud 22 will be released a week from tomorrow, and should update its PHP requirements here:

• https://docs.nextcloud.com/server/21/admin_manual/installation/source_installation.html#prerequisites-for-manual-installation
• https://docs.nextcloud.com/server/22/admin_manual/installation/source_installation.html#prerequisites-for-manual-installation

[jvonau]

Isn't wordpress a heavy user of php? I don't see any dependencies listed within wordpress's role.

Great question. Unfortunately I don't know the answer. In the past I had the impression WordPress has all its own PHP dependencies built in here somehow??

root@box:/library/wordpress# ls
index.php           wp-comments-post.php  wp-keys.php        wp-settings.php
license.txt         wp-config.php         wp-keys.php.BAK    wp-signup.php
readme.html         wp-config-sample.php  wp-links-opml.php  wp-trackback.php
wp-activate.php     wp-content            wp-load.php        xmlrpc.php
wp-admin            wp-cron.php           wp-login.php
wp-blog-header.php  wp-includes           wp-mail.php

But that may be nonsense. I don't know. Great if we can get an answer.

Why would you remove the old bloated php install if you don't know how it would affect all the roles that use php?

https://wpassist.me/wordpress-required-php-extensions/ has bundled/external notes

The list from #2832 (comment) and the above link looks suspiciously overlapping.

[jvonau]

Straight from upstream https://make.wordpress.org/hosting/handbook/server-environment/

[holta]

https://wpassist.me/wordpress-required-php-extensions/ has bundled/external notes

FWIW the above 3-way breakdown between core/bundled/external breakdown doesn't really correspond to reality on Ubuntu 21.04 and other distros as well.

Certainly WordPress works as part of IIAB as is.

Still, if one wanted to follow the above story as a starting point to support a richer set of WordPress plugins out-of-the-box (e.g. taking "Critical PHP Extensions" at face value?) then these 4 would/could easily be added to IIAB's wordpress/tasks/install.yml :

• phpX.Y-curl
• phpX.Y-mbstring
• php-imagick
• phpX.Y-zip

And if the above story is truly meaningful, then these 2 (already installed, as we discovered over recent days) could/should also be declared within IIAB's wordpress/tasks/install.yml, to make the accounting more honest:

• phpX.Y-mysql
• phpX.Y-xml

Finally, among the above so-called "Optional Extensions" everything's already covered except for gd, so this also can be added if it's universally appealing for IIAB's WordPress implementers:

• phpX.Y-gd

But it sounds like we should talk to more authoritative WordPress professionals first? e.g. experienced IIAB field implementers to ground-truth the most useful PHP modules for WordPress.

[holta]

Straight from upstream https://make.wordpress.org/hosting/handbook/server-environment/

Thanks sending that. I was about to look for that.