php newb seeing memory exhaustion composer-requiring into snipe-it

[bby-bishopclark]

Hi everyone,

Please, can you reassure me this bloat I'm seeing is normal? As mentioned, I'm a composer newbie, having done PHP dirty work in the php3 days and not much since.

I'm trying to build Snipe-IT with this library built-in, and I'm seeing memory exhaustion each time. I'm sure I'm invoking it wrongly, and that a simple correction and light heckling will get me from cursing to learning.

Plot twist. As per ISO27002:12.1.4e , we build RPM artefacts and install those to the test and then prod environments, so processes may look funny. Also, we're running dorky certs in a MiTM setup for SSL 'inspection', and that causes some SSL connections to barf eg when they're using HSTS properly.

When building my installable, I'm running a snippet like

%prep
%autosetup -p0 -n %{name}-%{?ghsha}%{?!ghsha:%{version}}
ls -ld composer.json
composer require "paragonie/sodium_compat":"^1.17"   #<-- trivial addition
composer install --no-dev --working-dir .

which launders to

Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.3MiQA9
+ umask 022
+ cd /dev/shm/BUILD
+ cd /dev/shm/BUILD
+ rm -rf snipe-it-5.3.10
+ /usr/bin/gzip -dc /home/gitlab-runner/builds/HbvDxSny/0/packaging/rpms/sources/snipe-it53/v5.3.10.tar.gz
+ /usr/bin/tar -xof -
+ STATUS=0
+ '[' 0 -ne 0 ']'
+ cd snipe-it-5.3.10
+ /usr/bin/chmod -Rf a+rX,u+w,g-w,o-w .
+ ls -ld composer.json
-rw-r--r--. 1 gitlab-runner gitlab-runner 3261 Feb 13  2022 composer.json
+ composer require 'paragonie/sodium_compat:^1.17'
Warning from https://repo.packagist.org: Support for Composer 1 is deprecated and some packages will not be available. You should upgrade to Composer 2. See https://blog.packagist.com/deprecating-composer-1-support/
Info from https://repo.packagist.org: #StandWithUkraine
./composer.json has been updated
Loading composer repositories with package information
Warning from https://repo.packagist.org: Support for Composer 1 is deprecated and some packages will not be available. You should upgrade to Composer 2. See https://blog.packagist.com/deprecating-composer-1-support/
Info from https://repo.packagist.org: #StandWithUkraine
Updating dependencies (including require-dev)
PHP Fatal error:  Allowed memory size of 1610612736 bytes exhausted (tried to allocate 335[54](https://prdgit01.city.burnaby.bc.ca.proxy/packaging/rpms/-/jobs/24326#L54)432 bytes) in /usr/share/php/Composer/DependencyResolver/RuleSetGenerator.php on line 129
Fatal error: Allowed memory size of 1610612736 bytes exhausted (tried to allocate 33[55](https://prdgit01.city.burnaby.bc.ca.proxy/packaging/rpms/-/jobs/24326#L55)4432 bytes) in /usr/share/php/Composer/DependencyResolver/RuleSetGenerator.php on line 129
Check https://getcomposer.org/doc/articles/troubleshooting.md#memory-limit-errors for more info on how to handle out of memory errors.error: Bad exit status from /var/tmp/rpm-tmp.3MiQA9 (%prep)
RPM build errors:
    Bad exit status from /var/tmp/rpm-tmp.3MiQA9 (%prep)

What I suspect from the barf above:

• Yes, I'm building in the RAMDisk /dev/shm on this 2gb buildmonkey
• I'm grabbing valid (older) source ball for the other project
• I'm in the right location, I think, as the composer.json's right there
• maybe I'm doing the composer-require invocation right?
• suddenly we need 1.5Gb space to hold this project, where previously .5gb was okay

The current build env:

[gitlab-runner@rocky8 ~]$ rpm -qf /etc/issue /etc/php.d/ `which composer`
rocky-release-8.7-1.2.el8.noarch
php-common-7.2.24-1.module+el8.4.0+413+c9202dda.x86_64
composer-1.10.26-1.el7.noarch  #[sic]

[gitlab-runner@rocky8 ~]$ sudo virt-what
virtualbox
kvm

[gitlab-runner@rocky8 ~]$ free -m
              total        used        free      shared  buff/cache   available
Mem:           1817         107        1287          17         422        1548
Swap:          2107         198        1909

Questions:

• Is all this expected?
• Can I get trivial confirmation I'm adding this in right?
• Is 1gb additional space expected, and should I (fill out the bizarre forms to instantiate and) push this build onto a really beefy SSD box with buckets of RAM to get through it and be okay?

[paragonie-security]

No, it's not expected.

Have you tried with Composer 2? The updates they made in the new major version may alleviate some of the memory usage issues you're seeing.

[bby-bishopclark]

Have you tried with Composer 2?

It's tricky. The enterprise load includes NO composer for PHP on this 'EL8' OS release. Really! None! So any solution flirts with the kind of supply chain issues we're trying to avoid. (We usually can get by from a common distro-adjacent channel, but it didn't have composer either; I know!)

But hey. I found one, and it worked quickly and without issue, just it's third-party and risky. So I have a solvable problem much more in my portfolio to resolve, and that's a great place to be.

Thanks for the spot of clarity when I needed it. Your simple question was exactly the right hint at the right time.