Sort releases in XML files by version instead of date

[mlocati]

I've started this discussion in the pecl-dev mailing list, but no one replied to it 😢 Let's see if we can start the discussion here instead... 😉

We can install PHP extensions by fetching them from the PECL website with

pecl install <package-handle>[-<spec>]

Where can be:

• a stability state (snapshot, devel, alpha, beta, or stable)
• a version marching ^\d+(\.\d+)*([a-zA-Z]+\d*)?$

if <spec> is not specified, it's assumed to be stable

When pecl resolves the actual version to be installed, it fetches http://pecl.php.net/rest/r/<package-handle>/allreleases.xml and returns the first version that's exactly the version specified, or has at least the stability specified (see here).

The versions in the allreleases.xml file are sorted by the release date.

That way, pecl considers as the "last" stable version the most recent one.

This works fine for packages that are published in a "linear" way.

Problems arise when there are multiple versions maintained for a package (grpc and protobuf are good examples).

For example, protobuf maintainers published version 3.21.6 on 2022-09-13 and version 3.18.3 one day later.
So, in its allreleases.xml we have that 3.18.3 comes before 3.21.6, and pecl would have picked up the former instead of the latter.
And this is a big problem: users aren't installing the most recent "stable" version.

I think that the easiest and cleanest solution would be to fix the code that generates the allreleases.xml file: it should sort the versions with the version_compare() PHP function.

PS: is this PR gets merged, someone who has access should regenerate the allreleases.xml files for every package published on the PECL archive (I think by executing the bin/generate-rest.php file).

[mlocati]

The underlying issue happened again today with the protobuf extension: they released version 3.23.0 on 2023-05-08 and version 3.22.5 on 2023-05-10, so its allreleases.xml file lists version 3.22.5 before 3.23.0, and pecl will pick up the former instead of the latter.

[mlocati]

@derickr so, this PR has been merged, right?

Also, reading for example https://pecl.php.net/rest/r/protobuf/allreleases.xml, it seems you already executed bin/generate-rest.php, right?

[derickr]

Hi,

Yes, this PR has been merged, and I did run bin/generate-rest.php. That was however not an easy task!
When running the script, it started randomly halting at the gnupg package, with comments that it couldn't read the temporary XML file that it created. This was odd, as there was no difference between the package.xml file for that specific release, as any of the others. And curiously, everything after that also started failing. After I can it for just GnuPG, it worked without problems.

In order to debug this, I wanted to run strace to see what was going wrong. However, the machine ran on an old version of Debian, and it refused to even download the latest source lists. I therefore had to upgrade Debian on the machine first, before I could even install strace.

Once done, I then ran strace and noticed the following:

access("/tmp/pear/temp/pearljw6B3/package.xml", F_OK) = -1 ENOENT (No such file or directory)
stat("/tmp/pear/temp/pearljw6B3", {st_mode=S_IFDIR|0700, st_size=4096, ...}) = 0
lstat("/tmp/pear/temp/pearljw6B3/package.xml", 0x7fffaf874250) = -1 ENOENT (No such file or directory)
lstat("/tmp/pear/temp/pearljw6B3", {st_mode=S_IFDIR|0700, st_size=4096, ...}) = 0
lstat("/tmp/pear/temp", {st_mode=S_IFDIR|0755, st_size=36864, ...}) = 0
lstat("/tmp/pear", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat("/tmp", {st_mode=S_IFDIR|S_ISVTX|0777, st_size=163840, ...}) = 0
openat(AT_FDCWD, "/tmp/pear/temp/pearljw6B3/package.xml", O_WRONLY|O_CREAT|O_TRUNC, 0666) = -1 EMFILE (Too many open files)

It turned out that there is some bug in the PEAR/PECL installer, where it doesn't close file handles… a file handle leak so to speak.

I solved it by setting the number of allowed open files to a higher number with ulimit -n, and then it ran fine. I bet that in the past (2010 :-) ) this wasn't an issue, as there were not enough releases yet to run into this!

cheers,
Derick

[mlocati]

@derickr the PHP community is very lucky to count you among its members.

Thank you!!!