[Bug]: Release artifacts for openvoxdb 8.9.1 have been overwritten

[smortex]

Is this a critical security issue?

• This is not a security issue.

Describe the Bug

The FreeBSD ports are build from the tar.gz artifact of openvoxdb, which is available at this address:
https://artifacts.voxpupuli.org/openvoxdb/8.9.1/openvoxdb-8.9.1.tar.gz

I have been working on FreeBSD ports for the openvox stack for some time, so I downloaded this artifact "a long time ago" (I don't know exactly when), and it is kept cached locally to avoid re-downloading again and again the same file. The FreeBSD ports ensure dowloaded files integrity by comparing the sha265 checksum of the distfile with the expected checksum in a distinfo file.

Yesterday, I pushed my WIP ports to the FreeBSD ports tree. This automatically trigger the build of packages. But today I received an e-mail form the build service telling me the openvoxdb8-8.9.1 and openvoxdb-terminus8-8.9.1 could not be built because the distfile was changed:

https://pkg-status.freebsd.org/beefy21/data/142i386-default/fc1a825d9035/logs/openvoxdb8-8.9.1.log
https://pkg-status.freebsd.org/beefy21/data/142i386-default/fc1a825d9035/logs/openvoxdb-terminus8-8.9.1.log

I re-downloaded the new distfile (to distfiles/openvoxdb-8.9.1.tar.gz) and compared it with the one I downloaded previously (/usr/ports/distfiles/openvoxdb-8.9.1.tar.gz):

$ ll /usr/ports/distfiles/openvoxdb-8.9.1.tar.gz distfiles/openvoxdb-8.9.1.tar.gz
-rw-r--r--  1 root   romain 61667810 19 mars  11:25 /usr/ports/distfiles/openvoxdb-8.9.1.tar.gz
-rw-r--r--  1 romain romain 61674414 19 juil. 01:41 distfiles/openvoxdb-8.9.1.tar.gz
$ sha256 /usr/ports/distfiles/openvoxdb-8.9.1.tar.gz distfiles/openvoxdb-8.9.1.tar.gz 
SHA256 (/usr/ports/distfiles/openvoxdb-8.9.1.tar.gz) = ac4b2ad8de6b95edb2a53d8c85421cd3aa1daafff3499b986a822b68b4186355
SHA256 (distfiles/openvoxdb-8.9.1.tar.gz) = f89190ea331e60ab62bb69f53b5af1aa19350c78aa9ffade89f3dbc7e0aabcbb
$ diff -u <(tar ztf /usr/ports/distfiles/openvoxdb-8.9.1.tar.gz | sort) <(tar ztf distfiles/openvoxdb-8.9.1.tar.gz | sort)
--- /tmp/zshafvl84	2025-07-20 08:13:10.256990000 -1000
+++ /tmp/zshjdLsVc	2025-07-20 08:13:10.497468000 -1000
@@ -9,11 +9,7 @@
 puppetdb-8.9.1/ext/cli/anonymize
 puppetdb-8.9.1/ext/cli/config-migration
 puppetdb-8.9.1/ext/cli/delete-reports
-puppetdb-8.9.1/ext/cli/foreground
-puppetdb-8.9.1/ext/cli/reload
 puppetdb-8.9.1/ext/cli/ssl-setup
-puppetdb-8.9.1/ext/cli/start
-puppetdb-8.9.1/ext/cli/stop
 puppetdb-8.9.1/ext/cli/upgrade
 puppetdb-8.9.1/ext/config/
 puppetdb-8.9.1/ext/config/bootstrap.cfg
@@ -31,7 +27,6 @@
 puppetdb-8.9.1/ext/debian/preinst
 puppetdb-8.9.1/ext/debian/prerm
 puppetdb-8.9.1/ext/debian/puppetdb.default_file
-puppetdb-8.9.1/ext/debian/puppetdb.init_script
 puppetdb-8.9.1/ext/debian/puppetdb.service_file
 puppetdb-8.9.1/ext/default
 puppetdb-8.9.1/ext/docs/
@@ -156,13 +151,8 @@
 puppetdb-8.9.1/ext/ezbake.manifest
 puppetdb-8.9.1/ext/fpm.rb
 puppetdb-8.9.1/ext/project_data.yaml
-puppetdb-8.9.1/ext/puppetdb.logrotate-legacy.conf
 puppetdb-8.9.1/ext/puppetdb.logrotate.conf
-puppetdb-8.9.1/ext/puppetdb.needrestart.conf
-puppetdb-8.9.1/ext/puppetdb.tmpfiles.conf
 puppetdb-8.9.1/ext/redhat/
-puppetdb-8.9.1/ext/redhat/init
-puppetdb-8.9.1/ext/redhat/init.suse
 puppetdb-8.9.1/ext/redhat/postinst
 puppetdb-8.9.1/ext/redhat/postrm
 puppetdb-8.9.1/ext/redhat/preinst
@@ -170,7 +160,6 @@
 puppetdb-8.9.1/ext/redhat/puppetdb.service
 puppetdb-8.9.1/Gemfile
 puppetdb-8.9.1/install.sh
-puppetdb-8.9.1/Makefile
 puppetdb-8.9.1/puppet/
 puppetdb-8.9.1/puppet/face/
 puppetdb-8.9.1/puppet/face/node/

As you can see, the date of the "original" distfile is the date of when the 8.9.1 tag was created, while the "new" one is from yesterday.
These changes seems to be related to OpenVoxProject/ezbake#5 which was merged in may.

In short, we happen to unexpectedly re-generate this distfile and upload it, overwriting the previous file.

Expected Behavior

Distfiles shall never be replaced.

[smortex]

As builds are not reproducible, I uploaded the original distfile here in case we can revert it:
https://people.freebsd.org/~romain/openvoxdb-8.9.1.tar.gz

[smortex]

It looks like the "Build openvoxdb" workflow upload the artifact to s3. The date of this run seems to match the date of the artifact:

https://github.com/OpenVoxProject/openvoxdb/actions/runs/16388252746/job/46310765151

[smortex]

It looks like the "Build openvoxdb" workflow upload the artifact to s3.

s/artifact/artifacts/: all openvoxdb artifacts have been unexpectedly replaced. My proposal is:

1. Remove all new artifacts as they do not reflect what has been tagged: no package is better than wrong packages;
2. If we have original artifacts, re-upload them to revert them: right package is better than no package. I only have the tar.gz I uploaded in [Bug]: Release artifacts for openvoxdb 8.9.1 have been overwritten #31 (comment)
3. Create a new release to supersede the current one which has missing files. We first need to fix this issue (quoting @bastelfreak on IRC):

   In https://github.com/OpenVoxProject/ezbake/pull/2/files we configured a java_bin option for ezbake. the systemd unit erb templates rendered EZBake::Config[:java_bin] to an empty string. So I thought we need to first add it as an official ezbake option, I did that in introduce java_bin ezbake option ezbake#24. now EZBake::Config[:java_bin] is always rendered to /usr/bin/java. I think we mixed up ezbake config options and optparse options in resources/puppetlabs/lein-ezbake/template/global/ext/fpm.rb Can we somehow pass the options from the fpm.rb to the ezbake.service.erb template? Or is the template rendered before resources/puppetlabs/lein-ezbake/template/global/ext/fpm.rb is even executed?
   tl;dr is ezbake doesn't provide a valid ExecStart= option right now for the openvox-server / openvoxdb units and I don't know how to properly fix it

4. Adjust our tooling to make it harder to inadvertently overwrite release files;
5. Setup backups / disaster recovery plan.

[bastelfreak]

Hi,

• I took a backup of https://artifacts.voxpupuli.org/openvoxdb/8.9.1/
• openvoxdb 8.9.1 related files on apt.voxpupuli.org:

# ls -la apt/pool/openvox8/o/openvoxdb*/openvoxdb*_8.9.1*
-rw-rw-r-- 1 www-data www-data 61586432 Jun  3 18:18 apt/pool/openvox8/o/openvoxdb/openvoxdb_8.9.1-1+debian10_all.deb
-rw-rw-r-- 1 www-data www-data 61586432 Jun  3 18:18 apt/pool/openvox8/o/openvoxdb/openvoxdb_8.9.1-1+debian11_all.deb
-rw-rw-r-- 1 www-data www-data 61586420 Jun  3 18:18 apt/pool/openvox8/o/openvoxdb/openvoxdb_8.9.1-1+debian12_all.deb
-rw-rw-r-- 1 www-data www-data 61586448 Jun  3 18:18 apt/pool/openvox8/o/openvoxdb/openvoxdb_8.9.1-1+ubuntu18.04_all.deb
-rw-rw-r-- 1 www-data www-data 61586446 Jun  3 18:18 apt/pool/openvox8/o/openvoxdb/openvoxdb_8.9.1-1+ubuntu20.04_all.deb
-rw-rw-r-- 1 www-data www-data 61586454 Jun  3 18:18 apt/pool/openvox8/o/openvoxdb/openvoxdb_8.9.1-1+ubuntu22.04_all.deb
-rw-rw-r-- 1 www-data www-data 61586450 Jun  3 18:18 apt/pool/openvox8/o/openvoxdb/openvoxdb_8.9.1-1+ubuntu24.04_all.deb
-rw-rw-r-- 1 www-data www-data    22086 Jun  3 18:18 apt/pool/openvox8/o/openvoxdb-termini/openvoxdb-termini_8.9.1-1+debian10_all.deb
-rw-rw-r-- 1 www-data www-data    22090 Jun  3 18:18 apt/pool/openvox8/o/openvoxdb-termini/openvoxdb-termini_8.9.1-1+debian11_all.deb
-rw-rw-r-- 1 www-data www-data    22068 Jun  3 18:18 apt/pool/openvox8/o/openvoxdb-termini/openvoxdb-termini_8.9.1-1+debian12_all.deb
-rw-rw-r-- 1 www-data www-data    22074 Jun  3 18:18 apt/pool/openvox8/o/openvoxdb-termini/openvoxdb-termini_8.9.1-1+ubuntu18.04_all.deb
-rw-rw-r-- 1 www-data www-data    22076 Jun  3 18:18 apt/pool/openvox8/o/openvoxdb-termini/openvoxdb-termini_8.9.1-1+ubuntu20.04_all.deb
-rw-rw-r-- 1 www-data www-data    22092 Jun  3 18:18 apt/pool/openvox8/o/openvoxdb-termini/openvoxdb-termini_8.9.1-1+ubuntu22.04_all.deb
-rw-rw-r-- 1 www-data www-data    22086 Jun  3 18:18 apt/pool/openvox8/o/openvoxdb-termini/openvoxdb-termini_8.9.1-1+ubuntu24.04_all.deb

• openvoxdb 8.9.1 related files on yum.voxpupuli.org:

 # ls -la yum/openvox8/*/*/x86_64/openvoxdb*-8.9.1-1*noarch.rpm
-rw-rw-r-- 1 www-data www-data 61660085 Jun  3 20:05 yum/openvox8/amazon/2023/x86_64/openvoxdb-8.9.1-1.amazon2023.noarch.rpm
-rw-rw-r-- 1 www-data www-data    28900 Jun  3 20:05 yum/openvox8/amazon/2023/x86_64/openvoxdb-termini-8.9.1-1.amazon2023.noarch.rpm
-rw-rw-r-- 1 www-data www-data 61659952 Jun  3 20:06 yum/openvox8/el/10/x86_64/openvoxdb-8.9.1-1.el10.noarch.rpm
-rw-rw-r-- 1 www-data www-data    28881 Jun  3 20:05 yum/openvox8/el/10/x86_64/openvoxdb-termini-8.9.1-1.el10.noarch.rpm
-rw-rw-r-- 1 www-data www-data 61659956 Jun  3 20:06 yum/openvox8/el/7/x86_64/openvoxdb-8.9.1-1.el7.noarch.rpm
-rw-rw-r-- 1 www-data www-data    28876 Jun  3 20:06 yum/openvox8/el/7/x86_64/openvoxdb-termini-8.9.1-1.el7.noarch.rpm
-rw-rw-r-- 1 www-data www-data 61659925 Jun  3 20:06 yum/openvox8/el/8/x86_64/openvoxdb-8.9.1-1.el8.noarch.rpm
-rw-rw-r-- 1 www-data www-data    28875 Jun  3 20:06 yum/openvox8/el/8/x86_64/openvoxdb-termini-8.9.1-1.el8.noarch.rpm
-rw-rw-r-- 1 www-data www-data 61659924 Jun  3 20:07 yum/openvox8/el/9/x86_64/openvoxdb-8.9.1-1.el9.noarch.rpm
-rw-rw-r-- 1 www-data www-data    28877 Jun  3 20:07 yum/openvox8/el/9/x86_64/openvoxdb-termini-8.9.1-1.el9.noarch.rpm
-rw-rw-r-- 1 www-data www-data 61659483 Jun  3 20:08 yum/openvox8/sles/15/x86_64/openvoxdb-8.9.1-1.sles15.noarch.rpm
-rw-rw-r-- 1 www-data www-data    28885 Jun  3 20:07 yum/openvox8/sles/15/x86_64/openvoxdb-termini-8.9.1-1.sles15.noarch.rpm

I copied both file lists back to https://artifacts.voxpupuli.org/openvoxdb/8.9.1/.

before:

# ls -la artifacts/openvoxdb/8.9.1/
total 843004
drwxrwxr-x 2 www-data www-data     4096 Jul 19 12:25 .
drwxrwxr-x 8 www-data www-data     4096 Mär 19 21:25 ..
-rw-rw-r-- 1 www-data www-data 61670341 Jul 19 11:40 openvoxdb-8.9.1-1.amazon2023.noarch.rpm
-rw-rw-r-- 1 www-data www-data 61591506 Jul 19 11:41 openvoxdb_8.9.1-1+debian10_all.deb
-rw-rw-r-- 1 www-data www-data 61591514 Jul 19 11:41 openvoxdb_8.9.1-1+debian11_all.deb
-rw-rw-r-- 1 www-data www-data 61591488 Jul 19 11:41 openvoxdb_8.9.1-1+debian12_all.deb
-rw-rw-r-- 1 www-data www-data 61670215 Jul 19 11:41 openvoxdb-8.9.1-1.el10.noarch.rpm
-rw-rw-r-- 1 www-data www-data 61670210 Jul 19 11:41 openvoxdb-8.9.1-1.el7.noarch.rpm
-rw-rw-r-- 1 www-data www-data 61670217 Jul 19 11:41 openvoxdb-8.9.1-1.el8.noarch.rpm
-rw-rw-r-- 1 www-data www-data 61670216 Jul 19 11:41 openvoxdb-8.9.1-1.el9.noarch.rpm
-rw-rw-r-- 1 www-data www-data 61669756 Jul 19 11:41 openvoxdb-8.9.1-1.sles15.noarch.rpm
-rw-rw-r-- 1 www-data www-data 61591514 Jul 19 11:41 openvoxdb_8.9.1-1+ubuntu18.04_all.deb
-rw-rw-r-- 1 www-data www-data 61591514 Jul 19 11:41 openvoxdb_8.9.1-1+ubuntu20.04_all.deb
-rw-rw-r-- 1 www-data www-data 61591510 Jul 19 11:41 openvoxdb_8.9.1-1+ubuntu22.04_all.deb
-rw-rw-r-- 1 www-data www-data 61591512 Jul 19 11:41 openvoxdb_8.9.1-1+ubuntu24.04_all.deb
-rw-rw-r-- 1 www-data www-data 61674414 Jul 19 11:41 openvoxdb-8.9.1.tar.gz
-rw-rw-r-- 1 www-data www-data    28957 Jul 19 11:40 openvoxdb-termini-8.9.1-1.amazon2023.noarch.rpm
-rw-rw-r-- 1 www-data www-data    21726 Jul 19 11:41 openvoxdb-termini_8.9.1-1+debian10_all.deb
-rw-rw-r-- 1 www-data www-data    21728 Jul 19 11:41 openvoxdb-termini_8.9.1-1+debian11_all.deb
-rw-rw-r-- 1 www-data www-data    21730 Jul 19 11:41 openvoxdb-termini_8.9.1-1+debian12_all.deb
-rw-rw-r-- 1 www-data www-data    28937 Jul 19 11:41 openvoxdb-termini-8.9.1-1.el10.noarch.rpm
-rw-rw-r-- 1 www-data www-data    28931 Jul 19 11:41 openvoxdb-termini-8.9.1-1.el7.noarch.rpm
-rw-rw-r-- 1 www-data www-data    28934 Jul 19 11:41 openvoxdb-termini-8.9.1-1.el8.noarch.rpm
-rw-rw-r-- 1 www-data www-data    28933 Jul 19 11:41 openvoxdb-termini-8.9.1-1.el9.noarch.rpm
-rw-rw-r-- 1 www-data www-data    28942 Jul 19 11:41 openvoxdb-termini-8.9.1-1.sles15.noarch.rpm
-rw-rw-r-- 1 www-data www-data    21734 Jul 19 11:41 openvoxdb-termini_8.9.1-1+ubuntu18.04_all.deb
-rw-rw-r-- 1 www-data www-data    21728 Jul 19 11:41 openvoxdb-termini_8.9.1-1+ubuntu20.04_all.deb
-rw-rw-r-- 1 www-data www-data    21730 Jul 19 11:41 openvoxdb-termini_8.9.1-1+ubuntu22.04_all.deb
-rw-rw-r-- 1 www-data www-data    21722 Jul 19 11:41 openvoxdb-termini_8.9.1-1+ubuntu24.04_all.deb

After:

# ls -la artifacts/openvoxdb/8.9.1/
total 842936
drwxrwxr-x 2 www-data www-data     4096 Jul 19 12:25 .
drwxrwxr-x 8 www-data www-data     4096 Mär 19 21:25 ..
-rw-rw-r-- 1 www-data www-data 61660085 Jul 22 18:28 openvoxdb-8.9.1-1.amazon2023.noarch.rpm
-rw-rw-r-- 1 www-data www-data 61586432 Jul 22 18:28 openvoxdb_8.9.1-1+debian10_all.deb
-rw-rw-r-- 1 www-data www-data 61586432 Jul 22 18:28 openvoxdb_8.9.1-1+debian11_all.deb
-rw-rw-r-- 1 www-data www-data 61586420 Jul 22 18:28 openvoxdb_8.9.1-1+debian12_all.deb
-rw-rw-r-- 1 www-data www-data 61659952 Jul 22 18:28 openvoxdb-8.9.1-1.el10.noarch.rpm
-rw-rw-r-- 1 www-data www-data 61659956 Jul 22 18:28 openvoxdb-8.9.1-1.el7.noarch.rpm
-rw-rw-r-- 1 www-data www-data 61659925 Jul 22 18:28 openvoxdb-8.9.1-1.el8.noarch.rpm
-rw-rw-r-- 1 www-data www-data 61659924 Jul 22 18:28 openvoxdb-8.9.1-1.el9.noarch.rpm
-rw-rw-r-- 1 www-data www-data 61659483 Jul 22 18:28 openvoxdb-8.9.1-1.sles15.noarch.rpm
-rw-rw-r-- 1 www-data www-data 61586448 Jul 22 18:28 openvoxdb_8.9.1-1+ubuntu18.04_all.deb
-rw-rw-r-- 1 www-data www-data 61586446 Jul 22 18:28 openvoxdb_8.9.1-1+ubuntu20.04_all.deb
-rw-rw-r-- 1 www-data www-data 61586454 Jul 22 18:28 openvoxdb_8.9.1-1+ubuntu22.04_all.deb
-rw-rw-r-- 1 www-data www-data 61586450 Jul 22 18:28 openvoxdb_8.9.1-1+ubuntu24.04_all.deb
-rw-rw-r-- 1 www-data www-data 61674414 Jul 19 11:41 openvoxdb-8.9.1.tar.gz
-rw-rw-r-- 1 www-data www-data    28900 Jul 22 18:28 openvoxdb-termini-8.9.1-1.amazon2023.noarch.rpm
-rw-rw-r-- 1 www-data www-data    22086 Jul 22 18:28 openvoxdb-termini_8.9.1-1+debian10_all.deb
-rw-rw-r-- 1 www-data www-data    22090 Jul 22 18:28 openvoxdb-termini_8.9.1-1+debian11_all.deb
-rw-rw-r-- 1 www-data www-data    22068 Jul 22 18:28 openvoxdb-termini_8.9.1-1+debian12_all.deb
-rw-rw-r-- 1 www-data www-data    28881 Jul 22 18:28 openvoxdb-termini-8.9.1-1.el10.noarch.rpm
-rw-rw-r-- 1 www-data www-data    28876 Jul 22 18:28 openvoxdb-termini-8.9.1-1.el7.noarch.rpm
-rw-rw-r-- 1 www-data www-data    28875 Jul 22 18:28 openvoxdb-termini-8.9.1-1.el8.noarch.rpm
-rw-rw-r-- 1 www-data www-data    28877 Jul 22 18:28 openvoxdb-termini-8.9.1-1.el9.noarch.rpm
-rw-rw-r-- 1 www-data www-data    28885 Jul 22 18:28 openvoxdb-termini-8.9.1-1.sles15.noarch.rpm
-rw-rw-r-- 1 www-data www-data    22074 Jul 22 18:28 openvoxdb-termini_8.9.1-1+ubuntu18.04_all.deb
-rw-rw-r-- 1 www-data www-data    22076 Jul 22 18:28 openvoxdb-termini_8.9.1-1+ubuntu20.04_all.deb
-rw-rw-r-- 1 www-data www-data    22092 Jul 22 18:28 openvoxdb-termini_8.9.1-1+ubuntu22.04_all.deb
-rw-rw-r-- 1 www-data www-data    22086 Jul 22 18:28 openvoxdb-termini_8.9.1-1+ubuntu24.04_all.deb

@smortex afterwards I replaced the openvoxdb-8.9.1.tar.gz with your version.

[smortex]

Artifacts uploaded by @bastelfreak have been replaced by the periodic sync.

@nmburgan re-uploaded these files at the right location so that mirrors get the correct files after sync. The sync has happened and we can now download the correct .tar.gz file, and signed packages that have the expected content:

romain@zappy …ects/FreeBSD/ports/databases/openvoxdb8 % make distclean 
===>  Cleaning for openvoxdb8-8.9.1
===>  Deleting distfiles for openvoxdb8-8.9.1
romain@zappy …ects/FreeBSD/ports/databases/openvoxdb8 % make fetch 
===>  License APACHE20 accepted by the user
===>   openvoxdb8-8.9.1 depends on file: /usr/local/sbin/pkg - found
=> openvoxdb-8.9.1.tar.gz doesn't seem to exist in /usr/home/romain/Projects/FreeBSD/ports/distfiles.
=> Attempting to fetch https://artifacts.voxpupuli.org/openvoxdb/8.9.1/openvoxdb-8.9.1.tar.gz
openvoxdb-8.9.1.tar.gz                                  58 MB 1159 kBps    52s
===> Fetching all distfiles required by openvoxdb8-8.9.1 for building
romain@zappy …ects/FreeBSD/ports/databases/openvoxdb8 % 

Thank you!

[bastelfreak]

update here: we have nightly VM snapshots for our mirror. it's just not very convinient to extract single files from it. We now also have a borg based setup for the whole system.