rpm 4.16.90 breaks perl-RPM4's testsuite (actually it's the sqlite backend)

[soig]

rpm 4.16.90 breaks perl-RPM4's testsuite.
I've tested with the fix for rpmv3 packages which makes no changes.
(It does however fix urpmi testsuite)

• Rollbacking to 4.16.1.3 (with default bdb backend) fixes it.
• Rollbacking to 4.16.1.3 with sqlite as default backend shows the same errors.

So it's the switch from the bdb to the sqlite backend that is the root cause.
RPM4 can be downloaded from:
https://metacpan.org/release/RPM4

Just:

1. unpack it
2. run "perl Makefile.PL"
3. run "make"
4. run "make test"

You can duplicate the offending bits by running:
perl -Iblib/lib/ -Iblib/arch/ t/05transaction.t

p2.diff.txt is a patch for forcing the BDB backend (which will only works with 4.16.1.3 obviously)
p1.diff.txt is the resulting diff in the test output.
We gain warning about "using deprecated bdb database backend" but all tests now pass.

dbg.diff.txt is a patch against the testsuite in order to display more info.
dbg.txt is the output of the t/05transaction.t with that patch.
It shows that the pkg is in the transaction.
But when the backend is sqlite, the package hasn't been installed: it doesn't show up in rpmdb!
(see the debug lines beginning with ">> "

One could argue that the manner of closing the db is an issue:
See https://metacpan.org/source/TVIGNAUD/RPM4-0.44/t/05transaction.t#L95
But I think this uncover a can of potential affected users in the wild.

dbg.txt
dbg.diff.txt
p2.diff.txt
p1.diff.txt

[pmatilai]

There must be something rather special about the usage in perl-RPM4 testsuite, because "normally" sqlite backend works just fine, it's in fairly wide real-world use now. Have you tested ndb?

[soig]

Strange, the same error happens with nbd backend too.
What's puzzling is that URPM::Transaction->DESTROY and RPM4::Transaction->DESTROY looks similar and both free the transaction:

• https://metacpan.org/release/URPM/source/URPM.xs#L2582
• https://metacpan.org/source/TVIGNAUD/RPM4-0.44/src/RPM4.xs#L1417

After looking again, the bug is the reverse of what I though:

• ndb/sqlite behaviour is sane as the transaction was run with JUSTDB!
• It's actually the bdb backend that exposes a bogus behaviour as the pkg can be found in the db whereas the transaction was run with JUSTDB flag!

See https://metacpan.org/source/TVIGNAUD/RPM4-0.44/t/05transaction.t#L79

I've inherited that code that I just maintain (because other tools rely on it) since the switch from jbj rpm 4.4 to rpm.org 4.6 but there's definitive oddness here!

dbg3.txt

[pmatilai]

JUSTDB does not affect database operations in any way, it's the everything else that it affects.

The header gets added to the database just fine, the problem appears to be in the iteration side. I don't know what the purpose of this condition in Ts_traverse() is but commenting out makes the transaction tests pass:

                if (tag == RPMDBI_PACKAGES && value != NULL) {
                    rc = 0;
                } else if (count == 1) {
                    rc = POPi;
                }

Not saying commenting it out is the right thing to do, if this works with bdb then it suggests we have a subtle behavior difference somewhere, and we need to understand the case better.

[pmatilai]

Doh, sorry for the accidental close. Managed to once again hit the wrong comment button in GH 🙄

[pmatilai]

Okay, the issue is related to database retrieval order. In this case, bdb consistently returns first the header that was added last (test-rpm-1.0-1mdk.noarch, offset 2) whereas ndb and sqlite return first the header that was added first (gpg-pubkey-a1df5c48-40e92691, offset 1). The order of stuff coming from RPMDBI_PACKAGES is undefined so there's an invalid assumption someplace, but I don't know enough Perl to determine whether it's the test or the traverse code in the Perl bindings that is wrong. I'd assume the test, because the order is generally undefined under bdb too.

[pmatilai]

So... I think we can safely conclude that this is not a bug in rpm, despite it initially seeming that way.
Thanks for reporting though.