TestSuite.test_psnr fails on s390x

[musicinmybrain]

On s390x, TestSuite.test_psnr fails:

=================================== FAILURES ===================================
_____________________________ TestSuite.test_psnr ______________________________
self = <tests.test_tiff2jp2.TestSuite testMethod=test_psnr>
    def test_psnr(self):
        """
        SCENARIO:  Convert TIFF file to JP2 with the irreversible transform.
    
        EXPECTED RESULT:  data matches, the irreversible transform is confirmed
        """
        with Tiff2Jp2k(
            self.minisblack_spp1_path, self.temp_jp2_filename,
            psnr=(30, 35, 40, 0)
        ) as j:
            j.run()
    
        j = Jp2k(self.temp_jp2_filename)
    
        d = {}
        for layer in range(4):
            j.layer = layer
            d[layer] = j[:]
    
        with warnings.catch_warnings():
            # MSE is zero for that first image, resulting in a divide-by-zero
            # warning
            warnings.simplefilter('ignore')
            psnr = [
                fixtures.skimage.metrics.peak_signal_noise_ratio(
                    fixtures.skimage.data.moon(), d[j]
                )
                for j in range(4)
            ]
    
        # That first image should be lossless.
>       self.assertTrue(np.isinf(psnr[0]))
E       AssertionError: False is not true
tests/test_tiff2jp2.py:584: AssertionError
=========================== short test summary info ============================
FAILED tests/test_tiff2jp2.py::TestSuite::test_psnr - AssertionError: False i...
================== 1 failed, 528 passed, 4 skipped in 49.92s ===================

I haven’t yet investigated this closely enough to form a theory about what is going wrong on this platform.

Since s390x is the only big-endian primary architecture in Fedora Linux, this bug could be endian-related.

[quintusdias]

So I'm working on it... tougher than expected to get a s390x virtual machine up and running

[musicinmybrain]

I had been hoping to offer you some additional detail, like the actual image data from this test, but I haven’t had enough time to revisit this bug yet.

Still, I can do interactive testing in qemu-user-static emulation quite easily, and non-interactive testing on real hardware via RPM scratch builds on Fedora infrastructure, if that helps. Just let me know what to try.

[musicinmybrain]

I ran the failing test in qemu-user-static emulation with a patch applied to write out the values of fixtures.skimage.data.moon() and d[j] for each j to JSON. I am attaching the results to this comment. Hopefully that will prove helpful.

moon.txt
d_0.txt
d_1.txt
d_2.txt
d_3.txt

[quintusdias]

How exactly did you get the emulation working?!? Everytime I've tried that I've failed.

…

On Tue, Aug 2, 2022 at 12:13 PM Ben Beasley ***@***.***> wrote: I ran the failing test in qemu-user-static emulation with a patch applied to write out the values of fixtures.skimage.data.moon() and d[j] for each j to JSON. I am attaching the results to this comment. Hopefully that will prove helpful. moon.txt <https://github.com/quintusdias/glymur/files/9244683/moon.txt> d_0.txt <https://github.com/quintusdias/glymur/files/9244685/d_0.txt> d_1.txt <https://github.com/quintusdias/glymur/files/9244686/d_1.txt> d_2.txt <https://github.com/quintusdias/glymur/files/9244687/d_2.txt> d_3.txt <https://github.com/quintusdias/glymur/files/9244688/d_3.txt> — Reply to this email directly, view it on GitHub <#546 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAVCIXT4VOGTL7IKCFG7HV3VXFQMDANCNFSM5RRI46SA> . You are receiving this because you commented.Message ID: ***@***.***>

-- John Evans

[musicinmybrain]

How exactly did you get the emulation working?!? Everytime I've tried that I've failed.

I’m working in Fedora 36 (bare metal install, but a VM would work just as well) and taking advantage of the tooling for working on Fedora packages.

• sudo dnf install fedora-packager
• Set up mock, i.e, sudo dnf install mock, sudo usermod -G mock -a $(whoami), log-out/log-in to refresh supplementary groups.
• sudo dnf install qemu-user-static
• fedpkg co python-glymur or, without a Fedora Account System account, git clone https://src.fedoraproject.org/rpms/python-glymur.git
• cd python-glymur
• Edit the spec file to stop skipping the failing test
• Add any patches
• fedpkg --release rawhide mockbuild --root fedora-rawhide-s390x

I can also do interactive tests in the emulated chroot, like:

$ mock -r fedora-rawhide-s390x -i python3-glymur
$ mock -r fedora-rawhide-s390x --shell
<mock-chroot> sh-5.1# python3
>>> import glymur
…

Or, I can do

$ mock -r fedora-rawhide-s390x -i $LOTS_OF_PACKAGES_I_WILL_NEED
$ mock -r fedora-rawhide-s390x --shell --enable-network

and run git clone in the chroot and then use a more typical Python workflow.

In theory you should be able to do all of these things too if you like. The approach is a little RPM-packaging-centric, but not too difficult to get working.

[quintusdias]

I can verify that the failing image produced by glymur on s390x is WAAAYYYY off... Before resorting to looking at the openjpeg test suite to see if it differs from s390x to x86_64, it will be worth the effort to try to see why none of the other PSNR tests in glymur fail.

[quintusdias]

So I've been trying to get an s390x VM up and running myself. No luck at all via virt-manager, it just locks up. Yesterday I had some better luck with just command-line qemu, but have been hitting errors just before the installation finishes. I'll keep pounding on it.

…

On Wed, Apr 6, 2022 at 7:28 AM Ben Beasley ***@***.***> wrote: I had been hoping to offer you some additional detail, like the actual image data from this test, but I haven’t had enough time to revisit this bug yet. Still, I can do interactive testing in qemu-user-static emulation quite easily, and non-interactive testing on real hardware via RPM scratch builds on Fedora infrastructure, if that helps. Just let me know what to try. — Reply to this email directly, view it on GitHub <#546 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAVCIXQCIDKKAOLB2OKSGH3VDVYOJANCNFSM5RRI46SA> . You are receiving this because you commented.Message ID: ***@***.***>

-- John Evans

[sharkcz]

@quintusdias , please send me an email to sharkcz [at] fedoraproject.org and I can provide you access to a real s390x hw.

[sharkcz]

@quintusdias, sent you an email, but please check your spam box, google often doesn't like me ...

[quintusdias]

This issue turned out to be far more convoluted than I could have imagined, with at least 4 issues at play. It seems to be fixed now, fixed "accidentally" as it would seem.

It's a bit of oversimplification, but when running the PSNR test, an unexpected but still valid code path was being taken, the one thru the RGBA interface. Fixing the RGBA interface issue (#603) fixes this issue. PSNR really didn't have anything to do with it.

I think that all that needs to be done to close this issue is to stop skipping the PSNR tests on big-endian architectures, add a lot of internal comments that fully explain what's going on, and think a bit on whether more tests are needed. This should be resolved soon.

[quintusdias]

I think I can close this now. Please re-open if you see fit. My apologies for this staying open as long as it did.