Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ClamAV 1.0.0-rc2 test failures on s390x #759

Open
opoplawski opened this issue Nov 17, 2022 · 5 comments
Open

ClamAV 1.0.0-rc2 test failures on s390x #759

opoplawski opened this issue Nov 17, 2022 · 5 comments

Comments

@opoplawski
Copy link
Contributor

Describe the bug

We are working on updating clamav to 1.0.0-rc2 in Fedora. We are getting the following test failure on s390x. It seems like 'Test.Import.Hash.UNOFFICIAL FOUND' is not being detected.

_______________________ TC.test_exe_imphash_plus_zipsfx ________________________
self = <allmatch_test.TC testMethod=test_exe_imphash_plus_zipsfx>
    def test_exe_imphash_plus_zipsfx(self):
        self.step_name('Test that clam will detect a string in text file, plus identify, extract, and alert on concatenated clam.zip containing clam.exe with an imp-hash sig.')
    
        # We can't use the hash sig for this clam.exe program because the hash goes out the window when we concatenate on the zip.
        (TC.path_tmp / 'clam.imp').write_text(
            "98c88d882f01a3f6ac1e5f7dfd761624:39:Test.Import.Hash\n"
        )
    
        # Build a file that is the clam.exe program with a zip concatinated on that contains the not_eicar test string file.
        clam_exe = TC.path_build / 'unit_tests' / 'input' / 'clamav_hdb_scanfiles' / 'clam.exe'
    
        not_eicar_zip = TC.path_tmp / 'not-eicar.zip'
        with ZipFile(str(not_eicar_zip), 'w', ZIP_DEFLATED) as zf:
            zf.writestr('not-eicar.txt', b"CLAMAV-TEST-STRING-NOT-EICAR")
    
        testfile = TC.path_tmp / 'clam.exe.not_eicar.zipsfx'
        testfile.write_bytes(clam_exe.read_bytes() + not_eicar_zip.read_bytes())
    
        command = '{valgrind} {valgrind_args} {clamscan} -d {clam_exe_db} -d {not_eicar_db} --allmatch {testfiles}'.format(
            valgrind=TC.valgrind, valgrind_args=TC.valgrind_args, clamscan=TC.clamscan,
            clam_exe_db=TC.path_tmp / 'clam.imp',
            not_eicar_db=TC.path_source / 'unit_tests' / 'input' / 'other_sigs' / 'Clamav-Unit-Test-Signature.ndb',
            testfiles=testfile,
        )
        output = self.execute_command(command)
    
        assert output.ec == 1  # virus
    
        expected_results = [
            'Test.Import.Hash.UNOFFICIAL FOUND',
            'NDB.Clamav-Unit-Test-Signature.UNOFFICIAL FOUND',
        ]
>       self.verify_output(output.out, expected=expected_results)
/builddir/build/BUILD/clamav-1.0.0-rc2/unit_tests/clamscan/allmatch_test.py:244: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/builddir/build/BUILD/clamav-1.0.0-rc2/unit_tests/testcase.py:248: in verify_output
    log_checker.verify_expected_output(expected, text, order=order)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <testcase.LogChecker object at 0x3ffa56f8b90>
expected_items = ['Test.Import.Hash.UNOFFICIAL FOUND', 'NDB.Clamav-Unit-Test-Signature.UNOFFICIAL FOUND']
output = '/builddir/build/BUILD/clamav-1.0.0-rc2/redhat-linux-build/unit_tests/TC-plyrzssf/clam.exe.not_eicar.zipsfx: NDB.Clama... 0.00 MB (ratio 0.00:1)\nTime: 0.005 sec (0 m 0 s)\nStart Date: 2022:11:17 00:22:02\nEnd Date:   2022:11:17 00:22:02\n'
order = 1
    def verify_expected_output(self, expected_items, output, order=STRICT_ORDER):
        """Check presence of regex patterns in output string.
    
        :Parameters:
            - `expected_items`: a list of regex patterns that should be found
                                in `output`.
            - `output`: a string with output to verify.
            - `order`: STRICT_ORDER, ANY_ORDER.
    
        :Exceptions:
            - `AssertionError`: is raised if:
                1)`output` is not a string.
                2) one of expected items was not found in `output`.
                3) items were found in wrong order.
        """
        if output != None and not isinstance(output, str):
            output = output.decode("utf-8", "ignore")
        assert isinstance(output, str), "`output` must be a string."
        expected_items = self._prepare_value(expected_items)
    
        last_found_position = 0
        for item in expected_items:
            pattern = re.compile(item)
            match = pattern.search(output)
>           assert match, "Expected item `%s` not found in output:\n%s" % (
                item,
                output,
            )
E           AssertionError: Expected item `Test.Import.Hash.UNOFFICIAL FOUND` not found in output:
E           /builddir/build/BUILD/clamav-1.0.0-rc2/redhat-linux-build/unit_tests/TC-plyrzssf/clam.exe.not_eicar.zipsfx: NDB.Clamav-Unit-Test-Signature.UNOFFICIAL FOUND
E           
E           ----------- SCAN SUMMARY -----------
E           Known viruses: 2
E           Engine version: 1.0.0-rc2
E           Scanned directories: 0
E           Scanned files: 1
E           Infected files: 1
E           Data scanned: 0.00 MB
E           Data read: 0.00 MB (ratio 0.00:1)
E           Time: 0.005 sec (0 m 0 s)
E           Start Date: 2022:11:17 00:22:02
E           End Date:   2022:11:17 00:22:02
/builddir/build/BUILD/clamav-1.0.0-rc2/unit_tests/testcase.py:794: AssertionError
----------------------------- Captured stdout call -----------------------------
[INFO]: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[INFO]: Test that clam will detect a string in text file, plus identify, extract, and alert on concatenated clam.zip containing clam.exe with an imp-hash sig.
[INFO]: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[DEBUG]: Run command:   /builddir/build/BUILD/clamav-1.0.0-rc2/redhat-linux-build/clamscan/clamscan -d /builddir/build/BUILD/clamav-1.0.0-rc2/redhat-linux-build/unit_tests/TC-plyrzssf/clam.imp -d /builddir/build/BUILD/clamav-1.0.0-rc2/unit_tests/input/other_sigs/Clamav-Unit-Test-Signature.ndb --allmatch /builddir/build/BUILD/clamav-1.0.0-rc2/redhat-linux-build/unit_tests/TC-plyrzssf/clam.exe.not_eicar.zipsfx 
[DEBUG]: Exit code: 1
[DEBUG]: stdout: /builddir/build/BUILD/clamav-1.0.0-rc2/redhat-linux-build/unit_tests/TC-plyrzssf/clam.exe.not_eicar.zipsfx: NDB.Clamav-Unit-Test-Signature.UNOFFICIAL FOUND
----------- SCAN SUMMARY -----------
Known viruses: 2
Engine version: 1.0.0-rc2
Scanned directories: 0
Scanned files: 1
Infected files: 1
Data scanned: 0.00 MB
Data read: 0.00 MB (ratio 0.00:1)
Time: 0.005 sec (0 m 0 s)
Start Date: 2022:11:17 00:22:02
End Date:   2022:11:17 00:22:02
[DEBUG]: stderr: 
______________________________ TC.test_many_sigs _______________________________
self = <allmatch_test.TC testMethod=test_many_sigs>
    def test_many_sigs(self):
        self.step_name('Test that each type of sig alerts in all-match mode')
    
        testfiles = TC.path_build / 'unit_tests' / 'input' / 'clamav_hdb_scanfiles' / 'clam.exe'
    
        command = '{valgrind} {valgrind_args} {clamscan} -d {path_db} {testfiles} --allmatch'.format(
            valgrind=TC.valgrind, valgrind_args=TC.valgrind_args, clamscan=TC.clamscan,
            path_db=TC.path_db,
            testfiles=testfiles,
        )
        output = self.execute_command(command)
    
        assert output.ec == 1  # virus
    
        expected_results = [
            'Test.LDB.UNOFFICIAL FOUND',
            'Test.NDB.UNOFFICIAL FOUND',
            'Test.MD5.Hash.UNOFFICIAL FOUND',
            'Test.MD5.Hash.NoSize.UNOFFICIAL FOUND',
            'Test.Sha1.Hash.UNOFFICIAL FOUND',
            'Test.Sha1.NoSize.UNOFFICIAL FOUND',
            'Test.Sha256.Hash.UNOFFICIAL FOUND',
            'Test.Sha256.Hash.NoSize.UNOFFICIAL FOUND',
            'Test.PESection.Hash.UNOFFICIAL FOUND',
            'Test.PESection.Hash.NoSize.UNOFFICIAL FOUND',
            'Test.Import.Hash.UNOFFICIAL FOUND',
            'Test.Import.Hash.NoSize.UNOFFICIAL FOUND',
        ]
>       self.verify_output(output.out, expected=expected_results)
/builddir/build/BUILD/clamav-1.0.0-rc2/unit_tests/clamscan/allmatch_test.py:88: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/builddir/build/BUILD/clamav-1.0.0-rc2/unit_tests/testcase.py:248: in verify_output
    log_checker.verify_expected_output(expected, text, order=order)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <testcase.LogChecker object at 0x3ffa55258d0>
expected_items = ['Test.LDB.UNOFFICIAL FOUND', 'Test.NDB.UNOFFICIAL FOUND', 'Test.MD5.Hash.UNOFFICIAL FOUND', 'Test.MD5.Hash.NoSize.UNOFFICIAL FOUND', 'Test.Sha1.Hash.UNOFFICIAL FOUND', 'Test.Sha1.NoSize.UNOFFICIAL FOUND', ...]
output = '/builddir/build/BUILD/clamav-1.0.0-rc2/redhat-linux-build/unit_tests/input/clamav_hdb_scanfiles/clam.exe: Test.NDB.UN... 0.00 MB (ratio 0.00:1)\nTime: 0.004 sec (0 m 0 s)\nStart Date: 2022:11:17 00:22:02\nEnd Date:   2022:11:17 00:22:02\n'
order = 1
    def verify_expected_output(self, expected_items, output, order=STRICT_ORDER):
        """Check presence of regex patterns in output string.
    
        :Parameters:
            - `expected_items`: a list of regex patterns that should be found
                                in `output`.
            - `output`: a string with output to verify.
            - `order`: STRICT_ORDER, ANY_ORDER.
    
        :Exceptions:
            - `AssertionError`: is raised if:
                1)`output` is not a string.
                2) one of expected items was not found in `output`.
                3) items were found in wrong order.
        """
        if output != None and not isinstance(output, str):
            output = output.decode("utf-8", "ignore")
        assert isinstance(output, str), "`output` must be a string."
        expected_items = self._prepare_value(expected_items)
    
        last_found_position = 0
        for item in expected_items:
            pattern = re.compile(item)
            match = pattern.search(output)
>           assert match, "Expected item `%s` not found in output:\n%s" % (
                item,
                output,
            )
E           AssertionError: Expected item `Test.Import.Hash.UNOFFICIAL FOUND` not found in output:
E           /builddir/build/BUILD/clamav-1.0.0-rc2/redhat-linux-build/unit_tests/input/clamav_hdb_scanfiles/clam.exe: Test.NDB.UNOFFICIAL FOUND
E           /builddir/build/BUILD/clamav-1.0.0-rc2/redhat-linux-build/unit_tests/input/clamav_hdb_scanfiles/clam.exe: Test.MD5.Hash.UNOFFICIAL FOUND
E           /builddir/build/BUILD/clamav-1.0.0-rc2/redhat-linux-build/unit_tests/input/clamav_hdb_scanfiles/clam.exe: Test.MD5.Hash.NoSize.UNOFFICIAL FOUND
E           /builddir/build/BUILD/clamav-1.0.0-rc2/redhat-linux-build/unit_tests/input/clamav_hdb_scanfiles/clam.exe: Test.Sha1.Hash.UNOFFICIAL FOUND
E           /builddir/build/BUILD/clamav-1.0.0-rc2/redhat-linux-build/unit_tests/input/clamav_hdb_scanfiles/clam.exe: Test.Sha1.NoSize.UNOFFICIAL FOUND
E           /builddir/build/BUILD/clamav-1.0.0-rc2/redhat-linux-build/unit_tests/input/clamav_hdb_scanfiles/clam.exe: Test.Sha256.Hash.UNOFFICIAL FOUND
E           /builddir/build/BUILD/clamav-1.0.0-rc2/redhat-linux-build/unit_tests/input/clamav_hdb_scanfiles/clam.exe: Test.Sha256.Hash.NoSize.UNOFFICIAL FOUND
E           /builddir/build/BUILD/clamav-1.0.0-rc2/redhat-linux-build/unit_tests/input/clamav_hdb_scanfiles/clam.exe: Test.LDB.UNOFFICIAL FOUND
E           /builddir/build/BUILD/clamav-1.0.0-rc2/redhat-linux-build/unit_tests/input/clamav_hdb_scanfiles/clam.exe: Test.PESection.Hash.UNOFFICIAL FOUND
E           /builddir/build/BUILD/clamav-1.0.0-rc2/redhat-linux-build/unit_tests/input/clamav_hdb_scanfiles/clam.exe: Test.PESection.Hash.NoSize.UNOFFICIAL FOUND
E           
E           ----------- SCAN SUMMARY -----------
E           Known viruses: 12
E           Engine version: 1.0.0-rc2
E           Scanned directories: 0
E           Scanned files: 1
E           Infected files: 1
E           Data scanned: 0.00 MB
E           Data read: 0.00 MB (ratio 0.00:1)
E           Time: 0.004 sec (0 m 0 s)
E           Start Date: 2022:11:17 00:22:02
E           End Date:   2022:11:17 00:22:02
/builddir/build/BUILD/clamav-1.0.0-rc2/unit_tests/testcase.py:794: AssertionError
----------------------------- Captured stdout call -----------------------------
[INFO]: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[INFO]:           Test that each type of sig alerts in all-match mode           
[INFO]: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[DEBUG]: Run command:   /builddir/build/BUILD/clamav-1.0.0-rc2/redhat-linux-build/clamscan/clamscan -d /builddir/build/BUILD/clamav-1.0.0-rc2/redhat-linux-build/unit_tests/TC-plyrzssf/database /builddir/build/BUILD/clamav-1.0.0-rc2/redhat-linux-build/unit_tests/input/clamav_hdb_scanfiles/clam.exe --allmatch 
[DEBUG]: Exit code: 1
[DEBUG]: stdout: /builddir/build/BUILD/clamav-1.0.0-rc2/redhat-linux-build/unit_tests/input/clamav_hdb_scanfiles/clam.exe: Test.NDB.UNOFFICIAL FOUND
/builddir/build/BUILD/clamav-1.0.0-rc2/redhat-linux-build/unit_tests/input/clamav_hdb_scanfiles/clam.exe: Test.MD5.Hash.UNOFFICIAL FOUND
/builddir/build/BUILD/clamav-1.0.0-rc2/redhat-linux-build/unit_tests/input/clamav_hdb_scanfiles/clam.exe: Test.MD5.Hash.NoSize.UNOFFICIAL FOUND
/builddir/build/BUILD/clamav-1.0.0-rc2/redhat-linux-build/unit_tests/input/clamav_hdb_scanfiles/clam.exe: Test.Sha1.Hash.UNOFFICIAL FOUND
/builddir/build/BUILD/clamav-1.0.0-rc2/redhat-linux-build/unit_tests/input/clamav_hdb_scanfiles/clam.exe: Test.Sha1.NoSize.UNOFFICIAL FOUND
/builddir/build/BUILD/clamav-1.0.0-rc2/redhat-linux-build/unit_tests/input/clamav_hdb_scanfiles/clam.exe: Test.Sha256.Hash.UNOFFICIAL FOUND
/builddir/build/BUILD/clamav-1.0.0-rc2/redhat-linux-build/unit_tests/input/clamav_hdb_scanfiles/clam.exe: Test.Sha256.Hash.NoSize.UNOFFICIAL FOUND
/builddir/build/BUILD/clamav-1.0.0-rc2/redhat-linux-build/unit_tests/input/clamav_hdb_scanfiles/clam.exe: Test.LDB.UNOFFICIAL FOUND
/builddir/build/BUILD/clamav-1.0.0-rc2/redhat-linux-build/unit_tests/input/clamav_hdb_scanfiles/clam.exe: Test.PESection.Hash.UNOFFICIAL FOUND
/builddir/build/BUILD/clamav-1.0.0-rc2/redhat-linux-build/unit_tests/input/clamav_hdb_scanfiles/clam.exe: Test.PESection.Hash.NoSize.UNOFFICIAL FOUND
----------- SCAN SUMMARY -----------
Known viruses: 12
Engine version: 1.0.0-rc2
Scanned directories: 0
Scanned files: 1
Infected files: 1
Data scanned: 0.00 MB
Data read: 0.00 MB (ratio 0.00:1)
Time: 0.004 sec (0 m 0 s)
Start Date: 2022:11:17 00:22:02
End Date:   2022:11:17 00:22:02
[DEBUG]: stderr: 
=========================== short test summary info ============================
FAILED clamscan/allmatch_test.py::TC::test_exe_imphash_plus_zipsfx - Assertio...
FAILED clamscan/allmatch_test.py::TC::test_many_sigs - AssertionError: Expect...
FAILED clamscan/allmatch_test.py::TC::test_pe_allmatch - AssertionError: Expe...
FAILED clamscan/allmatch_test.py::TC::test_regression_imphash_nosize - Assert...
FAILED clamscan/assorted_test.py::TC::test_pe_cert_block - AssertionError: Ex...
========================= 5 failed, 44 passed in 5.88s =========================
@micahsnyder
Copy link
Contributor

I almost asked if this fails in previous versions, but the test is new in 1.0.

We do not have any s390x systems to test with so it will be difficult to reproduce and demonstrate a fix on our end.

I am completely guessing here, but I wonder if the issue may be related to endianness when computing the import address table hashes.

@opoplawski
Copy link
Contributor Author

I'm happy to run any tests you need on s390x. I would definitely suspect endian issues somewhere in the stack.

@sebastianas
Copy link
Contributor

Is it still the case? We pass all tests for 1.0.0/1 on Debian, here a log for 1.0.1:
https://buildd.debian.org/status/fetch.php?pkg=clamav&arch=s390x&ver=1.0.1%2Bdfsg-1&stamp=1676664325&raw=0

We do have a few patches on top but we also have open pull reqs :)

@opoplawski
Copy link
Contributor Author

opoplawski commented May 3, 2023 •
edited

I'm still seeing these failures with 1.1.0. https://kojipkgs.fedoraproject.org//work/tasks/4907/100674907/build.log

@micahsnyder
Copy link
Contributor

I'm still seeing these failures with 1.1.0. https://kojipkgs.fedoraproject.org//work/tasks/4907/100674907/build.log

It looks like specifically the imphash signatures aren't matching for you. Very strange!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@opoplawski @sebastianas @micahsnyder