test_strings.py is failing on windows

[terriko]

I've enabled the windows CI! Unfortunately, since @wzao1515 's hard work on windows last summer, we've had a few regressions and the port is currently not working as expected.

• test/test_cli.py 👎
• test/test_csv2cve.py ✔️
• test/test_cvedb.py ✔️
• test/test_definitions.py ::heavy_check_mark:
  • but 0 tests running
• test/test_extract.py 🛑
  • disabled because this is where the loop happens
• test/test_file.py ✔️
• test/test_json.py ✔️
  • technically not running any tests because LONG_TESTS not enabled
• test/test_scanner.py 👎
• test/test_strings.py 👎
• test/test_util.py ✔️

This issue is to track the problem that we're encountering in test_strings.py. It looks to me like the problem is that the strings have an extra \r in them. Here's the beginning of the AssertionError that occurs:

AssertionError: '!This program cannot be run in DOS mode.\r' not found in ['MZ', '@', '!', 'L', '!This program cannot be run in DOS mode.', '$', 'PE', 'd',

As you can tell, the string "'!This program cannot be run in DOS mode." does appear, but the string "'!This program cannot be run in DOS mode.\r" does not. On linux, newlines are \n but on windows newlines are \r\n. We can almost certainly ignore any \r that we come across in the string.

The fix here needs to happen so that it works not only in the tests, but also in the parser itself so that we don't always fail to find strings because of an extra \r character.

[terriko]

Note for students: I don't regularly work on Windows, so if you do use Windows primarily, this is a great chance to show the benefits of operating system diversity!

[Niraj-Kamdar]

one of the error in test_scanner.py is in test_does_not_scan_symlinks function because in windows os.symlink is privilege operation.

[terriko]

@Niraj-Kamdar Good find. We can probably have that test disabled when on windows, then.

[Niraj-Kamdar]

one other error in test_scanner.py is while asserting package in list(cves.keys()) because package ffmpeg isn't in cve.keys()

[Niraj-Kamdar]

What is pstring here? I didn't find pstring.py in cve_bin_tool

try:
    import cve_bin_tool.pstring as pstring
except ImportError:
    pstring = None

[Niraj-Kamdar]

In test_cli why are we asserting not equal here.

    def test_extract_curl_7_20_0(self):
        """Scanning curl-7.20.0"""
        self.assertNotEqual(
            main(["cve-bin-tool", "-l", "debug", "-x", self.tempdir]), 0
        )

It should be assertEqual zero case successful function call returns 0

[terriko]

Cve-bin-tool returns work like this:

• 0 means it ran and found no cves
• positive numbers indicate a number of cves (this was a specific request from a user so that CI systems would fail when cves were found)
• negative numbers indicate errors that caused the tool not to run

So yes, scanning on that old version of curl should find cves and return a positive non-zero number.

[terriko]

Pstring is the strings implementation for Windows. It's in the .c file because it was converted to c to improve performance (considerably).

[terriko]

Don't know what's up with the ffmpeg error though.

[Niraj-Kamdar]

@terriko test_extract_curl_7_20_0 in test_cli is failing because of the same problem of "\r" in the end. Here's truncated output of scanner ['/lib64/ld-linux-x86-64.so.2\r', 's{^^\r']

[Niraj-Kamdar]

@terriko Why output from strings command and our Strings library is different

        s = Strings(filename)
        o = s.parse()
        lines = o.split("\n")
        t = subprocess.check_output(['strings', filename]).decode().split("\n")[0:2]
        LOGGER.debug(f"\n\n\n{t} ,{lines[0:2]}\n\n")

debug output looks like this....

['/lib64/ld-linux-x86-64.so.2\r', 's{^^\r'] ,['\x7fELF', '>']

[terriko]

It shouldn't be different. This is likely a bug. We should probably strip all \r from output in both cases.

[Niraj-Kamdar]

@terriko Why don't we use compiled strings binary from GNU mingw? Its size is only 1MB and results are similar to linux GNU strings.

[wzao1515]

@terriko Why don't we use compiled strings binary from GNU mingw? Its size is only 1MB and results are similar to linux GNU strings.

Because previously we were thinking about using a single method to support all platforms, so that we could get rid of both linux strings and mingw strings. But for now I think it is fine to use packages from third party.

[Niraj-Kamdar]

@wzao1515 glad to hear that because using strings from GNU, we can avoid some problems.

[Niraj-Kamdar]

@terriko I have open PR which fixes test_checker, test_strings and test_extract but there is still problem with test_scanner and test_cli . extractor isn't extracting cpio file in Windows and there is also problem of "\r\n" in cli module and fixing it the way I fixed test_string breaking bunch of things.

[Niraj-Kamdar]

New python string parser is working fine on windows.

[Niraj-Kamdar]

test_cli , test_scanner and test_string are working fine now on windows. so, only thing left is test_extract and for that i have opened issue #667. We should close this issue now.

[terriko]

Closing!