mtest: fix xml chars discouraged to use

[nbyavuz]

Fixes #9894.

I copied most of the implementation from there. Instead of filtering them, I replaced such characters with their printable representation.

For example:

project('tutorial', 'c')
test('mytest',find_program('python'), args: ['-c','print(u"Test\x07")'])

was failing with:

builddir/meson-logs/testlog.junit.xml:2: parser error : PCDATA invalid Char value 7
e name="mytest" classname="tutorial" time="0.06694579124450684"><system-out>Test

now it doesn't fail, instead it produces:

<?xml version="1.0" encoding="utf-8"?>
<testsuites tests="1" errors="0" failures="0"><testsuite name="tutorial" tests="1" errors="0" failures="0" skipped="0" time="0.008198976516723633"><testcase name="mytest" classname="tutorial" time="0.008198976516723633"><system-out>Test\x07</system-out></testcase></testsuite></testsuites>

TBD:
I am not sure if this solution is suitable. If you think it is okay and tests are necessary I can add test cases.

[tristan957]

Could you add a test case? Should be fairly simple since your PR has a reproducer. Your test case should probably involve all the illegal characters.

[nbyavuz]

Could you add a test case? Should be fairly simple since your PR has a reproducer. Your test case should probably involve all the illegal characters.

Test cases are added.

[nbyavuz]

"decode" is a pretty ambiguous name to import from another module I guess. It might be better to do from mesonbuild import mtest and refer to mtest.decode

passing known invalid input into mtest.decode() and checking that it matches what we expect it to be transformed to

Updated based on these comments.

[tristan957]

If you are unable to come back and update this PR, I would be more than happy to take it over and push it over the finish line.

[nbyavuz]

If you are unable to come back and update this PR, I would be more than happy to take it over and push it over the finish line.

Thanks for the review. I updated the PR based on your comments.

[tristan957]

CI is now running. Trying to get an actual reviewer for this.

[jpakkane]

Test failure is related:

    def test_replace_unencodable_xml_chars_unit(self):
        '''
        Test that unencodable xml chars are replaced with their
        printable representation
        https://github.com/mesonbuild/meson/issues/9894
        '''
        if not shutil.which('xmllint'):
            raise SkipTest('xmllint not installed.')
        testdir = os.path.join(self.unit_test_dir, '110 replace unencodable xml chars')
        self.init(testdir)
>       self.run_tests()
E       subprocess.CalledProcessError: Command '['D:\\a\\_temp\\msys64\\mingw32\\bin\\ninja.EXE', '-w', 'dupbuild=err', '-d', 'explain', 'test', 'benchmark']' returned non-zero exit status 1.

unittests\allplatformstests.py:447: CalledProcessError
---------------------------- Captured stdout call -----------------------------
$ D:\a\_temp\msys64\mingw32\bin\python3.exe D:\a\meson\meson\meson.py setup --backend=ninja "D:\a\meson\meson\test cases/unit\110 replace unencodable xml chars" D:\a\meson\meson\tmpamahhywo --prefix /usr --libdir lib
stdout:
The Meson build system
Version: 1.1.99
Source dir: D:\a\meson\meson\test cases\unit\110 replace unencodable xml chars
Build dir: D:\a\meson\meson\tmpamahhywo
Build type: native build
Project name: replace unencodable xml chars
Project version: >= 0.55
Host machine cpu family: x86_64
Host machine cpu: x86_64
Program script.py found: YES (D:\a\_temp\msys64\mingw32\bin\python3.exe D:\a\meson\meson\test cases\unit\110 replace unencodable xml chars\script.py)
Build targets in project: 0

replace unencodable xml chars >= 0.55

  User defined options
    backend: ninja
    libdir : lib
    prefix : /usr

Found ninja.EXE-1.11.1 at D:\a\_temp\msys64\mingw32\bin\ninja.EXE

$ D:\a\_temp\msys64\mingw32\bin\ninja.EXE -w dupbuild=err -d explain test benchmark
stdout:
[0/2] Running all tests.
1/1 main FAIL            0.36s   exit status 1
>>> MALLOC_PERTURB_=21 D:\a\_temp\msys64\mingw32\bin\python3.exe D:\a\meson\meson\test cases\unit\110 replace unencodable xml chars\script.py
------------------------------------- 8< -------------------------------------
stderr:
Traceback (most recent call last):

  File "D:\a\meson\meson\test cases\unit\110 replace unencodable xml chars\script.py", line 28, in <module>

    print(invalid_string)

  File "D:\a\_temp\msys64\mingw32\lib\python3.10\encodings\cp1252.py", line 19, in encode

    return codecs.charmap_encode(input,self.errors,encoding_table)[0]

UnicodeEncodeError: 'charmap' codec can't encode characters in position 30-126: character maps to <undefined>

------------------------------------------------------------------------------