OFT gives an exception while running with Xen code

[ayankuma]

Description

When we run OFT with xen codebase (((https://xenbits.xen.org/gitweb/?p=xen.git;a=summary)) , we get the following issue

Exception in thread "main" org.itsallcode.openfasttrace.api.importer.ImporterException: Error reading "xen/xen/drivers/video/font_8x14.c" at line 1751
        at org.itsallcode.openfasttrace.importer.tag.LineReader.readLines(LineReader.java:37)
        at org.itsallcode.openfasttrace.importer.tag.TagImporter.runImport(TagImporter.java:51)
        at java.base/java.util.Optional.ifPresent(Optional.java:183)
        at org.itsallcode.openfasttrace.core.importer.MultiFileImporterImpl.lambda$importRecursiveDir$5(MultiFileImporterImpl.java:93)
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
        at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
        at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177)
        at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
        at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177)
        at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177)
        at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
        at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
        at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
        at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
        at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
        at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497)
        at org.itsallcode.openfasttrace.core.importer.MultiFileImporterImpl.importRecursiveDir(MultiFileImporterImpl.java:92)
        at org.itsallcode.openfasttrace.core.importer.MultiFileImporterImpl.importAny(MultiFileImporterImpl.java:63)
        at org.itsallcode.openfasttrace.core.OftRunner.importItems(OftRunner.java:35)
        at org.itsallcode.openfasttrace.core.cli.commands.AbstractCommand.importItems(AbstractCommand.java:94)
        at org.itsallcode.openfasttrace.core.cli.commands.TraceCommand.run(TraceCommand.java:36)
        at org.itsallcode.openfasttrace.core.cli.CliStarter.run(CliStarter.java:116)
        at org.itsallcode.openfasttrace.core.cli.CliStarter.main(CliStarter.java:54)
        at org.itsallcode.openfasttrace.core.cli.CliStarter.main(CliStarter.java:35)
Caused by: java.nio.charset.MalformedInputException: Input length = 1
        at java.base/java.nio.charset.CoderResult.throwException(CoderResult.java:274)
        at java.base/sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:339)
        at java.base/sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
        at java.base/java.io.InputStreamReader.read(InputStreamReader.java:181)
        at java.base/java.io.BufferedReader.read1(BufferedReader.java:210)
        at java.base/java.io.BufferedReader.read(BufferedReader.java:287)
        at java.base/java.io.BufferedReader.fill(BufferedReader.java:161)
        at java.base/java.io.BufferedReader.readLine(BufferedReader.java:326)
        at java.base/java.io.LineNumberReader.readLine(LineNumberReader.java:203)
        at org.itsallcode.openfasttrace.importer.tag.LineReader.readLines(LineReader.java:29)

Steps to Reproduce

1. git clone https://xenbits.xen.org/gitweb/?p=xen.git;a=summary
2. java -jar <path_to>/openfasttrace-3.7.1.jar trace -o html -f trace.html xen

Environment

• OFT: [3.7.1]
• OS: [Ubuntu 20.04]
• Java Version: [e.g. openjdk version 11.0.20.1]
• Maven Version (in case of build problems only): [e.g. 3.5.0]

Additional context

Add any other context about the problem here.

[orzelmichal]

I can confirm. The only problematic files are:
-font_8x14.c
-font_8x16.c
-font_8x8.c
Removing them (just for a test) results in a successful OFT run (i.e. 0 findings).

[kaklakariada]

I assume this is caused by invalid UTF-8 content in font_8x14.c, e.g.

    /* 254 0xfe '�' */
...
    /* 255 0xff '�' */

I will investigate how to fix this.

[kaklakariada]

The minimal reproduction is importing a file with the following two bytes: 0x9F 0x88.

Root cause: Java's Files.newBufferedReader() uses a decoder that fails on malformed input. Creating the reader directly with new InputStreamReader(...) solves this. Also see https://stackoverflow.com/a/43446789