[c] Parse exception with inline assembly

[penguin42]

Affects PMD Version:

7.0.0-rc3

Description:

I've been trying pmd on the Linux kernel source, and I'm getting a crash on the arch/arc subdirectory, I've narrowed it down to the file arch/arc/include/asm/entry-arcv2.h
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/arc/include/asm/entry-arcv2.h

A clear and concise description of what the bug is.

The following crashes, originally found with just --dir arch but I narrowed it down to that file, which is actually arc asm.

$ pmd cpd --minimum-tokens 100 --language c --dir arch/arc/include/asm/entry-arcv2.h

[main] INFO net.sourceforge.pmd.cli.commands.internal.AbstractPmdSubcommand - Log level is at INFO
java.lang.IllegalArgumentException: Begin column must be >= 1, got 0
	at net.sourceforge.pmd.util.AssertionUtil.mustBe(AssertionUtil.java:203)
	at net.sourceforge.pmd.util.AssertionUtil.mustBe(AssertionUtil.java:199)
	at net.sourceforge.pmd.util.AssertionUtil.requireOver1(AssertionUtil.java:132)
	at net.sourceforge.pmd.lang.document.FileLocation.<init>(FileLocation.java:55)
	at net.sourceforge.pmd.lang.document.FileLocation.<init>(FileLocation.java:48)
	at net.sourceforge.pmd.lang.document.FileLocation.caret(FileLocation.java:169)
	at net.sourceforge.pmd.lang.ast.TokenMgrError.location(TokenMgrError.java:61)
	at net.sourceforge.pmd.lang.ast.FileAnalysisException.positionToString(FileAnalysisException.java:79)
	at net.sourceforge.pmd.lang.ast.FileAnalysisException.getMessage(FileAnalysisException.java:63)
	at java.base/java.lang.Throwable.getLocalizedMessage(Throwable.java:397)
	at java.base/java.lang.Throwable.toString(Throwable.java:496)
	at java.base/java.lang.String.valueOf(String.java:4216)
	at java.base/java.lang.StringBuilder.append(StringBuilder.java:173)
	at picocli.CommandLine.executeUserObject(CommandLine.java:2050)
	at picocli.CommandLine.access$1500(CommandLine.java:148)
	at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2461)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2453)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2415)
	at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2273)
	at picocli.CommandLine$RunLast.execute(CommandLine.java:2417)
	at picocli.CommandLine.execute(CommandLine.java:2170)
	at net.sourceforge.pmd.cli.PmdCli.main(PmdCli.java:18)

Code Sample demonstrating the issue:

see above header from kernel

Steps to reproduce:

Please provide detailed steps for how we can reproduce the bug.

1. Check out a Linux git from the git listed above, I doubt it's actually that version sensitive but I'm at 1ef6663a587ba3e57dc5065a477db1c64481eedd
2. pmd cpd --minimum-tokens 100 --language c --dir arch/arc/include/asm/entry-arcv2.h

Running PMD through: [CLI | Ant | Maven | Gradle | Designer | Other]

CLI

running on Fedora 38 Linux x86-64
$ java --version
openjdk 17.0.7 2023-04-18
OpenJDK Runtime Environment (Red_Hat-17.0.7.0.7-4.fc38) (build 17.0.7+7)
OpenJDK 64-Bit Server VM (Red_Hat-17.0.7.0.7-4.fc38) (build 17.0.7+7, mixed mode, sharing)

[oowekyala]

Thanks for the report. I could reproduce it in master and also in #4397.

The problem is the following fragment (https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/arc/include/asm/entry-arcv2.h#n184):

	;    but that doesn't really matter
	bz	1f

The lexer doesn't know about assembly so it doesn't recognize the first line as a comment. It then tries to parse the part after the apostrophe as a character literal, which fails when it consumes up to the line ending.

There are two problems here

• The lexer produces an error with a column number of zero, which is invalid (and causes the reported exception). That we can fix easily.
• The lexer doesn't know about assembly-specific syntax. It looks like the .macro ... .endm delimiters are specific to the GNU assembler. Maybe we could recognize them in the grammar.

[penguin42]

Thanks for the report. I could reproduce it in master and also in #4397.

The problem is the following fragment (https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/arc/include/asm/entry-arcv2.h#n184):

	;    but that doesn't really matter
	bz	1f

The lexer doesn't know about assembly so it doesn't recognize the first line as a comment. It then tries to parse the part after the apostrophe as a character literal, which fails when it consumes up to the line ending.

There are two problems here

* The lexer produces an error with a column number of zero, which is invalid (and causes the reported exception). That we can fix easily.

Great.

* The lexer doesn't know about assembly-specific syntax. It looks like the `.macro ... .endm` delimiters are specific to the [GNU assembler](https://ftp.gnu.org/old-gnu/Manuals/gas-2.9.1/html_chapter/as_7.html#SEC109). Maybe we could recognize them in the grammar.

I wouldn't bother recognising the asm necessarily; I think as long as you don't choke on it, then it's OK.

[adangel]

I think, this can be reproduced for java with the following snipped:

public class Foo {
    String s = "abc";"
    String s2 = "xyz";
}

Notice the unbalanced quote in line 2...

[adangel]

The lexer produces an error with a column number of zero, which is invalid (and causes the reported exception). That we can fix easily.

This is fixed with #4694

The lexer doesn't know about assembly-specific syntax. It looks like the .macro ... .endm delimiters are specific to the GNU assembler. Maybe we could recognize them in the grammar.

This is still open.