Bad characters rebuild whole PsiTree

[MrTsepa]

Hello everyone, I'm not sure if this question related to previous issues or not, but I have a difficulty.

In my verilog support plugin, I got a grammar from here and started to implement the plugin. I successfully implemented highlighting, referencing and completion, but I sometimes get strange behavior, when proposed variants in completion proposals are duplicated.

It happens because if I insert incomplete name it is not only considered as a bad character bud rebuild PsiTree.

For example, right tree

and tree with bad character

As you can see a parent of highlighted flop1 differs and lead to duplicates in completion proposals. It's because all elements of type IdentifierPsiNode are proposed now (if everything is correct they appear only in declarations so it's okay).

So my question is if there any ways to bypass this problem, for example, stopping tree rebuilding if bad characters are found. It's desirable to find decision which doesn't involve correcting grammar.

Thanks.

[bjansen]

@MrTsepa is this issue still relevant?

This might be caused by the fact that your Bad_character token is not sent to the HIDDEN channel. Instead it is sent to the parser, which of course does not know what to do with it.

See this comment in ANTLRv4Lexer:

// -----------------
// Illegal Character
//
// This is an illegal character trap which is always the last rule in the
// lexer specification. It matches a single character of any value and being
// the last rule in the file will match when no other rule knows what to do
// about the character. It is reported as an error but is not passed on to the
// parser. This means that the parser to deal with the gramamr file anyway
// but we will not try to analyse or code generate from a file with lexical
// errors.
//
ERRCHAR
: . -> channel(HIDDEN)
;

Some tweaks have to be applied to existing ANTLRv4 grammars in order to make them work correctly in an IntelliJ plugin. For example, you should never -> skip tokens, and you should send tokens you don't want to parse (comments, etc) to the HIDDEN channel. I guess we should clarify that in a tutorial or something like that.