Enable regular expression flags for terminal rules

[spoenemann]

See #126 (comment). JavaScript supports flags to control the behavior of regular expressions. Shall we enable in the grammar language?

[dhuebner]

I don't know how useful are other flags, but those seen to be useful:

s	Allows . to match newline characters.
i	Case-insensitive search.

Will mean to change the RegexLiteral to:
\/(?![*+?])(?:[^\r\n\[/\\]|\\.|\[(?:[^\r\n\]\\]|\\.)*\])+\/[a-z]?

[a-z]? is if regex will become more options in the future.

[spoenemann]

This idea does not work well with the changes made in #288. Let's postpone it for now.

[msujew]

Regarding case sensitivity, there's also a group flag which allows to make search case insensitive. I would argue, that users which are interested in this feature should use the group flag instead of the global regex flag.

[dhuebner]

What about a new keyword for multiline matching?

[msujew]

@dhuebner Multiline matching is always enabled, no flag necessary :) See the multiline-comment terminal rules.

[dhuebner]

@msujew
I meant the s flag for: Allows . to match newline characters.

[dhuebner]

Keyword is not a good idea, because this flag regex specific

[Josh-Cena]

Hi, just want to mention that the u flag is quite useful. For example, all JS identifiers use the syntax /[$_\p{ID_Start}][$\u200c\u200d\p{ID_Continue}]*/u, but at the moment there's no easy way to specify Unicode character classes.

[msujew]

For now, the unicode flag isn't supported in Chevrotain. It is part of a larger lexer issue, see Chevrotain/chevrotain#1670.

Note that custom pattern functions can still be used to achieve the same effect, so it's still possible to achieve this functionality in Langium.