Problem with line ending immediately after javadoc tag

[voidfist]

$ cat test.javadoc (CRLF line endings)

* @exception
      Exception     Exception-description
      spanning multiple lines

java -jar master-8.2-all.jar -j test.javadoc

JAVADOC -> JAVADOC [0:0]
|--LEADING_ASTERISK -> * [0:0]
|--WS ->   [0:1]
|--JAVADOC_TAG -> JAVADOC_TAG [0:2]
|   |--EXCEPTION_LITERAL -> @exception [0:2]
|   |--NEWLINE -> \r\n [0:13]
|   |--WS ->        [1:0]
|   `--DESCRIPTION -> DESCRIPTION [1:6]
|       |--TEXT -> Exception     Exception-description [1:6]
|       |--NEWLINE -> \r\n [1:41]
|       `--TEXT ->       spanning multiple lines [2:0]
`--EOF -> <EOF> [2:29]

ALSO please note that CLASS_NAME is not being recognized in the above example which can be resolved by adding NEWLINE to corresponding lexer mode

cat test.javadoc (with LF line endings)

* @exception
      Exception     Exception-description
      spanning multiple lines

java -jar master-8.2-all.jar -j test.javadoc

JAVADOC -> JAVADOC [0:0]
|--LEADING_ASTERISK -> * [0:0]
|--WS ->   [0:1]
|--JAVADOC_TAG -> JAVADOC_TAG [0:2]
|   |--EXCEPTION_LITERAL -> @exception [0:2]
|   |--NEWLINE -> \n [1:0]
|   |--WS ->        [2:0]
|   `--DESCRIPTION -> DESCRIPTION [2:6]
|       |--TEXT -> Exception     Exception-description [2:6]
|       |--NEWLINE -> \n [2:41]
|       `--TEXT ->       spanning multiple lines [3:0]
`--EOF -> <EOF> [3:29]

SIMILAR CASES

@param

cat test.javadoc (with CRLF line endings)

* @param
       parameter

java -jar master-8.2-all.jar -j test.javadoc

JAVADOC -> JAVADOC [0:0]
|--LEADING_ASTERISK -> * [0:0]
|--WS ->   [0:1]
|--JAVADOC_TAG -> JAVADOC_TAG [0:2]
|   |--PARAM_LITERAL -> @param [0:2]
|   |--NEWLINE -> \r\n [0:9]
|   |--WS ->         [1:0]
|   `--DESCRIPTION -> DESCRIPTION [1:7]
|       `--TEXT -> parameter [1:7]
`--EOF -> <EOF> [1:16]

ALSO please note that PARAM_NAME is not being recognized in the above example which can be resolved by adding NEWLINE to corresponding lexer mode

WITH LF line endings

java -jar master-8.2-all.jar -j test.javadoc

JAVADOC -> JAVADOC [0:0]
|--LEADING_ASTERISK -> * [0:0]
|--WS ->   [0:1]
|--JAVADOC_TAG -> JAVADOC_TAG [0:2]
|   |--PARAM_LITERAL -> @param [0:2]
|   |--NEWLINE -> \n [1:0]
|   |--WS ->         [2:0]
|   `--DESCRIPTION -> DESCRIPTION [2:7]
|       `--TEXT -> parameter [2:7]
`--EOF -> <EOF> [2:16]

@throws

cat test.javadoc (with CRLF line endings)

* @throws
     Exception     Exception-description
         spanning multiple lines

java -jar master-8.2-all.jar -j test.javadoc

JAVADOC -> JAVADOC [0:0]
|--LEADING_ASTERISK -> * [0:0]
|--WS ->   [0:1]
|--JAVADOC_TAG -> JAVADOC_TAG [0:2]
|   |--THROWS_LITERAL -> @throws [0:2]
|   |--NEWLINE -> \r\n [0:10]
|   |--WS ->       [1:0]
|   `--DESCRIPTION -> DESCRIPTION [1:5]
|       |--TEXT -> Exception     Exception-description [1:5]
|       |--NEWLINE -> \r\n [1:40]
|       `--TEXT ->          spanning multiple lines [2:0]
`--EOF -> <EOF> [2:32]

ALSO please note that CLASS_NAME is not being recognized in the above example which can be resolved by adding NEWLINE to corresponding lexer mode

WITH LF line endings

java -jar master-8.2-all.jar -j test.javadoc

JAVADOC -> JAVADOC [0:0]
|--LEADING_ASTERISK -> * [0:0]
|--WS ->   [0:1]
|--JAVADOC_TAG -> JAVADOC_TAG [0:2]
|   |--THROWS_LITERAL -> @throws [0:2]
|   |--NEWLINE -> \n [1:0]
|   |--WS ->       [2:0]
|   `--DESCRIPTION -> DESCRIPTION [2:5]
|       |--TEXT -> Exception     Exception-description [2:5]
|       |--NEWLINE -> \n [2:40]
|       `--TEXT ->          spanning multiple lines [3:0]
`--EOF -> <EOF> [3:32]

The position of NEWLINE immediately after the javadoc tag is getting affected by the type of line endings in the file.

This behaviour repeats with *7.6.2-SNAPSHOT-all.jar too. So it's not something due to any recent changes.

This issue should relate to #4824 and the fact that there is no recognition for NEWLINE in the corresponding Lexer modes.

All the lexer modes with no NEWLINE should display the same behaviour. One easy fix would be to introduce NEWLINE in those lexer modes.

[romani]

@PS-SP , please send draft fix for review.
Regression is critical to evaluate a fix.

[Vladlis]

As a result of this fix gitattributes has to be modified #4938 (comment)