Bash: "different styles between \r and \n" after comments and heredoc delims

[rdipardo]

LexBash may switch styles before complete EOL termination in files with CRLF line endings.

This affects SCE_SH_COMMENTLINE :

as well as SCE_SH_HERE_Q:

All cases can be traced to how the value of StyleContext::atLineEnd ¹ is determined, with CRLFs apparently giving a result like this: <false>CR<true>LF.

Since DOS EOLs would only corrupt a genuine shell script, the problem (if any) is strictly limited to prospective lexer tests for Bash (*), currently absent from the source tree.

Simply not testing LexBash would be the quickest solution; in which case the enclosed patch set can be ignored.

LexBash-Prefer-MatchLineEnd-to-atLineEnd.zip

---

(*) The proposed x.sh currently generates these (non-fatal) warnings:

/home/dev/lexilla/test/examples/bash/x.sh:1: different styles between \r and \n at 20: 2, 0
/home/dev/lexilla/test/examples/bash/x.sh:3: different styles between \r and \n at 109: 2, 0
/home/dev/lexilla/test/examples/bash/x.sh:4: different styles between \r and \n at 191: 2, 0
/home/dev/lexilla/test/examples/bash/x.sh:5: different styles between \r and \n at 276: 2, 0
/home/dev/lexilla/test/examples/bash/x.sh:6: different styles between \r and \n at 360: 2, 0
/home/dev/lexilla/test/examples/bash/x.sh:7: different styles between \r and \n at 441: 2, 0
/home/dev/lexilla/test/examples/bash/x.sh:19: different styles between \r and \n at 705: 0, 13
/home/dev/lexilla/test/examples/bash/x.sh:21: different styles between \r and \n at 769: 13, 0

/home/dev/lexilla/test/examples/bash/x.sh:1: has different styles with \n versus \r\n line ends

As this would add the first ever Bash lexing test, an effort was made to demonstrate a full suite of lexical styles, along with those affected by the topic issue; hence the nondescript file name.

Footnotes

1. https://github.com/ScintillaOrg/lexilla/blob/a35a59845e793d9d37d249cf097e71fecc5f4bcd/lexlib/StyleContext.h#L35-L40 ↩

[nyamatongwe]

It will help to include bash in testing and also to make the lexer consistent with different line ends.

Lexilla includes .sh files so .gitattributes includes **.sh text eol=lf causing *.sh to download with Unix line ends everywhere. To ensure the bash examples download with local line ends (which is standard for tests) the .bsh extension could be used and added to .gitattributes with behaviour text. To ensure everything works, the .gitattributes change should be a separate commit and occur before any tests.

[nyamatongwe]

Committed the change with example files using .bsh extension.

It could be argued that \r in shell scripts is almost always an error so the lexer should highlight them rather than hide them. However, some environments (MSYS2, apparently) can treat \r\n as equivalent to \n. If an application or user wants to highlight \r then SCI_SETVIEWEOL can be called, possibly with SCI_SETREPRESENTATION to emphasize \r over \n.

[nyamatongwe]

Included in 5.1.5 release.