Allow multi-line lookahead in modes

[marijnh]

Since patch 846d043 , the considerations that made lookahead past the current line problematic have largely been made irrelevant.

See discussion here

A mode would still have to declare its maxmimum lookahead, so that the parser frontier can be backed up appropriately when the document is edited. And it does muddle up the StringStream API a bit more (i.e. the various pieces of code that run parsers, such as runMode, need to somehow expose the whole document to the stream), but it's probably worth it.

[ForbesLindesay]

Did this ever get looked at? I've been trying to update the JavaScript mode so that it recognizes functions as "in scope" even if they were declared after they were used. It's not proving easy...

[marijnh]

This was only ever intended for simple, limited lookahead. Arbitrary lookahead like what you'd need to find all locals in a block won't be possible -- you'd have to do that with an out-of-line background scan, though wiring that up to a mode is also currently not trivial.

[cben]

What about simple limited lookahead?
I'm specifically interested in markdown and rst underlined headers.
Could also be addressed with background scan + change event hooks doing markText(), but that's way harder to write correctly.
Also, if the scan wants to call getTokenAt() to use mode's parsing (e.g. to skip literal blocks) without replicating the whole mode, then it's no longer background; the sanest approach I see is replacing scan with renderLine (when mode has already run), calling markText under setTimeout(0)...

[superlou]

Also interested in limited look-ahead, particularly for Fountain.io markup. I've been able to mostly implement highlighting by updating state with blank lines, but I think the canonical syntax definitions rely on look-ahead.

[marijnh]

This was finally implemented in 5.27.0 (via the StringStream.lookahead method)