Implement indentation in terms of SMIE

[bbatsov]

SMIE pretty much aims to end the practice of every package rolling out custom indentation logic. You can see ruby-mode in Emacs's trunk for a good example of SMIE usage.

[ap4y]

I spent couple days trying to understand smie and implement it for swift-mode. Unfortunately, it didn't go well and it looks to me that it will be really hard use it with swift.

Limitations of the supported BNF grammars along with concise syntax of swift (i.e. lack of unique terminators) make it really hard to express even simple swift statements without producing lots of grammar conflicts. Common pattern in smie (as I saw in other packages in trunk and github) is to use ; as terminator and implicitly provide it from lexer. I quite a few hours trying to figuring out where to put terminator for smie. It's such a strange process, since language itself doesn't expose this conception and obviously language reference doesn't have this information.

Grammars defined in language reference in most cases impossible to translate into supported BNF syntax, again due to limitations of having 2 non terminal is grammar definition. I had to find workarounds even for simple constructs like if and for.

Provided default indentation rules don't work with swift. It's not hard to modify this rules, but it removes most of benefits of implementing lexer and grammars.

[ap4y]

After some initial struggling, I was able to find workarounds for most of the issues.

[bbatsov]

That's great news. I noticed there was a similar thread on the emacs-devel mailing list (regarding elixir-mode).

[mattdeboard]

Sorry for the thread necromancy here, but I'm looking for some info on the --implicit-semi-p functions in ruby-mode and here in swift-mode. Can you help me understand what the implicit semicolon is for in concrete terms? The SMIE documentation talks about how whitespace-sensitive languages can use it to delineate significant whitespace but I'm not clear on what's happening under the hood. Understanding these --implicit-semi-p funcs seems like a good handle to try to figure that out.

Thanks

[mattdeboard]

BTW your SMIE grammar specification here is a real treat because of its conciseness @ap4y

[ap4y]

Thanks for the kind words about my implementation @mattdeboard!

Regarding implicit semicolons: While concept of implicit semicolon can be confusing initially, idea behind it is pretty simple. As you probably know, ruby and swift don't use classical statement terminators(like semicolon) and instead use lexical rules. In simple case we may say that new line character (\n) is a statement terminator. As you probably know, default lexer implementation for smie doesn't handle new line, so we have to manually mark new line in custom lexer as terminator. Conveniently, semicolon has appropriate semantic for this case, so several modes started marking implicit terminators as ;.

Unfortunately, new line doesn't always works as terminator for the statements. Obvious examples are case statement and multi-line statements (a +\n b). I borrowed this neat function --implicit-semi-p from ruby-mode to handle such situations. Basically, it decides if lexer should put implicit semicolon at the current position.

Hope this will help you.

[mattdeboard]

Indeed. I actually ripped out the previous implementation of the
forward/backward token functions that was the wrong approach and replaced
with much simpler implementation that leans on SMIE's machinery. I modeled
it on your work. Thanks again.

p.s. I'm referring to elixir-mode: https://github.com/elixir-lang/emacs-elixir

On Tue, Aug 19, 2014 at 12:15 AM, Arthur Evstifeev <notifications@github.com

wrote:

Thanks for the kind words about my implementation @mattdeboard
https://github.com/mattdeboard!

Regarding implicit semicolons: While concept of implicit semicolon can be
confusing initially, idea behind it is pretty simple. As you probably know,
ruby and swift don't use classical statement terminators(like semicolon)
and instead use lexical rules. In simple case we may say that new line
character (\n) is a statement terminator. As you probably know, default
lexer implementation for smie doesn't handle new line, so we have to
manually mark new line in custom lexer as terminator. Conveniently,
semicolon has appropriate semantic for this case, so several modes started
marking implicit terminators as ;.

Unfortunately, new line doesn't always works as terminator for the
statements. Obvious examples are case statement and multi-line statements
(a +\n b). I borrowed this neat function --implicit-semi-p from ruby-mode
to handle such situations. Basically, it decides if lexer should put
implicit semicolon at the current position.

Hope this will help you.

—
Reply to this email directly or view it on GitHub
#18 (comment)
.