[Haskell] Rewrite Haskell syntax

[FichteFoll]

After working with Haskell a bit, I couldn't help but notice how poor the syntax highlighting for it was. It got the job done, but there were and still are several things blatantly missing or using the wrong scopes.

This PR is in a mergeable state. I'm not done or satisfied with it, but I submitted it anyway to collect some feedback if people happen to stumble upon it. I am by no means a Haskell pro. However, this is already a significant improvement over the current syntax.

Notably, the following things still need attention:

• class definitions
• type definitions
• lambdas & \case
• curlies & semicolon
• records
• separate subscopes for known operators, such as assignment
• | and -> are actual keywords, but context-dependant. Others?
• unicode operators
• accessors
• (builtin) types
• import scope names
• more I don't remember or know about yet

Fixes #214.

[djspiewak]

Shouldn't this be keyword.operator.assignment.haskell? Also perhaps keyword.operator.declaration.haskell.

[djspiewak]

With extensions, Haskell also permits arbitrary unicode operators, so this regex is actually overly-specific.

[FichteFoll]

Not exactly arbitrary, I believe. I intended to go through the list I linked in OP and add those to this variable.

[djspiewak]

Unicode should probably be considered here as well.

[FichteFoll]

How would I differentiate a unicode name and a unicode operator?

[:upper:][:alnum:] ? Do you happen to know if there is a lexical description of the Haskell syntax somewhere?

I did most on this based on my 2 day experience and some guides on areas I haven't dug into yet.

[djspiewak]

How would I differentiate a unicode name and a unicode operator?

Hmm, I'll see if I can find a more comprehensive description of Haskell's lexical syntax in this area. It's entirely possible that you can't. In the Scala mode, for example, I gave up entirely on this and just matched identifiers as… identifiers, since there's no meaningful distinction. Haskell does distinguish both at definition site and at call site, but only in a context-sensitive fashion:

• At definition site, parens/backticks may distinguish names from operators, as well as infix usage of the type-declared name (this is the context-sensitive bit)
• At call site, parens/backticks may also be found, but also the use of an identifier in infix position rather than prefix, which should theoretically be parseable

There's got to be some distinguishing lexical feature though, which should give us something to hang our hats on. I'll do some digging

[djspiewak]

This seems insufficient. For example, it would not catch the following:

Prelude> {- {- test -} 42 -} 42
42

[djspiewak]

Should the variable.function scope really cover the whole thing? I generally think of the parentheses more as operators in this context (similar to the backticks in the ascii case) and the contents as the actual function.

[FichteFoll]

Same problem as #2145 (comment).

[djspiewak]

This scoping is inconsistent with Scala, which has a similar import mechanism and which is itself consistent with JavaScript, Python, and C#. For reference:

   import Data.Vector.{Mutable => MutableVector}
// ^^^^^^ meta.import.scala keyword.other.import.scala
//        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ meta.import.scala
//                    ^ meta.import.scala meta.import.selector.scala punctuation.section.group.begin.scala
//                     ^^^^^^^^^^^^^^^^^^^^^^^^^ meta.import.scala meta.import.selector.scala
//                             ^^ meta.import.scala meta.import.selector.scala keyword.operator.arrow.scala

[djspiewak]

I do like scoping this as variable.function, but it is inconsistent with other import scopes.

[djspiewak]

Are the arrows scoped? They should be.

[FichteFoll]

They are matched by the operator pattern.

[djspiewak]

I think it's probably worth scoping them specifically, since operators should not be scoped as keyword, but -> should.

[FichteFoll]

I agree, ->should be a proper keyword, much like guards are. It is used in many different situations and you can't assign to it.

[djspiewak]

Probably worth also adding a test for the 0x prefix, just to ensure the case insensitivity is working.

[FichteFoll]

@djspiewak please note that none of the things you commented on have been changed in the PR, but I appreciate the comments. I'll translate them into todo items for the OP.

[FichteFoll]

Seeing how I probably won't be able to return back to this any time soon, I'm removing the draft status. As mentioned in the OP, the PR is mergeable and already an improvement, but there is much more to be improved on.

[moodmosaic]

This looks great, much more precise than the current syntax. ❤️ 😍 💯 🥇 Thank you so much, @FichteFoll.

See comments about meta.group.haskell and meta.sequence.haskell scopes.

[moodmosaic]

Notably, the following things still need attention:

After merging those latest pull requests, apart from

• unicode operators, which I haven't thoroughly checked
• and the groups discussion in [Haskell] Rewrite Haskell syntax #2225 (comment)

the rest look good enough after doing a comparison with @lierdakil's atom-haskell/language-haskell grammar (which I consider one of the most precise grammars for Haskell).

[lierdakil]

Fwiw, I think both atom-haskell/language-haskell and Sublime's Haskell grammar are more or less based on the same TextMate grammar originally. Which in theory means that it should be possible to backport atom-haskell changes here. Or alternatively, convert language-haskell to tmlanguage. Although I think sublime grammar is strictly more powerful than tmlanguage, from which Haskell grammar could benefit a lot -- my implementation has a few limitations that stem from the lack of direct control over context, which I hear sublime's grammar engine allows.

[FichteFoll]

I'm pretty sure Sublime's grammar was copied from Textmate and basically never touched since then.

Trying to backport atom's grammer doesn't look like a good option because of how convoluted its regular expressions are and I'd like to take advantage of ST's power with the context stack and branching where feasible, which is better done from scratch. Thanks for the heads-up, however.

On the PR at hand, I incorporated the changes suggested by @moodmosaic. I still believe record syntax to be pretty broken, as it's basically unhandled right now, but the syntax doesn't break on it and has some highlighting at least.

[moodmosaic]

@FichteFoll

I still believe record syntax to be pretty broken, as it's basically unhandled right now, but the syntax doesn't break on it and has some highlighting at least.

There's only one thing missing to fix the record syntax, just don't match the first field and be done:

As you can see above, we're matching the first field as meta.function.type-declaration.haskell but if we can avoid that, we're all set.

[AllanZyne]

This syntax file is what I wrote when I was interested in Haskell. It might be helpful as reference for unicode operator, which I studied from The Haskell 2010 language report by some effort.

  id_char: '[\p{Ll}\p{Lu}\p{Lt}\p{Nd}_'']' 
  con_id: '[\p{Lu}\p{Lt}]{{id_char}}*'
  var_id: '[\p{Ll}_]{{id_char}}*'
  reserved_id : 'case|class|data|default|deriving|do|else|foreign|if|import|in|infix|infixl|infixr|instance|let|module|newtype|of|then|type|where|_'
  mod_id: (?:(?:{{con_id}}.)*{{con_id}})
  symbol: '[\p{S}\p{P}&&[^(),;\[\]`{}_"'']]'

  qcon_id: '{{con_id}}(?:\.{{con_id}})*'
  qvar_id: (?:{{qcon_id}}\.)?{{var_id}}
  qvar_sym: (?:{{mod_id}}\.)?{{symbol}}+

  qvar: \b(?:{{qvar_id}}|\({{qvar_sym}}\))
  qvar_op: \b({{qvar_sym}}|`{{qvar_id}}`)

  operator_char: (?:[\p{S}\p{P}&&[^(),;\[\]`{}_"']])
  operator: '{{operator_char}}+'
  operator_fun: (?:\((?!--+\)){{operator}}\))

[moodmosaic]

@FichteFoll

There's only one thing missing to fix the record syntax, just don't match the first field and be done:

What I mean is that we go from this:

To this:

---

Here's the above code

data Post =
  Post {
      postId :: PostId
    , postId :: PostId
    , postUserId :: UserId
    , postTitle :: Text
    , postBody :: Text
    , postCreatedAt :: UTCTime
    } deriving (Eq, Ord, Show)

[moodmosaic]

Ping

[FichteFoll]

It's still on my radar, but so are many other things. I don't have any updates to share atm.

[moodmosaic]

@FichteFoll, @wbond, this should be merged, even without the improvement described in #2225 (comment). If there's anything I can do to help further, let me know.

[wbond]

@FichteFoll So you are still of the opinion that this is a pretty significant step forward?

[FichteFoll]

Definitely, it's mergeable and a significant improvement. The unchecked items in the OP are merely a checklist of things to also check out that I haven't gotten around to yet and probably won't in the near future either. They are no blockers.

[wbond]

Thanks for the work on writing this and the reviews!