parsing identifier which name starts with reserved keyword

[semio]

Dhall fails to parse identifier when the identifier's name starts with a reserved keyword, for example:

➔ dhall <<< "let mergedRec = { a = 1 } // { b = 2 } in mergedRec"

Error: Invalid input

(stdin):2:1:
  |
2 | <empty line>
  | ^
unexpected end of input
expecting "!=", "&&", "++", "->", "//", "//\\", "/\", "==", "||", "→", "∧", "⩓", "⫽", '#', '(', '*', '+', '.', '?', '@', built-in expression, double literal, import, integer literal, label, list literal, natural literal, record type or literal, text literal, the rest of label, or union type or literal

But it works if I rename the variable name

➔ dhall <<< "let _mergedRec = { a = 1 } // { b = 2 } in _mergedRec"
{ b = 2, a = 1 }

Also I found dhall format gives incorrect result when the identifier starts with constructors:

➔ dhall format <<< "let constructorsRec = { a = 1, b = 2 } in constructorsRec"
let constructorsRec = { a = 1, b = 2 } in constructors Rec    
##                                                    ^ a space was inserted

[Gabriella439]

@semio: Yeah, this is a bug in the standard grammar since it doesn't require a space after the merged keyword, so the Haskell implementation is parsing your expression as:

let mergedRec = { a = 1 } // { b = 2 } in merged Rec

That's also why it's inserting whitespace when you begin a name with constructors because it is parsing constructorsRec as constructors Rec, so the parser is incorrect but the formatting logic is correct.

I will put up a proposal to change the grammar to require non-empty whitespace after reserved keywords to fix this ambiguity in the grammar

[ocharles]

But isn't True a reserved keyword? The file that is just True<EOF> should parse, so I guess "non-empty whitespace" can also be EOF?

[Gabriella439]

Oh yeah, good point. Let me think more about this

[Gabriella439]

Here's another idea: we can require non-empty whitespace after the following key words since they cannot be followed by end-of-file:

• if
• then
• else
• let
• in
• as
• using
• merge
• constructors

This appears to work because all of the remaining reserved identifiers are built-in functions and values which don't exhibit this problem because the grammar already correctly supports identifiers that begin with built-in names thanks to dhall-lang/dhall-lang#122

$ dhall <<< 'let Truex = 1 in Truex'
Natural

1
$ dhall <<< 'let Natural/foldx = 1 in Natural/foldx'
Natural

1

I will propose that tomorrow if nobody has any objections

Also, cc: @f-f since I know he is interested in removing ambiguities from the grammar

[f-f]

@Gabriel439 yeah, I think that is the right solution 👍 Would it break anything? (any valid expression, that is)

[Gabriella439]

@f-f: Yes, it would break expressions like these:

if(True) then 1 else 2

merge{ x = λ(_ : {}) → 1 } < x = {=} >

[Gabriella439]

Here is the proposed change to the standard grammar: dhall-lang/dhall-lang#222

[Gabriella439]

Here's the matching change to the Haskell implementation: #551