Show unexpected token in error message

[bennofs]

Trifecta doesn't show the unexpected token in the error message like parsec does in the following example:

Program

import Text.Parser.Token
import Text.Parser.Char
import Text.Parser.Combinators
import Text.Parser.Token.Highlight

import qualified Text.Parsec as ParsingLib
-- import qualified Text.Trifecta as ParsingLib

import Control.Applicative
import Data.HashSet (fromList)

identStyle :: CharParsing m => IdentifierStyle m
identStyle = IdentifierStyle
  { _styleName = "identifier"
  , _styleStart = letter
  , _styleLetter = letter
  , _styleReserved = fromList ["if", "then", "else"]
  , _styleHighlight = Identifier
  , _styleReservedHighlight = ReservedIdentifier
  }

data Term = If Term Term Term
          | Id String
  deriving Show

identifier :: (Monad m, TokenParsing m) => m String
identifier = ident identStyle <?> "identifier"

reserved :: (Monad m, TokenParsing m) => String -> m ()
reserved = reserve identStyle

ifExpr :: (Monad m, TokenParsing m) => m Term
ifExpr = If <$> (reserved "if" *> term) <*> (reserved "then" *> term) <*> (reserved "else" *> term)

term :: (Monad m, TokenParsing m) => m Term
term = ifExpr <|> (Id <$> identifier)

main :: IO ()
main = do
  ParsingLib.parseTest term "if true then false else null"
  putStrLn "--"
  ParsingLib.parseTest term "if true then false then false"
  putStrLn "--"
  ParsingLib.parseTest term "else"

Parsec

If (Id "true") (Id "false") (Id "null")
--
parse error at (line 1, column 20):
unexpected "t"
expecting "else"
--
parse error at (line 1, column 5):
unexpected reserved identifier "else"
expecting identifier

Trifecta

If (Id "true") (Id "false") (Id "null")
--
(interactive):1:20: error: expected: "else"
if true then false then false<EOF> 
                   ^               
--
(interactive):1:1: error: expected: "if", identifier
else<EOF> 
^

The unexpected token is very useful in this case because it shows the reason why else is not a valid identifier (because it is reserved).

[bennofs]

I think the reason for this behaviour is the definition of ident from parsers:

ident :: (TokenParsing m, Monad m, IsString s) => IdentifierStyle m -> m s
ident s = fmap fromString $ token $ try $ do
  name <- highlight (_styleHighlight s)
          ((:) <$> _styleStart s <*> many (_styleLetter s) <?> _styleName s)
  when (HashSet.member name (_styleReserved s)) $ unexpected $ "reserved " ++ _styleName s ++ " " ++ show name
  return name

My guess is that when trifecta reaches the unexpected call, it backtracks (because of the try) and thus looses the reason given to unexpected.

Parsec has a different behaviour and preserves the error message when backtracking in some way.

This seems similar to #28.

[ekmett]

Yes, the fact that the unexpected is inside of the try block is what kills the message here. try erases all the information about the error. We might try using LookAheadParsing to grab the word, then parse it twice, once to check for keywords, and once to take the value, but this will result in a clunkier more expensive parse with a worse type signature.