Literals should be valid token types

[Naddiseo]

I have a ply.lex class in which I've defined "{" and "}" as literals via the "literals" attribute, but I also want to attach an action to these tokens to match nesting.

I thought I would be able to do it as follows, but it complains about an unknown token type. Am I doing this correctly? I'd also like to avoid having a named token for something that really is a literal.

class Lexer(object):
    def __init__(self):
        self.lexer = ply.lex.lex(module=self)
        self.nesting = 0

    literals = [ '{', '}' ]

    def t_LBRACE(self, t):
        r'\{'
        self.nesting += 1
        t.type = '{'
        return t

    def t_RBRACE(self, t):
        r'\}'
        self.nesting += 1
        t.type = '}'
        return t

I'm not sure if this is intended design, or an oversight, so I thought I'd report it.

[dabeaz]

Honestly, this use case has never come up before, but it's interesting. To support it, there would have to be some way to map the rule names LBRACE and RBRACE back to the literals set. Let me think about it.

[blastrock]

+1
I would have liked that feature

[dabeaz]

Fixed in PLY-3.5.