lextab.py and yacctab.py

[masics]

I tried to read and understand the documentation about creating and using those file. My goal is not to create those files every time user runs my app (command line). What is the right procedure?

[dabeaz]

The current Github version of PLY writes no such files. In older versions, there are optional arguments to lex() and yacc() that control this. I can't recall what they are off the top of my head, but honestly, you should probably be using the Github version of code instead.

[masics]

I tried to use a latest version but it throws me errors:

ERROR: Rule 't_STRING_LITERAL' defined for an unspecified token STRING_LITERAL
ERROR: Rule 't_LBRACKET' defined for an unspecified token LBRACKET
ERROR: Rule 't_PERIOD' defined for an unspecified token PERIOD
ERROR: Rule 't_RBRACKET' defined for an unspecified token RBRACKET
ERROR: Rule 't_LNOT' defined for an unspecified token LNOT
ERROR: Rule 't_NOT' defined for an unspecified token NOT

and won't allow to proceed.

It is working fine with the pervious version. And I have a lot other tokens.

[dabeaz]

These errors have nothing to do with files and are probably due to a bad tokens specification. Can't say without any example code.

[masics]

I don't know what could be wrong here (and it worked fine with previous version):

tokens = reserved + (
    # Literals (identifier, integer constant, float constant, string constant,
    # char const)
    'ID', 'SCONST',

    # constants
    'INT_CONST_DEC', 'INT_CONST_OCT', 'INT_CONST_HEX', 'INT_CONST_BIN',

    # Operators (+,-,*,/,%,|,&,~,^,<<,>>, ||, &&, !, <, <=, >, >=, ==, !=)
    'PLUS', 'MINUS', 'TIMES', 'DIVIDE', 'MOD',
    'OR', 'AND', 'NOT', 'XOR', 'LSHIFT', 'RSHIFT',
    'LOR', 'LAND', 'LNOT',
    'LT', 'LE', 'GT', 'GE', 'EQ', 'NE',

    # Assignment (=, *=, /=, %=, +=, -=, <<=, >>=, &=, ^=, |=)
    'EQUALS', 'TIMESEQUAL', 'DIVEQUAL', 'MODEQUAL', 'PLUSEQUAL', 'MINUSEQUAL',
    'LSHIFTEQUAL', 'RSHIFTEQUAL', 'ANDEQUAL', 'XOREQUAL', 'OREQUAL',

    # Increment/decrement (++,--)
    'PLUSPLUS', 'MINUSMINUS',

    # Structure dereference (->)
    'ARROW',

    # Conditional operator (?)
    'CONDOP',

    # Delimeters ( ) [ ] { } , . ; :
    'LPAREN', 'RPAREN',
    'LBRACKET', 'RBRACKET',
    'LBRACE', 'RBRACE',
    'COMMA', 'PERIOD', 'SEMI', 'COLON',

    # Ellipsis (...)
    'ELLIPSIS',
)

# Completely ignored characters
t_ignore = ' \t\x0c'

# Newlines


def t_NEWLINE(t):
    r'\n+'
    t.lexer.lineno += t.value.count("\n")

# Operators
t_PLUS = r'\+'
t_MINUS = r'-'
t_TIMES = r'\*'
t_DIVIDE = r'/'
t_MOD = r'%'
t_OR = r'\|'
t_AND = r'&'
t_NOT = r'~'
t_XOR = r'\^'
t_LSHIFT = r'<<'
t_RSHIFT = r'>>'
t_LOR = r'\|\|'
t_LAND = r'&&'
t_LNOT = r'!'
t_LT = r'<'
t_GT = r'>'
t_LE = r'<='
t_GE = r'>='
t_EQ = r'=='
t_NE = r'!='

[dabeaz]

You don't show how you're creating the lexer and the error messages reference variables that aren't even defined in the above code sample. So, hard to say.

[masics]

I build it as:

        self.clex = lexer(
            error_func=self._lex_error_func,
            on_lbrace_func=self._lex_on_lbrace_func,
            on_rbrace_func=self._lex_on_rbrace_func,
            type_lookup_func=self._lex_type_lookup_func)

        self.clex.build()

[dabeaz]

I don't know what this lexer() function/object is, but it's not part of PLY (old or new). So, I have no idea. Sorry.

[masics]

this is a class with all the definitions. Like this

[dabeaz]

Will have to investigate further. Do you know the last known version of PLY that worked with this?

[masics]

This version worked:

# PLY package
# Author: David Beazley (dave@dabeaz.com)

__version__ = '3.9'
__all__ = ['lex','yacc']

[dabeaz]

And what are you doing exactly? Is the goal to simply use pycparser or is something else going on? Are you extending it?

[masics]

I just used it as a template for my own parser (for another custom language).

[masics]

I made changes and it is working now.