Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

parser error with typedef #523

Closed
kdschlosser opened this issue Dec 2, 2023 · 10 comments
Closed

parser error with typedef #523

kdschlosser opened this issue Dec 2, 2023 · 10 comments

Comments

@kdschlosser
Copy link

kdschlosser commented Dec 2, 2023

The following code is being use to keep memory use in check when returning an enumeration from a function. The 3 byte savings is needed. The issue is when generating documentation unless I change the type to the actual enumeration the path back to the enumeration will not be completed. The following code fixes that issue when documentation is generated for the C code.

There is a binding that is made for Python that uses pycparser to read the C code. The binding makes things more pythonic and in a class arrangement. I would like to be able to pull the documentation from the c library and align it correctly with the python code. This shouldn't be an issue except when I set the macro for DOXYGEN and run it through pycparser it errors out on typedef _lv_res_t lv_res_t;. It doesn't error if I do not have the DOXYGEN macro set. I need it set in order to make the correct link in the documentation in order to have the enum values correctly tied to a functions return value in the documentation.

enum _lv_res_t {
    LV_RES_INV = 0, /*Typically indicates that the object is deleted (become invalid) in the action
                      function or an operation was failed*/
    LV_RES_OK,      /*The object is valid (no deleted) after the action*/
};

#ifdef DOXYGEN
typedef _lv_res_t lv_res_t;
#else
typedef uint8_t lv_res_t;
#endif /*DOXYGEN*/

Here is the debugging output from pycparser

State  : 2
Stack  : translation_unit . LexToken(ENUM,'enum',42,6926)
Action : Shift and goto state 35

State  : 35
Stack  : translation_unit ENUM . LexToken(ID,'_lv_res_t',42,6931)
Action : Shift and goto state 116

State  : 116
Stack  : translation_unit ENUM ID . LexToken(LBRACE,'{',42,6941)
Action : Shift and goto state 119

State  : 119
Stack  : translation_unit ENUM ID LBRACE . LexToken(ID,'LV_RES_INV',43,6947)
Action : Reduce rule [brace_open -> LBRACE] with ['{'] and goto state 195
Result : <str @ 0x7f39071164b0> ('{')

State  : 195
Stack  : translation_unit ENUM ID brace_open . LexToken(ID,'LV_RES_INV',43,6947)
Action : Shift and goto state 199

State  : 199
Stack  : translation_unit ENUM ID brace_open ID . LexToken(EQUALS,'=',43,6958)
Action : Shift and goto state 328

State  : 328
Stack  : translation_unit ENUM ID brace_open ID EQUALS . LexToken(INT_CONST_OCT,'0',43,6960)
Action : Shift and goto state 162

State  : 162
Stack  : translation_unit ENUM ID brace_open ID EQUALS INT_CONST_OCT . LexToken(COMMA,',',43,6961)
Action : Reduce rule [constant -> INT_CONST_OCT] with ['0'] and goto state 158
Result : <Constant @ 0x7f390688a240> ("Constant(type='int',\n         value='0 ...)

State  : 158
Stack  : translation_unit ENUM ID brace_open ID EQUALS constant . LexToken(COMMA,',',43,6961)
Action : Reduce rule [primary_expression -> constant] with [<Constant @ 0x7f390688a240>] and goto state 153
Result : <Constant @ 0x7f390688a240> ("Constant(type='int',\n         value='0 ...)

State  : 153
Stack  : translation_unit ENUM ID brace_open ID EQUALS primary_expression . LexToken(COMMA,',',43,6961)
Action : Reduce rule [postfix_expression -> primary_expression] with [<Constant @ 0x7f390688a240>] and goto state 147
Result : <Constant @ 0x7f390688a240> ("Constant(type='int',\n         value='0 ...)

State  : 147
Stack  : translation_unit ENUM ID brace_open ID EQUALS postfix_expression . LexToken(COMMA,',',43,6961)
Action : Reduce rule [unary_expression -> postfix_expression] with [<Constant @ 0x7f390688a240>] and goto state 146
Result : <Constant @ 0x7f390688a240> ("Constant(type='int',\n         value='0 ...)

State  : 146
Stack  : translation_unit ENUM ID brace_open ID EQUALS unary_expression . LexToken(COMMA,',',43,6961)
Action : Reduce rule [cast_expression -> unary_expression] with [<Constant @ 0x7f390688a240>] and goto state 141
Result : <Constant @ 0x7f390688a240> ("Constant(type='int',\n         value='0 ...)

State  : 141
Stack  : translation_unit ENUM ID brace_open ID EQUALS cast_expression . LexToken(COMMA,',',43,6961)
Action : Reduce rule [binary_expression -> cast_expression] with [<Constant @ 0x7f390688a240>] and goto state 140
Result : <Constant @ 0x7f390688a240> ("Constant(type='int',\n         value='0 ...)

State  : 140
Stack  : translation_unit ENUM ID brace_open ID EQUALS binary_expression . LexToken(COMMA,',',43,6961)
Action : Reduce rule [conditional_expression -> binary_expression] with [<Constant @ 0x7f390688a240>] and goto state 139
Result : <Constant @ 0x7f390688a240> ("Constant(type='int',\n         value='0 ...)

State  : 139
Stack  : translation_unit ENUM ID brace_open ID EQUALS conditional_expression . LexToken(COMMA,',',43,6961)
Action : Reduce rule [constant_expression -> conditional_expression] with [<Constant @ 0x7f390688a240>] and goto state 457
Result : <Constant @ 0x7f390688a240> ("Constant(type='int',\n         value='0 ...)

State  : 457
Stack  : translation_unit ENUM ID brace_open ID EQUALS constant_expression . LexToken(COMMA,',',43,6961)
Action : Reduce rule [enumerator -> ID EQUALS constant_expression] with ['LV_RES_INV','=',<Constant @ 0x7f390688a240>] and goto state 198
Result : <Enumerator @ 0x7f390688b040> ("Enumerator(name='LV_RES_INV',\n         ...)

State  : 198
Stack  : translation_unit ENUM ID brace_open enumerator . LexToken(COMMA,',',43,6961)
Action : Reduce rule [enumerator_list -> enumerator] with [<Enumerator @ 0x7f390688b040>] and goto state 324
Result : <EnumeratorList @ 0x7f390688aac0> ("EnumeratorList(enumerators=[Enumerator( ...)

State  : 324
Stack  : translation_unit ENUM ID brace_open enumerator_list . LexToken(COMMA,',',43,6961)
Action : Shift and goto state 327

State  : 327
Stack  : translation_unit ENUM ID brace_open enumerator_list COMMA . LexToken(ID,'LV_RES_OK',45,6968)
Action : Shift and goto state 199

State  : 199
Stack  : translation_unit ENUM ID brace_open enumerator_list COMMA ID . LexToken(COMMA,',',45,6977)
Action : Reduce rule [enumerator -> ID] with ['LV_RES_OK'] and goto state 456
Result : <Enumerator @ 0x7f3906889700> ("Enumerator(name='LV_RES_OK',\n          ...)

State  : 456
Stack  : translation_unit ENUM ID brace_open enumerator_list COMMA enumerator . LexToken(COMMA,',',45,6977)
Action : Reduce rule [enumerator_list -> enumerator_list COMMA enumerator] with [<EnumeratorList @ 0x7f390688aac0>,',',<Enumerator @ 0x7f3906889700>] and goto state 324
Result : <EnumeratorList @ 0x7f390688aac0> ("EnumeratorList(enumerators=[Enumerator( ...)

State  : 324
Stack  : translation_unit ENUM ID brace_open enumerator_list . LexToken(COMMA,',',45,6977)
Action : Shift and goto state 327

State  : 327
Stack  : translation_unit ENUM ID brace_open enumerator_list COMMA . LexToken(RBRACE,'}',46,6979)
Action : Reduce rule [enumerator_list -> enumerator_list COMMA] with [<EnumeratorList @ 0x7f390688aac0>,','] and goto state 324
Result : <EnumeratorList @ 0x7f390688aac0> ("EnumeratorList(enumerators=[Enumerator( ...)

State  : 324
Stack  : translation_unit ENUM ID brace_open enumerator_list . LexToken(RBRACE,'}',46,6979)
Action : Shift and goto state 205

State  : 205
Stack  : translation_unit ENUM ID brace_open enumerator_list RBRACE . LexToken(SEMI,';',46,6980)
Action : Reduce rule [brace_close -> RBRACE] with ['}'] and goto state 454
Result : <str @ 0x7f39071164f0> ('}')

State  : 454
Stack  : translation_unit ENUM ID brace_open enumerator_list brace_close . LexToken(SEMI,';',46,6980)
Action : Reduce rule [enum_specifier -> ENUM ID brace_open enumerator_list brace_close] with ['enum','_lv_res_t','{',<EnumeratorList @ 0x7f390688aac0>,'}'] and goto state 31
Result : <Enum @ 0x7f390688bf00> ("Enum(name='_lv_res_t',\n     values=Enu ...)

State  : 31
Stack  : translation_unit enum_specifier . LexToken(SEMI,';',46,6980)
Action : Reduce rule [type_specifier -> enum_specifier] with [<Enum @ 0x7f390688bf00>] and goto state 24
Result : <Enum @ 0x7f390688bf00> ("Enum(name='_lv_res_t',\n     values=Enu ...)

State  : 24
Stack  : translation_unit type_specifier . LexToken(SEMI,';',46,6980)
Action : Reduce rule [declaration_specifiers -> type_specifier] with [<Enum @ 0x7f390688bf00>] and goto state 12
Result : <dict @ 0x7f390688bdc0> ("{'qual': [], 'storage': [], 'type': [En ...)

State  : 12
Stack  : translation_unit declaration_specifiers . LexToken(SEMI,';',46,6980)
Action : Reduce rule [empty -> <empty>] with [] and goto state 82
Result : <NoneType @ 0x5617b57043e0> (None)

State  : 82
Defaulted state 82: Reduce using 19
Stack  : translation_unit declaration_specifiers empty . LexToken(SEMI,';',46,6980)
Action : Reduce rule [init_declarator_list_opt -> empty] with [None] and goto state 78
Result : <NoneType @ 0x5617b57043e0> (None)

State  : 78
Defaulted state 78: Reduce using 85
Stack  : translation_unit declaration_specifiers init_declarator_list_opt . LexToken(SEMI,';',46,6980)
Action : Reduce rule [decl_body -> declaration_specifiers init_declarator_list_opt] with [<dict @ 0x7f390688bdc0>,None] and goto state 13
Result : <list @ 0x7f390688a540> ("[Decl(name=None,\n     quals=[\n        ...)

State  : 13
Stack  : translation_unit decl_body . LexToken(SEMI,';',46,6980)
Action : Shift and goto state 89

State  : 89
Stack  : translation_unit decl_body SEMI . LexToken(TYPEDEF,'typedef',48,6983)
Action : Reduce rule [declaration -> decl_body SEMI] with [<list @ 0x7f390688a540>,';'] and goto state 6
Result : <list @ 0x7f390688a540> ("[Decl(name=None,\n     quals=[\n        ...)

State  : 6
Stack  : translation_unit declaration . LexToken(TYPEDEF,'typedef',48,6983)
Action : Reduce rule [external_declaration -> declaration] with [<list @ 0x7f390688a540>] and goto state 63
Result : <list @ 0x7f390688a540> ("[Decl(name=None,\n     quals=[\n        ...)

State  : 63
Stack  : translation_unit external_declaration . LexToken(TYPEDEF,'typedef',48,6983)
Action : Reduce rule [translation_unit -> translation_unit external_declaration] with [<list @ 0x7f39068f1240>,<list @ 0x7f390688a540>] and goto state 2
Result : <list @ 0x7f39068f1240> ("[Typedef(name='size_t',\n        quals= ...)

State  : 2
Stack  : translation_unit . LexToken(TYPEDEF,'typedef',48,6983)
Action : Shift and goto state 56

State  : 56
Stack  : translation_unit TYPEDEF . LexToken(ID,'_lv_res_t',48,6991)
Action : Reduce rule [storage_class_specifier -> TYPEDEF] with ['typedef'] and goto state 21
Result : <str @ 0x7f39068885b0> ('typedef')

State  : 21
Stack  : translation_unit storage_class_specifier . LexToken(ID,'_lv_res_t',48,6991)
Action : Reduce rule [empty -> <empty>] with [] and goto state 97
Result : <NoneType @ 0x5617b57043e0> (None)

State  : 97
Stack  : translation_unit storage_class_specifier empty . LexToken(ID,'_lv_res_t',48,6991)
Action : Reduce rule [declaration_specifiers_no_type_opt -> empty] with [None] and goto state 100
Result : <NoneType @ 0x5617b57043e0> (None)

State  : 100
Stack  : translation_unit storage_class_specifier declaration_specifiers_no_type_opt . LexToken(ID,'_lv_res_t',48,6991)
Action : Reduce rule [declaration_specifiers_no_type -> storage_class_specifier declaration_specifiers_no_type_opt] with ['typedef',None] and goto state 25
Result : <dict @ 0x7f3906888740> ({'qual': [], 'storage': ['typedef'], 'ty ...)

State  : 25
Stack  : translation_unit declaration_specifiers_no_type . LexToken(ID,'_lv_res_t',48,6991)
Action : Shift and goto state 27

State  : 27
Stack  : translation_unit declaration_specifiers_no_type ID . LexToken(ID,'lv_res_t',48,7001)
ERROR: Error  : translation_unit declaration_specifiers_no_type ID . LexToken(ID,'lv_res_t',48,7001)
@kdschlosser
Copy link
Author

I expanded the output for the debugging results. It look like for some reason it is marking the _lv_res_t enum as a typedef when it isn't.

State  : 100
Stack  : translation_unit storage_class_specifier declaration_specifiers_no_type_opt . LexToken(ID,'_lv_res_t',52,6995)
Action : Reduce rule [declaration_specifiers_no_type -> storage_class_specifier declaration_specifiers_no_type_opt] with ['typedef',None] and goto state 25
Result : <dict @ 0x7f37f6256600> ({'qual': [], 'storage': ['typedef'], 'type': [], 'function': [], 'alignment': []})

@eliben
Copy link
Owner

eliben commented Dec 3, 2023

Please add a minimal, reproducible example

A small .c file without any preprocessor directives that you observe the issue on.

@kdschlosser
Copy link
Author

It's right there. I just gave you an example.

enum _lv_res_t {
    LV_RES_INV = 0, /*Typically indicates that the object is deleted (become invalid) in the action
                      function or an operation was failed*/
    LV_RES_OK,      /*The object is valid (no deleted) after the action*/
};

typedef _lv_res_t lv_res_t;

@eliben
Copy link
Owner

eliben commented Dec 3, 2023

This example still has comments that need to be removed manually. I asked for code that can be fed to pycparser directly without the preprocessor (which often obscures issue reports).

In any case, the code is invalid standard C, I believe, because the compiler does not know what _lv_res_t is.

Trying your fragment with gcc -std=c99, I get:

error: unknown type name ‘_lv_res_t’
    7 | typedef _lv_res_t lv_res_t;
      |         ^~~~~~~~~

The right way to write this typedef would be:

typedef enum _lv_res_t lv_res_t;

@kdschlosser
Copy link
Author

OK so I have done what you stated and this is what happens.

enum _lv_meter_indicator_type_t {
    LV_METER_INDICATOR_TYPE_NEEDLE_IMG,
    LV_METER_INDICATOR_TYPE_NEEDLE_LINE,
    LV_METER_INDICATOR_TYPE_SCALE_LINES,
    LV_METER_INDICATOR_TYPE_ARC,
};

typedef enum _lv_meter_indicator_type_t lv_meter_indicator_type_t;

Decl(name=None,
     quals=[
           ],
     align=[
           ],
     storage=[
             ],
     funcspec=[
              ],
     type=Enum(name='_lv_meter_indicator_type_t',
               values=EnumeratorList(enumerators=[Enumerator(name='LV_METER_INDICATOR_TYPE_NEEDLE_IMG',
                                                             value=None
                                                             ),
                                                  Enumerator(name='LV_METER_INDICATOR_TYPE_NEEDLE_LINE',
                                                             value=None
                                                             ),
                                                  Enumerator(name='LV_METER_INDICATOR_TYPE_SCALE_LINES',
                                                             value=None
                                                             ),
                                                  Enumerator(name='LV_METER_INDICATOR_TYPE_ARC',
                                                             value=None
                                                             )
                                                 ]
                                     )
               ),
     init=None,
     bitsize=None
     )

TypeDecl(declname='lv_meter_indicator_type_t',
         quals=[
               ],
         align=None,
         type=Enum(name='_lv_meter_indicator_type_t',
                   values=None
                   )
         )

see the issue there.

lv_meter_indicator_type_t has the type of _lv_meter_indicator_type_t but it has no values.

Those values should be apart of the lv_meter_indicator_type_t type because lv_meter_indicator_type_t is an alias for _lv_meter_indicator_type_t

@kdschlosser
Copy link
Author

the type for lv_meter_indicator_type_t should be the same instance as the type used in the Decl at the top.

@kdschlosser
Copy link
Author

Oh and I am not sure why I am able to run the code through the preprocessor and it doesn't error at all. It also works with DOXYGEN without any issues. The original example I am referring to.

@eliben
Copy link
Owner

eliben commented Dec 5, 2023

Yes, that's how it works. pycparser just produces an AST for you, it doesn't do any semantic or type analysis. Note also that just:

typedef enum _lv_meter_indicator_type_t lv_meter_indicator_type_t;

... is perfectly valid C. The enum type doesn't have to be declared.


The AST you have from pycparser has all the information you need -- you can see the type the typedef is referring to, find that type in the AST and extract whatever you need from it.

@eliben eliben closed this as completed Dec 5, 2023
@kdschlosser
Copy link
Author

There is no link between the 2 enumerations made?.. what you are telling me is I have to iterate over the entire ast again in order to locate the type used in the typedef? That's crazy to have to do that. There should be some kind of a link between a typedef and it's parent type if there is one available.

@kdschlosser
Copy link
Author

so if I use this code

typedef enum _example {
    some_enumeration=0,
} example;

is some_enumeration a descendant of example?
I believe it is.

enum _example {
    some_enumeration=0,
};

typedef enum _example example;

but it's not if I do this?...

They are both the same thing. They do the same thing...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants