Redeclared types

[Syeberman]

While using pycparser to parse a large, existing codebase, I immediately came upon the typedef-name problem. The changes in this pull request resolve and test for the issues encountered; in particular:

• Reusing typedef names as structure/union member names
• Reusing typedef names as variables names in inner scopes
• Reusing typedef names as parameter names in declarations and definitions
• Duplicated typedef declarations (non-standard, but apparently common and syntactically
  similar to the above)

There is a corner case regarding parameter name scoping that required access to yacc.py's lookahead token (see p_direct_declarator_5 in c_parser.py for details). There are three solutions as I see it:

• Modify yacc.py to expose the lookahead token as an attribute of the parser (...but requiring future PLY updates to be merged, not copied)
• Keep track of the most-recent token via a custom tokenfunc (...but trusting that the parser's lookaheadstack is empty)
• Use the inspect module to grab the value of the parser's lookahead variable (...but direct
  inspection of Python frames and local dictionaries is pretty evil)

For the purpose of this change I decided to use the inspect module, as it is the least error-prone (it can inspect lookaheadstack to ensure it's empty) and the least invasive to the codebase. I can instead make lookahead/lookaheadstack attributes, if you're willing to support the merging of PLY.

[eliben]

In the description of alternatives, you've mentioned:

Keep track of the most-recent token via a custom tokenfunc (...but trusting that the parser's lookaheadstack is empty)

I'm not sure we care much about lookaheadstack here, as it seems to be only used for error recovery. Your code just asserts it anyway. Would this make the idea of using a custom tokenfunc more feasible?

[Syeberman]

I'll look into it.

[eliben]

In the description of alternatives, you've mentioned:

Keep track of the most-recent token via a custom tokenfunc (...but trusting that the parser's lookaheadstack is empty)

I'm not sure we care much about lookaheadstack here, as it seems to be only used for error recovery. Your code just asserts it anyway. Would this make the idea of using a custom tokenfunc more feasible?

[Syeberman]

I'll be updating the pull request shortly to use tokenfunc rather than inspect.

As for separating the struct field namespace to a separate patch: it's quite dependent on this change. "unsigned TT" looks very similar to the parser whether TT is being redeclared as a parameter, a variable, or a structure member.

[eliben]

If separating struct field namespace lookups is too hard, then never mind - you can keep them combined. I'm looking forward to the new patch with tokenfunc.

[Syeberman]

The pull request has been updated for tokenfunc: see 2f84c0d.

[eliben]

Looks better, thanks. Can you now rebase the whole thing vs. tip so it's merge-able?

[Syeberman]

It might take a few weeks to make that change, though. (Other priorities.) Is the current version sufficient for now?

[eliben]

The current version with tokenfunc is fine. I just need a single commit for the merge (squash).

[Syeberman]

The squashed commit should be available now.

[eliben]

Merged. Thanks!