Macro expansion #17
Comments
|
Here're related RFCs:
|
|
The current parser will return the If we just store its So I think we may have to maintain 2 copies of each macro definition. This is not the best way, but could be optimized in the future refactoring. Any better idea? |
|
Wouldn’t you be able to store a reference to the MacroItem (or rather a reference to a unique_ptr containing the MacroItem) in the table? Alternatively, you could parse the macro definition into a different form during the expansion phase and put the results of that into the table, and maybe even remove the macro definition item from the constructed AST after that stage if it isn’t required anymore. Or just move the MacroItem unique_ptr into the table while in the expansion phase, so that the MacroItem is still in the AST during the AST dump phase and so remains in the dump. |
Yes, I can, but sounds not safe to store the unwrapped reference into the table, since it's not unique anymore.
I think you're suggesting a pass to eliminate dead macros, yes I can do that.
But I have to search for the MacroItem in expansion by traversing AST for the first time. Let me elaborate on the situation for other readers here: the parser interfaces are all unified to return unique_ptr, so I have no chance to move it into a lookup table before return it. But I can do it in expansion since the reference is unique. |
|
Yeah, I think I misunderstood what you were trying to say. I assumed you originally meant that you'd do a "macro definition finding" pass over the AST that located all definitions and stored a parsed form in a table.
Not even mainly centred around "eliminating" macros rather than just "detecting" them, but yes, that's what I was suggesting. It would be the same kind of thing as a pass for name resolution - the AST is recursively traversed and any macro definitions found can be parsed to a usable form and stored in a table (or possibly some kind of reverse-linked list of tables to preserve scope or something).
We could take the scorched-earth approach and make everything shared_ptr if we can't find any other satisfactory options, though I'm a bit worried about the performance implications of that. |
I think we may have to do it in the
Definitely, I just want to make the minimum change. So far, I think unique_ptr is not a bad choice, we don't have to change if the macro is the only concern. |
|
@NalaGinrut Are you currently working on this? If not, I was intending to have a try at creating some of the macro expansion code. |
|
Currently, no. I'm very busy on my startup issues. Feel free to try what you like. I'll be back ASAP. |
|
One issue I have discovered while attempting to start introducing macro expansion is how the parsing of Essentially, Anyway, the issue I encountered was in implementing this "matching" feature. I have thought of a few approaches:
Does anyone else have any opinions or thoughts on the possible options we could take here? I'm a bit unsure on how to proceed, but I'm leaning towards the first option (at least initially). |
|
My gut feeling would be approach 1 or 3. I doubt many people have much experience with Earley parsers. Approach 3 seems reasonable and might be simpler to implement compared to approach 1, but i don't have any real opinion either way. |
|
Before I make my comment, I have to mention that we're still in brainstorm, so I'm not pushing any of my suggestions. The Can we make our design more flexible that we can replace the algorithm in the future? If so, we may use |
I was also leaning towards implementing a "backtracking"-style algorithm (i.e. option 1). As for replacing the algorithm, I'm sure that the macro parser can be implemented in such a way that whatever calls the |
|
@SimplyTheOther I think your opinion is to hygienically abstract it to |
This is the first pass at implementing macros more testcases are needed. This does not support repetition matchers but it supports simple declarative macros and transcribes them. The approach taken here is that we reuse our existing parser to call the apropriate functions as specified as part of the MacroFragmentType enum if the parser does not have errors parsing that item then it must be a match. Then once we match a rule we have a map of the token begin/end offsets for each fragment match, this is then used to adjust and create a new token stream for the macro rule definition so that when we feed it to the parser the tokens are already substituted. The resulting expression or item is then attached to the respective macro invocation and this is then name resolved and used for hir lowering. Fixes #17 #22 Addresses #573
This is the first pass at implementing macros more testcases are needed. This does not support repetition matchers but it supports simple declarative macros and transcribes them. The approach taken here is that we reuse our existing parser to call the apropriate functions as specified as part of the MacroFragmentType enum if the parser does not have errors parsing that item then it must be a match. Then once we match a rule we have a map of the token begin/end offsets for each fragment match, this is then used to adjust and create a new token stream for the macro rule definition so that when we feed it to the parser the tokens are already substituted. The resulting expression or item is then attached to the respective macro invocation and this is then name resolved and used for hir lowering. Fixes #17 #22 Addresses #573
This is the first pass at implementing macros more testcases are needed. This does not support repetition matchers but it supports simple declarative macros and transcribes them. The approach taken here is that we reuse our existing parser to call the apropriate functions as specified as part of the MacroFragmentType enum if the parser does not have errors parsing that item then it must be a match. Then once we match a rule we have a map of the token begin/end offsets for each fragment match, this is then used to adjust and create a new token stream for the macro rule definition so that when we feed it to the parser the tokens are already substituted. The resulting expression or item is then attached to the respective macro invocation and this is then name resolved and used for hir lowering. Fixes #17 #22 Addresses #573
This is the first pass at implementing macros more testcases are needed. This does not support repetition matchers but it supports simple declarative macros and transcribes them. The approach taken here is that we reuse our existing parser to call the apropriate functions as specified as part of the MacroFragmentType enum if the parser does not have errors parsing that item then it must be a match. Then once we match a rule we have a map of the token begin/end offsets for each fragment match, this is then used to adjust and create a new token stream for the macro rule definition so that when we feed it to the parser the tokens are already substituted. The resulting expression or item is then attached to the respective macro invocation and this is then name resolved and used for hir lowering. Fixes #17 #22 Addresses #573
938: First pass at declarative macro expansion r=philberty a=philberty
This does not support repetition matchers but it supports simple
declarative macros and transcribes them. The approach taken here is that
we reuse our existing parser to call the apropriate functions as specified
as part of the MacroFragmentType enum if the parser does not have errors
parsing that item then it must be a match.
Then once we match a rule we have a map of the token begin/end offsets
for each fragment match, this is then used to adjust and create a new token
stream for the macro rule definition so that when we feed it to the parser
the tokens are already substituted. The resulting expression or item is
then attached to the respective macro invocation and this is then name
resolved and used for hir lowering.
Fixes #17 #22
Addresses #573
Co-authored-by: Philip Herron <philip.herron@embecosm.com>
The macro expansion happens before the resolution.
This issue is opened for the record of pipeline in kanban.
The text was updated successfully, but these errors were encountered: