Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Folder #85

Merged
merged 1 commit into from
Jan 29, 2017
Merged

Implement Folder #85

merged 1 commit into from
Jan 29, 2017

Conversation

gnzlbg
Copy link
Contributor

@gnzlbg gnzlbg commented Jan 26, 2017

Closes #43 .

Adds a new feature: "fold".

The Folder trait allows implementing AST->AST transformations.

If the way it is implemented looks good, adding the documentation is trivial, but adding some tests will take some work.

@dtolnay
Copy link
Owner

dtolnay commented Jan 27, 2017

Thanks! This is exactly what I wanted. I don't have time to review today but I will get to it tomorrow.

@gnzlbg
Copy link
Contributor Author

gnzlbg commented Jan 27, 2017

@dtolnay I think we should strive to provide a MacroExpander Folder in syn that is able to expand all macros (otherwise Folder is really hard to use correctly for anything). I guess that basically would require reimplementing libsyntax::ext::expand on top of Folder.

@mystor
Copy link
Collaborator

mystor commented Jan 27, 2017

@gnzlbg I think that a MacroExpander Folder would fit more in an external helper crate which can be used with syn than directly inside of syn. I feel like syn should keep itself mostly concerned only with parsing, and not with how the results of those parses will be used.

The main reason why I think fold and visit are in syn is because their implementation is so tightly tied to each version of syn's internal representation.

This is all up to @dtolnay though - so if he thinks that MacroExpander has a place in syn, then so be it :).

@gnzlbg
Copy link
Contributor Author

gnzlbg commented Jan 27, 2017

@mystor I haven't written MacroExpander yet, so I don't know how tightly it would need to be integrated with syn. Ideally one could write it as a Folder, but this is not the way libsyntax does it.

So... I am a bit stumbled about how to proceed. I don't know enough about macro expansion to know if even attempting to write it as a Folder make sense. Maybe it should be "it's own thing" like in libsyntax, and more tightly integrated with parsing, since maybe macro expansion should just be a TokenTree -> TokenTree transformation, instead of an AST -> AST transformation.

Even if we were to implement it as a Folder, I guess we still would need to have some expand function that performs the TokenTree -> TokenTree transformation and is used by the Folder. I would expect this function to be tightly integrated with syn.

@dtolnay
Copy link
Owner

dtolnay commented Jan 27, 2017

I don't think syn needs to be in the business of pretending that we can fully expand macros in your crate. The reality is that only rustc can make that guarantee. Procedural macros need to accept this and work in a way that handles unexpanded macros correctly.

I would be in favor of having fold_mac implemented as noop_fold_mac by default. The situation in libsyntax is a bit different - if rustc is folding something that contains unexpanded macros, that is probably a bug. In syn the expected use cases are ones where macro expansion is not available so folders need to be written in a way that supports unexpanded macros. In other words finding unexpanded macros in the input is the expected state of the world and does not indicate a bug the way it would in rustc.

/// If you want to ensure that your code handles every variant explicitly, you
/// need to override each method and monitor future changes to `Folder` in case
/// a new method with a new default implementation gets introduced.
pub trait Folder: Sized {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need this Sized bound. Unclear why libsyntax has it.

Copy link
Contributor Author

@gnzlbg gnzlbg Jan 29, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

syn::Visitor has this as well, without it, I cannot call any of the noop_ functions in the default methods.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is just a matter of accepting <F: ?Sized + Folder> in the noop functions. I will see if I can make it work.

fn fold_ty_param_bound(&mut self, bound: TyParamBound) -> TyParamBound {
noop_fold_ty_param_bound(self, bound)
}
fn fold_poly_trait_ref(&mut self,
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's take only the PolyTraitRef as input here, the way libsyntax does.

-> PolyTraitRef {
noop_fold_poly_trait_ref(self, trait_ref, modifier)
}
fn fold_variant_data(&mut self,
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's take only the VariantData as input.

fn fold_field(&mut self, field: Field) -> Field {
noop_fold_field(self, field)
}
fn fold_variant(&mut self, variant: Variant, generics: Generics) -> Variant {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's take only the Variant as input.

noop_fold_stmt(self, stmt)
}

/// Prefer overriding fold_stmt instead!
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why? Block shows up in lots of other places.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think one often wants to fold over a Block, so Folder should offer a .fold_block method, but since a Block is just a vector of Stmts, one typically wants to override fold_stmt instead (and not fold_block).

@gnzlbg
Copy link
Contributor Author

gnzlbg commented Jan 29, 2017

@dtolnay I've fixed all issues you mentioned and rebased the PR :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants