consider forwards compatibility

[nrc]

I realise you're still pre-1.0, but it is probably worth considering the forwards compatibility story early. A reason that macros are moving to tokens/strings rather than AST is for the stability of procedural macros. This requires that libraries such as Syn have a forwards compatibility policy.

The idea is that when we add a feature to Rust which changes the syntax, you need to add this to Syn, and to handle such code downstream proc macros must upgrade to the new version. But, the important thing is that if downstream macros do not want to handle the new feature, they can upgrade Syn without breaking.

E.g., say we add unions (well, we already have, but imagine we hadn't), then where previously an Item enum had Struct and Enum variants, now it needs Union too. That would be a breaking change, since exhaustive matches are no longer exhaustive.

Exactly how you handle forwards compatibility is an open question. You could combine unenforced policy that clients have to abide be with some degree of coding techniques, or you could use solely code (but that might not be possible with some other design decisions).

One example might be that clients should not pattern match structs, Syn avoids struct variants, and every enum has an Unknown variant which clients should not match.

[dtolnay]

A reason that macros are moving to tokens/strings rather than AST is for the stability of procedural macros. This requires that libraries such as Syn have a forwards compatibility policy.

I don't think that follows. Procedural macros don't declare a dependency on a specific compiler version, but they do declare a dependency on a specific Syn version and Cargo is good at managing those. By choosing a version they choose a snapshot of the Rust syntax to support. This is no different from using Syntex which was sort of the plan for how Macros 1.1 would be used, specifically mentioned in the RFC.

The biggest benefit over Syntex after compile time is that I expect Syn to be easier to maintain by a factor of 10x to 50x. This is from the person who has been maintaining Syntex for the past several months. The current release is out of date by 2 weeks just because the merge conflict is so bad. I do not foresee that happening with Syn because every syntax change will be just a few minutes to implement. That means, in contrast to Syntex, procedural macros will be able to support new syntax almost immediately rather than waiting for potentially weeks.

I agree there is something around pattern matching that could be done in a better way. I will keep thinking about that. Thanks for flagging this!

[nrc]

First of all, s/forwards/backwards, sorry about the confusion there.

You are right that Cargo can manage the dependency issues without intervention, and we avoid most of the back-compat nightmare that is current macros simply by not depending on the compiler version.

I guess what I'm talking about is steps you can take to make having to bump a major version number less likely. I.e., it would be nice if adding say struct field defaults only meant going from v3.2 to 3.3 rather than 4.0.

[dtolnay]

I don't have a good answer to this. We will bump a major version when syntax changes require it, as infrequently as reasonable, and count on Cargo to sort out dependencies on syn.