Tracking issue for quote 1.0

[dtolnay]

Current prerelease: quote-next = "1.0.0-rc3"
Release notes in progress:

---

This release fixes some longstanding limitations of the quote! macro, bringing quote better in line with the patterns that macro authors are used to from working with macro_rules.

Duplicate interpolations in a repetition

In past versions of quote, interpolating the same variable multiple times inside of one repeating block was not allowed.

For example if we had an iterator of local variable names and wanted the generated code to do a clone of each one, that wouldn't compile:

error[E0416]: identifier `var` is bound more than once in the same pattern
 --> src/main.rs:7:27
  |
7 |         #( let #var = #var.clone(); )*
  |                        ^^^ used in a pattern more than once

Macros usually worked around this by having the same sequence of interpolated values named two different things:

// old workaround 1
let var = input.fields.iter().map(|field| &field.ident);
let var2 = var.clone();

quote! {
    #( let #var = #var2.clone(); )*
}

or by giving up on repetitions and doing more of the work within the iterator chain:

// old workaround 2
let var_clones = input
    .fields
    .iter()
    .map(|field| {
        let var = &field.ident;
        quote! {
            let #var = #var.clone();
        }
    });

quote! {
    #(#var_clones)*
}

As of quote 1.0, these workarounds are no longer necessary and values can be used as many times as necessary in a repetition. This fix is thanks to a technique and implementation by @mystor.

// quote 1.0
let var = input.fields.iter().map(|field| &field.ident);

quote! {
    #( let #var = #var.clone(); )*
}

Non-repeating interpolations in a repetition

In past versions of quote, every interpolated variable in a repetition had to have an IntoIterator impl, where each iteration of the repetition would interpolate the next item from each of the iterators. Thus it wasn't possible to mix repeating and non-repeating values inside the same repetition.

// previously nonworking code
let obj = &input.ident;
let field = input.fields.iter().map(|field| &field.ident);

quote! {
    #( #obj.#field.print(); )*
}

error[E0599]: no method named `into_iter` found for type `proc_macro2::Ident` in the current scope
 --> src/main.rs:8:13
  |
8 |     #( #obj.#field.print(); )*
  |         ^^^

What's much worse, the IntoIterator impl could have extremely surprising and undesirable behavior when placing a TokenStream (such as produced by quote) inside a repetition in a subsequent quote.

// previously insane behavior
let default_impl = quote!(unimplemented!());
let names = &input.function_names;

quote! {
    #(
        fn #names() { #default_impl }
    )*
}

What the macro author probably wanted the example above to produce was:

fn fname1() { unimplemented!() }
fn fname2() { unimplemented!() }
...

Instead what they would have gotten is:

fn fname1() { unimplemented }
fn fname2() { ! }
fn fname3() { () }

where because the default_impl TokenStream is used in the repetition, quote calls its IntoIterator impl which for TokenStreams produces one token of the stream at a time. The tokens of the default_impl thus get smeared across consecutive iterations of the repetition.

All of this is fixed in quote 1.0. Repetition is no longer based on the IntoIterator trait. Instead we separately handle any interpolated value that implements Iterator (by interpolating one item from the iterator per repetition), any value that implements ToTokens (by interpolating the entire thing each repetition), and a small additional set of common types like &[T] and Vec<T> and BTreeSet<T> which interpolate one element per iteration but themselves are not an Iterator.

This fix is thanks to a technique by @Goncalerta and an implementation by @mystor.

Reusing values across repetitions

As a side benefit of no longer relying on the IntoIterator trait, owned values like Vec are no longer consumed the first time they are used in a repetition.

// code that did not compile before, but works in quote 1.0
let field = vec![a, b, c];

quote! {
    struct S {
        #( #field: String, )*
    }
    
    impl S {
        fn print(&self) {
            #( self.#field.print(); )*
        }
    }
}

error[E0382]: use of moved value: `field`
  --> src/main.rs:10:13
   |
11 |         struct S {
12 |             #( #field: String, )*
   |                 ^^^^^ value moved here
...  
17 |                 #( self.#field.print(); )*
   |                          ^^^^^ value used here after move

No more deep recursion

The quote macro used to perform potentially deep recursive invocations in the style of a "tt muncher" macro. For large invocations this would sometimes require placing #![recursion_limit = "256"] or even higher in the proc macro crate in order for it to compile.

This is no longer necessary because quote no longer performs one-per-token recursive invocations. In quote 1.0 the recursion depth is proportional to the nesting depth (maximum number of nested parens/brackets/braces in the input) rather than total number of tokens, and nesting depth is typically far less than the compiler's default recursion limit.

The fix is thanks to a technique and implementation by @qnighy.

Macro for creating an ident

Quote 1.0 adds a format_ident! macro which works like format! but produces an Ident instead of string. When interpolated inside quote, idents are unquoted (like variable names) where strings would be quoted (like string literals).

let arg = format_ident!("arg{}", i);

// previously would have been written more verbosely as:
let arg = Ident::new(&format!("arg{}", i), Span::call_site());

There are some additional Ident-specific features of format_ident, like control over the Span of the identifier for hygiene and error reporting purposes. Refer to the rustdoc for details.

The idea for this macro and the implementation were contributed by @mystor.

Breaking changes

• Minimum required Rust version is raised from rustc 1.15 to 1.31.

• Repetition is no longer based on the IntoIterator trait.

  As discussed above. If you had interpolated a value inside a repetition that implemented IntoIterator but not Iterator, and was not Vec or &[T] or any of the other types that continue to be explicitly supported, then you will need to turn it into an iterator yourself. Usually this will be a matter of inserting a .iter() call.

• Repetitions with nothing interpolated no longer compile.

  This is intentional to help catch cases where the macro author writes a repetition but forgets to place any interpolation inside of it:

  quote! {
      // should have been #var
      #( let var = Default::default(); )*
  }

  Formerly this would silently produce an empty token stream.

• The quote_spanned macro requires the span to be a Span.

  The quote_spanned! macro accepts a span argument to use as the default span instead of call_site for all tokens originating within the macro invocation. Formerly, when this fallback span was never used because the entire macro body consisted of interpolations, its type was never checked and may have been something other than a Span. In quote 1.0 the span given to quote_spanned must be of type Span no matter what, even when it happens to not be used.

[dtolnay]

https://github.com/dtolnay/quote/releases/tag/1.0.0