Tracking issue for `concat_idents`

[aturon]

Tracks stabilization for the concat_idents macro.

[retep998]

concat_idents is so fundamentally useless at the moment because macros cannot be used in ident positions.

[Toby-S]

Yeah it is pretty much useless right now (see #12249 #13294).

[nrc]

Relevant blog for future plans: http://www.ncameron.org/blog/untitledconcat_idents-and-macros-in-ident-position/

[eddyb]

A solution that has been thrown around on IRC (but never ended up anywhere else AFAIK):
Have a macro invocation form for "early expansion", i.e. the following would work without macros in ident position:

macro_rules! wrap {
    ($name:ident) => { struct concat_idents!!(Wrapped, $name)($name); }
}

The "early expansion" macro_name!!(args) syntax was originally $*macro_name!(args) and is a subject of bikeshed.

If we move towards having all macros produce token streams that are parsed on-demand, such "early expansion" could be used in many more locations without adding macro support for each one.
Generic parameters, ADT fields, function signatures and match arms come to mind - there are so many recursive push-down hacks macros can end up with, just to construct lists of things and whatnot, that would simply not be necessary with "early expansion".

The only disadvantage is that concat_idents!! and friends would only work inside the RHS of macro_rules!, but I don't really see why you would need to use such a capability outside of a macro.

[skade]

https://github.com/mikkyang/rust-blas uses it for quick notation of FFI functions that follow a certain scheme: https://github.com/mikkyang/rust-blas/blob/master/src/prefix.rs https://github.com/mikkyang/rust-blas/blob/master/src/matrix/ll.rs

[skade]

Well, given that mikkyang/rust-blas#12 is a nicer solution within the current language, I'd also like to raise my hand for "useless".

[nrc]

I think we should never stabilise the current version - as others have noted it is useless.

[aidanhs]

Since this is the tracking issue, other who stumble across it might be interested in https://github.com/SkylerLipthay/interpolate_idents which works on nightly rust.

[retep998]

Unfortunately that solution will always depend on nightly Rust (until the day that syntax extensions become stable and pigs fly).

[skade]

Given that the consensus seems to be "useless" and theres nicer solutions (e.g. [1], maybe we can remove this feature?

[1] https://github.com/mikkyang/rust-blas/pull/12/files

[eddyb]

See rust-lang/rfcs#1628.

[aidanhs]

@retep998 yes, just noting it as a stopgap.

@skade it's the implementation rather than the idea that's useless. If concat_idents is removed rather than fixed, something else needs to fill its place (your solution doesn't work for all use-cases). I like the look of the rfc @eddyb linked, will follow that.

[skade]

@aidanhs that still means that we should drop the feature to make sure no one uses it.

I agree that my solution doesn't cover all edge-cases, but all uses that I've currently seen in the wild. Also, but this is outside of my competence to decide, I think this is a perfect case for the use of a code generator.

[aidanhs]

@skade afaict nobody can use it, so that's not a big concern...but equally there's no reason to keep it around. My main motivation for leaving it was as a reminder that people do want the functionality, but I'm content to follow the RFC and have changed my position to "I don't mind either way".

Regarding use cases, here's the motivating example that brought me to this issue in the first place. Codegen's awesome for things like phf, but it feels a bit like overkill when I just want to generate some repetitive methods to extract tuples from enum variants. I guess it's down to personal preference.

[ExpHP]

An amusing fact about the current implementation of concat_idents! is that it will accept an empty argument list, making it possible to construct the empty string as an identifier:

error[E0425]: cannot find value `` in this scope
 --> src/main.rs:5:5
  |
5 |     concat_idents!();
  |     ^^^^^^^^^^^^^^^^^ not found in this scope

(good luck actually doing anything with it)

[petrochenkov]

construct the empty string as an identifier

Empty string is used as a reserved identifier with special meaning in several places in the compiler, so this can potentially cause issues.

[jonhoo]

The linked RFC has been closed as postponed. Should concat_idents! now be removed? Is there any chance of seeing something in its place that supports generating identifiers?

[durka]

The RFC should be reopened. It never should have been closed, as no better solution was proposed.

[skade]

@durka I don't agree. There's ample reason to expect a new one when moving towards macros 2.0. RFCs are not necessarily closed to be replaced immediately, they are closed to stop following that line of discussion. Future RFCs might refer to it.

[alexreg]

I'm with @durka on this.

[petrochenkov]

@rfcbot concern: Audit hygienic context of the produced identifier, add relevant tests

Context of the ident should probably be the context of concat_idents! macro invocation (#50122), aka Span::call_site().

[alexreg]

@durka Hygiene opt-out/escaping landing, I mean.

[durka]

OK, that's a lot more uses than I thought! I guess it's hard to replace them all with mashup because you'd have to generate so many macros?

…

On Thu, May 3, 2018, 09:12 Alexander Regueiro ***@***.***> wrote: @duroa Hygiene opt-out/escaping landing, I mean. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#29599 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAC3n3SwnrGXRRIvhOjPm6wStlrGBXBOks5tuwJWgaJpZM4GcPel> .

[alexreg]

@durka Yeah. I feel concat_idents + early expansion + hygiene escaping is the more elegant and general solution that fits all potential use cases.

[dtolnay]

@petrochenkov currently concat_idents cannot refer to a captured local variable, only an item and items do not have hygiene. Do you think concat_idents is different enough from other cases of non-atomic code fragments that we need to block stabilization until the non-atomic hygiene story is worked out? I would think that can happen later and does not need to be a blocker.

[petrochenkov]

@dtolnay

currently concat_idents cannot refer to a captured local variable

That's a bug.

and items do not have hygiene.

All identifiers have hygiene with macros >= 1.2!
concat_idents looks like a typical use case for Span::call_site hygiene.

Do you think concat_idents is different enough from other cases of non-atomic code fragments that we need to block stabilization until the non-atomic hygiene story is worked out?

concat_idents is not special, #50122 (or rather its minimal subset about call-site for macro invocations) is a blocker for it the same degree as for Macros 1.2 using Span::call_site in general.

[durka]

What should happen with concat_idents!($a, $b) when $a and $b come from different hygiene contexts?

…

On Thu, May 3, 2018 at 1:55 PM, Vadim Petrochenkov ***@***.*** > wrote: @dtolnay <https://github.com/dtolnay> currently concat_idents cannot refer to a captured local variable That's a bug. and items do not have hygiene. All identifiers have hygiene with macros >= 1.2! concat_idents looks like a typical use case for Span::call_site hygiene. Do you think concat_idents is different enough from other cases of non-atomic code fragments that we need to block stabilization until the non-atomic hygiene story is worked out? concat_idents is not special, #50122 <#50122> (or rather its minimal subset about call-site for macro invocations) is a blocker for it the same degree as for Macros 1.2 in general. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#29599 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAC3n8i0Von0gNWwefFkN6Z9NQOoOqFeks5tu0SUgaJpZM4GcPel> .

[petrochenkov]

@durka
I'd expect contexts of components to be completely lost during concatenation, it's a purely string-content based operation after all.

[alexreg]

@durka I'd expect the same as @petrochenkov. The identifier would then take the syntax context of its call site (i.e. where it's used in code), unless escaped (e.g. using # syntax). We have discussed the #[escapes(...)] approach to escaping hygiene as well, but unless we allow macro expansion within escapes(...), this becomes a real problem.

[durka]

OK, that makes sense. I just remember that being a question that came up around this macro before.

…

On Thu, May 3, 2018 at 2:23 PM, Alexander Regueiro ***@***.*** > wrote: @durka <https://github.com/durka> I'd expect the same as @petrochenkov <https://github.com/petrochenkov>. The identifier would then take the syntax context of its call site (i.e. where it's used in code), unless escaped (e.g. using # syntax). We have discussed the #[escapes(...)] approach to escaping hygiene as well, but unless we allow macro expansion within escapes(...), this becomes a real problem. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#29599 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAC3n0lFw5QraOUxg9z5wJVGp2ANhu8Cks5tu0sigaJpZM4GcPel> .

[nrc]

@concern premature

I don't think we should stabilise concat_idents, between the macro system and the macro itself, this is not the macro that most macro authors want, but it is occupying prime naming real estate. I would prefer to wait until we have a macro that does do what people want before stabilising something.

Is there a strong reason to stabilise this now when we've been OK without stabilising it for years? It does seem like a really sub-optimal thing to stabilise.

[alexreg]

@nrc It does what it says on the tin though, as has been pointed out. Shouldn't the aim just be to work on early (eager) expansion now?

[durka]

So the main issue with "it doesn't do what macro authors want" is that it only works in expressions, right? It doesn't work for defining things because macros aren't accepted in ident position. Some possible mitigations: 1. Just make that extremely clear in the documentation 2. Change the name somehow to reflect it (concat_idents_for_expr? make_expr_name?). If eager expansion actually happens the name would be awkward, but we could deprecate and change the name back in that case. 3. Make the error message better when discovering a macro invocation in ident position (currently it's just unexpected `!` with no notes or helps)

…

On Sun, May 6, 2018 at 12:14 PM, Alexander Regueiro < ***@***.***> wrote: @nrc <https://github.com/nrc> It does what it says on the tin though, as has been pointed out. Shouldn't the aim just be to work on early (eager) expansion now? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#29599 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAC3n4JGRm5swlmIdV-0ryEkiAjWQU1yks5tvyFKgaJpZM4GcPel> .

[alexreg]

1. or 3. sound best right now.

[dtolnay]

I continue to believe this macro in its current form serves as a valuable building block for API design despite being limited to solving a specific set of use cases and not them all. There comes a time in an experienced macro author's life when they have seen many dozen situations that require exactly this functionality and wonder what the holdup could be after almost 7 years. But I can see that changing so many minds is not going to be a good use of two teams' time. Thanks for the discussion everyone! 🙂

@rfcbot fcp cancel

[rfcbot]

@dtolnay proposal cancelled.

[alexreg]

@dtolnay Sounds fair. Maybe we can visit this when hygiene escape support and eager expansion land, both of which I'm currently working on.