macros: expand items before their `#[derive]`s #41029

jseyfried · 2017-04-03T05:57:07Z

For example, after this PR the following are equivalent:

#[derive(Example)] // for any `#[proc_macro_derive]` or built-in derive `Example`
struct S(i32);

macro_rules! m { () => { i32 } }
#[derive(Example)]
struct S(m!());

Note that today, #[cfg]s in an item are processed before expanding #[derive]s. For example,

#[derive(Example)] // `Example` will see `struct S();`
struct S(#[cfg(any())] i32);

After this PR, we can consider #[cfg] to be a normal attribute macro (assuming attribute macros were allowed everywhere #[cfg] is allowed).

Since type macros and #[proc_macro_derive] are stable, this is a [breaking-change].

r? @nrc

jseyfried · 2017-04-03T05:59:43Z

cc #38356
cc @keeperofdakeys @dtolnay @sgrif

nrc · 2017-04-04T02:19:08Z

cc @alexcrichton @rust-lang/lang

nrc · 2017-04-04T02:20:01Z

Other than bringing cfg into line, what is the motivation for this change?

Could we Crater this?

jseyfried · 2017-04-04T07:47:10Z

@nrc
Yeah, the motivation is to allow user defined cfg-like attribute procedural macros to be as powerful as cfg, as opposed to being "second-class".

Also, derives usually want expanded code, e.g.

if we didn't cfg-process first, we'd break most derives
we no longer have to forbid type macros in #[derive(<built-in>)] structs ('rustc' panicked at 'visit_mac disabled by default' #32950) after this PR

alexcrichton · 2017-04-04T17:45:13Z

Seems fine to me!

nikomatsakis · 2017-04-05T17:12:31Z

Seems like we could detect some breakage here via a crater run, for sure.

sgrif · 2017-04-05T20:04:55Z

To be clear, this change would mean that #[cfg(foo)] on a struct field would require custom code from every derive that could be affected by it? Big 👎 from me in that case, then. It would certainly break Diesel

withoutboats · 2017-04-05T20:18:44Z

@sgrif my understanding of this change is that cfg attributes will not change at all. Its that other macros inside a struct will be expanded in the same way that cfg is.

jseyfried · 2017-04-05T20:33:11Z

@withoutboats exactly.

sgrif · 2017-04-05T21:21:37Z

Ah. Big 👍 from me then. XD

nrc

r=me with the tests fixed and a comment on PartialExpansion. I think that in general the expansion code is getting pretty complex, if you get a chance it would be great if you could add some docs giving an overview of the process and data structures, and some detailed comments around the place to explain what is going on (in another PR, doesn't need to block this one).

nrc · 2017-04-09T22:13:17Z

src/librustc_resolve/macros.rs

@@ -156,16 +156,20 @@ impl<'a> base::Resolver for Resolver<'a> {
        self.whitelisted_legacy_custom_derives.contains(&name)
    }

-    fn visit_expansion(&mut self, mark: Mark, expansion: &Expansion, derives: &[Mark]) {
+    fn visit_expansion(&mut self, mark: Mark, expansion: &Expansion, derives: &[(Mark, Path)]) {


I would think about making this new tuple into a struct - it's OK for now, but I think it is a bit opaque to identify a derive by a (Mark, Path).

nrc · 2017-04-09T22:16:32Z

src/libsyntax/ext/expand.rs

    monotonic: bool, // c.f. `cx.monotonic_expander()`
 }

+struct PartialExpansion {


Could you add a comment here please - it would be good to explain how partial expansions come about - are they just intermediate state or are they stored like this, in what circumstances is a macro partially rather than fully expanded, and are there invariants that can be assumed about partial expansions.

Will do. Partial expansion are macro expansions that have unexpanded macro invocations in them. That is, the macro itself has resolved and been expanded, but it created more macro invocations that have yet to be expanded. This is in contrast to "fully expanded" AST, which has no macro invocations left.

carols10cents · 2017-04-17T12:17:01Z

Friendly ping to keep this PR on your radar @jseyfried! ❤️

shepmaster · 2017-04-21T14:34:41Z

Looks like there's some minor tweaks requested, @jseyfried — do you think you will be able to address those soon?

jseyfried · 2017-04-21T18:02:18Z

@carols10cents @shepmaster Yeah, this is still on my radar -- I plan on fixing the tweaks and r+ing this weekend (been super busy these last two weeks).

alexcrichton · 2017-05-04T18:26:10Z

I'm going to close this out of inactivity for now, but @jseyfried you're of course welcome to resubmit at any time! (just trying to help clear out the queue)

abonander · 2018-02-19T13:02:22Z

@jseyfried are you going to revive this PR?

…henkov Expand items before their derives continuation of rust-lang#41029 At this point all I've done is naively rebased on top of `master` and fixed the code enough to compile. I'm not even sure if it works because my local build stopped here: ``` C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE\VC\VCTargets\Microsoft.CppBuild.targets(317,5): error MSB3491: Could not write lines to file "emscripten -optimizer.dir\Release\emscript.63209ED8.tlog\emscripten-optimizer.lastbuildstate". The specified path, file name, or both are too long. The fully qualified file name must be less than 260 characters, and the directory name must be less than 248 characters. [C:\Users\cyber\Rust\rust\build\x86_64-pc-windows-msvc\stage0-rustc\x86_64-pc-windows-msvc\release\bui ld\rustc_binaryen-aaf5e3b308543526\out\build\src\emscripten-optimizer\emscripten-optimizer.vcxproj] ``` So I'm going to be relying on Travis to run tests unless I can fix this error.

rust-highfive assigned nrc Apr 3, 2017

jseyfried mentioned this pull request Apr 3, 2017

Tracking issue for RFC 1566: Procedural macros #38356

Closed

31 tasks

jseyfried force-pushed the expansion_order branch 3 times, most recently from 536d411 to 370a837 Compare April 3, 2017 06:46

jseyfried added 2 commits April 3, 2017 07:58

Expand items before their #[derive]s.

df1405d

Add test.

3b6b779

jseyfried force-pushed the expansion_order branch from 370a837 to 3b6b779 Compare April 3, 2017 07:58

nrc approved these changes Apr 9, 2017

View reviewed changes

Mark-Simulacrum added the S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. label Apr 14, 2017

alexcrichton closed this May 4, 2017

abonander mentioned this pull request Feb 23, 2018

Expand items before their derives #48465

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

macros: expand items before their `#[derive]`s #41029

macros: expand items before their `#[derive]`s #41029

jseyfried commented Apr 3, 2017 •

edited

jseyfried commented Apr 3, 2017

nrc commented Apr 4, 2017

nrc commented Apr 4, 2017

jseyfried commented Apr 4, 2017

alexcrichton commented Apr 4, 2017

nikomatsakis commented Apr 5, 2017

sgrif commented Apr 5, 2017

withoutboats commented Apr 5, 2017 •

edited

jseyfried commented Apr 5, 2017

sgrif commented Apr 5, 2017

nrc left a comment

nrc Apr 9, 2017

nrc Apr 9, 2017

jseyfried Apr 10, 2017

carols10cents commented Apr 17, 2017

shepmaster commented Apr 21, 2017

jseyfried commented Apr 21, 2017

alexcrichton commented May 4, 2017

abonander commented Feb 19, 2018

macros: expand items before their #[derive]s #41029

macros: expand items before their #[derive]s #41029

Conversation

jseyfried commented Apr 3, 2017 • edited

jseyfried commented Apr 3, 2017

nrc commented Apr 4, 2017

nrc commented Apr 4, 2017

jseyfried commented Apr 4, 2017

alexcrichton commented Apr 4, 2017

nikomatsakis commented Apr 5, 2017

sgrif commented Apr 5, 2017

withoutboats commented Apr 5, 2017 • edited

jseyfried commented Apr 5, 2017

sgrif commented Apr 5, 2017

nrc left a comment

Choose a reason for hiding this comment

nrc Apr 9, 2017

Choose a reason for hiding this comment

nrc Apr 9, 2017

Choose a reason for hiding this comment

jseyfried Apr 10, 2017

Choose a reason for hiding this comment

carols10cents commented Apr 17, 2017

shepmaster commented Apr 21, 2017

jseyfried commented Apr 21, 2017

alexcrichton commented May 4, 2017

abonander commented Feb 19, 2018

macros: expand items before their `#[derive]`s #41029

macros: expand items before their `#[derive]`s #41029

jseyfried commented Apr 3, 2017 •

edited

withoutboats commented Apr 5, 2017 •

edited