New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tracking issue for "Macros 1.1" (RFC #1681) #35900

Closed
nikomatsakis opened this Issue Aug 22, 2016 · 268 comments

Comments

Projects
None yet
@nikomatsakis
Contributor

nikomatsakis commented Aug 22, 2016

Tracking issue for rust-lang/rfcs#1681.

cc @alexcrichton

Stabilization TODO

Litmus tests:

Features:

  • Crate name, currently is proc_macro
  • Crate type, currently proc-macro
  • The #[proc_macro_derive(Foo)] attribute
  • Loading proc-macro crates with -L and #[macro_use] to load them
  • shadowing is an error
  • no hygiene
  • passing a token stream for the entire struct and receiving it all back
  • Cargo manifest attribute, currently proc-macro = true

Known bugs:

Implementation TODO

  • - Create a rustc_macro crate
    • - Have librustc_macro link to libsyntax. Depend on librustc_macro in librustc_driver
    • - Tag rustc_macro as unstable with our standard header.
    • - Only tag rustc_macro with #![crate_type = "rlib"], do not produce a dylib.
    • - Implement the API of rustc_macro using libsyntax's TokenStream internally
    • - tag rustc_macro with a TokenStream lang item so the compiler knows about it.
  • - Add rustc_macro_derive attribute
    • - validate it's of the exact form foo(bar), no other arguments/formats
    • - verify it's only applied to functions
    • - verify it's only applied to functions in the root module
    • - verify the signature with the TokenStream lang item added above
    • - encode all function symbols with rustc_macro_derive into metadata along with the derive mode they're used for.
  • - Add a rustc-macro crate type for other crates
    • - wire it up to produce a dylib
    • - ensure the dylib gets metadata
    • - ensure rustc-macro crates cannot be linked as dylibs
    • - ensure there are no reachable items other than those tagged with #[rustc_macro_derive]
    • - Add cfg(rustc_macro) as an unstable cfg directive, set it for the rustc-macro crate type
    • - ensure that rustc-macro crates link dynamically to libsytnax
  • - Fill in #[macro_use] support for rustc-macro crates
    • - extend library loader to find rustc-macro crates separately from dylib/rlib tracked today when loding crates
    • - Parse metadata for rustc-macro crates to learn about symbol/derive mode pairings
    • - dlopen the dylib
    • - generate an error if any derive mode would shadow any other.
  • - Add cargo integeration
    • - recognize rustc-macro similar to plugin = true
    • - pass --crate-type=rustc-macro when depending on it
    • - plumb the same host/target logic for rustc-macro crates as is present for plugin crates (e.g. always compile rustc-macro crates for hosts)
  • - Tests
    • - smoke test loading a rustc-macro, dummy #[derive] trait
    • - name conflicts are an error
    • - compiling for the wrong architecture is an error
    • - cannot emit crate type rustc-macro and anything else (e.g. rustc-macro+dylib) is an error
    • - span information not being horrendous
    • - removing attributes from a struct, as well as fields
    • - adding impls next to a struct
    • - cross compilation looks for the host rustc-macro crate type
    • - don't load vanilla dylibs as rustc-macro crate types
    • - can't have public exports beyond macro derive functions in a rustc-macro crate
    • - derive macros must have required signature
    • - load two macro crates in one compilation
@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Aug 22, 2016

Member

I've updated the issue description with a checklist of what needs to be done. It's likely not exhaustive but it should get us 90% of the way there hopefully

Member

alexcrichton commented Aug 22, 2016

I've updated the issue description with a checklist of what needs to be done. It's likely not exhaustive but it should get us 90% of the way there hopefully

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Aug 22, 2016

Member

Ok, I'm going to look into this today and see how far I get.

Member

alexcrichton commented Aug 22, 2016

Ok, I'm going to look into this today and see how far I get.

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Aug 24, 2016

Member

cc #35957, an implementation

Member

alexcrichton commented Aug 24, 2016

cc #35957, an implementation

@nrc

This comment has been minimized.

Show comment
Hide comment
@nrc

nrc Aug 29, 2016

Member

From #35957: we should bikeshed the name of the librustc_macro crate some more. In particular, this is intended to be a long-lived crate which will have essentials for all macro authors in it, so limiting to rustc_macro (which in my mind, at least) is just about the 1.1 idea seems bad. I'd previously wanted libmacro for this, but since macro is a reserved word (and we might want it as a keyword in the future) that is impossible. @cgswords and I have been using libproc_macro, and I think that is not a bad name, although I am not 100% happy with it.

Member

nrc commented Aug 29, 2016

From #35957: we should bikeshed the name of the librustc_macro crate some more. In particular, this is intended to be a long-lived crate which will have essentials for all macro authors in it, so limiting to rustc_macro (which in my mind, at least) is just about the 1.1 idea seems bad. I'd previously wanted libmacro for this, but since macro is a reserved word (and we might want it as a keyword in the future) that is impossible. @cgswords and I have been using libproc_macro, and I think that is not a bad name, although I am not 100% happy with it.

@eternaleye

This comment has been minimized.

Show comment
Hide comment
@eternaleye

eternaleye Aug 29, 2016

Contributor

@nrc: Okay, my immediate thoughts on naming:

  • extern crate macros; - short and sweet, but might be read as containing macros, rather than support code for writing them
  • extern crate macro_runtime; - exactly what it says on the tin
  • extern crate metarust; - write Rust, about Rust, for operating on Rust
  • extern crate bikeshed; - with procedural macros, you can have whatever color of Rust you want!
  • extern crate macrame; - sounds like "macro make[r]"; possibly better left for a future "nice" API over the raw library.
Contributor

eternaleye commented Aug 29, 2016

@nrc: Okay, my immediate thoughts on naming:

  • extern crate macros; - short and sweet, but might be read as containing macros, rather than support code for writing them
  • extern crate macro_runtime; - exactly what it says on the tin
  • extern crate metarust; - write Rust, about Rust, for operating on Rust
  • extern crate bikeshed; - with procedural macros, you can have whatever color of Rust you want!
  • extern crate macrame; - sounds like "macro make[r]"; possibly better left for a future "nice" API over the raw library.
@aturon

This comment has been minimized.

Show comment
Hide comment
@aturon

aturon Aug 30, 2016

Member

@nrc It seems like an important aspect of this question is the naming of our various macro styles over all -- in particular, if we go with libproc_macro, we're making a hard commitment to the "procedural macro" terminology. I don't have a strong opinion here, but I'm not sure if we've openly explored the space of terminology.

To be clear, are you thinking that today's macro_rules would simply be "macros", i.e. the default thing we mean by macros, while you have to qualify "procedural macros"? That seems like a pretty reasonable plan to me. And in that world, I'd argue that libproc_macro is better than libmacros.

Member

aturon commented Aug 30, 2016

@nrc It seems like an important aspect of this question is the naming of our various macro styles over all -- in particular, if we go with libproc_macro, we're making a hard commitment to the "procedural macro" terminology. I don't have a strong opinion here, but I'm not sure if we've openly explored the space of terminology.

To be clear, are you thinking that today's macro_rules would simply be "macros", i.e. the default thing we mean by macros, while you have to qualify "procedural macros"? That seems like a pretty reasonable plan to me. And in that world, I'd argue that libproc_macro is better than libmacros.

@nrc

This comment has been minimized.

Show comment
Hide comment
@nrc

nrc Aug 30, 2016

Member

My thinking here is that all macros are "macros", and when we need to make a distinction based on the implementation (the usage should be exactly the same for all kinds) we use "procedural macros" vs "macros by example". I would like to banish "syntax extension" and "compiler plugin" entirely (and one day re-use the latter for actual plugins).

But, yes, I strongly want to get behind the "procedural macro" terminology.

Member

nrc commented Aug 30, 2016

My thinking here is that all macros are "macros", and when we need to make a distinction based on the implementation (the usage should be exactly the same for all kinds) we use "procedural macros" vs "macros by example". I would like to banish "syntax extension" and "compiler plugin" entirely (and one day re-use the latter for actual plugins).

But, yes, I strongly want to get behind the "procedural macro" terminology.

@aturon

This comment has been minimized.

Show comment
Hide comment
@aturon

aturon Aug 30, 2016

Member

@nrc Makes good sense to me! While "macros by example" is a bit unwieldy, it's also a very evocative/intuitive term. My one worry about "procedural macro" is that "procedure" is not a thing in Rust. I wonder if there's a way of making the connection more direct. "Function macro" isn't quite right, but maybe gives you a sense of what I mean?

Member

aturon commented Aug 30, 2016

@nrc Makes good sense to me! While "macros by example" is a bit unwieldy, it's also a very evocative/intuitive term. My one worry about "procedural macro" is that "procedure" is not a thing in Rust. I wonder if there's a way of making the connection more direct. "Function macro" isn't quite right, but maybe gives you a sense of what I mean?

@nrc

This comment has been minimized.

Show comment
Hide comment
@nrc

nrc Aug 30, 2016

Member

Yeah, it's not quite perfect, but given that it is a well-known/used term outside of Rust I think it is better than coining our own term. "Programmatic macro" is possible, but I prefer "procedural".

Member

nrc commented Aug 30, 2016

Yeah, it's not quite perfect, but given that it is a well-known/used term outside of Rust I think it is better than coining our own term. "Programmatic macro" is possible, but I prefer "procedural".

@eternaleye

This comment has been minimized.

Show comment
Hide comment
@eternaleye

eternaleye Aug 30, 2016

Contributor

Perl's term for the closest equivalent it has to "procedural macros" is "source filters", which (especially with the shift from AST to tokens) is a pretty fitting description.

Contributor

eternaleye commented Aug 30, 2016

Perl's term for the closest equivalent it has to "procedural macros" is "source filters", which (especially with the shift from AST to tokens) is a pretty fitting description.

@mystor

This comment has been minimized.

Show comment
Hide comment
@mystor

mystor Aug 30, 2016

Contributor

Perhaps 'Syntax transformers' or 'programmatic macros' would work well as names? I don't have a problem with procedural macros however.

Contributor

mystor commented Aug 30, 2016

Perhaps 'Syntax transformers' or 'programmatic macros' would work well as names? I don't have a problem with procedural macros however.

@jimmycuadra

This comment has been minimized.

Show comment
Hide comment
@jimmycuadra

jimmycuadra Aug 31, 2016

Contributor

"Procedural" is already what people call this, and I think it's clearly understood what it means, especially in contrast to "macros by example." I wouldn't worry about trying to find a different name.

Contributor

jimmycuadra commented Aug 31, 2016

"Procedural" is already what people call this, and I think it's clearly understood what it means, especially in contrast to "macros by example." I wouldn't worry about trying to find a different name.

@nikomatsakis

This comment has been minimized.

Show comment
Hide comment
@nikomatsakis

nikomatsakis Aug 31, 2016

Contributor

I like the term "procedural macro" for regular use (or maybe "custom macro"). I particularly like using the word macro so that it's clear that they can (eventually...) be used in the same way as "regular macros". For this reason, I don't like "source filters" (I also expect a filter to just drop data, not transform it, though I know the term is used for both).

I am fine with either libproc_macro or libmacros. I sort of prefer the latter just because I don't love having _ in crate names when it can be readily avoided. =)

Contributor

nikomatsakis commented Aug 31, 2016

I like the term "procedural macro" for regular use (or maybe "custom macro"). I particularly like using the word macro so that it's clear that they can (eventually...) be used in the same way as "regular macros". For this reason, I don't like "source filters" (I also expect a filter to just drop data, not transform it, though I know the term is used for both).

I am fine with either libproc_macro or libmacros. I sort of prefer the latter just because I don't love having _ in crate names when it can be readily avoided. =)

@nikomatsakis

This comment has been minimized.

Show comment
Hide comment
@nikomatsakis

nikomatsakis Aug 31, 2016

Contributor

One question: do we ever expect to have "support routines" that could be used from non-procedural macros? I know of no such plans, but if we did, and we wanted them in the same crate, then libmacros would be a better name. =)

(I am thinking a bit about e.g., @dherman's comment from the eager-expansion RFC.)

Contributor

nikomatsakis commented Aug 31, 2016

One question: do we ever expect to have "support routines" that could be used from non-procedural macros? I know of no such plans, but if we did, and we wanted them in the same crate, then libmacros would be a better name. =)

(I am thinking a bit about e.g., @dherman's comment from the eager-expansion RFC.)

@eternaleye

This comment has been minimized.

Show comment
Hide comment
@eternaleye

eternaleye Aug 31, 2016

Contributor

@nikomatsakis: A related but subtly different question is a use case I brought up in rust-lang/rfcs#1561 (comment) - will we want procedural macros' implementation functions to be able to call other procedural macros' implementation functions?

I can easily see wanting to allow one custom derive to call another, and that would essentially make procedural macro definitions themselves capable of being used as such "support routines"

But yes, I think @dherman's gensym example is pretty compelling. Of course, if the answer to my question above is "yes", gensym is both a macro (which could be used by macros-by-example) and a support function (which could be used by procedural macros).

Contributor

eternaleye commented Aug 31, 2016

@nikomatsakis: A related but subtly different question is a use case I brought up in rust-lang/rfcs#1561 (comment) - will we want procedural macros' implementation functions to be able to call other procedural macros' implementation functions?

I can easily see wanting to allow one custom derive to call another, and that would essentially make procedural macro definitions themselves capable of being used as such "support routines"

But yes, I think @dherman's gensym example is pretty compelling. Of course, if the answer to my question above is "yes", gensym is both a macro (which could be used by macros-by-example) and a support function (which could be used by procedural macros).

@dtolnay

This comment has been minimized.

Show comment
Hide comment
@dtolnay

dtolnay Sep 1, 2016

Member

I have a cargo PR rust-lang/cargo#3064 which should check all the "cargo integration" boxes in the checklist.

Member

dtolnay commented Sep 1, 2016

I have a cargo PR rust-lang/cargo#3064 which should check all the "cargo integration" boxes in the checklist.

@Ericson2314

This comment has been minimized.

Show comment
Hide comment
@Ericson2314

Ericson2314 Sep 1, 2016

Contributor

I left a comment on the cargo PR, but I think we want a different type of dependency, not just a different type of package. Firstly, I think this just is just better aestheticly and ergnomicly, but that's just my opinion. But I have two concrete reasons too.

  • In a future with public and private deps, it's important to know that its to know that public deps of procedural deps and regular public deps need not coincide. E.g.

  • In a future with inline procedural macros/quasi-quoting, any library can be used at compile time.

  • will we want procedural macros' implementation functions to be able to call other procedural macros' implementation functions?

    As this shows the same crate could be a regular dep of another crate of procedural macros, or a macro dep of another crate.

Contributor

Ericson2314 commented Sep 1, 2016

I left a comment on the cargo PR, but I think we want a different type of dependency, not just a different type of package. Firstly, I think this just is just better aestheticly and ergnomicly, but that's just my opinion. But I have two concrete reasons too.

  • In a future with public and private deps, it's important to know that its to know that public deps of procedural deps and regular public deps need not coincide. E.g.

  • In a future with inline procedural macros/quasi-quoting, any library can be used at compile time.

  • will we want procedural macros' implementation functions to be able to call other procedural macros' implementation functions?

    As this shows the same crate could be a regular dep of another crate of procedural macros, or a macro dep of another crate.

@dtolnay

This comment has been minimized.

Show comment
Hide comment
@dtolnay

dtolnay Sep 2, 2016

Member

Does serde work?

Yes https://github.com/serde-rs/serde/releases/tag/v0.8.6

(except for container attributes in some cases #36211)

Member

dtolnay commented Sep 2, 2016

Does serde work?

Yes https://github.com/serde-rs/serde/releases/tag/v0.8.6

(except for container attributes in some cases #36211)

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Sep 2, 2016

Member

Awesome, thanks for the updates @dtolnay!

Member

alexcrichton commented Sep 2, 2016

Awesome, thanks for the updates @dtolnay!

@White-Oak

This comment has been minimized.

Show comment
Hide comment
@White-Oak

White-Oak Sep 2, 2016

Contributor

Are there docs on these macros? I suppose, the only example of using them is in serde, am I right?

Contributor

White-Oak commented Sep 2, 2016

Are there docs on these macros? I suppose, the only example of using them is in serde, am I right?

@Ericson2314

This comment has been minimized.

Show comment
Hide comment
@Ericson2314

Ericson2314 Sep 6, 2016

Contributor

OK cargo stuff has landed. That's fine, but it would be nice to revisit #35900 (comment) sometime before stabilization. [For what its worth, I meant to bring this up in the original RFC but forgot.]

Contributor

Ericson2314 commented Sep 6, 2016

OK cargo stuff has landed. That's fine, but it would be nice to revisit #35900 (comment) sometime before stabilization. [For what its worth, I meant to bring this up in the original RFC but forgot.]

@jessstrap

This comment has been minimized.

Show comment
Hide comment
@jessstrap

jessstrap Sep 9, 2016

I think "macros by example" and "procedural macros" could be better catagorized as "declarative macros" and "imperative macros" respectively. This gives informative parallels to the more widely known declarative/imperative categorization of programming languages. As imperative is treated as a superset or synonym of procedural, it should be close enough for people used to "procedural macro" terminology to make the jump. It should also avoid any confusion with procedure/function/method concepts in rust itself.
This naming scheme gives us a macro_imp crate and module to parallel macro_rules. macro_rules could eventually become a module of a more general macro_dec crate.

@nrc, When you refer to "actual plugins" are you including things like metacollect and clippy, things like rustw, rustfmt, and the Rust Language Server, or some other category of program?

jessstrap commented Sep 9, 2016

I think "macros by example" and "procedural macros" could be better catagorized as "declarative macros" and "imperative macros" respectively. This gives informative parallels to the more widely known declarative/imperative categorization of programming languages. As imperative is treated as a superset or synonym of procedural, it should be close enough for people used to "procedural macro" terminology to make the jump. It should also avoid any confusion with procedure/function/method concepts in rust itself.
This naming scheme gives us a macro_imp crate and module to parallel macro_rules. macro_rules could eventually become a module of a more general macro_dec crate.

@nrc, When you refer to "actual plugins" are you including things like metacollect and clippy, things like rustw, rustfmt, and the Rust Language Server, or some other category of program?

@SimonSapin

This comment has been minimized.

Show comment
Hide comment
@SimonSapin

SimonSapin Sep 9, 2016

Contributor

For what it’s worth, I mildly dislike the "by example" name (since $foo patterns aren’t "examples" in my mind). Declarative vs imperative sounds better to me.

Contributor

SimonSapin commented Sep 9, 2016

For what it’s worth, I mildly dislike the "by example" name (since $foo patterns aren’t "examples" in my mind). Declarative vs imperative sounds better to me.

@sgrif

This comment has been minimized.

Show comment
Hide comment
@sgrif

sgrif Sep 9, 2016

Contributor

Playing around with this I've noticed one issue. It looks like Rust's provided derives sometimes add attributes which the compiler knows to ignore. This context gets lost when it goes through a custom derive. I'd expect the identify function given as a custom derive to be a no-op, but it will cause errors around the #[structural_match] attribute getting added.

Reproduction script

(In a crate named demo_plugin)

#![feature(rustc_macro, rustc_macro_lib)]

extern crate rustc_macro;

use rustc_macro::TokenStream;

#[rustc_macro_derive(Foo)]
pub fn derive_foo(input: TokenStream) -> TokenStream {
    input
}

(in another crate)

#![feature(rustc_macro)]

#[macro_use] extern crate demo_plugin;

#[derive(PartialEq, Eq, Foo)]
struct Bar {
    a: i32,
    b: i32,
}

Removing the #[derive(Eq)] causes everything to work fine.

Contributor

sgrif commented Sep 9, 2016

Playing around with this I've noticed one issue. It looks like Rust's provided derives sometimes add attributes which the compiler knows to ignore. This context gets lost when it goes through a custom derive. I'd expect the identify function given as a custom derive to be a no-op, but it will cause errors around the #[structural_match] attribute getting added.

Reproduction script

(In a crate named demo_plugin)

#![feature(rustc_macro, rustc_macro_lib)]

extern crate rustc_macro;

use rustc_macro::TokenStream;

#[rustc_macro_derive(Foo)]
pub fn derive_foo(input: TokenStream) -> TokenStream {
    input
}

(in another crate)

#![feature(rustc_macro)]

#[macro_use] extern crate demo_plugin;

#[derive(PartialEq, Eq, Foo)]
struct Bar {
    a: i32,
    b: i32,
}

Removing the #[derive(Eq)] causes everything to work fine.

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Sep 9, 2016

Member

@sgrif ah yes indeed thanks for reminding me!

So there's a few things going on here:

  • With #[derive(PartialEq, Eq)], the compiler is silently adding #[structural_match] because it statically understands that the PartialEq derivation does indeed fulfill that contract.
  • I believe a similar attribute arises with #[derive(Copy, Clone)] with #[rustc_copy_clone_marker] being added.
  • Currently this works because the span of these attributes says "allows internal unstable". We lose the span information, however, when parsing and reparsing.

So, some solutions we could do:

  • Conventionally say custom derive comes first, e.g. #[derive(Foo, Eq, PartialEq)]
  • Rely on custom derive to omit these attributes
  • Don't emit custom attributes if we detect custom derive
  • Use an entirely different mechanism in the compiler to communicate what these attributes are saying, but not through the AST itself.

I'd be in favor of either not emitting these attributes or using a different mechanism. This is really tricky, though, because #[derive(Copy, Foo, Clone)] also needs to work (where custom code runs in the middle that could change definitions).

My preferred course of action would be to just not emit these attributes if we detect custom derive. Even that, though, may not be trivial. For now a convention of "custom first and standard last" should suffice, but I do think we should fix this before stabilizing.

Member

alexcrichton commented Sep 9, 2016

@sgrif ah yes indeed thanks for reminding me!

So there's a few things going on here:

  • With #[derive(PartialEq, Eq)], the compiler is silently adding #[structural_match] because it statically understands that the PartialEq derivation does indeed fulfill that contract.
  • I believe a similar attribute arises with #[derive(Copy, Clone)] with #[rustc_copy_clone_marker] being added.
  • Currently this works because the span of these attributes says "allows internal unstable". We lose the span information, however, when parsing and reparsing.

So, some solutions we could do:

  • Conventionally say custom derive comes first, e.g. #[derive(Foo, Eq, PartialEq)]
  • Rely on custom derive to omit these attributes
  • Don't emit custom attributes if we detect custom derive
  • Use an entirely different mechanism in the compiler to communicate what these attributes are saying, but not through the AST itself.

I'd be in favor of either not emitting these attributes or using a different mechanism. This is really tricky, though, because #[derive(Copy, Foo, Clone)] also needs to work (where custom code runs in the middle that could change definitions).

My preferred course of action would be to just not emit these attributes if we detect custom derive. Even that, though, may not be trivial. For now a convention of "custom first and standard last" should suffice, but I do think we should fix this before stabilizing.

@colin-kiegel

This comment has been minimized.

Show comment
Hide comment
@colin-kiegel

colin-kiegel Sep 9, 2016

Disclaimer: I only have an outside view of the compiler - so there is a lot I don't know. ^^

But from what I understand, the current approach of macros 1.1 custom derive is more like a temporary workaround. Temporary workaround might translate to "quite a while". But in the long run (=macros 2.0) we won't do the string-parsing-round-trip anymore, which currently leads to the loss of span-information. I wonder if these hidden attributes in the AST were such a bad thing in a macros 2.0 world. Maybe someone with more inside-knowledge about the compiler can tell. If these hidden attributes actually make sense in that future world, I would argue to go for the workaround by conventionally putting custom derives up front. The whole thing already is a workaround anyway, isn't it?

colin-kiegel commented Sep 9, 2016

Disclaimer: I only have an outside view of the compiler - so there is a lot I don't know. ^^

But from what I understand, the current approach of macros 1.1 custom derive is more like a temporary workaround. Temporary workaround might translate to "quite a while". But in the long run (=macros 2.0) we won't do the string-parsing-round-trip anymore, which currently leads to the loss of span-information. I wonder if these hidden attributes in the AST were such a bad thing in a macros 2.0 world. Maybe someone with more inside-knowledge about the compiler can tell. If these hidden attributes actually make sense in that future world, I would argue to go for the workaround by conventionally putting custom derives up front. The whole thing already is a workaround anyway, isn't it?

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Sep 9, 2016

Member

@colin-kiegel I believe you're right in the sense that what we're doing right now is not future-proof. Let's say you have, for example:

#[derive(Eq, Foo, PartialEq)]

Today we'd add the Eq implementation, then run the custom code for Foo, and then add an implementation of PartialEq. The structure could change between Eq and PartialEq, so the #[structural_match] added by Eq may not actually be correct after Foo runs.

In that sense I agree that they're not necessarily future-proof in any case!

Member

alexcrichton commented Sep 9, 2016

@colin-kiegel I believe you're right in the sense that what we're doing right now is not future-proof. Let's say you have, for example:

#[derive(Eq, Foo, PartialEq)]

Today we'd add the Eq implementation, then run the custom code for Foo, and then add an implementation of PartialEq. The structure could change between Eq and PartialEq, so the #[structural_match] added by Eq may not actually be correct after Foo runs.

In that sense I agree that they're not necessarily future-proof in any case!

@colin-kiegel

This comment has been minimized.

Show comment
Hide comment
@colin-kiegel

colin-kiegel Sep 9, 2016

My feeling is, that custom derives which rely on structural changes generally won't compose very well, independently of those hidden attributes. Will a v2.0 custom derive be able to change the structure of the item, or will it be somehow limited to decorating only?

colin-kiegel commented Sep 9, 2016

My feeling is, that custom derives which rely on structural changes generally won't compose very well, independently of those hidden attributes. Will a v2.0 custom derive be able to change the structure of the item, or will it be somehow limited to decorating only?

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Sep 9, 2016

Member

Yeah the ability will always be there I believe (inherent in the TokenStream -> TokenStream interface) although I suspect any reasonable implementation of #[derive] would retain the structure of the original structure.

Member

alexcrichton commented Sep 9, 2016

Yeah the ability will always be there I believe (inherent in the TokenStream -> TokenStream interface) although I suspect any reasonable implementation of #[derive] would retain the structure of the original structure.

alexcrichton added a commit to alexcrichton/rust that referenced this issue Jan 2, 2017

rustc: Stabilize the `proc_macro` feature
This commit stabilizes the `proc_macro` and `proc_macro_lib` features in the
compiler to stabilize the "Macros 1.1" feature of the language. Many more
details can be found on the tracking issue, #35900.

Closes #35900

steveklabnik added a commit to steveklabnik/rust that referenced this issue Jan 3, 2017

Document custom derive.
These are some bare-bones documentation for custom derive, needed
to stabilize "macros 1.1",
rust-lang#35900

The book chapter is based off of a blog post by @cbreeden,
https://cbreeden.github.io/Macros11/

Normally, we have a policy of not mentioning external crates in
documentation. However, given that syn/quote are basically neccesary
for properly using macros 1.1, I feel that not including them here
would make the documentation very bad. So the rules should be bent
in this instance.

steveklabnik added a commit to steveklabnik/rust that referenced this issue Jan 3, 2017

Document custom derive.
These are some bare-bones documentation for custom derive, needed
to stabilize "macros 1.1",
rust-lang#35900

The book chapter is based off of a blog post by @cbreeden,
https://cbreeden.github.io/Macros11/

Normally, we have a policy of not mentioning external crates in
documentation. However, given that syn/quote are basically neccesary
for properly using macros 1.1, I feel that not including them here
would make the documentation very bad. So the rules should be bent
in this instance.

GuillaumeGomez added a commit to GuillaumeGomez/rust that referenced this issue Jan 4, 2017

Rollup merge of #38770 - steveklabnik:doc-custom-derive, r=nikomatsakis
Document custom derive.

These are some bare-bones documentation for custom derive, needed
to stabilize "macros 1.1",
rust-lang#35900

The book chapter is based off of a blog post by @cbreeden,
https://cbreeden.github.io/Macros11/

Normally, we have a policy of not mentioning external crates in
documentation. However, given that syn/quote are basically neccesary
for properly using macros 1.1, I feel that not including them here
would make the documentation very bad. So the rules should be bent
in this instance.

So far, this PR includes only docs; @alexcrichton said in rust-lang#35900 that he'd be okay with landing them before stabilization; I don't mind either way.

bors added a commit that referenced this issue Jan 4, 2017

Auto merge of #38783 - alexcrichton:stabilize-proc-macro, r=nikomatsakis
rustc: Stabilize the `proc_macro` feature

This commit stabilizes the `proc_macro` and `proc_macro_lib` features in the
compiler to stabilize the "Macros 1.1" feature of the language. Many more
details can be found on the tracking issue, #35900.

Closes #35900

@bors bors closed this in #38783 Jan 4, 2017

steveklabnik added a commit to steveklabnik/rust that referenced this issue Jan 4, 2017

Document custom derive.
These are some bare-bones documentation for custom derive, needed
to stabilize "macros 1.1",
rust-lang#35900

The book chapter is based off of a blog post by @cbreeden,
https://cbreeden.github.io/Macros11/

Normally, we have a policy of not mentioning external crates in
documentation. However, given that syn/quote are basically neccesary
for properly using macros 1.1, I feel that not including them here
would make the documentation very bad. So the rules should be bent
in this instance.

bors added a commit that referenced this issue Jan 4, 2017

Auto merge of #38770 - steveklabnik:doc-custom-derive, r=alexcrichton
Document custom derive.

These are some bare-bones documentation for custom derive, needed
to stabilize "macros 1.1",
#35900

The book chapter is based off of a blog post by @cbreeden,
https://cbreeden.github.io/Macros11/

Normally, we have a policy of not mentioning external crates in
documentation. However, given that syn/quote are basically neccesary
for properly using macros 1.1, I feel that not including them here
would make the documentation very bad. So the rules should be bent
in this instance.

So far, this PR includes only docs; @alexcrichton said in #35900 that he'd be okay with landing them before stabilization; I don't mind either way.

bors added a commit that referenced this issue Jan 5, 2017

Auto merge of #38770 - steveklabnik:doc-custom-derive, r=alexcrichton
Document custom derive.

These are some bare-bones documentation for custom derive, needed
to stabilize "macros 1.1",
#35900

The book chapter is based off of a blog post by @cbreeden,
https://cbreeden.github.io/Macros11/

Normally, we have a policy of not mentioning external crates in
documentation. However, given that syn/quote are basically neccesary
for properly using macros 1.1, I feel that not including them here
would make the documentation very bad. So the rules should be bent
in this instance.

So far, this PR includes only docs; @alexcrichton said in #35900 that he'd be okay with landing them before stabilization; I don't mind either way.

nikomatsakis added a commit to nikomatsakis/rust that referenced this issue Jan 6, 2017

rustc: Stabilize the `proc_macro` feature
This commit stabilizes the `proc_macro` and `proc_macro_lib` features in the
compiler to stabilize the "Macros 1.1" feature of the language. Many more
details can be found on the tracking issue, #35900.

Closes #35900

nikomatsakis added a commit to nikomatsakis/rust that referenced this issue Jan 6, 2017

Document custom derive.
These are some bare-bones documentation for custom derive, needed
to stabilize "macros 1.1",
rust-lang#35900

The book chapter is based off of a blog post by @cbreeden,
https://cbreeden.github.io/Macros11/

Normally, we have a policy of not mentioning external crates in
documentation. However, given that syn/quote are basically neccesary
for properly using macros 1.1, I feel that not including them here
would make the documentation very bad. So the rules should be bent
in this instance.

nikomatsakis added a commit to nikomatsakis/rust that referenced this issue Jan 6, 2017

rustc: Stabilize the `proc_macro` feature
This commit stabilizes the `proc_macro` and `proc_macro_lib` features in the
compiler to stabilize the "Macros 1.1" feature of the language. Many more
details can be found on the tracking issue, #35900.

Closes #35900

nikomatsakis added a commit to nikomatsakis/rust that referenced this issue Jan 6, 2017

Document custom derive.
These are some bare-bones documentation for custom derive, needed
to stabilize "macros 1.1",
rust-lang#35900

The book chapter is based off of a blog post by @cbreeden,
https://cbreeden.github.io/Macros11/

Normally, we have a policy of not mentioning external crates in
documentation. However, given that syn/quote are basically neccesary
for properly using macros 1.1, I feel that not including them here
would make the documentation very bad. So the rules should be bent
in this instance.

nikomatsakis added a commit to nikomatsakis/rust that referenced this issue Jan 6, 2017

rustc: Stabilize the `proc_macro` feature
This commit stabilizes the `proc_macro` and `proc_macro_lib` features in the
compiler to stabilize the "Macros 1.1" feature of the language. Many more
details can be found on the tracking issue, #35900.

Closes #35900

nikomatsakis added a commit to nikomatsakis/rust that referenced this issue Jan 6, 2017

Document custom derive.
These are some bare-bones documentation for custom derive, needed
to stabilize "macros 1.1",
rust-lang#35900

The book chapter is based off of a blog post by @cbreeden,
https://cbreeden.github.io/Macros11/

Normally, we have a policy of not mentioning external crates in
documentation. However, given that syn/quote are basically neccesary
for properly using macros 1.1, I feel that not including them here
would make the documentation very bad. So the rules should be bent
in this instance.
@tomaka

This comment has been minimized.

Show comment
Hide comment
@tomaka

tomaka Jan 13, 2017

Contributor

If the user writes #[derive(Foo)] #[foo_def = "definition.json"] struct MyStruct; the macro handler has no way to know what "the current directory" is and thus can't find definition.json.

This is by-design and thus wouldn't be easy to fix, and I guess it's too late to fix this anyway.

Contributor

tomaka commented Jan 13, 2017

If the user writes #[derive(Foo)] #[foo_def = "definition.json"] struct MyStruct; the macro handler has no way to know what "the current directory" is and thus can't find definition.json.

This is by-design and thus wouldn't be easy to fix, and I guess it's too late to fix this anyway.

@eddyb

This comment has been minimized.

Show comment
Hide comment
@eddyb

eddyb Jan 13, 2017

Member

You can go Span -> FileMap -> filename -> directory. All that's missing is accessing the information through proc_macro.

Member

eddyb commented Jan 13, 2017

You can go Span -> FileMap -> filename -> directory. All that's missing is accessing the information through proc_macro.

@tomaka

This comment has been minimized.

Show comment
Hide comment
@tomaka

tomaka Jan 13, 2017

Contributor

You would also need to tell the compiler to add a dependency to definition.json so that the build is dirty if it gets modified.

Contributor

tomaka commented Jan 13, 2017

You would also need to tell the compiler to add a dependency to definition.json so that the build is dirty if it gets modified.

@dtolnay

This comment has been minimized.

Show comment
Hide comment
@dtolnay

dtolnay Jan 13, 2017

Member

The proc macro can use env::var("CARGO_MANIFEST_DIR") to get the directory containing Cargo.toml of the crate containing the macro invocation. Presumably foo_def is relative to that. See dtolnay/syn#70 (comment).

Member

dtolnay commented Jan 13, 2017

The proc macro can use env::var("CARGO_MANIFEST_DIR") to get the directory containing Cargo.toml of the crate containing the macro invocation. Presumably foo_def is relative to that. See dtolnay/syn#70 (comment).

@est31

This comment has been minimized.

Show comment
Hide comment
@est31

est31 Jan 13, 2017

Contributor

@tomaka that can be done by mutating FileMap, e.g. this is how the include_str! macro does it.

Contributor

est31 commented Jan 13, 2017

@tomaka that can be done by mutating FileMap, e.g. this is how the include_str! macro does it.

@tomaka

This comment has been minimized.

Show comment
Hide comment
@tomaka

tomaka Jan 13, 2017

Contributor

Presumably foo_def is relative to that.

I think it's not very intuitive to have to put the path relative to the Cargo.toml.

Contributor

tomaka commented Jan 13, 2017

Presumably foo_def is relative to that.

I think it's not very intuitive to have to put the path relative to the Cargo.toml.

@tomaka

This comment has been minimized.

Show comment
Hide comment
@tomaka

tomaka Jan 13, 2017

Contributor

that can be done by mutating FileMap, e.g. this is how the include_str! macro does it.

Yeah I know it can be done, it just can't be done with the current procedural macros API.

The parameter is an Item. It would be acceptable to add a method to retrieve a span from that Item, but adding a method to Item to ask the compiler to add a dependency would be a hack IMO.

Contributor

tomaka commented Jan 13, 2017

that can be done by mutating FileMap, e.g. this is how the include_str! macro does it.

Yeah I know it can be done, it just can't be done with the current procedural macros API.

The parameter is an Item. It would be acceptable to add a method to retrieve a span from that Item, but adding a method to Item to ask the compiler to add a dependency would be a hack IMO.

@SimonSapin

This comment has been minimized.

Show comment
Hide comment
@SimonSapin

SimonSapin Jan 13, 2017

Contributor

You can go Span -> FileMap -> filename -> directory.

Are these APIs (in particular FileMap) on a path to be stabilized?

Contributor

SimonSapin commented Jan 13, 2017

You can go Span -> FileMap -> filename -> directory.

Are these APIs (in particular FileMap) on a path to be stabilized?

@eddyb

This comment has been minimized.

Show comment
Hide comment
@eddyb

eddyb Jan 14, 2017

Member

They don't have to be, in fact I wouldn't want to stabilize any of the internals. We can, instead, stabilize APIs that extract information about a Span (e.g. line, column, filename).

Member

eddyb commented Jan 14, 2017

They don't have to be, in fact I wouldn't want to stabilize any of the internals. We can, instead, stabilize APIs that extract information about a Span (e.g. line, column, filename).

@alexreg

This comment has been minimized.

Show comment
Hide comment
@alexreg

alexreg Feb 2, 2017

Contributor

I just got this error in a crate of mine. What's going on?

error: Cannot use `#![feature(proc_macro)]` and `#![feature(custom_attribute)] at the same time
Contributor

alexreg commented Feb 2, 2017

I just got this error in a crate of mine. What's going on?

error: Cannot use `#![feature(proc_macro)]` and `#![feature(custom_attribute)] at the same time
@abonander

This comment has been minimized.

Show comment
Hide comment
@abonander

abonander Feb 2, 2017

Contributor

@alexreg If you're using #[derive], it's stable now. You don't need #![feature(proc_macro)].

Contributor

abonander commented Feb 2, 2017

@alexreg If you're using #[derive], it's stable now. You don't need #![feature(proc_macro)].

@jseyfried

This comment has been minimized.

Show comment
Hide comment
@jseyfried

jseyfried Feb 2, 2017

Contributor

@alexreg
proc_macro_derives (macros 1.1) are now stable -- you can just remove #![feature(proc_macro)].

#[proc_macro_attribute] recently landed behind the #![feature(proc_macro)] feature gate; these are incompatible with #![feature(custom_attribute)]. #![feature(custom_attribute)] will be deprecated once the replacement lands (rust-lang/rfcs#1755).

Contributor

jseyfried commented Feb 2, 2017

@alexreg
proc_macro_derives (macros 1.1) are now stable -- you can just remove #![feature(proc_macro)].

#[proc_macro_attribute] recently landed behind the #![feature(proc_macro)] feature gate; these are incompatible with #![feature(custom_attribute)]. #![feature(custom_attribute)] will be deprecated once the replacement lands (rust-lang/rfcs#1755).

@abonander

This comment has been minimized.

Show comment
Hide comment
@abonander

abonander Feb 2, 2017

Contributor

@jseyfried I think we should change the tracking issue on proc_macro since this is closed and doesn't contain relevant information.

Contributor

abonander commented Feb 2, 2017

@jseyfried I think we should change the tracking issue on proc_macro since this is closed and doesn't contain relevant information.

@alexreg

This comment has been minimized.

Show comment
Hide comment
@alexreg

alexreg Feb 2, 2017

Contributor

Thanks guys. That makes sense.

Contributor

alexreg commented Feb 2, 2017

Thanks guys. That makes sense.

@jseyfried

This comment has been minimized.

Show comment
Hide comment
@jseyfried

jseyfried Feb 2, 2017

Contributor

@abonander Yeah, #![feature(proc_macro)] should definitely point to #38356 now -- I should have verified that when reviewing #38842. Could you submit a PR?

Contributor

jseyfried commented Feb 2, 2017

@abonander Yeah, #![feature(proc_macro)] should definitely point to #38356 now -- I should have verified that when reviewing #38842. Could you submit a PR?

phansch added a commit to phansch/rust-clippy that referenced this issue Mar 26, 2018

Fix false positive in empty_line_after_outer_attribute
`empty_line_after_outer_attribute` produced a false positive warning when
deriving `Copy` and/or `Clone` for an item.

It looks like the second point in [this comment][that_comment] is related,
as the attribute that causes the false positive has a path of
`rustc_copy_clone_marker`.

Fixes #2475

[that_comment]: rust-lang/rust#35900 (comment)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment