Static Metaprogramming

Metaprogramming refers to code that operates on other code as if it were data. It can take code in as parameters, reflect over it, inspect it, create it, modify it, and return it. Static metaprogramming means doing that work at compile-time, and typically modifying or adding to the program based on that work.

Today it is possible to do static metaprogramming completely outside of the language - using packages such as build_runner to generate code and using the analyzer apis for introspection. These separate tools however are not well integrated into the compilers or tools, and it adds a lot of complexity where this is done. It also tends to be slower than an integrated solution because it can't share any work with the compiler.

Sample Use Case - Data Classes

The most requested open language issue is to add data classes. A data class is essentially a regular Dart class that comes with an automatically provided constructor and implementations of ==, hashCode, and copyWith() (called copy() in Kotlin) methods based on the fields the user declares in the class.

The reason this is a language feature request is because there’s no way for a Dart library or framework to add data classes as a reusable mechanism. Again, this is because there isn’t any easily available abstraction that lets a Dart user express “given this set of fields, add these methods to the class”. The copyWith() method is particularly challenging because it’s not just the body
of that method that depends on the surrounding class’s fields. The parameter list itself does too.

We could add data classes to the language, but that only satisfies users who want a nice syntax for that specific set of policies. What happens when users instead want a nice notation for classes that are deeply immutable, dependency-injected, observable, or differentiable? Sufficiently powerful static metaprogramming could let users define these policies in reusable abstractions and keep the slower-moving Dart language out of the fast-moving methodology business.

Design

See this intro doc for the general design direction we are exploring right now.

Update January 2025

We have an unfortunate update on macros/metaprogramming. We have invested significant time and resources to prototype macros over the past couple years. Unfortunately, each time we solved a major technical hurdle, we saw new ones pop up. At this point, we are not seeing macros converging anytime soon toward a feature we are comfortable shipping, with the quality and developer-time performance we want.

After considering the opportunity cost — in particular, the features we could be shipping to the community instead — we’ve made the difficult decision to stop our work on macros.

For additional details, please see the blog post:
https://medium.com/dartlang/an-update-on-dart-macros-data-serialization-06d3037d4f12

Would static function composition be in the scope of this feature?

At the moment, higher-order functions in Dart are fairly limited since they require knowing the full prototype of the decorated function.

An example would be a debounce utility:

void Function() debouce(Duration duration, void Function() decorated) {
  Timer? timer;
  return () {
    timer?.cancel();
    timer = Timer(duration, () => decorated());
  };
}

which allows us to, instead of:

class Example {
  void doSomething() {

  }
}

write:

class Example {
  final doSomething = debounce(Duration(seconds: 1), () {
  
   });
}

but that comes with a few drawbacks:

obvious readability decrease
our debounce utility is not very reusable. It works only on void Function(), but we'd like it to work for all functions.

With static meta-programming, our debounce could inject code in the class at compilation, such that we could write:

class Example {
  @Debounce(Duration(seconds: 1))
  void doSomething() {
    print('doSomething');
  }
  
  @Debounce(Duration(seconds: 1))
  void doSomethingElse(int value, {String named}) {
    print('doSomethingElse $value named: $named');
  }
}

There is a delicate balance re: static function composition, but there are certainly many useful things that could be done with it. I think ultimately it is something we would like to support as long as we can make it obvious enough that this wrapping is happening.

The specific balance would be around user confusion - we have a guiding principle that we don't want to allow overwriting of code in order to ensure that programs keep their original meaning. There are a lot of useful things you could do by simply wrapping a function in some other function (some additional ones might include uniform exception handling, analytics reporting, argument validation, etc). Most of these things would not change the meaning really of the original function, but the code is being "changed" in some sense by being wrapped.

Ultimately my sense is this is something we should try to support though. I think the usefulness probably outweighs the potential for doing weird things.

I like Lisp approach (in my opinion, the utmost language when it comes to meta-programming). Instead of defining a @Debounce or something alike, we would define new syntax that would simply expand to a regular method at compile-time. I don't know, however, how much complex is to make something like this considering Dart syntax.

For something like debounce, a more aspect-like approach seems preferable. Say, if you could declaratively wrap a function body with some template code:

class Example {
  void doSomething() with debounce(Duration(seconds: 1)) {
    print('doSomething');
  }
  
  void doSomethingElse(int value, {String named}) with debounce(Duration(seconds: 1)) {
    print('doSomethingElse $value named: $named');
  }
}

template debounce<R>(Duration duration) on R Function {
  template final Stopwatch? sw;
  template late R result;
  if (sw != null && sw.elapsed < duration) {
    return result;
  } else {
    (sw ??= Stopwatch()..start()).reset();
    return result = super;
  }
}

This defines a "function template" (really, a kind of function mixin) which can be applied to other functions.
It cannot change the signature of the function, but it can access arguments (by forwarding them as with templateName(arg)), and it can do things before and after the original body is run.
The template variables are per-template instantiation variables (just as we could declare static variables inside normal functions).

(Maybe we just need AspectD for Dart.)

It cannot change the signature of the function, but it can access arguments

But an important part of function composition is also the ability to inject parameters and ask for more parameters.

For example, a good candidate is functional stateless-widgets, to add a key parameter to the prototype and inject a context parameter.
This means the user would define:

@statelessWidget
Widget example(BuildContext context, {required String name}) {
  return Text(name);
}

and the resulting prototype after composition would be:

Widget Function({Key? key, required String name})

where the final code would be:

class _Example extends StatelessWidget {
  Example({Key? key, required String name}): super(key: key);

  final String name;

  @override
  Widget build(BuildContext) => originalExampleFunction(context, name: name);
}

Widget example({Key? key, required String name}) {
  return _Example(key: key, name: name);
}

I definitely agree we don't want to allow for changing the signature of the function from what was written. I don't think that is prohibitive though as long as you are allowed to generate a new function/method next to the existing one with the signature you want. The original function might be private in that case.

I don't think that is prohibitive though as long as you are allowed to generate a new function/method next to the existing one with the signature you want

That's what functional_widget does, but the consequence is that the developer experience is pretty bad.

A major issue is that it breaks the "go to definition" functionality because instead of being redirected to their function, users are redirected to the generated code

It also causes a lot of confusion around naming. Because it's common to want to have control on whether the generated class/function is public or private, but the original function to always be private.

By modifying the prototype instead, this gives more control to users over the name of the generated functions.

Allowing the signature to be modified has a lot of disadvantages as well. I think its probably worse to see a function which is written to have a totally different signature than it actually has, than to be navigated to a generated function (which you can then follow through to the real one). You can potentially blackbox those functions in the debugger as well so it skips right to the real one if you are stepping through.

I suppose this will allow generating fromJson and toJson methods at compile time for Json serialization ?

@bouraine

@bouraine

I suppose this will allow generating fromJson and toJson methods at compile time for Json serialization ?

Yes.

@tatumizer This issue is just for the general problem of static metaprogramming. What you describe would be one possible solution to it, although we are trying to avoid exposing a full AST api because that can make it hard to evolve the language in the future. See https://github.com/dart-lang/language/blob/master/working/static%20metaprogramming/intro.md for an intro into the general design direction we are thinking of here which I think is not necessarily so far off from what you describe (although the mechanics are different).

Great intro & docs.

Hopefully we'll stay (far far) away from annotations to develop/work with static meta programming?!

so it looks like copyWith is seen as a crown jewel of the upcoming facility

The main reason we use this as an example is its well understood by many people, and it is also actually particularly demanding in terms of features to actually implement due to the public api itself needing to be generated :).

The language needs some mechanism of dealing with default values, which has been a showstopper in dart from day one.

Can you elaborate? Default values for parameters are getting some important upgrades in null safe dart (at least the major loophole of being able to override them accidentally by passing null explicitly is closed).

@davidmorgan

@davidmorgan In addition to this list, I'd like to add this bug to the pile.

Again, I'm sure addressing build_runner issues will uncover a lot of tech debt

@lucavenir

Thanks @lucavenir, I've marked it as a sub-issue of dart-lang/build#3806

You know, the biggest benefit macros provided was the live on-the-spot code gen

But the actual API was... Not great.

Is there any chance we could get just the instant gen and just leave the actual implementation and introspection to the ecosystem to refine on its own?

Also, could I get an enumeration of the exact reasons the attempt failed as-is? Just so I don't re-tread covered grounds.

Because sometimes I want to make a quick extension on a function type, multiple times, but not enough to want to pull out build_runner.

Also, could I get an enumeration of the exact reasons the attempt failed as-is?

I would also really like to know this, because if the problem is just performance, this could be improved over the years, seeing as the Rust compilation has improved over the years, in addition to better software optimizations, there are also hardware improvements over time.

I think it might be interesting if the dart SDK (compiler/analyzer) had an extension plugin API with hooks so that the community could develop a macro package that could be coupled to the SDK with fewer barriers than build_runner, in a more integrated way so that a package that depends on it doesn't force the package user to have to put build_runner as a dependency and have to run dart run build_runner build, that is, so that this becomes transparent to the package user, eliminating this barrier would be very good and it would work almost like the native macros of the dart SDK.

Also, could I get an enumeration of the exact reasons the attempt failed as-is?

I would also really like to know this, because if the problem is just performance, this could be improved over the years, seeing as the Rust compilation has improved over the years, in addition to better software optimizations, there are also hardware improvements over time.

You can look through the closed issues tagged static-metaprogramming on the language repo, there were plenty of large open issues to choose from.

The biggest functional piece missing was the capability to pass configuration to macros, e.g. JsonSerializable(key: 'foo'). We had ideas for this but not much progress on an actual implementation--it's hard for a lot of reasons.

Macros would have run as you type in the IDE, it simply does not make sense to release a macros or macros-like feature and promise it will be faster later :)

The biggest functional piece missing was the capability to pass configuration to macros, e.g. JsonSerializable(key: 'foo'). We had ideas for this but not much progress on an actual implementation--it's hard for a lot of reasons.

i feel like extension types could have been great for that.

also a lot of the issues are regarding api and analysis.

i dont think we even "had" the feature for long enough to even care about that yet.

I can easily imagine:

macro class MyMacro {
  // you could get the actual value, but this is more useful for emitting to straight to code. basically a Reviver'd version, by my understanding
  final Literal<String> key;
  // information, no coercion or anything
  final String info;
  // the type itself is useless, but as its a literal, it secretly retained the associated element, so we can still use it for type analysis
  // possibly has a `.identifier` getter
  final Literal<Type> supertype;
  // possibly used for non-const superpowers, if we want the revivable, but dont care to read the value directly in the macro.
  final Code</*ExpectedType*/> something;
}

Literal() is Code

seems simple enough, but i'm not deep enough in implementation code to actually say that i suppose.

seems simple enough, but i'm not deep enough in implementation code to actually say that i suppose.

It's very hard.

constant evaluation can't run before macros, because macros are part of constant evaluation; so macros can't see fully evaluated parameters
if what the macros sees is some representation of the source code, this is something that has to be able to change as the language evolves
full source code isn't necessarily available while compiling: dependencies can be provided as kernel or as analyzer summaries; so any source code representation has to be baked into the toolchain including at least these two intermediate formats

Codegen just fails if it asks for ASTs where they're not available, but that's not an option for a language feature.

Codegen just breaks when ASTs change :) also not an option for a language feature.

Macros would have run as you type in the IDE

This seems like a bad idea. To me, the IDE should ignore macros and only the compiler should analyze and process the macros and generate code when compiling the code. Making the IDE analyze and generate code when typing is absurdly expensive. The user experience doesn't need to be magical. See, when you use very dynamic things, you only know for sure the result when running or compiling, and that's not the end of the world. You don't need to have the best UX in the world to be very useful.

Macros would have run as you type in the IDE

This seems like a bad idea. To me, the IDE should ignore macros and only the compiler should analyze and process the macros and generate code when compiling the code. Making the IDE analyze and generate code when typing is absurdly expensive. The user experience doesn't need to be magical. See, when you use very dynamic things, you only know for sure the result when running or compiling, and that's not the end of the world. You don't need to have the best UX in the world to be very useful.

That's a great way to completely shut down analyzer-level type safety.

How are you supposed to use a generated type otherwise?

There is no miracle, there is always a compromise, there are always pros and cons in everything, it is not feasible to make the IDE analyze the entire project with each keystroke, that is madness, but having said that there is no loss of type safety once the compiler finds an error it will show it.

There is no miracle, there is always a compromise, there are always pros and cons in everything, it is not feasible to make the IDE analyze the entire project with each keystroke, that is madness, but having said that there is no loss of type safety once the compiler finds an error it will show it.

And how is the ide meant to find an error for a type it doesn't yet know exists because it hasn't been macro-expanded?

If perhaps we reanalyzed after we pause typing, or when we ask for it (configurable? On-save? Oh hi build_runner watch) then that would at least make sense.

Hey, maybe we should go and abuse custom_lint quick-fixes as crappy macros! Heh.

Could we get more transparency on why hygienic macros were abandoned? It seems more like a resource issue rather than a fundamental technical challenge. Considering that, given what we've seen so far, Dart macros could have been essentially the first time that metaprogramming was "done right", i.e. well designed and focused on predictable runtime behavior and so it doesn't make sense to me to completely abandon it. Could we get some kind of indication that the general macros design could be reconsidered in the future given improvements in the perf of hot reload and the analyzer? It didn't sound like there were any fundamental performance limits. I would like to add that I was basing a lot of work on this feature and that I'm much less likely to use Dart in the future without the macros feature. It would really help to get more information on whether the performance limits were fundamental or simply out of reach for the team for now. I would also say that performance improvements in the analyzer and hot reload would also be really appreciated in any case, even if they're not exactly problematic as is. Dart is a special language and it's hard to see it miss out on a feature that could complete its programming model.

Fundamentally, so much work was put into the macros system and as far as I can see, there weren't any issues with the system itself so permanently shelving it because of tooling performance issues seems like a really bad tradeoff. If some of the finality of this decision was lifted, I would be much more at ease along with many in the Dart/Flutter ecosystem about the future of the language.

Here are the top level problems with macros aside from performance, from my point of view as one of the people working on macros in 2024:

Configuring macros. Macros/codegen are usually configured/triggered using annotations. Annotations are constants. Constant evaluation is defined in terms of a complete program, so it "runs" at the end of the compile. But macros need to run before that. So we ended up in an undefined area where there is nothing in the language saying what to do. Codegen works here because the analyzer has built up, over the years, an ad-hoc way to interpret incomplete programs that works for lints, hints, codegen. You can see a sketch of the path we were on to try to address this here, but it was only a sketch.
Where macros get "built". The path we were on was that the analyzer and the CFE had their own separate implementations of macros. As part of your dev workflow, macros would have run twice, via two different implementations. Except, you probably also had some old style codegen, so macros would have run at least three times. Any need to run codegen repeatedly and the number of macro runs would have exploded. The fundamental problem here is that we needed the properties of a correctly scalable build system, inside multiple single-process, in-RAM tools.
Limitations of macros. It was inherent in the macros design that there were limitations to what types of code could be output in each "phase". This would for example prohibit outputting classes based on fields of classes. And, it was inherent in the design that only Dart code could be an input and only Dart code could be an output. So for example no conversion to/from schemas. These two limitations together meant that macros could not actually address many of the use cases addressed by codegen today, which would mean we would need to also support codegen forever, which would make problem 2 significantly more serious.

Codegen already has an answer for all of these, but solving them in the macros space was going to either take a very long time or be impossible. So I think we can deliver much more value by pushing on codegen. Of course, once we hit the end of that path we'll have to find something else to work on :)

One more note, enhanced parts and augmentations were originally part of the macros feature, they will be launched as separate language features. These are going to be a huge win for codegen, bringing it a lot closer to what macros was promising. Along with improvements to codegen tooling I think we're looking at 2026 being a great year for Dart metaprogramming.

Navigation Menu

Repository navigation

Static Metaprogramming #1482

Sample Use Case - Data Classes

Design

Update January 2025

583 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Static Metaprogramming #1482

Description

Sample Use Case - Data Classes

Design

Update January 2025

Activity

rrousselGit commented on Mar 2, 2021

jakemac53 commented on Mar 2, 2021

mateusfccp commented on Mar 2, 2021

lrhn commented on Mar 2, 2021

rrousselGit commented on Mar 2, 2021

jakemac53 commented on Mar 2, 2021

rrousselGit commented on Mar 2, 2021

jakemac53 commented on Mar 2, 2021

bouraine commented on Mar 2, 2021

mateusfccp commented on Mar 2, 2021

jakemac53 commented on Mar 2, 2021

idkq commented on Mar 3, 2021

jakemac53 commented on Mar 3, 2021

583 remaining items

lucavenir commented on Feb 25, 2025

davidmorgan commented on Feb 26, 2025

TekExplorer commented on Mar 10, 2025

insinfo commented on Mar 18, 2025

insinfo commented on Mar 18, 2025

davidmorgan commented on Mar 18, 2025

TekExplorer commented on Mar 18, 2025

davidmorgan commented on Mar 18, 2025

insinfo commented on Mar 18, 2025

TekExplorer commented on Mar 18, 2025

insinfo commented on Mar 18, 2025

TekExplorer commented on Mar 18, 2025

jonaird commented on Dec 18, 2025

davidmorgan commented on Jan 5, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions