Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow using dart:mirrors #1150

Closed
stevenroose opened this Issue Jan 8, 2016 · 43 comments

Comments

Projects
None yet
@stevenroose
Copy link

stevenroose commented Jan 8, 2016

I know that mirrors is blocked now because it causes the generated packages to be very large.

However, on the long term, it should remain a goal to find a solution for this. Many libraries use mirrors so not being able to use mirrors for Flutter apps will make the framework a lot less powerful.

The trade-off between large package sizes and using mirrors should be left to the user. A possible solution might be to use an explicit --allow-mirrors flag. But of course the ultimate solution should be to find a way to use mirrors without blowing up package sizes.

Java has reflection and Objective-C has Runtime Reference, so why would Flutter not have a reflection system?

@Hixie

This comment has been minimized.

Copy link
Contributor

Hixie commented Jan 8, 2016

Allowing mirrors would mean we couldn't remove unused code (since all the code is implicitly used). That's what causes the code size increase.

Java apps are huge, that's one of the things we want to avoid. Objective C, as I understand it, doesn't trim unused members from classes either. It also doesn't strip any classes that are exported, because it uses dynamic linking.

Dart uses static linking (effectively), and using static analysis we can strip out anything that isn't used ("tree shaking"). If you import a huge Dart library but only use a self-contained two-line method, then you only pay the cost of the two-line method, even if that Dart library itself imports dozens and dozens of other libraries.

(At least, in theory. I'm not sure how much of this we implement yet.)

We could enable mirrors only for classes that you opt into (similar to Delphi's $M+). That would require support from the language.

@stevenroose

This comment has been minimized.

Copy link
Author

stevenroose commented Jan 8, 2016

@Hixie Isn't that what the @MirrorsUsed annotation is for? I thing dart2js also does tree shaking using the @MirrorsUsed annotation.

@Hixie

This comment has been minimized.

Copy link
Contributor

Hixie commented Jan 8, 2016

I think that's more about avoiding name mangling, which is a whole other kettle of fish (we don't do any name mangling). But it would be a similar kind of thing, yes. To my knowledge the snapshotter doesn't support any annotation of that kind currently.

@stevenroose

This comment has been minimized.

Copy link
Author

stevenroose commented Jan 8, 2016

I quote from this article from @sethladd

Mirrors are really powerful, but they introduce challenges to dart2js's tree shaking abilities. Normally, dart2js can look at the entire app and deduce what code is required to run the app. However, as of the time of this writing, mirrors (due to their API design and highly dynamic nature) more-or-less disable dart2js's tree shaking.

Enter @MirrorsUsed, a metadata annotation, which explicitly states what parts of the program are reflected. Tools like dart2js can use @MirrorsUsed to re-enable tree shaking.

I think that the guys that wrote tree shaking for dart2js can really help you out here since it seems they had the same problem and managed to fix it. Of course, package size will still be larger when using mirrors, but the difference won't be as significant when tree shaking is done fairly efficiently.

@Hixie

This comment has been minimized.

Copy link
Contributor

Hixie commented Jan 8, 2016

Ah, yeah, I didn't see 'targets' and 'metaTargets' before. Those could be used to do this, if the snapshotter supported them.

@sethladd

This comment has been minimized.

Copy link
Contributor

sethladd commented Jan 8, 2016

MirrorsUsed is not supported and has edge cases it doesn't cover. I wouldn't build a solution around it.

@stevenroose

This comment has been minimized.

Copy link
Author

stevenroose commented Jan 8, 2016

@sethladd I noticed it said "experimental", yes.
But is it still the only thing that helps dart2js tree shaking with mirrors? Or are there other strategies.

Anyways, dart2js and Flutter seem to have a similar issue when it comes to mirrors usage, so I guess they can use the same "solution", however complete that solution might be.

I believe dart2js still warns users of large output size when it sees mirrors usage, that is a good practice. Better than preventing users from using it...

@floitschG

This comment has been minimized.

Copy link
Contributor

floitschG commented Jan 9, 2016

There is also a reflectable package. It can either run untransformed, using
dart:mirrors, or transformed, where it extracts the information beforehand.
It is not complete, though. For example there is no way to access private
identifiers.
It also adds an additional build step...
On Jan 8, 2016 23:49, "Steven Roose" notifications@github.com wrote:

@sethladd https://github.com/sethladd I noticed it said "experimental",
yes.
But is it still the only thing that helps dart2js tree shaking with
mirrors? Or are there other strategies.

Anyways, dart2js and Flutter seem to have a similar issue when it comes to
mirrors usage, so I guess they can use the same "solution", however
complete that solution might be.

I believe dart2js still warns users of large output size when it sees
mirrors usage, that is a good practice. Better than preventing users from
using it...


Reply to this email directly or view it on GitHub
#1150 (comment).

@eernstg

This comment has been minimized.

Copy link

eernstg commented Jan 12, 2016

For completeness of reflectable, please check out the 'Known limitations' section at the end of https://github.com/dart-lang/reflectable/tree/master/reflectable.

@Hixie Hixie modified the milestone: No Milestone Necessary Feb 16, 2016

@abarth

This comment has been minimized.

Copy link
Contributor

abarth commented Oct 16, 2016

We seem to have gotten pretty far without dart:mirrors. I think we can safely close this issue.

@abarth abarth closed this Oct 16, 2016

@stevenroose

This comment has been minimized.

Copy link
Author

stevenroose commented Oct 16, 2016

I don't agree.

We seem to have gotten pretty far without dart:mirrors.

You know that it makes no sense to say that, right? People know that mirrors is not supported so they either leave the platform (I don't think you can deny that traction is not really great), removed features from their application or used alternative libraries.

Both Android and iOS support reflection, which means that as a competitor against their default development platforms, this will always be a setback for some developers.

While reflectable is a decent enough way to bridge the gap for now, it is far from ideal and as I already pointed out, dart:mirrors + MirrorsUsed should be able to get exactly the same performance than reflectable.

@abarth

This comment has been minimized.

Copy link
Contributor

abarth commented Oct 16, 2016

The approach we use when considering which features to implement is to compare the use cases for those features against the cost of implementing and maintaining the feature. In the case of dart:mirrors, we've largely found less costly ways of addressing these use cases. It's possible that we'll eventually encounter a compelling use case that requires dart:mirrors, in which case we can reconsider implementing that feature, but having this bug open doesn't help us make progress.

Rather than bugs requesting particular solutions, we prefer to work from bugs that state problems, which gives us the flexibility to address those problems in a different ways and potentially find better solutions.

@stevenroose

This comment has been minimized.

Copy link
Author

stevenroose commented Oct 16, 2016

Well, I agree that this is not of a particular priority, that's for sure.
But is there a problem withs issues that live a long time? You can tag them
as improvement or postponed or low priority. I just think closing an issue
that still is one just because you want to reduce the backlog, is a
strategy I don't really agree with.

On Sun, Oct 16, 2016 at 6:03 PM, Adam Barth notifications@github.com
wrote:

The approach we use when considering which features to implement is to
compare the use cases for those features against the cost of implementing
and maintaining the feature. In the case of dart:mirrors, we've largely
found less costly ways of addressing these use cases. It's possible that
we'll eventually encounter a compelling use case that requires
dart:mirrors, in which case we can reconsider implementing that feature,
but having this bug open doesn't help us make progress.

Rather than bugs requesting particular solutions, we prefer to work from
bugs that state problems, which gives us the flexibility to address those
problems in a different ways and potentially find better solutions.


You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#1150 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AA0F3MfDSMonLKa1XGIvYprTX2049D_0ks5q0krTgaJpZM4HBcjM
.

@Hixie

This comment has been minimized.

Copy link
Contributor

Hixie commented Oct 16, 2016

The problem is this isn't really an issue. It's not describing something you can't do, it's just a request for a particular solution. We prefer to track the actual issues. You should definitely feel free to file bugs describing the things you want to do that you can't do without mirrors.

@stevenroose

This comment has been minimized.

Copy link
Author

stevenroose commented Nov 4, 2016

@Hixie To quickly come back to this: the concrete issue it not about a specific functionality that is missing, but more the inability to use any library that relies on mirrors.

I think a lot of libraries will have to start doing things like this: PointyCastle/pointycastle#107

@Hixie

This comment has been minimized.

Copy link
Contributor

Hixie commented Nov 4, 2016

Those libraries are going to stop working in general once Dart drops mirrors entirely, so I'm not too worried about that.

There are certain things that aren't available to a Dart library in Flutter. Some of those things are obvious and nobody thinks about them, like "C++ syntax" or "Perl libraries". Others are less obvious initially, like "no dart:js" or "no dart:html", but they make sense with a quick examination. And then there's some that aren't obvious at all unless you really know how it works, like "no dart:mirrors". There's libraries affected by all of these. I don't think it really matters on the long run. People will just avoid the libraries that don't work with Flutter because of using dart:mirrors just like they avoid using the libraries that don't work with Flutter because they're written in Lua, or whatever.

@stevenroose

This comment has been minimized.

Copy link
Author

stevenroose commented Nov 4, 2016

Are there plans to discontinue mirrors entirely?

I completely disagree with your reasoning btw. For the things you
mention there are valid reasons that prevent these features being
available in Flutter: Dart does not support Perl syntax or Flutter apps
have no notion of a DOM. There is nothing that prevents mirrors from
being available in Flutter. The reason it is not supported is binary
size, which is exactly the same concern for dart2js. In dart2js however
this trade-off is left to the user.

Don't get me wrong, I totally understand that support for mirrors is a
lot of work and there might be valid arguments that it's not worth it.
However, it is a feature that people may want and it is possible to
support, so the place for it should be on the long-term wish list.

On 4 Nov 2016 20:29, "Ian Hickson" <notifications@github.com
mailto:notifications@github.com> wrote:

Those libraries are going to stop working in general once Dart drops
mirrors entirely, so I'm not too worried about that.

There are certain things that aren't available to a Dart library in
Flutter. Some of those things are obvious and nobody thinks about
them, like "C++ syntax" or "Perl libraries". Others are less obvious
initially, like "no dart:js" or "no dart:html", but they make sense
with a quick examination. And then there's some that aren't obvious
at all unless you really know how it works, like "no dart:mirrors".
There's libraries affected by all of these. I don't think it really
matters on the long run. People will just avoid the libraries that
don't work with Flutter because of using dart:mirrors just like they
avoid using the libraries that don't work with Flutter because
they're written in Lua, or whatever.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<https://github.com/flutter/flutter/issues/1150#issuecomment-258526243>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AA0F3NmHdkceWwtsxMvu8hbjhf8Xz2LKks5q64eLgaJpZM4HBcjM>.
@Hixie

This comment has been minimized.

Copy link
Contributor

Hixie commented Nov 5, 2016

Nothing prevents us from having a Perl interpreter either. :-)

The question is, what can't you do with Flutter today that you could do if you had mirrors? "Use libraries that use mirrors" is not a good answer to that, because it has the simple retort "just rewrite the libraries not to use mirrors", just like "You should support Perl libraries so that I can use Perl libraries" isn't a good description of a problem (just write those libraries in Dart instead).

What actual thing can't you do today that you could do with mirrors? You should file a bug saying that you can't do that specific thing. If mirrors are the best way to address that problem, and if the problem is worth addressing, then we'd figure out how to introduce mirrors.

Are there plans to discontinue mirrors entirely?

I don't know about specific plans but with the push towards strong mode it would make sense to me to drop things like mirrors. (Personally I'd like to replace mirrors with metaclasses, so we get all the benefits of mirrors with none of the tree-shaking costs, but that's also probably not going to happen.)

@eernstg

This comment has been minimized.

Copy link

eernstg commented Nov 8, 2016

Are there plans to discontinue mirrors entirely?

Drive-by comment: This is the Flutter repo, so this discussion is distinct from discussions about how/whether Dart will support reflection on other platforms. Besides, as long as it's possible to edit Dart code manually it will also be possible to use techniques like the ones in reflectable to obtain some level of support for reflection. In that sense it isn't even possible to discontinue mirrors entirely.

@branflake2267

This comment has been minimized.

Copy link

branflake2267 commented Apr 23, 2017

Adding a note for future searches. I can't use the json_object library, it depends on dart_mirrors, which is blocked. Would be nifty if the IDE would detect blacklisted libraries or transitives that are blocked.

pubsec.yaml "json_object: any"
import 'package:json_object/json_object.dart';

@xster

This comment has been minimized.

Copy link
Contributor

xster commented Jan 11, 2018

Semi tangential: seems like json_object does 2 things:

  1. Access json map like an object
  2. Serialize dart objects into json

I assume use case 1 is the most popular and it only has a mirrors dependency from this line which is 5 years old. It should be still doable without mirrors like mockito

@zoechi

This comment has been minimized.

Copy link
Contributor

zoechi commented Jan 12, 2018

@branflake2267 I think pub.dartlang.org is taking this into account now and doesn't show the flutter tag when mirrors is used.
https://pub.dartlang.org/packages/json_object

I think currently
https://pub.dartlang.org/packages/json_serializable
https://pub.dartlang.org/packages/built_value
are the best options for JSON serialization
besides several others (jaguar, dson, ...)

@konifar konifar referenced this issue Jan 14, 2018

Merged

Add unit test #9

@CrazyStaff

This comment has been minimized.

Copy link

CrazyStaff commented Feb 24, 2018

I think this might be a good case to support mirrors, so that Json could be easily
serialized to objects.

https://github.com/redstone-dart/redstone_mapper

@matejthetree

This comment has been minimized.

Copy link

matejthetree commented Feb 27, 2018

Also for dependecy injection like in this package.
In big apps with a lot of business logic, this is a dream.
https://github.com/dotdotcommadot/dart_robotlegs_di

@asxhl2007

This comment has been minimized.

Copy link

asxhl2007 commented Mar 28, 2018

@stevenroose I will leave this platform, really!😂

@enif-lee

This comment has been minimized.

Copy link

enif-lee commented Jun 20, 2018

Any news? Really??? 😢😢

@zoechi

This comment has been minimized.

Copy link
Contributor

zoechi commented Jun 20, 2018

@jinseoung-lee any news about what?
It was a deliberate decision to not include dart:mirrors for concrete reasons.
There are alternatives listed above like code custom generation or the reflectable package.

@enif-lee

This comment has been minimized.

Copy link

enif-lee commented Jun 20, 2018

@zoechi I understand why they decided so. but.. If the reason for disabling the mirror system is only due to size, IMHO it's better option to let the each developer choose.

if in-house application(user is specified), size is not a problem compare to usual cases
and we still need to use IoC Container library for enterprise application(or developers 😉).
If possible, it would be a good idea to provide separate packages.

I also think the flutter team is still under consideration.

@enif-lee

This comment has been minimized.

Copy link

enif-lee commented Jun 20, 2018

Oh, There are also dagger like compile-time ioc container library released by Google.

@zoechi

This comment has been minimized.

Copy link
Contributor

zoechi commented Jun 20, 2018

@jinseoung-lee it's not so easy to enable/disable such a feature.
My understanding is that the inject package is in a very early stage and might not yet work.

@kidcosmic

This comment has been minimized.

Copy link

kidcosmic commented Jun 23, 2018

Just throwing this out there:

It's immediately apparent @MirrorsUsed is a sufficient fix for an implementation for protecting tree shaking. Deliberate elimination of candidacy from tree shaking is not only an easy implementation, but sufficient in letting the shake flourish through the rest of the codebases. Mirror usage in the world is almost exclusively confined to data/model classes which are never shaken as it is, injection which pretty much only has seemingly shakeable constructors and event bus implementations. These MirrorsUsed cases only apply application side, such as in a widget and subscriptions to an event bus, but not package side, . The only exceptions there are monster application-like packages (ie have your data and render your ui). Incorrect @MirrorsUsed in packages are the only real threat to proper tree shaking, but that can and will likely be rooted out by the populace as well. Also mirrored data classes are by far better dev experience/speed than not, but that's debatable. Learning curves aren't usually an excuse.

Tldr; there is no real good reason for no mirrors implementation.

That being said, I'm in the camp of who gives a shit. Codegen can get the job done. So screw it, codegen.

@Hixie

This comment has been minimized.

Copy link
Contributor

Hixie commented Jun 23, 2018

If annotations are sufficient then you can also just implement a registration mechanism where in your main.dart you call a function that registers your class with your RTTI library (no code gen). Depends exactly what you want mirrors for, of course.

@zoechi

This comment has been minimized.

Copy link
Contributor

zoechi commented Jun 24, 2018

@kidcosmic @MirrorsUsed wasn't satisfying for dart2js at all, this is why this direction was not pusued further.

@kidcosmic

This comment has been minimized.

Copy link

kidcosmic commented Jun 24, 2018

@Hixie ??? annotations = compile time, vs runtime. Annotations inform compiler? Funny thing is, I need to be able to the compiler "don't fuck with this code, i know better than you", since our compilers are not omniscient. If our compilers were omnisicient_, then they would know what's being reflected and would be able to tree shake with reflections anyway. Annotations are a hack from point A to point B. The best implementation of mirroring with tree shaking support is of course our omniscient compilers. Also, where's my proguard.txt?

@zoechi never looked at their implementation. I was more basing the argument off of a signal compiler to remove candidacy implementation. Nuking candidacy is actually the quickest way to achieve mirrors + tree shaking Mind you, it kind of looks like @MirrorsUsed kind of fell apart because of it's language. An implementation like proguards seems like it would flourish.

@Hixie

This comment has been minimized.

Copy link
Contributor

Hixie commented Jun 24, 2018

@kidcosmic A registration mechanism does the same thing (because registering the identifier is a way to use it, and thus prevents it from being tree-shaken away).

@kidcosmic

This comment has been minimized.

Copy link

kidcosmic commented Jun 27, 2018

@Hixie ah i see, never had experience with RTTI libs, only recently started diving into native. From what I'm seeing in RTTI cpp examples (wikis) though is no registration mech in main? You saying that mirrors for example would need one? RTTI looks to be the same as a mirror/reflection lib O_o I guess you might be referring to reflectabe? Which i agree, does the job pretty damn well.

@Hixie

This comment has been minimized.

Copy link
Contributor

Hixie commented Jun 27, 2018

RTTI is the same as mirrors or reflection or type inspection or any of the other ways to refer to things being discussed in this thread. It's just the more generic term.

I don't know exactly what you want mirrors for, but let's take the common case of wanting to be able to build instances of objects from a file on disk. A trivial runtime type information library for this purpose could look like:

// RTTI LIBRARY

typedef T Constructor<T>();

final Map<String, Constructor<Object>> _constructors = <String, Constructor<Object>>{};

void register<T>(Constructor<T> constructor) {
  _constructors[T.toString()] = constructor;
}

dynamic build(String type) {
  return _constructors[type]();
}

Here's how it could be used (here I've hard-coded the strings but they could come from disk or the network or whatever just as easily):

// THE VARIOUS CLASSES

class A { void test() => print('a'); }
class B { void test() => print('b'); }
class C { void test() => print('c'); }

void registerClasses() {
  // Register these classes with the RTTI library.
  // This is equivalent to putting an annotation on the classes.
  register<A>(() => new A());
  register<B>(() => new B());
  register<C>(() => new C());
}


// THE TEST
 
void test() {
  build('A').test(); // prints "a"
  build('B').test(); // prints "b"
  build('C').test(); // prints "c"
}


// THE APP

void main() {
  registerClasses();
  test();
}
@kidcosmic

This comment has been minimized.

Copy link

kidcosmic commented Jun 27, 2018

Ah I see, pretty clean with the constructors, so KV init would look like:

typedef T KVInit<T>(Map<String, dynamic> data);

dynamic build(String type, Map<String, dynamic> data);

class A { String name; }
class B { A a; }

void registerClasses() {
  register<A>((data) {
    var a = new A();
    a.name = data["name"];
    return a;
  });
  register<B>((data) {
    var b = new B();
    b.a = build('A', data['a']);
    return b;
  });
}

Not too shabby, make the classes pretty immediately available to db ORMs & json IO.

@Hixie

This comment has been minimized.

Copy link
Contributor

Hixie commented Jun 27, 2018

You can also put the constructors in the classes and have them be immutable:

class A { A(this.name); final String name; }
class B { B(this.a); final A a; }

void registerClasses() {
  register<A>((data) => new A(data['name']));
  register<B>((data) => new B(build('A', data['a'])));
}
@jerinho

This comment has been minimized.

Copy link

jerinho commented Dec 16, 2018

I know that mirrors is blocked now because it causes the generated packages to be very large.

However, on the long term, it should remain a goal to find a solution for this. Many libraries use mirrors so not being able to use mirrors for Flutter apps will make the framework a lot less powerful.

The trade-off between large package sizes and using mirrors should be left to the user. A possible solution might be to use an explicit --allow-mirrors flag. But of course the ultimate solution should be to find a way to use mirrors without blowing up package sizes.

Java has reflection and Objective-C has Runtime Reference, so why would Flutter not have a reflection system?

where to put that --allow-mirrors ?

@matejthetree

This comment has been minimized.

Copy link

matejthetree commented Dec 16, 2018

@zoechi

This comment has been minimized.

Copy link
Contributor

zoechi commented Dec 16, 2018

@matejthetree as already mentioned above #1150 (comment) (and the following discussion)

@rrousselGit

This comment has been minimized.

Copy link
Contributor

rrousselGit commented Feb 13, 2019

I don't think #26125 is a duplicate of this one, but I'll put my points here.

I personally agree that we do not need dart:mirror in a shipped application, as code-generation solves most of it.

But there's a catch: code-generators themselves may want to depend on flutter

A typical example is a package of mine: https://github.com/rrousselGit/functional_widget

It is a code generator that generates Widget subclass and debugFillProperties overrides. As such, functional_widget heavily depends on Flutter features.

But due to dart:mirror being disabled, it is impossible for such code generator to depend on Flutter as some generations dependencies use mirroring (such as build_runner and source_gen).

It currently works thanks to multiple workarounds. But there are many limitations


Overall, I think all dev dependencies should be allowed to use mirroring. The use-case is not limited to code generation, and it has no impact on what is shipped in the final app.

A typical use case is to add custom testing features similar to Jest. With mirroring, it becomes possible to use dart2json within tests and therefore make a toMatchSnapshot.

It may also be possible to use reflection for more advanced features, like running only tests impacted by the git local changes (like Jest again).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.