-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Dynamic module loading #10530
Comments
Added Area-Language, Triaged labels. |
This comment was originally written by @seaneagan Have you seen DeferredLibrary yet: http://api.dartlang.org/docs/bleeding_edge/dart_async/DeferredLibrary.html is it tracked in issue #3940. it's currently limited to 2 files, but that should be changing soon. |
This comment was originally written by @dam0vm3nt That's great. But what do you mean with 'limited to two files'? I'll do some experiment. |
This comment was originally written by increo...@gmail.com You can load code not known at compile time through Isolates and spawnUri(). |
This comment was originally written by @dam0vm3nt #4: I already knew you can use Isoletes and spawnUri, but this is not the solution because:
|
This comment was originally written by @dam0vm3nt #2 : I've checked the pointer and this is noy (YET) what I mean. Infact the compiler still needs to know the library at compile time, hence the need of a constant constructor (thus not allowing for truly dynamical library loading) and declaring an import statement (that requires to know the module at compile time). So I've proposed some changes to that feature and posted a comment on that thread. For commodity I repeat here the example that should work to satisfy this issue: queryServerForLibToBeLoaded().then(loadIt); loadIt(libName) { moduleLoaded(loadedLibraryMirror) { |
This comment was originally written by @stevehsu77 i propose an easy definition of dynamic load. ======================================================================== funntionmirror=reflect(function); var result=functionmirror.invoke(arg1, arg2); so that it doesn't need to load a new library and handle dynamic type processing. |
This comment was originally written by jirkad...@gmail.com Going forward, do we really want to use this? There is much[1] talk[2] recently[3] about the security model on the web and being able to dynamically inject stuff into the code of the main app (which is easy in JS) is being frowned upon. XSS, you know. There is even the extreme case[4] of some academics trying to use the first and second W3C standard [1,3] I mentioned to implement in JS something akin to Dart isolates. As an alternative, the current isolate model could be made more developer friendly by making some API wrapper or so. There is something in these lines in Java. It could be made with the security model of Least Privilege and Information Hiding in mind that is being proposed[2, 4, 5] for the web. [1] The sandbox attribute http://www.whatwg.org/specs/web-apps/current-work/multipage/the-iframe-element.html#attr-iframe-sandbox |
This comment was originally written by @dam0vm3nt I think this is, not only desiderable, but also of extreme importance for Dart needs a way to create extensible and modular application. Isolates can Let's make an example. I have a project that consist in a new fantastic If I'm not missing something at the moment if we want to implement this
I hope I'm wrong and someone could teach me a third way because so far this Security should not be an excuse for not implementing important features |
This comment was originally written by @dam0vm3nt Maybe this example is more clear of what IMHO it is needed: // We call our backend and ask for a module to load // Backend provides us with the url of the lib to load // The module is loaded we can create object using classes defined |
This comment was originally written by @stevehsu77 It's concerned about class run-time types while loading whole library. class a extends b {} //but b is in another dynamic library we just loaded var myObj = loadlibrarymirror.class['lib.a'].create(); because dart is not a pre-compiled language, it does't have any header file or dependency file. and it use language-based vm, does't own any intermediate language like as bytecode or msil. many things have been done when vm is starting. that's why i assume loading dynamic library will make vm developers to think more about security, performance, convenience, and more then more. and that's why i propose an easy way to load dynamic functions. it does't to be considered about type and many other things in vm. and easy to modify code in vm in 3 steps. (load source string, parse to object, instance it) typedef int function(String arg1,int arg2); //as a delegate, supported now closuremirror=reflect(function); //reflect it, need a little modify don't do think too much about runtime-typed, memory heap, garbage collection. and etc. and then support dynamic function, so that we can wrap it as a module. |
This comment was originally written by @stevehsu77 believe i would so strongly except dart can support dynamic library. so i can move my c# and as3 architecture to dart. and i was trying to do this. i built everything is class in the project (https://github.com/yunist/yun). but i change my mind since dart 1.0 (there are many changes of mirrors and isolate), i think everything is object in dart as js. so i define structure use map, ans use class as interface(like go language) in the new project (https://github.com/yuner/surebet). now dynamic function is enough for me. dart is not like other language, you need redesign your architecture to suit it. |
This comment was originally written by @dam0vm3nt Another common usecase where we absolutely want this is : a main module that loads a dynamic module that creates a web component to be returned to the main module. This is not possible with current mirror/isolates. Example: the main module implements a browser of a collection. when clicking an item we want to dynamic load a module that will create an editor for the item to be displayed on the current page in a specific div. |
This comment was originally written by @stevehsu77 code as below Dynamic_Class (this.Source_File) even you can implement different code for each div by special "class_name"; Map<String div_class, Dynamic_Class> div_wrapper; is it easy? for vm developer , it's very easy to do this modification. |
This comment was originally written by @stevehsu77 and i will put dynamic_class to an independent isolate to manager all dynamic classes for security. dynamic code may cause many injection problems in web apps. for a "div" application, we often listen it's events, so pass the event to another isolate, and then process it in dynamic module, finally, use massage to transfer the result to main isolate will keep everything under control and safety. however, it will cause lots of memory-copied actions and duplicated data. but, vm use a solution like as snapshot(not sure) to solve this problem. |
This comment was originally written by nigel.magn...@gmail.com I'm not sure Isolates, or the definition of DeferredLibrary is the answer. For a relevant example in the JS world, look at this: https://github.com/dluksza/angular-gerrit - client-side modules, integrating into a bigger application. Comment #9 is spot-on. Many applications - ones that dart is a good fit for - need this kind of modularisation ability. Now, I don't really mind if the modules themselves suffer from 'poor minification'. In the (say) Java world, I might load a module into a new classloader, and be able to reference classes from my 'parent' - without having to bundle those classes all over again. I can see how that would be extremely difficult in the Dart case. But at the moment - somewhat embarrasingly - if I want to reference dart 'module A' from 'dart application B', about the only tractable solution is to compile moduleA entirely into JS, then use the JS interop to access it (!). Yuck! It would be nice to have even an iterim solution, otherwise dart's going to be a non-starter :-(. |
This comment was originally written by danil.kornishev@gmail.com In Ruby/php you can import a file from anywhere in the code. Why can't I do this in dart? I'd like to be able to scan a directory and import all dart files found, for example in my dart cucumber project. That seems not possible. Had to resort to generating a dart runner file with all the found files imported and using spawnUri. |
This comment was originally written by imre.gabes...@gmail.com I'm looking forward to this feature extension. Currently this shortage is the reason why we use JS instead of Dart. And JS is way uglier and slower than Dart but at least that's dynamic... |
any update on this? |
+1, I really need this to be able to use Dart for my project instead of ES6/TypeScript. |
@hermanbergwerf thanks for the comment. Could you provide specific details on exactly what problem you need solved? |
@sethladd I want to build a plugin framework where plugins can be executed during runtime so I can move features out of the core into separate modules. Additionally, third party developers will be able to develop and run plugins. In JS I could inject the plugin script which could then access a global object from the core that holds an API. It seems spawnDomURI from Dart can achieve this too (I think I could pass the API object via args), but there is no dart2js implementation yet. I suppose the compiled dart2js could also be injected and the plugin could import the core so it can use the API object (which is really just a class with a lot of functions, this should be passed by reference though). |
@hermanbergwerf thanks! Just for clarity, do you need this only for your web apps, or are you writing VM-based apps, too? Also for clarity, it sounds like you need to dynamically load a library into an existing isolate, such that the loaded library has access to all the same memory as the isolate it was loaded into? (where, isolate == app or web app, in this case) cc @floitschG |
I think @mraleph 's Thoughts would be great to hear about it. |
If the objects are handled correctly, you should not have a memory problem. |
Yes, it would be very interesting to know the thoughts of @mraleph :) |
They say AOT compilation principles, excludes possibility of reflection, but I don't really have deep understanding of those principles. |
If you are using thousands of complex custom widgets there is a big chance you will came across problems, at least in an android app case you will have battarey consumption very high and it is a big problem. |
@campfire5 Neither all the objects nor all the windows of an application are used simultaneously, if someone does that, they have poorly developed their application. |
I think you should be able to use reflection and reflectable on windows fluently its a problem of flutter mostly, although you will not have ability of static reflection, So let's define a problem, if you are using reflaction dart executable needs to be aware of all of its body all of it's code to introspect it and create reflections of an object, So executables size gets bigger because during compilation dart needs to compile everything for the sake of reflection. With reflectable you are just telling it which class should be compiled fully and not only its parts, it means wherever you have imported file which includes reflectable class, that class will be compiled fully which undermines the tree shaking principle only for that class or classes. So basically, tree shaking is a kinda oposite of reflection in dart, and because of that you don't have modules as interfaces, but if you undermine it and use reflaction basically you will have modules or classes as interfaces but it will be loaded in memory and your executable will be pretty large. Reflactable package just makes it more specific and light weight. |
Let's hope they say something from the dart development team. :) |
I really need a framework-plugin system just like a C++ executable dynamically loading DLLs. BUT: How can such kind of functionality be secure in Flutter? As soon as my Flutter app loads an infected module, this module has power over the whole device. Things will go bad. We need a plugin concept working with permissions. A module should not use the camera or even render a widget (e.g. fake login system) unless the app/the user has authorized it to do so. In fact, this must be pretty hard to implement securely. But what about an interpreted approach: The Flutter app can read and parse a script with little functionality. Any special or performance critical function can then be exported from the script engine to the script. BOOM: Full control on the app side. |
Actually, any programming language can have the same problem, the applications that are developed can access most of the functionalities of a computer. |
This security concern could be solved by signing these dynamic libraries with a certificate. So that only trusted code could be loaded in your app. |
@campfire5 I need to create a deferred component out of a custom package as it contains huge assets and the aab exceeds the 150MB Play Store limit. The deferred libraries you were talking about would be exactly what I need, can you tell me a bit more about them? |
Exactly. Any news on this? Dart and Flutter really needs plugins/modules system. |
I would also be interested to hear if the devs have plans to implement this |
A lot of time has passed and everything is still the same, apparently nobody is interested in this functionality. :( |
I need this functionality. |
Yeah, currently most of the software we use everyday has a system that allows you to add additional third party functionality. |
I am waiting for this functionality for a long time. |
If we can just hold in there until May of next year, we will have gone a whole decade hoping for this. That's sorta an achievement in itself. |
Has any work been started on this? I'm happy to see if I can contribute to the core to get modular deployables implemented. I'm working on a Flutter SPA that integrates with clients' existing backend ERP, CMS, and/or databases. In addition to those externals being different per client, clients also want UI customization that requires source-level changes Not all clients will be subscribing to all features. Normally I would modularize all the components and have a list of modules that load per client. Without modularization I either have to maintain one monolithic product per client, or one one single monolithic product that includes everything for every client and runtime logic to call the correct functionality. Imagine there was no way to dynamically load images and every user you have wants their avatar photo displayed in an App/SPA. You'd have to either build your package with every photo in the assets folder and rebuild and redeploy every time a photo is added or changed, or build unique deployables for each client with their customized avatar.png. C/C++ has a simple interface for dynamically loading compiled code. Compiling a modular object includes a symbol table that can be used with dlopen/dlmopen and dlsym/dlmsym to interact with components. Given that Dart allows function pointers to be passed as arguments and be stored in Lists/Maps, could a base "DynamicModule" object load a compiled Dart module file and store all the symbol names and pointers in a Map for access? (This could also allow for hot-swapping running code without restart which is useful for services.) |
What you say feel a little bit complicated to me, but the flutter app example is good. I see that loading additional app modules / parts would really help. |
I also need it. With this functionality, I would definitely use Flutter in my project. Without it, I have to seek other solutions. |
The team currently does not have any plans to provide support for modular AOT deployments (at least in short term). If you need this sort of functionality you either need to run in JIT (which supports dynamic code loading) or run your modules in separate isolates - which is supported in AOT - you can use |
It isn't may yet, but github shows that it's 10 years already. Someone should make a meme with that skeleton sitting on a chair underwater. |
That's sad, Isolates won't work for this. They may be good for doing background work, but not for adding new features to an app (and rendering widgets). |
If you want to allow new features to bring in arbitrary widgets then things get complicated, yes. But if you limit your features to a fixed set of widgets they can use (those built into the main app), then I think it is possible to use isolates: you just need to develop an API for building widget trees out of supported widgets and sending them to the main isolate to render. It's a bit of boilerplate work (you would have to duplicate API surface of all widget classes you need to use), but it is hardly infeasible. Alternatively you could build on top of something like https://pub.dev/packages/rfw instead. |
Found this project that may be a adequate (albeit very heavy) substitute but it's still WIP: https://pub.dev/packages/dart_eval _dart_eval is an extensible bytecode compiler and interpreter for the Dart language, written in Dart, enabling dynamic codepush for Flutter and Dart AOT. The primary aspect of dart_eval's goal is to be interoperable with real Dart code. Classes created in 'real Dart' can be used inside the interpreter with a wrapper, and classes created in the interpreter can be used outside it by creating an interface and bridge class. dart_eval's compiler is powered under the hood by the Dart analyzer, so it achieves 100% correct and up-to-date parsing (although compilation and evaluation aren't quite there yet.) Currently dart_eval implements a decent amount of the Dart spec, but there are still missing features like generators, Sets and extension methods. In addition, much of the standard library hasn't been implemented._ |
This issue was originally filed by @dam0vm3nt
This feature was initally described in
Issue #3819: Provide a Method to Dynamicaly Load Classes
But it has been closed pretending it was done. IT IS NOT as Mr. gbracha honestly admits: "... where the code being loaded is not known statically. This will require the ability advanced capabilities like mirror builders. This should be a separate issue."
So I'm posting that separate issue hoping sometime in the future will be adressed as it is the only reason stopping me to adopt dart.
In other word: We need some mechanism to dyamically load classes not know at compile time by another module. Something like java class loaders / Flex Module / JS AMD.
The text was updated successfully, but these errors were encountered: