-
Notifications
You must be signed in to change notification settings - Fork 202
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Effects on modular compilation of compile time user code #1483
Comments
cc @natebosch |
Proposed Solution: Split compile time and runtime depsThe general idea is to provide full Dart files (or full kernel files) for dependencies which are imported (transitively) by any library which is used at compile time (in a macro or as a part of constant evaluation). All other deps would only be provided as outlines. This doesn't require any language feature, it could be fully done through other static configuration (in bazel) or potentially automatic detection (in build_runner). We may want to investigate a language feature to specify compile time imports but the benefit would be potentially more consistent tree-shaking of compile time imports in the runtime application and not likely anything else. This does not fully solve the issue, but it should significantly mitigate the issue:
Concrete proposal for specific build systemsBazel and similar build systemsFor static metaprogramming librariesIn build systems such as bazel where we require up front configuration prior to reading any Dart files, the For constant evaluationIn build systems such as bazel where we require up front configuration prior to reading any Dart files, we should split up the cc @davidmorgan Build_runner and more dynamic build systemsBuild systems that have the ability to more dynamically choose inputs after reading Dart files, may choose to automatically do detection of libraries that define static metaprogramming features or do const evaluation, removing the need for explicit configuration. This is likely actually a requirement to be able to use libraries that were created without knowledge of build_runner. |
Can someone explain to me what the characteristics of each of these terms are? What do you mean when you say 'metaprogramming' and what do you mean when you say 'constant evaluation'?
Am I the only one worried about the usage at compile time just increasing until eventually everything has to be full files? What is suggested here - as I see it - is basically to create a feature we will then (have to) tell the user not to use (or stuff will become slow). (as a semi-unreleated side-note - if I recall correctly - I saw somewhere that the constant evaluation should also be allowed to read files which would mean - for things to work as if it was a single-shot compile - that that file (and it's content) would have to be tracked as well. It goes without saying that I - and the complexity of all 'build systems' as well as the compile time speed - wouldn't like that.)
(Which will probably grow over time to eventually be all of them.)
Please remember that the incremental compiler is also impacted by this (it too is basically split into outline and full when running with the 'experimental invalidation' which is becoming default). You might argue that it's just a "dynamic build system", but I certainly think it's more helpful to have it top of mind. |
MetaprogrammingThe general idea here is that you have code defined in some library that supports "metaprogramming". We don't have a concrete proposal but you could imagine defining a function that takes an ast node and modifies it. The important distinction here is that the only code running at compile time is code that lives in the library that implements metaprogramming features, and not code that uses those metaprogramming libraries. In blaze etc we could just create for instance a Constant evaluationSpecifically here I am talking about allowing you to invoke arbitrary functions in constant expressions. This is more concerning because it could happen in any library anywhere in the program, causing all of the transitive deps of any apis they call to be pulled in as full kernel files. Note the distinction though for
I don't share the same level of concern here, for a few reasons:
I think this is heavily mitigated by the explicit compile time deps separation.
It is worth noting that all kernel compilation outside of DDC today uses full kernel files already. So I don't forsee any sort of apocalyptic scenario here with regards to build times. For instance we should also be able to do the same "used input" tracking we already do during the interpretation of these constants, and mark most of the transitive inputs as not actually used, which would help invalidation a lot for some of the more pathological cases.
Allowing the reading of files will be separately addressed and if we can't do it efficiently (and safely) then we just won't allow it. To start all of |
I don't understand. If we have a package I somewhat understand how this could work with code generation currently - we have a separate action that depends on |
If an arbitrary function is executed at compile time, and that function is changed we don't know what it has been changed into. It might now call other functions that it didn't before. In fact it might call all functions that it can possibly reach. Those might not have been called before and they too might call whatever they can reach. That certainly seems like the potential for all transitive dependencies to me. If looking only at the incremental compiler for a minute, (with advanced invalidation) what happens is this:
Said another way, if you modify a "regular function" we will only recompile that file. With the proposed constant evaluation (as I understand it) this no longer works.
Except - as noted above - we can't know what is now needed for constant evaluation before having compiled.
This is very specific to (I'm guessing) blaze.
Again this is about something like blaze. Also - and I may be reading this wrong - it seems to say it's okay to make all DDC users wait (much?) longer.
As noted internally (and above) I don't think it's a given that we can do this. |
Right, what I was trying to call out is the amount of code that needs to be executed at compile time doesn't grow with the size of the app, its always just
None of this is clear yet, we may still want to output real files to disk as there are a lot of advantages to that. It makes the IDE experience more seamless if it can navigate to real files as an example. Likely the code generation would happen in the same analyzer action though, and it would likely also happen in the kernel actions. Possibly we would try to share that but its likely faster in blaze at least to just duplicate the work. |
Yes, the set of functions actually invoked at the compile time could be identical to the set of transitive dependencies, but that is unlikely. That would essentially imply the entire program being one const object. Maybe a few programs would fall under that category, but not most (and the ones that did it would be explicitly what the user was asking for anyways?).
I don't understand why this would be challenging. You could add a single boolean flag to each function ast node, indicating if it was evaluated at compile time. This would get serialized to kernel as a part of the node (some question here about modular builds, it could likely be omitted there though). In the constant evaluator (or macro expander) whenever you handle a function invocation you flip the bit on that ast node to true. Whenever a function body is modified, you check the bit and invalidate all transitive deps if it was set, and then you flip the bit back to false. I might be glossing over something here but it sounds pretty tractable to me?
This is why we require an explicit compile time deps section in bazel, separate from normal deps. In build systems that require static configuration of inputs we would have to require the user to configure this (tools can fill this in though, but it needs to be static build config).
We don't know what the impact would be. My implication here is that DDC is actually the outlier - all the other workflows already use full dill files today. Ultimately no we would not accept a significant regression in DDC performance for this feature. Note that in external flutter DDC users use the frontend server (and incremental compiler). We could invest in switching internal users over to this as well, although it doesn't play nicely with codegen so its a lot of work. Codegen is heavily used for angular apps internally so it would need to work well and that is the main reason we haven't switched yet. |
The entire program does not have to be one const object (or I'm misunderstanding what you mean by that). There has to be one thing used from each library (at least in the CFE where we invalidate/recompile at library level, not finer grained than that). Of course "every library" might be unlikely but if it's n or n/2 doesn't really matter.
To me, even if it's just some select users, or only once in a while, making it slower because of a feature that we give them is not okay. I doubt the user would explicitly ask for the compile to be slow. The user has been given a feature that the user then uses --- and being penalized for it.
What you're suggesting might work if we always started from scratch (and always 'went forward'), but we don't: |
This isn't as black and white as you are painting it, its all about the trade-offs. Most language features are this way, and come at a cost. For instance one could argue In this case, one concrete trade-off is doing more work at compile time in order to do less work at runtime. That very well might be worth it, especially if it means your production app is faster. We should be evaluating the costs (build time, implementation, etc) versus the benefits and the proposal should live or die based on those metrics.
I think I am missing something here, in all the examples I have seen using the frontend server there is only a single dill file. Is that being incrementally serialized to disk? This sounds similar to the modular build problem though. I would suggest that whenever we are in a mode where we are outputting kernel files in a depth-first order (in terms of the library dependency graph), that we should not serialize this bit at all. Whenever initializing from a dill file that does not have this bit set, you would revert the very first build to the more pessimistic invalidation strategy.
Can you explain this case a bit more? I am not familiar with the details of how it works. |
You could make the exact same argument about static types. Static checking makes compilation a lot slower, and they definitely didn't ask for it to be slower. The more types they use, the more types have to be checked. Dart would compile much faster if it was fully dynamically typed. Of course, it would compile to slower code, but the compilation would be faster. Which is also exactly the case with being able to do more execution at compile time that currently has to happen at runtime. You could also make the same argument about compiler optimizations. But users regularly build with You're right that users don't want slow compiles. But they do want the useful behavior that the compiler is busy spending its time doing for them. It's up to us to design features that provide as much value as possible and implementations that spend that time as efficiently. But it's not the case that any additional work happening at compile time is strictly user hostile. It's simply another kind of cost that needs to provide enough benefit. Users like stuff happening at build time because that catches and reports errors early and makes their runtime smaller and faster. They just want to spend that build time as efficiently as possible. |
Yes, but... we make a lot of compromises in the Dart type system in the name of fast compilation. Perhaps rightly, perhaps wrongly, but we do. I sort of think this discussion is getting a little philosophical in a way that I'm not sure is moving us forward. I think the concern that @jensjoha is raising is entirely valid (and I think we are all broadly on agreement on it): namely, that we cannot afford to ship a feature here which results in significant degradation of the user experience. Our goal here (and this thread has been really helpful towards this) is to understand what we can do, and what it will cost. Personally, I think I would set out a few concrete goals here for anything that we ship in this space:
I realize that the last in particular is very squishy. What is a reasonable use? What is commensurate? I think looking at existing codegen uses is a good start. If existing uses of codegen can be eliminated, and made faster in the process, then that's a clear win under this criteria. There is an obvious danger, already called out here though, that the convenience of the feature may lead to much more ubiquitous usage. This is something we will need to try to account for analytically or experimentally. Good discussion! |
To my mind these are very different.
Sure. Everything is a trade-off. What I'm worried about is that when looking at this, the looking is very narrow. For production apps where - I'm assuming - incremental compilation is not really a thing (I'm, maybe naively, assuming that one does a single-shot build for a release which also suddenly has to go through aot-phases etc) the trade-off looks very different than for the hot-reload scenario.
Well... Things are complicated. We also do "incremental serialization" so that we don't have to re-serialize (for instance) the flutter package if it doesn't change. Your suggestion allows user code to change used packages (e.g. the flutter package) and we thus couldn't do that anymore.
Now we're again - it seems - moving to a blaze or blaze-like build system.
The protocol is that the server is asked to compile something and the vm then responds with either accept or reject.
I'd argue yes and no here. Yes, static types makes it slower. But - at least to my knowledge - it's proportional. Every library becomes some factor slower to compile.
This makes me more uneasy, because this - to me at least - focuses very much on a different scenario that the one I worry about. The way I read this you're talking about a one-shot (AOT) compile where trade-offs are naturally entirely different.
That's not the argument I'm trying to make. The argument I'm trying to make is that I'm afraid the trade-offs involved is being looked at one-sidedly.
Again this depends very much on the scenario. For hot-reload and test runs etc I don't think this is generally true. Also I don't see how the constant evaluation proposed catches and reports more errors at compile time --- the way I see it it will only make debugging any errors they find later that more challenging.
Everything else being equal, for the incremental compilation to still be sound we would (at least):
We might to some extend be able to engineer our way out of some of this (but certainly not all), but I think it will come at a significant cost both in engineering time (building out the necessary tracking) and in runtime cost and memory usage (in the extra overhead for doing the tracking). |
This is still paying for actual usage, but the usage is at a distance yes, not at the function definition site. If the tests don't do any const eval using that function, it won't incur any cost. It is true that won't take into account the runtime usage of constants.
This particular issue was filed to look at the effects on modular compilation which is why the discussion/proposal I had is more focused on that explicitly. The general proposal does call out all the other use cases, such as the frontend server (or incremental compiler in general) and expression evaluation as well. We are definitely taking a broad look here, but we hadn't had any concerns raised yet on the incremental compilation side of things and so we didn't yet have an issue filed. I think you have raised enough concerns that it should definitely have its own issue/solutions proposed, but that is why this issue/proposal seems so one-sided :). I will go ahead and create that issue today with my proposed solution and we can iterate on that there.
It doesn't have to happen in this issue, but I am curious to understand more here, in what scenarios would the VM reject a change? However, I think this gives further weight to the idea of never serializing this bit. It does not have any semantic meaning for the program, it is purely used for better invalidation. If the bit is never serialized, the VM would never see it no matter what state it is in, and it wouldn't matter if the bit internally changed in the incremental compiler between runs.
Ya I agree the release compiles have entirely different trade-offs than dev builds. We could create a separate issue to track any concerns with those types of builds as well, I don't expect this proposal to have as much of an effect on them though.
I think the proposal here does allow both of these things to be enabled. Or at least I am not convinced it doesn't :). |
I agree very much with your general point that we need to take incremental/hot-reload iteration time into account when doing cost/benefit analysis. I think that is implicit in how Jake and I think about the feature—we fully understand that hot reload is one of Dart's most valuable features—but it's always good to have it re-emphasized.
Any "runtime" error thrown during constant evaluation becomes a compile time error. This means that existing code that does validation and throws on failure could potentially turn into compile-time validation. For example, right now the language gives you no compile-time validation that your RegExp strings are actually valid regular expressions. If you forget to escape a If we are able to make that constructor Basically, any class that does useful validation in its constructor body can't make that constructor const today, which means that validation only happens at runtime. Enhanced const allows const constructors with interesting bodies, and that in turn lets that validation run at compile time. Conversely, some classes simply don't validate everything that they could in order to enable their constructors to be const. For example, the Tolerance class in Flutter is documented to say: /// The arguments should all be positive values. But it doesn't actually validate that at all. Enhanced const would let them write: const Tolerance({
this.distance = _epsilonDefault,
this.time = _epsilonDefault,
this.velocity = _epsilonDefault,
}) {
if (distance <= 0.0) throw ArgumentError.name('distance');
if (time <= 0.0) throw ArgumentError.name('time');
if (velocity <= 0.0) throw ArgumentError.name('velocity');
} |
The same thing is already possible without enhanced const: const Tolerance({
this.distance = _epsilonDefault,
this.time = _epsilonDefault,
this.velocity = _epsilonDefault,
}) : assert(distance >= 0),
assert(time >= 0),
assert(velocity >= 0); which gives:
when trying something like (And also - to me - this makes it seem like you expect there to be lots of usage, which again fires my worries.) |
Interesting. I thought that was prohibited because you can't perform operations on doubles in const expressions. Maybe comparisons are allowed? |
@jakemac53, @munificent, how compatible would you say #1565 is with these concepts? (Details for the proposal are under the "Implementation" section.) Borrowing from @scheglov, this is roughly what it looks like:
So I didn't get into incremental or modular compilation. Instead, I kept the code generation separate from the actual code, such that each phase has its own compilation and execution. |
@Levi-Lesches that sounds similar to how the current build_runner codegen works - it is possible we may want some shared thing that expands macros but it would come at a very large cost. That thing has to fully parse and analyze the program in order to expand macros, and if that parsing/analysis is not shared with the compiler or analyzer then it is all duplicate work. Basically, yes it could help the analysis/compilation steps but the new steps which are added still have the same problem that analysis/compilation had before, and the total number of steps is roughly doubled. Each "step" in modular compilation carries a fair bit of overhead, and so doubling the size of the "critical path" (the longest sequential path of actions required to compile the app), does come with its own problems and is probably not desirable. |
Makes sense. So instead of starting and stopping, we'd like to find a way to say "we know everything we need to generate this code", then once it's generated, pick up where we left off? |
There are a couple pieces to this - but yes we would want to avoid restarting the compiler entirely after generating each piece of code as is currently suggested. This isn't actually directly related to the problem at hand in this issue though - which is the problem of needing full source inputs in order to run macros, which negatively affects invalidation and the size and quantity of inputs. Your proposal doesn't make that fundamental piece any better or worse really (as far as I can tell at least). |
Okay that's good. I don't have any experience working with compilers so I tried going for as neutral as possible. I'm gonna stay subscribed to this issue and hopefully learn something. |
I suspect this is obsolete, if not please reopen. |
A general concern for using Dart as a static metaprogramming language is that we will need the full code available for any dependencies that are executed at compile time.
Today both of our modular build systems (bazel and build_runner) declare all their dependencies up front, and when compiling for the web as well as running analysis with the analyzer (bazel only) we only supply what we refer to as the outline of the dependencies. These outlines define the api surface area, and enough information to evaluate constant expressions (or read already evaluated constants), but not method bodies, comments, or other unnecessary information.
In order to support running dart code at compile time (either as a part of static metaprogramming or enhanced constant evaluation), we would have to provide the full kernel or Dart source code for anything deps that are a part of compile time execution.
Downsides of providing full Dart inputs
There are some downsides to simply providing the full dart files for all transitive dependencies:
Invalidation
The largest bonus we get from using outlines is actually that it significantly improves our invalidation story. Because they contain less information, build systems which use file contents for invalidation (build_runner, bazel) get major benefits. It means that libraries that depend on the library you are editing only have to be recompiled if you change the api shape of your library (edit something public, basically, although its a bit more complicated). Editing method bodies or comments is very efficient. We would lose these properties if we just switched to using full Dart inputs without mitigating this issue somehow.
Individual action efficiency
Outlines are significantly smaller than full Dart files, and they also typically encapsulate the outline of several dart libraries into a single file. Not using outlines for analyzer means that we would have both more inputs and larger inputs to all actions.
For kernel compilation we could still use a kernel file as input (so same total # of files) but they would be larger than the outlines.
The text was updated successfully, but these errors were encountered: