Design new language/runtime architecture

[joeduffy]

There are many questions swirling about with respect to our languages and runtime work (and, related to that, our package management story). Although we won't get to the actual work in 0.4, we need to end 0.4 with a plan for a plan during 0.5. This work item tracks the plan for a plan.

[joeduffy]

We have a plan. See below for an overview:

I've been mulling over languages and runtimes a lot.

I am increasingly hopeful that, with the new execution model put in place last sprint, and our recent decision on multi-language, we can get by with mostly unmodified native runtimes for all the languages we wish to support. In other words, V8 for JavaScript, CPython for Python, and so on.

The bad news is that this upends quite a bit of code we've written. The good news is we get to toss lots of it and we end up with perfect language compatibility and cheaper language bring-ups.

In a nutshell, we have the Lumi configuration and planning engine on the left side of an RPC interface. And the collection of supported native language runtimes on the right side. This connection looks a lot like the new execution model rendezvous between the planning and interpreter goroutines. But obviously out-of-proc rather than in-proc.

To pull this off, we need to hook certain events in the language runtimes to perform the RPC calls. The most important is resource object allocation. In some cases, we can do this without modifying the runtime; for example, in the LumiJS case, I suspect we can have a native C shim. Even this may require some light touch edits to the runtime, but it would be minimal.

The most complex thing to support is all the output properties work we did recently, due to the speculative nature. As I mentioned in #90, there is a fascinating relationship between these properties and promises. In fact, we can eradicate all runtime magic simply by representing them as promises. And that's precisely what I recommend we do: all output properties become promises. This not only allows us to eliminate some extremely complex magical runtime machinery, but is also easier to understand and more transparent about what is actually happening.

This lands us in a pretty nice place:

• No more LumiIL required; we just accept the native language's format (source for dynamic languages)
• BYOC: no more LumiJS compiler required; use TypeScript if you want, or your favorite transpiler, we don't care
• We can leverage all of the native package managers
• Add this to our bridge for Terraform plugins, and man are we cooking with gas, in terms of instant breadth across many scenarios.

From there, we can decide to add restrictions on what you can do. For example, similar to Google's App Engine and Deployment Manager, perhaps we disallow I/O syscalls for purposes of configuration. This requires that we maintain a fork of the respective runtimes, but the quantity of edits will be far less extreme than what would've been required for output properties, for instance.

This is a huge change in direction and so I am not taking it lightly. I wanted to seed the ideas so we can begin discussing as we approach 0.5, when I propose that we do the work.

[joeduffy]

Here is the basic plan of attack.

There will be an RPC connection between a monitor and the program running in the runtime of choice. The monitor is what watches resource object operations and performs deployment planning. The runtime runs the program, calling back into the monitor at select points.

To evaluate a program, we no longer run an interpreter. Instead, we invoke a shim that wires up the RPC connection back to the monitor and lets the program loose. At select points in the program, the code will call back through the RPC interface to the monitor. This includes:

• Reading config.
• Resource object allocation.

This is a slight change from our current model in a few ways.

Instead of magically poking config variables, reading a config variable will read from the Pulumi monitor state.

Resource object allocation is interesting. I envision that every resource constructor will terminate with a call to something like lumi.registerResource(this). Internally, this will scrape the state and use it as inputs, and present the object allocation to the monitor. It will also swap out all output properties with promises. The monitor will then later be capable of scheduling promise resolutions into the runtime's message loop, thereby resolving them when the cloud operation finishes. Because of the use of promises, this permits us to parallelize as much as we'd like on the monitor side, while keeping the runtime side simple and idiomatic for that language. The switch to output properties as promises is the major user-visible impact of this change.

At this point in the plan of attack, there is no runtime fork. But it's still unclear how to get the lambda closure serialization to work. This may push us towards a fork, although we will attempt to resist it. In the long run, a fork is probably inevitable, so we can enforce things like "no I/O", execution limits when run in a hosted environment, and so on.

[joeduffy]

I'm not sure promises will work out. We want something like a promise, but not precisely a promise. The problem with a promise is that we'd have some computations that could hang forever during planning since we won't actually carry out the actions. All it takes is a simple:

let res = new RandomResource(...);
let v = await res.outputProperty; // oops 😢 

I am thinking of introducing a new type, Computed<T>, that is like a promise but not awaitable. It never fails and is always unresolved during planning and always gets resolved at the right moment during deployments. It is printable and it has a single function on it, then, that is a little like Promise<T>'s then function, except that it just exists to create a derived computed value: it takes a function that receives the value and returns a derived Promise<U>. E.g. someValue.then(s => s + "-blargh"). There is also a Computed.all in case you need to join multiple values together.

The resolution of these things still happens at message loop boundaries, exactly like a Promise<T>, and permits the Pulumi deployment layer to do everything in parallel as it sees fit.

[lukehoban]

I am thinking of introducing a new type, Computed, that is like a promise but not awaitable.

This feels strange. await is not some fundamental thing, it's just a convenience for a call to .then(...). So saying that we want it to be a promise but not be awaitable doesn't really make sense. Any problem we can run into with await we can just as easily run into with .then.

I think you are trying to create some deeper distinction where there are two separate execution phases (run the program, then run the deployment), and you don't want the first phase to ever block on the second phase.

But if that's the goal, then as soon as you expose the the ability to wait on values that are outputs of the first phase into the user code, you are opening up to this ability to accidentally depend on it.

This would also seem to introduce a third execution context for code:

1. deployment planning
2. deployment resource allocations (user code like s + "-blargh" above runs here?)
3. runtime

Can you describe what the ideal execution model looks like?

[joeduffy]

It's true the mechanics are similar (although not identical since you won't be able to physically await one of these). The intended uses are fundamentally different. The only thing in then you are supposed to do is compute a new value, not perform side effects or otherwise block the program. In practice, this is exactly how computed properties are used.

[lukehoban]

What does "physically await" mean?

FWIW - if you have a then method, you can be awaited:

var x = {}
x.then = function(f) { setTimeout(f, 1000); }
var f = async function() { await x; console.log(3); }
f() // prints 3 after a delay

But even if you take away await (renaming the method I suppose) - JS developers seem likely to just go with that they would typically write instead:

let f = new Function();
f.arn.then(arn => {
    let target = new EventTarget({
        arn: "target-" + arn,
    });
});

Which I believe has the same problems. I believe you are hoping they will write this instead:

let f = new Function();
let target = new EventTarget({
    arn: f.arn.then(arn => "target-"+arn),
});

But I'm not sure that's what you'll most often end up with. This kind of Promises-as-values programming isn't very common in JS.

[joeduffy]

Basically, promise and computed have very different use cases. Even if the implementations were identical, I wouldn't want to confuse people by calling them the same thing. Computed values are meant to be used in purely a dataflow style, nothing more, nothing less. They are conceptually distinct.

I realize "promises-as-values" isn't very common in JS, but then again, declaratively provisioning infrastructure by diffing objects isn't either ... I actually don't worry too much here, because it will be rather uncommon to do this. And furthermore, I will learn much more seeing how this gets used in practice than debating this conceptually. I am just not smart enough to envision it all.

I got the impression from everything I read that the ability to await is based on the Promise constructor, not duck typing then methods, but if this is true, I guess we'll need to pick another name. (Yay duck typing.) I was going to dig into specs tomorrow, so thank you for saving me time!

Another alternative if we want to constrain things (which I do) is to eschew the then method in favor of things like thenAdd(5), etc., essentially mirroring the speculative operators we support today. Although we may end up there, I'd actually prefer to use then and see how people want to use it. Especially because you may want to do things like concat, substring, ... and so many more things we can't possible envision in advance. Can you shoot yourself in the foot? Yes. Will we do our best to guide you down the better path while still giving you what you need to get your job done? Yes. This feels like the sweet spot to me IMHO.

[joeduffy]

We have decided on a design. It will be documented thoroughly in #323 and implemented as part of #311.